As noted in #36073, having support for the different protocols for
exporting OpenTelemetry data would be useful, especially if this is
leading to users seeing issues when they have auto-injected
configuration being set.
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
* test: replace manual process.env mutation with vi.stubEnv
Enable Vitest's `unstubEnvs` so env stubs are reverted automatically
before every test, and convert the manual `process.env.X = ...` /
`delete process.env.X` pairs in specs to `vi.stubEnv()`.
This removes the hand-written `afterEach` cleanup blocks and the
`const OLD_ENV = process.env` / `process.env = { ...OLD_ENV }`
save-restore dance, which silently leaked state between tests when a
test set a variable without a matching delete.
Left as-is (incompatible with `unstubEnvs`, which unstubs before each
test, not after):
- module top-level env writes (cocoapods, gomod, post-update, pip-compile
and bundler artifacts specs)
- `beforeAll` env writes -- the two in `util/http` moved to `beforeEach`
* test: stop asserting on ambient environment variables
Three specs asserted on values that the surrounding environment can
override, so they only passed on a machine that happened not to set them:
- `util/exec/env`: `getChildProcessEnv()` forwards every entry of
`basicEnvVars`, but the spec only seeded a subset. Any other forwarded
variable that was set ambiently (e.g. a lowercase `http_proxy`) leaked
into the `toEqual` assertion. Clear all of them first.
- `util/git/index`: `process.env.GIT_SSH_COMMAND` deliberately overrides
the default that the two env-inheritance tests assert on. Clear it.
- `logger/pretty-stdout`: the colorized level strings are built at import
time, so the `beforeEach` setting `FORCE_COLOR` could never affect them
-- it was dead code, and the test silently assumed a terminal without
colour support. Derive the expectation the same way the module does.
* test(datasource/nuget): use a throwaway cache dir for nupkg downloads
The `determine source URL from nupkg` tests do not mock the filesystem,
so they really wrote downloaded packages into a fixed `/tmp/cache` path.
That fails wherever the system tmpdir is not writable, and two checkouts
running the suite at once would share the same directory.
Use `tmp.dir({ unsafeCleanup: true })` and clean it up afterwards, which
is the pattern the other specs that touch disk already use.
* feat(lint): add renovate/prefer-stub-env rule
Enforces in specs what the migration to `vi.stubEnv` established:
- assigning to or deleting `process.env.X` inside `beforeEach`, `afterEach`,
`it` or `test` is reported, with a fixer that rewrites it to `vi.stubEnv()`
- module scope, `describe` bodies and `beforeAll` / `afterAll` are left alone:
`unstubEnvs` reverts stubs *before* each test, so a stub created there is
gone by the time the first test runs, and those must keep assigning directly
- replacing the `process.env` object is reported everywhere and never fixed.
`vi.stubEnv()` captures the original object when the worker starts, so after
a reassignment its deletes silently target the old object
The rule found 28 further violations in specs the earlier sweep missed, since
that one was anchored on `delete process.env`. Twelve were auto-fixed; the
rest replaced `process.env` wholesale and are now `clearEnv()` from test/util.
`util/exec/index.spec.ts` opts out with a comment: it asserts on the whole
child env, and `vi.stubEnv()` cannot express that because it maps `PROD`,
`DEV` and `SSR` onto '' instead of deleting them.
* feat(instrumentation): allow exporting OTLP JSON file
We've recently been investing in improvements to Renovate's
OpenTelemetry instrumentation to improve debuggability and
supportability.
However, not everyone has an immediately available OpenTelemetry
Collector, so the added instrumentation isn't useful to those users.
Instead, we can provide a way to export to the OTLP file format, which
provides a means to export the data to disk and then consume it later.
This will also support use-cases like running in a CI platform like
GitHub Actions and GitLab CI, where OpenTelemetry Collectors may not be
exposed in that environment, but a post-run execution can export it.
We now need to make sure that `RENOVATE_USE_CLOUD_METADATA_SERVICES` is
never set, as `isTracingEnabled` will now resolve true, and the Cloud
Metadata Resource Detectors will be used.
Co-authored-by: Claude Sonnet 4.6 <jamie.tanna+claude-code@mend.io>
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
* fixup! feat(instrumentation): allow exporting OTLP JSON file
* fixup! fixup! feat(instrumentation): allow exporting OTLP JSON file
Co-authored-by: Claude Sonnet 5 <jamie.tanna+claude-code@mend.io>
* fixup! feat(instrumentation): allow exporting OTLP JSON file
---------
Co-authored-by: Claude Sonnet 4.6 <jamie.tanna+claude-code@mend.io>
As part of recent changes to how our logger is initialised, we
accidentally broke the propagation of OpenTelemtry `span_id` and
`trace_id` into the Bunyan log context.
We already registered `BunyanInstrumentation`, but because of the
`import`, the instrumentation could not man-in-the-middle the import and
register the instrumentation.
This is slightly more complex, and brittle, test, but does capture the
current behaviour of `BunyanInstrumentation`, if it were removed.
Co-authored-by: Claude Sonnet 4.6 <jamie.tanna+claude-code@mend.io>
* feat: timing statistics for getReleases
Adds collection and reporting of timing statistics for
`datasource.getReleases()` calls. Each call records the datasource, registryUrl,
packageName, and duration. The intent is to visibility into where
renovate is retrieving release information and how long that might take.
The report consists of a nested data-structure that provides increasing
detail as we descend datasource, registryUrl, and packageName.
The full report is logged at trace level
`getReleases statistics with packages`.
A summary report including only datasources and registryUrls is logged
at debug level `getReleases statistics summary`.
* add GetReleasesSpanProcessor
- Re-instrument getReleases calls using open telemetry.
- Adds GetReleasesSpanProcessor, which can adapt
`getReleases` spans to GetReleasesStats.
* fixup instrumentation tests
* pr feedback
* pr feedback: naming
* pr feedback: comments
* fix(instrumentation): always process Git Operation spans
As part of b5cd0317cf we added the ability
to report the `Git Operations Statistics` at the end of the Renovate
run, by introducing a `SpanProcessor`.
However, this didn't correctly take into account the fact that this
would only be enabled if Renovate was configured to log/export traces,
which is not the case in most deployments.
As noted in #42118, this leads to a blank log line output, which isn't
very useful.
To correct this, we nede to always process these spans, even if tracing
is not enabled.
Co-authored-by: Claude Sonnet 4.6 <jamie.tanna+claude-code@mend.io>
* fixup! fix(instrumentation): always process Git Operation spans
* fixup! fixup! fix(instrumentation): always process Git Operation spans
* fixup! fixup! fix(instrumentation): always process Git Operation spans
---------
Co-authored-by: Claude Sonnet 4.6 <jamie.tanna+claude-code@mend.io>
As noted by Michael, this was likely fixed as part of other changes he'd
made recently around logging.
Co-authored-by: Michael Kriese <1798109+viceice@users.noreply.github.com>
Now we have instrumented our Git operations with tracing, we can make
sure that users who are not using OpenTelemetry still get insights into
their Git operation usage using a debug log output.
Following how other insights are produced, we can introduce a new type
in `lib/util/stats` for `GitOperationStats`, which will track the
overall execution for each operation type, and output them in the same
style as we use for other timing reports.
At the end of a Renovate run, we'll then make sure that this is logged
into a debug log for further investigation.
To create the report, we can make sure that we introduce a new
OpenTelemetry `SpanProcessor`, which will run when we end our Run, and
store any Git operations in `GitOperationStats`.
This adapts code from GPT-4.1 and code from [0].
Co-authored-by: gpt-4.1 <jamie.tanna+github-copilot@mend.io>
[0]: https://github.com/yurisouza/monitoria-sdk/blob/ca9d751fa7c3a785a04f87b8966bb782b4617021/src/adapters/nestjs/span-duration-tracker.ts#L18-L27
When instrumenting, we specifically handle Promises. However, if we have
an object, like SimpleGit's `Response`:
```
export type Response<T> = SimpleGit & Promise<T>;
```
This does not get correctly instrumented, and we end up not waiting for
the function call to complete.
We can make sure we use `isPromise` to correctly check if it's a native
Promise, or something with a Promise-like API.
As part of being able to fully instrument our Git operations - using
SimpleGit - we can introduce some Git-specific instrumentation helpers.
These helpers will be used as part of a future change, where we will
instrument methods of SimpleGit that we're using.
This provides a set of types which will be used as OpenTelemetry
Attributes to categorise command types.
A number of subcommands that aren't currently deemed to be too important
are bucketed into an `other` category.
As the current method only allows decorating on a class method, but it's
convenient to be able to wrap a whole function.
This isn't technically a decorator, but is a wrapper function.
Co-authored-by: gpt-4.1 <jamie.tanna+github-copilot@mend.io>
As part of a step towards better OpenTelemetry instrumentation as part
of #38609, we can start by introducing a known span attribute for the
repository "split" we're currently processing.
This requires a slight refactor of the `SpanOptions`, to use our custom
type, which enforces the `RenovateSplit` type when using the span name.
This adds it into each of the splits we currently instrument.
Co-authored-by: secustor <sebastian@poxhofer.at>
Without this, we cannot call `instrument` with the 4 parameters,
otherwise we receive a Typescript error.
Co-authored-by: gpt-4.1 <jamie.tanna+github-copilot@mend.io>
Jamie Tanna