Now we have instrumented our Git operations with tracing, we can make
sure that users who are not using OpenTelemetry still get insights into
their Git operation usage using a debug log output.
Following how other insights are produced, we can introduce a new type
in `lib/util/stats` for `GitOperationStats`, which will track the
overall execution for each operation type, and output them in the same
style as we use for other timing reports.
At the end of a Renovate run, we'll then make sure that this is logged
into a debug log for further investigation.
To create the report, we can make sure that we introduce a new
OpenTelemetry `SpanProcessor`, which will run when we end our Run, and
store any Git operations in `GitOperationStats`.
This adapts code from GPT-4.1 and code from [0].
Co-authored-by: gpt-4.1 <jamie.tanna+github-copilot@mend.io>
[0]: https://github.com/yurisouza/monitoria-sdk/blob/ca9d751fa7c3a785a04f87b8966bb782b4617021/src/adapters/nestjs/span-duration-tracker.ts#L18-L27
As a means to provide more insight into how long package manager
extractions take when using instrumentation, we can wrap the individual
manager's extractions, as well as the filtering of packageFiles for each
manager.
As a means to provide more insight into how long package manager lookups
take when using instrumentation, we can wrap the fetching of updates in
a trace per manager.
As part of ongoing work to improve OpenTelemetry instrumentation of
Renovate in #38609, we can instrument our branch creation with the
branch name (and relevant Semantic Convention attributes) for
visibility.
When calling `extractDependencies`, we actually perform both `extract`
and `lookup` splits. However, we are instrumenting the whole method call
as the `extract` split, which results in incorrect tracing data.
To correct this, we can wrap the specific code responsible for the
extraction into `extract`, and then the separate code for `lookup`
accordingly.
This also makes sure that `emptyExtract` sets repository splits.
As part of a step towards better OpenTelemetry instrumentation as part
of #38609, we can start by introducing a known span attribute for the
repository "split" we're currently processing.
This requires a slight refactor of the `SpanOptions`, to use our custom
type, which enforces the `RenovateSplit` type when using the span name.
This adds it into each of the splits we currently instrument.
Co-authored-by: secustor <sebastian@poxhofer.at>
* feat: Cleanup mutexes before each run
* Update setup.ts
* Update mutex.spec.ts
* Update setup.ts
* Unmock mutex in deb datasource spec to prevent race condition
* Update index.spec.ts
* feat: add configValidationError option to log config validation errors as errors instead of warnings
* Moved `configValidationError` docs to correct file.
Refactors `getNewValue` logic to replace it with an optional
`getPinnedValue()` where rangeStrategy=pin. This way each versioning
module doesn't need to add and test extensive "pin" logic.
This should not be breaking, but out of an abundance of caution, we'll
mark it as such.
As noted in #37952, we've seen a significant increase in recent years on
a number of package ecosystems being targeted with supply chain security
attacks.
We're in a particularly strong position to improve the level of security
across the world by setting a default for `minimumReleaseAge` in the
`best-practices` config preset.
To do this, we can introduce a new preset,
`security:minimum-release-age` to house these recommended options.
We use new presets `security:minimum-release-age` and
`security:minimum-release-age-npm`, to allow consumers to utilise
`ignorePresets` to undo this configuration, if they so wish.
As a first step towards securing our users, we can set a default for
npm, which as a large ecosystem with an increased number of packages
that depend on other packages (compared to other ecosystems), has been a
more targeted area.
In the future, we can expand this for other managers, as it makes sense,
and as support is added for them to use `minimumReleaseAge`.
Closes#38457.
As part of future changes, where it's more important to know what
`updateType` is being filtered, we will want to have this information
output to the logs.
This also changes the log field outputting, as `depNames` no longer
makes sense now we have both `depName` and `updateType`.
* chore: debug log message when timestamp is optional
When setting minimumReleaseAgeBehaviour to optional
it is intended that packages that does not have a release timestamp
will get updated. Writing a warning to the console is spam for an
intentional setting.
* chore: re-add WARN log, once
To make sure that users are still aware, we can make sure a single
warning log message is emitted.
---------
Co-authored-by: Jamie Tanna <jamie.tanna@mend.io>
* fix: do not modify # within codeblocks
* fix: parse fenced section in markdown
* add test
* Update lib/workers/repository/update/pr/changelog/release-notes.ts
* use re2
* Optimize version filtering logic in filterVersions and lookupUpdates functions
* Simplifies version filtering optimization to cover conditions by existing tests.
* Further reduction of versioning API usage.
* Keeping compatibility of exported function filterVersions()
As part of a step towards better OpenTelemetry instrumentation as part
of #38609, we need to capture the `init` repository split, as the other
splits are currently instrumented.
Note that we've duplicated the error handling, as the `localDir` may be
unset, and we need to handle it.
* use branchPrefix if multiple base branches expected
* fix test
* fix coverage
* apply suggestions
* apply suggestions
* use utility fn
* refactor tests
* fix test
* major overhaul: use baseBranches field internally
suggested by @secustor
* refactor: cleanup
* test: use better descriptions
* add tests
* fix logic
* fix test
* exclude unlisted versions when finding latestStable
---------
Co-authored-by: Rhys Arkins <rhys@arkins.net>
As noted in #38290, #38348, a "rough edge" of the current implementation
of `minimumReleaseAge` is that:
- if we have set `minimumReleaseAge` to apply to a given dependency
- and we have 3 dependencies that need an update
- 2 of which have a `releaseTimestamp` that is not yet "stable"
- 1 of which does not have a `releaseTimestamp`
The dependency update that does not have a `releaseTimestamp` will be
treated as "stable", and will receive an update immediately.
To make this a little more predictable, and explicit, we should add a
new configuration item, `minimumReleaseAgeBehaviour`, to control this
behaviour.
This adds the new configuration field, as well as making it available as
a strongly typed field in the codebase.
In the case that `minimumReleaseAgeBehaviour=timestamp-optional` is
being used, we will WARN on any packages that are being updated, but
maybe shouldn't be, as a way to proactively inform the user of possible
misconfiguration.
Closes#38290, #38348.
While working on #38607 and #38633, I noticed that this doesn't actually
successfully return.
We should document this more clearly for the next person interacting
with it.
While working on #38607 and #38633, I noticed that this doesn't actually
override the value of `localDir`, as `GlobalConfig.set` overrides any
existing configuration, and when we call `renovateRepository`, this also
calls `GlobalConfig` and overrides the value.
When performing lookups in Mend's Merge Confidence APIs, our contract
requires that we use the `packageName` not the `depName`.
In 9b7ae6aa98 and
b67eae2180 we made changes to make sure
that this wasn't the case, but we ended up missing a couple of cases.
chore: add Repository Result metadata to log messages
The `Repository result` log message is useful for understanding
at-a-glance why a Renovate run may have not worked.
Adding the metadata that currently exists in the log line as Meta keys
would allow for logging tools to be able to more meaningfully
interpolate the data, without requiring parsing of the underlying
string.
Co-authored-by: Jamie Tanna <jamie.tanna@elastic.co>
As part of discussion in #38290, #38348 and #38324, the current
behaviour of `minimumReleaseAge` has been flagged as not particularly
great, as when we run with `minimumReleaseAge` enforced, a package that
does not return a `releaseTimestamp` will no longer be treated as
release that is bound by `minimumReleaseAge`'s checks to make it
pending.
As noted in the discussions, this can lead to cases where users are in a
"false sense of security", as they feel comfortable with
`minimumReleaseAge` in place, but it is not being enforced, only when
the `releaseTimestamp` is present.
Before we change this, we should add a test to document the existing
behaviour.
[0]: https://github.com/renovatebot/renovate/discussions/38290
[1]: https://github.com/renovatebot/renovate/discussions/38348
Adam Setch