In the case that we are reading binary files in repositories, we should
not log a WARN that there are "hidden" characters in it, as it's a
binary file.
We can use a common heuristic (at least according to Claude Sonnet 4.5)
to look for null bytes at the beginning of our files, and not log the
warning in that case.
Co-authored-by: Claude Sonnet 4.5 <jamie.tanna+github-copilot@mend.io>
fix(logger): don't include filename in "Hidden Unicode" warning
As noted in #41467, if a repository has many files with these warnings,
the Dependency Dashboard can be flooded, which can reduce the utility,
especially if some of these may be false positives.
We can remove the filename from the WARN message, which will ensure that
only a single message is logged.
However, this will reduce the utility of the log message, as then users
need to check their logs to see the `file`.
As noted in #41381, it's common for a Byte Order Mark (BOM) Unicode
character to exist in a codebase that's used on Windows.
If we only find a BOM, we will report a TRACE log instead of the WARN,
but if any other characters are found, we'll continue with our WARN.
feat(utilgit): log warning if file(s) contain hidden Unicode characters
With the rise of prompt injection, there is an increased number of
attacks occurring using hidden Unicode characters.
These hidden characters can also confuse other tools, so it's best for
Renovate to surface these to repository owners, as we read files that
may be problematic.
To avoid multiple reads of files, we can do this every time our
`readLocalFile` function is called, as we already have the file content.
An alternate option would be that in `workers/repository` we could check
the read package files, but given `extractAllPackageFiles` may be used
by a Manager, we can't inject the logic in, without Managers needing to
implement the logic themselves.
This should be performant, and not introduce too many false-positives,
but this will also depend on how (package) files are set up in
repositories.
The list of suspicious characters has been largely compiled by Claude
Sonnet 4.5.
Co-authored-by: Claude Sonnet 4.5 <jamie.tanna+github-copilot@mend.io>
Jamie Tanna