Files
renovate/lib/util/emoji.ts
Sebastian Poxhofer a1971afa78 fix(linters): add linting rule to enforce usage of our regEx utility (#44690)
* ci(linter): add renovate/require-regex-util rule

Adds a custom oxlint rule that flags raw RegExp usage in lib/
production code: any `new RegExp(...)` call and any regex literal
that is not the direct first argument of `regEx(...)`. The regEx()
helper from lib/util/regex.ts uses the RE2 engine for ReDoS-safe
matching and caches compiled patterns. lib/util/regex.ts itself is
exempt.

The rule is disabled for **/*.spec.ts, test/**, and tools/** via
config overrides, because vitest APIs such as expect.stringMatching
need bare RegExp instances (a past commit reverted regEx() there).

Severity is set to "warn" instead of "error": the repo currently has
197 pre-existing violations (above the agreed mechanical-fix
threshold of 80), and a number of them are deliberately
RE2-incompatible patterns (lookahead/lookbehind/named backreferences,
see TODO #12872/#12875 markers in e.g. lib/modules/versioning/rez/
pattern.ts and lib/modules/manager/dockerfile/extract.ts) that need
per-site judgment before they can be wrapped in regEx() or suppressed
with a justified disable comment. No violation fixes are included in
this commit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* ci(linter): enforce renovate/require-regex-util at error

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor: use regEx() for regex literals and RegExp construction in lib

Burn down all renovate/require-regex-util violations (197):

- wrap RE2-compatible regex literals and RegExp constructions in regEx()
- rewrite lookaround patterns where behavior is provably preserved:
  goproxy separator split, sbt-package link extraction, leiningen
  comment stripping and keyword search, npm GitHub owner validation
  (unit-count bound replaced by an explicit length check)
- keep native RegExp with disable comments where RE2 cannot be used:
  moo / good-enough-parser lexer patterns (moo rejects RE2 instances),
  MigrationsService propertyName (matched via isRegExp, which RE2
  instances fail), rez named backreferences and lookaheads, GitHub
  markdown link negative lookbehind, emoji astral-plane matching
- drop stale TODO #12875 comments for patterns current re2 handles

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor: reduce require-regex-util disable comments

Rewrite the github massage-markdown-links URL regex to capture an
optional `api.` prefix in a named group (skipped in code) instead of
relying on a negative lookbehind, so both the base and global variants
can use regEx()/RE2 and their two disable comments are removed. Add
spec cases for api-prefixed URLs that match the URL shape.

Consolidate the four related line disables in rez/pattern.ts (bound and
range patterns emulating Python conditional patterns via named
backreferences and lookaheads, which RE2 cannot express) into a single
block disable/enable pair, matching the treatment of the lexer files.

Disable comment count drops from 13 to 8.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(datasource/sbt-package): use named capture group in link regex

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(manager/cake): restore original multiLineComment regex spelling

The [^] -> [\\s\\S] rewrite was only needed for an RE2 wrap that was
later reverted; keep the diff minimal.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(datasource/go): skip empty GOPROXY segments

The switch from a lookahead split to a capture-group split (needed because
RE2 has no lookahead support) changed how empty GOPROXY segments parse:
`a|,b` yielded `{ url: ',', fallback: '|' }` before and
`{ url: '', fallback: ',' }` after. The comma fallback only continues on
404/410, so a failing proxy ahead of an empty segment silently stopped the
lookup instead of falling through to the remaining proxies.

Empty segments carry no url to query, so drop them.

* refactor(datasource/go): use isTruthy for empty segment filter

* fix(datasource/go): filter empty segments before mapping

Filtering after the map broke the contextual typing from the
GoproxyItem[] annotation, widening fallback to string.

* refactor(platform/github): use regEx() for merge queue query stripping

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-14 14:30:46 +00:00

147 lines
4.0 KiB
TypeScript
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
import mathiasBynensEmojiRegex from 'emoji-regex';
import {
fromCodepointToUnicode,
fromHexcodeToCodepoint,
fromUnicodeToHexcode,
} from 'emojibase';
import emojibaseEmojiRegex from 'emojibase-regex/emoji.js';
import SHORTCODE_REGEX from 'emojibase-regex/shortcode.js';
import { z } from 'zod/v4';
import type { RenovateConfig } from '../config/types.ts';
import dataFiles from '../data-files.generated.ts';
import { logger } from '../logger/index.ts';
import { regEx } from './regex.ts';
import { Result } from './result.ts';
import { Json } from './schema-utils/index.ts';
import { coerceString } from './string.ts';
let unicodeEmoji = true;
let mappingsInitialized = false;
const shortCodesByHex = new Map<string, string>();
const hexCodesByShort = new Map<string, string>();
const EmojiShortcodes = Json.pipe(
z.record(
z.string(),
z.union([z.string().transform((val) => [val]), z.array(z.string())]),
),
);
type EmojiShortcodes = z.infer<typeof EmojiShortcodes>;
const patchedEmojis: EmojiShortcodes = {
'26A0-FE0F': ['warning'], // Colorful warning (⚠️) instead of black and white (⚠)
'2139-FE0F': ['information_source'], // Colorful info (ℹ️) instead of black and white (ℹ)
};
function initMapping(mapping: EmojiShortcodes): void {
for (const [hex, shortcodes] of Object.entries(mapping)) {
const mainShortcode = `:${shortcodes[0]}:`;
shortCodesByHex.set(hex, mainShortcode);
shortCodesByHex.set(stripHexCode(hex), mainShortcode);
for (const shortcode of shortcodes) {
hexCodesByShort.set(`:${shortcode}:`, hex);
}
}
}
function lazyInitMappings(): void {
if (!mappingsInitialized) {
const githubShortcodes = dataFiles.get(
'node_modules/emojibase-data/en/shortcodes/github.json',
);
Result.parse(githubShortcodes, EmojiShortcodes)
.onValue((data) => {
initMapping(data);
initMapping(patchedEmojis);
mappingsInitialized = true;
})
.onError(
/* istanbul ignore next */
(err) => {
logger.warn({ err }, 'Unable to parse emoji shortcodes');
},
);
}
}
export function setEmojiConfig(config: RenovateConfig): void {
unicodeEmoji = !!config.unicodeEmoji;
}
const shortCodeRegex = regEx(
(SHORTCODE_REGEX as unknown as RegExp).source,
'g',
);
export function emojify(text: string): string {
if (!unicodeEmoji) {
return text;
}
lazyInitMappings();
return text.replace(shortCodeRegex, (shortCode) => {
const hexCode = hexCodesByShort.get(shortCode);
return hexCode
? fromCodepointToUnicode(fromHexcodeToCodepoint(hexCode))
: shortCode;
});
}
const emojiRegexSrc = [emojibaseEmojiRegex, mathiasBynensEmojiRegex()].map(
(r) => (r as RegExp).source,
);
// oxlint-disable-next-line renovate/require-regex-util -- RE2 compiles this pattern but does not match astral-plane emoji correctly, TODO #12875
const emojiRegex = new RegExp(`(?:${emojiRegexSrc.join('|')})`, 'g');
const excludedModifiers = new Set([
'20E3',
'200D',
'FE0E',
'FE0F',
'1F3FB',
'1F3FC',
'1F3FD',
'1F3FE',
'1F3FF',
'1F9B0',
'1F9B1',
'1F9B2',
'1F9B3',
]);
export function stripHexCode(input: string): string {
return input
.split('-')
.filter((modifier) => !excludedModifiers.has(modifier))
.join('-');
}
export function unemojify(text: string): string {
if (unicodeEmoji) {
return text;
}
lazyInitMappings();
return text.replace(emojiRegex, (emoji) => {
const hexCode = stripHexCode(fromUnicodeToHexcode(emoji));
const shortCode = shortCodesByHex.get(hexCode);
return coerceString(shortCode, '�');
});
}
function stripEmoji(emoji: string): string {
const hexCode = stripHexCode(fromUnicodeToHexcode(emoji));
// `hexCode` could be empty if `emoji` is a modifier character that isn't
// modifying anything
if (hexCode) {
const codePoint = fromHexcodeToCodepoint(hexCode);
return fromCodepointToUnicode(codePoint);
}
return '';
}
export function stripEmojis(input: string): string {
return input.replace(emojiRegex, stripEmoji);
}