* feat(security): one-click managed Trivy install
Add a Vulnerability Scanner card to Settings, Security with install,
update, uninstall, and auto-update controls (Admiral-only). The installer
downloads a verified Trivy release into the existing data volume at
/app/data/bin/trivy and defaults the cache to /app/data/trivy-cache, so
no host mounts or extra env vars are required. Detection probes the
managed path, a TRIVY_BIN override, and the host PATH, distinguishing
managed vs host installs. A daily scheduled check surfaces available
Trivy updates, installs them automatically when opted in, and dedupes
notifications per version.
* fix(frontend): silence react-hooks/set-state-in-effect in useTrivyStatus
The initial status fetch and managed-source update check both call
setState from the effect body. Match the existing pattern used in
useDashboardData / SSOSection and disable the rule at the call site.
* fix(security): harden Trivy scan lifecycle, logging, and docs
- Call TrivyService.initialize() at startup so capability state is
accurate before first request; add periodic re-detect to the scheduler
so newly installed Trivy binaries light up without a restart.
- Add markStaleScansAsFailed sweep (+ idx_vuln_scans_status index) to
recover any scan row left in_progress after a crash or timeout; sweep
runs before the paid-tier gate so every tier self-heals.
- Split scanImage persistence into beginScan/finishScan so the manual
scan route owns a single code path and can return a scanId synchronously
while work continues asynchronously.
- Validate image refs on /api/security/scan and /sbom via new utility;
defense-in-depth against shell-metacharacter payloads.
- Dispatch a warning-level alert when a post-deploy scan fails so the
operator has a user-visible path to the failure instead of a silent log.
- Share DIGEST_CACHE_TTL_MS and severity ordering across service and
route layers; remove dead invalidateDetection().
- Add [Trivy:diag] logging gated behind developer_mode for support
diagnostics; production logs unchanged.
- Frontend: defensive toast fallback chain, sr-only SheetDescription,
and a truncation badge when the 500-item detail fetch is capped.
- Tests: extend trivy-service and vulnerability-db suites; add
image-ref and severity unit tests.
- Docs: expand vulnerability-scanning troubleshooting with recovery,
re-detect, and diagnostic-log guidance; link Dockerfile comment to
trivy-setup.
* fix(security): drop unnecessary escape in image-ref forbidden-char regex
Adds a workflow that fires on every v* tag push, computes the window
since the latest release post's anchor version in the sencho-website
repo, and (when the window reaches 5) opens a draft PR in
sencho-website with a fully pre-filled scaffold: grouped
ChangelogSection blocks parsed from CHANGELOG.md, version field set to
the new anchor, and a checklist body covering intro, screenshot, docs
link, and reading time. No auto-publish; the human writes the
narrative.
Uses the same sencho-token-app GitHub App as release-please, with its
installation extended to include sencho-website.
* feat(images): Trivy-powered vulnerability scanning
Scan container images for known CVEs via Trivy. On-demand scanning and
severity badges are available on every tier; scheduled scans, scan
policies, SBOM generation, and scan history are gated to Skipper+.
- New TrivyService (binary detection, per-image scan, SBOM, digest cache)
- Three new tables: vulnerability_scans, vulnerability_details, scan_policies
- 12 routes under /api/security (scan, results, summaries, SBOM, policies, compare)
- Post-deploy async scans wired into all five deploy paths, with a
per-deploy opt-out toggle in the App Store deploy sheet
- "scan" action type added to SchedulerService for fleet-wide recurring scans
- Frontend: severity badges in Resources Hub with animated cursor detail,
scan results drawer with vulnerability table and filters, and a new
Security section in Settings for scan policy CRUD
- Policy threshold violations dispatch a warning or critical alert based on
the policy's block_on_deploy flag; deploys themselves are never blocked
* fix(security): compute scan age in useEffect to satisfy react-hooks/purity
Add docs/security.mdx with the full auth stack as the lead section:
password auth, SSO (5 providers + generic OIDC), TOTP MFA, JWT sessions,
API tokens, RBAC (5 roles), audit logging, AES-256-GCM encryption,
rate limiting, and node-to-node authentication.
Includes tier availability matrix, permission matrix, production
hardening checklist, and vulnerability reporting section. Links to
dedicated feature pages for configuration details.
Add navigation entry in docs.json Reference group.
* feat: add Custom OIDC provider and move SSO to Community tier
Add a generic Custom OIDC provider that works with any spec-compliant
OIDC identity provider (Keycloak, Authentik, Authelia, Zitadel, KanIDM,
Pocket ID, etc.) via standard discovery. Supports configurable claim
mapping for User ID, Username, and Email fields to handle non-standard
providers.
Move all SSO functionality (LDAP and OIDC) from the Admiral tier to the
Community tier so every user has access to identity provider integration.
Backend: add oidc_custom to AuthProvider type, extend SSOService with
claim mapping fields and env-var seeding, add oidc_custom to route
validation, remove requireAdmiral guards from SSO config endpoints.
Frontend: add Custom OIDC card with Display Name, Issuer URL, and claim
mapping fields to SSOSection; add KeyRound icon on login page; remove
AdmiralGate wrapper and lock icon from SSO settings nav.
Tests: update tier guard expectations, add oidc_custom authorize/config/
provisioning tests and claim mapping coverage. All 992 tests pass.
Docs: add Custom OIDC configuration reference, provider-specific setup
examples, troubleshooting section, and updated screenshots.
* fix: settings dialog close button overlap and combobox styling
Reposition the close button in Settings Hub above the scroll area so
it stays fixed when content scrolls. Increase dialog height to
accommodate the growing number of setting sections.
Fix combobox trigger styling to match Input component tokens
(border-glass-border, bg-input) and eliminate the gap between trigger
and dropdown list (top-full -mt-px). Apply the same fixes to
multi-select-combobox for consistency.
Add items-start to the Scopes/Default Role grid so the combobox
aligns with the adjacent input field. Add showClose prop to
DialogContent for consumers that need custom close button placement.
Update SSO doc screenshots at 1920x900.
* feat(convert): harden /api/convert endpoint with auth, validation, and tests
Applies authMiddleware to the docker run to compose endpoint, validates
that the payload is a non-empty string within an 8192 character budget,
rejects inputs containing null bytes, and wraps composerize in a
try/catch that surfaces a 422 with a clear message when the library
cannot produce a services block. Adds a Vitest suite covering the auth
gate, happy path, common flag coverage, boundary and null byte
placement variants, and malformed command handling.
* feat(editor): add From Docker Run tab to create stack dialog
Introduces a third tab in the Create New Stack dialog that accepts a
docker run command, calls the converter endpoint, and previews the
returned compose YAML before writing it to a new stack directory. Uses
the defensive toast pattern, clears the stale preview when the input
changes, and rolls back the empty stack directory if saving the
converted YAML fails so the user never ends up with an orphan stack.
* docs(stack-management): document docker run to compose converter
Adds a Convert from a docker run command section to the stack
management page covering how to use the new tab, the list of supported
flags, and troubleshooting for unparseable inputs. Screenshots show the
empty tab, a successful conversion with the compose preview, the
resulting stack in the editor, and the error toast surfaced when the
input cannot be converted. Appends a matching entry to the
troubleshooting page.
* feat(mfa): auto-submit 6-digit TOTPs and normalize pasted backup codes
Match the UX every major MFA prompt has (GitHub, GitLab, 1Password): the
challenge screen and every code-entry dialog now submit automatically once
the sixth TOTP digit lands, and the backup-code input accepts pastes with
smart-dashes, trailing whitespace, or mixed case without silently
truncating the value. Also caps the backup-code input at the correct
11 characters (10 plus a single separator) instead of 12.
Shared normalization helpers live in frontend/src/lib/mfa.ts so the
challenge and the three account-settings dialogs stay in lockstep.
* feat(mfa): warn users when backup codes run low
The Account & Security card silently showed a dim count of backup codes
remaining, which meant users could drift toward zero without noticing
until their phone was already lost. The card now surfaces a warning tone
with an alert icon when 1 or 2 codes remain, and swaps to a dedicated
destructive warning card with a "Regenerate now" action when the user
has used every code.
* feat(mfa): gate diagnostic logs behind developer mode
Reuses the existing isDebugEnabled() gate so operators investigating a
2FA support ticket can flip Developer Mode on to get per-branch
diagnostics (login path taken, replay check outcome, failure counter
after a verify, replay-table purge counts), and flip it back off when
they are done. Standard lifecycle logs stay on by default: enrolment
completed, 2FA disabled, backup codes regenerated, admin reset, SSO
bypass toggled, lockout engaged. Nothing that could reveal a TOTP code,
base32 secret, backup-code cleartext, or partial-auth JWT is ever
logged.
* test(mfa): cover drift, invalid formats, lockout recovery, and paste normalization
Backend: a TOTP generated for a window that has already slid out is
rejected, malformed backup codes (too short, non-alphanumeric, 11-char
alphanumeric that matches no hash) all increment failed_attempts, a
successful verify clears a below-threshold failure streak, a successful
verify after locked_until has passed clears the lockout, a second
enroll/start overwrites the prior pending secret, and the backup-code
normalizer treats en-dash/em-dash/figure-dash with stray whitespace the
same as the canonical form.
E2E: low-backup-codes warning renders in the warning tone and the
exhausted-codes state flips to the dedicated warning card, a 6-digit
TOTP auto-submits without a button click, and a backup code pasted
without the separator still signs in.
* docs(mfa): auto-submit, paste guidance, and expanded troubleshooting
Document that the challenge screen submits automatically on the sixth
digit, that backup codes accept the separator and any case, and that
the Account & Security card nudges at low code counts. Expands the
troubleshooting section with entries for lost or exhausted backup codes
and adds a short note to the admin guide about surfacing auth
diagnostics via Developer Mode.
* feat(auth): add TOTP two-factor authentication with backup codes
Adds RFC 6238 time-based one-time password support to every tier,
integrated with the existing password and SSO login paths.
Backend:
- New MfaService wrapping otplib with a plus or minus 1 step tolerance,
base32 secret generation, and hashed single-use backup codes (bcrypt).
- user_mfa and mfa_used_tokens tables in DatabaseService. The second
table is a DB-backed replay blacklist, purged on a 60s interval.
- authMiddleware now recognizes an mfa_pending scope. A token carrying
that scope is rejected on every route except the MFA challenge and
logout, so no API surface is reachable before the second factor
clears.
- /api/auth/login issues only a short-lived mfa_pending cookie when the
user has MFA enrolled. /api/auth/login/mfa consumes that cookie,
verifies the code (or backup code), and swaps in a real session.
- /api/auth/mfa/* routes for status, enrol/start, enrol/confirm,
disable, backup-code regenerate, and SSO-bypass opt-in.
- Admin recovery path: POST /api/users/:id/mfa/reset clears the target's
MFA state, bumps token_version, and writes an audit log entry.
- CLI emergency fallback: backend/src/cli/resetMfa.ts is wired via
`npm run reset-mfa <username>` and also exported for tests.
- SSO flows (LDAP and OIDC) gate on user_mfa.sso_enforce_mfa before
issuing a session; default behaviour keeps the SSO path frictionless.
- Per-user lockout after 5 consecutive failed codes (15 min).
Frontend:
- AppStatus gains an mfa-challenge branch driven by /api/auth/status.
- New MfaChallenge screen, MfaEnrollDialog (QR plus manual secret plus
backup codes), MfaDisableDialog, MfaBackupCodesDialog.
- Account section shows a Two-factor authentication card with enrol,
regenerate, disable, and the SSO-enforce toggle (shown only when SSO
providers are configured).
- Users section gains a Reset 2FA action for admins.
Docs:
- New user guide at features/two-factor-authentication.mdx.
- New admin guide at operations/two-factor-admin.mdx.
- SSO page cross-links to the 2FA doc.
* fix(mfa): drop unused TEST_PASSWORD import and stale eslint disable
* fix(mfa): simplify e2e openAccountSettings helper to match working pattern
* fix(mfa): make e2e suite self-contained and always clean up
Test #2 called loginAs() before the MFA challenge step, which waited for
the dashboard indicator that never appears once the previous test enrolled
the user. That timeout skipped the rest of the serial block, including
the disable step, leaving MFA enabled and breaking every later spec.
Two fixes:
- Tests #2 and #3 now navigate directly to the login page instead of
piggybacking on loginAs, which only handles the password-only path.
- A new afterAll hook unconditionally disables MFA via the API using two
unused backup codes, so the DB is reset even if a test fails midway.
* fix(e2e): use backup code for mfa recovery to avoid totp replay race
The final recovery step in the backup-code replay test previously
generated a fresh TOTP to sign back in. When the timing landed inside
the same 30-second window that test #2 consumed, the server's replay
blacklist correctly rejected it, producing a ~50% flake rate. Backup
codes are single-use and sidestep the replay window, so the recovery
becomes deterministic.
* fix(e2e): drive mfa disable test through the challenge screen
Test #4 called loginAs after test #3 left MFA enabled, but loginAs
waits for the dashboard indicator and does not handle the challenge
screen, so it timed out. Drive the login manually, satisfy the
challenge with a backup code, and use a backup code for the disable
step too to avoid any TOTP replay-window race against earlier tests
in the serial block.
* feat(git-sources): surface LFS and submodule warnings on create
Creating a stack from a Git repo now detects two common anomalies and
tells the user about them rather than silently producing broken stacks.
- LFS-pointer compose/env files fail early with a clear error instead
of writing a 130-byte pointer stub to disk as real content.
- Repositories containing .gitmodules produce a non-fatal warning so
the user knows build contexts or volumes inside submodules will be
empty at deploy time.
Also refines the create dialog: sr-only DialogDescription for a11y,
short commit SHA suffix on the success toast, env-path hint under the
"Sync .env" checkbox showing which path will be read, and a route-level
diagnostic log line gated on developer mode for support debugging.
* test(git-sources): cover LFS, submodule, and nested env_path paths
Adds unit coverage for the new LFS-pointer rejection and submodule
warning plumbing, plus a nested compose_path case that exercises the
default env_path resolution ("apps/web/compose.yaml" with sync_env on
and env_path unset writes "apps/web/.env" both to disk and to the DB).
Extends the E2E suite with a happy-path assertion that the full-length
commit SHA is returned in the create response, and a UI flow that
verifies the short-SHA suffix appears in the success toast.
* docs(git-sources): add troubleshooting for LFS, submodules, HTTPS-only
Adds troubleshooting entries for the newly surfaced LFS and submodule
anomalies, expands the clone-timeout entry with the bounded-fetch
explanation, and adds a dedicated HTTPS-only entry. Also consolidates
the known limitations into a single list covering LFS, submodules,
branch-tracking, and HTTPS-only.
* fix(settings): use Route icon for notification routing
The routing section in Settings previously used GitBranch, which now
clashes with the Git Source feature's icon across the editor. Switch
to Route (a branching-flow glyph) so routing rules have a distinct
visual identity and aren't visually conflated with Git-backed stacks.
* fix(git-sources): return 400 for upstream auth failures and disambiguate 404s
Upstream git-host auth failures were mapping to HTTP 401, which the frontend
apiFetch treats as a Sencho session expiry and fires the global logout event.
They now return 400 with code=AUTH_FAILED in the body so the UI can branch on
the discriminator without logging the user out. The status mapping moved into
utils/gitSourceHttp so it can be unit-tested without booting the app.
mapGitError also relied on the HttpError class alone, so any non-2xx response
(including 404) was classified as auth failure. It now inspects the numeric
status on err.data and considers whether a token was supplied, producing more
actionable messages for missing repos, private repos, and wrong-scope tokens.
* refactor(git-sources): extract GitSourceFields from GitSourcePanel
Pure extraction of the repo/branch/path/auth/apply-mode form fields into a
reusable controlled component so the upcoming Create Stack from Git flow can
render the same form in the Create Stack dialog. No behavior change.
* feat(git-sources): create a stack from a Git repository
Add a From Git tab to the Create Stack dialog so users can name a new
stack, point it at a repo + branch + compose path, and have the compose
fetched, validated, written to disk, and linked in one shot. Optional
deploy-after-create runs the initial bring-up when requested.
Backend: new POST /api/stacks/from-git route gated by stack:create.
GitSourceService.createStackFromGit() fetches and validates before
touching disk, then creates the stack, writes the compose (and .env if
sync is enabled), and seeds the git source row with the fetched commit
so future pulls produce a clean diff. Runs under the per-stack lock so
a concurrent webhook cannot race the create. Deploy failure is
non-fatal and surfaced to the caller.
Frontend: the existing Create Stack dialog is now tabbed, with Empty
keeping the original single-field flow unchanged.
* test(git-sources): cover create-from-git endpoint and e2e flow
Service tests verify createStackFromGit seeds the last_applied columns
on success, writes the env file when sync is enabled, refuses an
invalid apply-matrix without fetching, rejects invalid compose
without leaving orphan state, and rolls back the on-disk stack dir
when a post-create step fails.
Route tests cover auth, missing stack_name, invalid stack name,
http:// rejection, oversized repo_url, and the 409 collision guard.
E2E adds a Create-stack-from-Git block covering tab visibility,
client-side HTTPS check, backend .git/config rejection, and a
happy-path fetch against a public demo repo (skipped on network
failure).
* docs(git-sources): document create-stack-from-git tab
Add a new section near the top describing the From Git tab in the
Create Stack dialog: what it does, the Deploy after create checkbox,
and the four failure modes (name collision, unreachable repo,
invalid compose, deploy-after-create failure).
* fix(git-sources): harden validation, RBAC, concurrency, and deploy recovery
Tightens the surface area around the Git source feature:
- Enforce HTTPS-only repo URLs server-side (regex was permissive).
- Add stack:read permission check on git-source reads and filter the
list endpoint by callable permission.
- Validate stack names before permission checks on mutation routes so
scoped lookups never see unvalidated input.
- Cap repo_url / branch / compose_path / env_path / token lengths and
require the stack directory to exist before upsert.
- Wrap pull() in the per-stack mutex to eliminate the pull/delete race
that could orphan pending data.
- Block .git/ path components in compose_path / env_path so a
misconfigured clone cannot leak repo metadata.
- Return {applied, deployed, deployError?} on deploy failure instead of
throwing, and surface deployError as a warning toast so the user can
retry deploy without re-pulling.
- Always clean the stack_git_sources row on stack delete even when the
file deletion step fails.
- Add shadow-card-bevel to the pending alert and metadata card per the
design system.
- Handle the new 403 response on the panel fetch gracefully.
- Add diagnostic logging gated on developer_mode (isDebugEnabled) across
fetch / pull / apply / webhook paths with credential scrubbing.
* test(git-sources): expand coverage for hardening and route validation
- New route-level suite covers HTTPS enforcement, required fields,
max-length caps on repo_url / branch / compose_path / env_path /
token, the stack-existence 404 guard, and GET authz.
- Service tests cover the .git metadata guard on compose and env
paths (including nested and substring-containing "git"), pull and
apply rejections when no source is configured or pending is
cleared, the sha-mismatch branch, and the deploy-failure return
shape that now carries deployError.
- E2E adds three server-side contract assertions: PUT against a
missing stack returns 404, http:// is rejected with 400, and
.git/config is rejected as compose_path.
* docs(git-sources): document deploy-failure recovery path
Adds a Troubleshooting entry explaining that when apply succeeds but
the subsequent deploy fails, the compose content is already on disk
and the user can retry deploy from the stack editor without
re-pulling.
* docs(git-sources): add configuration, diff, pending, and webhook screenshots
* feat(git-sources): link stacks to Git repositories with diff-and-apply workflow
Add Git Sources so any stack can point at an HTTPS Git repository, branch, and
compose file path. Pulls fetch + validate the incoming commit, store a
diffable pending snapshot, and apply writes only after explicit confirmation
(or automatically, per the configured apply mode). Sibling .env sync is
optional. Works on the Community tier.
Apply modes:
- Review only: mark pending, wait for manual apply in the diff dialog
- Auto-write: write compose + env to disk, do not redeploy
- Auto-deploy: write files and run docker compose up -d
Webhook integration: webhooks can target the new "git-pull" action to trigger
a sync from CI. Per-source debounce prevents runaway pipelines from hammering
the repository host. Tokens are encrypted at rest and never returned to the
frontend.
Docs and tests included. Screenshots and Playwright E2E flows to follow.
* fix(git-sources): drop unnecessary useMemo on commit sha slice
React Compiler's lint rule rejected the manual dependency list because the
inferred dep ('pull') was less specific than the written one ('pull?.commitSha').
The computation is a cheap 7-char slice, so drop the useMemo entirely rather
than fight the rule.
* test(git-sources): add Playwright E2E flows and drop orphan source rows on stack delete
- E2E coverage: non-HTTPS URL rejected client-side, unreachable repo surfaces
a toast error on save, and configure+remove walks the AlertDialog confirm path.
- Deleting a stack now also drops its linked Git source row so a future stack
with the same name starts clean rather than inheriting a stale config.
* feat(registries): add stateless test endpoint, ECR caching, URL and host hardening
Adds a POST /api/registries/test endpoint so credentials can be verified
before being persisted. Caches ECR authorization tokens in memory until
their AWS-reported expiry (minus a safety margin) instead of fetching on
every compose invocation. Normalizes registry URLs on save so the
stored values match the keys Docker expects in ~/.docker/config.json,
fixes a bidirectional host-match bug in getAuthForRegistry that could
cross-match overlapping hostnames, and surfaces per-registry decryption
failures as warnings in the deploy log stream instead of swallowing
them. Also strips the Authorization header on cross-host redirects in
the test probe, rejects non-http(s) schemes on save, and validates the
shape of returned ECR authorization tokens before use.
* refactor(registries): align UI with design system and add in-form test button
Swaps the registry type dropdown from shadcn Select to the project's
Combobox, applies the canonical card bevel and top-border hover styling
to the form container and each registry row, restyles the delete
button to the ghost + muted destructive pattern, uses strokeWidth 1.5
on every Lucide icon, and routes all toast errors through the
standard defensive chain. Adds a Test connection button inside the
form so credentials can be verified before saving.
* test(registries): cover RegistryService and deploy warnings surface
Adds unit coverage for URL normalization, the encrypt/decrypt round
trip through create and resolveDockerConfig, exact-host matching in
getAuthForRegistry, resolveDockerConfig warnings on decryption
failure, ECR token cache hit/miss and invalidation on update, the
stateless testWithCredentials path for 200, 401 with and without a
challenge, network errors, and ECR success and failure including
malformed tokens. Extends the ComposeService tests to verify that
warnings from resolveDockerConfig reach the deploy log stream.
* docs(registries): document test-before-save flow and troubleshooting
Describes the in-form Test connection button, the two-point testing
flow from the registries list, the cached ECR token behavior during
deploys, the per-registry warning Sencho emits when a stored secret
cannot be decrypted, and adds a Troubleshooting section covering
common 401 causes, ECR token handling, warning interpretation, and
per-node credential scoping.
* fix(notifications): stop Sencho version notifications from silently skipping
Three independent defects combined to make version-update notifications
silently fail even while Fleet overview correctly surfaced an update button:
- The in-memory 6-hour cooldown was advanced before the network fetch,
so a single transient failure at boot could lock the check for the
rest of the container lifetime. Moved the cooldown update inside the
success branch so failures retry on the next eval cycle.
- MonitorService called the raw version fetch directly, bypassing the
CacheService wrapper (TTL, inflight dedup, stale-on-error) that Fleet
uses, so the two paths could diverge. Unified both on a shared
getLatestVersion() helper in utils/version-check.ts.
- The dedup key could carry stale state from a previous build and never
self-clear. It now self-heals when the running version reaches the
previously-notified version, so future releases re-fire as expected.
Added diagnostic logs gated on debug mode for each skip branch, plus
three regression tests covering cooldown-on-failure, cooldown-on-success,
and dedup self-heal.
* docs(notifications): drop legacy-upgrade framing from alerts troubleshooting
Sencho has not shipped publicly, so troubleshooting entries written in
'this used to happen but now does Y' mode reference a past that does not
exist for any reader. Rewrote the version-notification, image-update,
and crash-alert troubleshooting entries to describe current behavior
positively without referring to prior builds, upgrade paths, or legacy
fixes.
* chore(security): accept CVE-2026-33810 in bundled Docker CLI 29.4.0
Trivy now flags CVE-2026-33810 (Go stdlib crypto/x509 DNS constraint
bypass, fixed in Go 1.26.2) in the Docker CLI static binary we ship.
Docker CLI 29.4.0 is the latest upstream release and still links Go
1.26.1; no newer static binary exists yet.
Same exposure profile as the already-accepted CVE-2026-32280: the
Docker CLI and compose plugin only validate certificates from
well-known registry CAs and the local Docker socket, not from
attacker-controlled CAs with crafted DNS name constraints. Revisit
on the next Docker CLI release that rebuilds against Go 1.26.2 or
later.
* fix(docker-events): harden crash detection against edge cases
- Isolate per-container failures in the reconcile path so one failed
container inspect cannot abort classification for the rest of the
batch after a reconnect.
- Fall back to inspecting State.OOMKilled when a container exits with
code 137 and no oom event preceded the die, so cgroup OOM kills that
lose the oom event are still classified correctly. The dedup check
runs before the fallback so crashloops do not hammer the daemon.
- Guard the intentional-kill window against wildly future-dated Docker
timestamps by switching to a signed age comparison with a bounded
negative-skew tolerance, so clock skew cannot flip a genuine crash
into an intentional stop.
- Align diagnostic logging with the codebase [Service:diag] convention
and add one informational lifecycle log on boot and shutdown so
operators running in developer mode can confirm the watcher started.
- New tests cover gap-inspect isolation, the OOM inspect fallback (both
success and failure paths), duplicate die events collapsing within
the grace window, and clock-skew bounds on the classifier.
* docs(alerts): add troubleshooting entries for crash detection toggle and rate limits
* fix(notifications): replace polling with Docker event stream for container lifecycle detection
Replaces the 30-second MonitorService crash-detection poll with a causal,
per-node Docker events stream. Eliminates false crash alerts on intentional
stops (docker stop, compose down, stack restart/update), detects OOM kills
as a distinct alert category, and surfaces real crashes in real time.
A new DockerEventManager spawns one DockerEventService per local node. Each
service consumes the filtered container event stream, classifies die events
against recent kill/oom state, and reconciles container state via snapshot
diffing on connect and reconnect. Rate limiting, exponential backoff with
jitter, and parse-error tolerance keep the stream resilient under load and
during daemon interruptions.
MonitorService retains host limits, janitor, version check, and stack metric
alerts; crash and healthcheck detection move out entirely.
* fix(tests): silence require-imports lint in hoisted mock factory
Anso