feat(images): Trivy-powered vulnerability scanning (#635)

* feat(images): Trivy-powered vulnerability scanning

Scan container images for known CVEs via Trivy. On-demand scanning and
severity badges are available on every tier; scheduled scans, scan
policies, SBOM generation, and scan history are gated to Skipper+.

- New TrivyService (binary detection, per-image scan, SBOM, digest cache)
- Three new tables: vulnerability_scans, vulnerability_details, scan_policies
- 12 routes under /api/security (scan, results, summaries, SBOM, policies, compare)
- Post-deploy async scans wired into all five deploy paths, with a
  per-deploy opt-out toggle in the App Store deploy sheet
- "scan" action type added to SchedulerService for fleet-wide recurring scans
- Frontend: severity badges in Resources Hub with animated cursor detail,
  scan results drawer with vulnerability table and filters, and a new
  Security section in Settings for scan policy CRUD
- Policy threshold violations dispatch a warning or critical alert based on
  the policy's block_on_deploy flag; deploys themselves are never blocked

* fix(security): compute scan age in useEffect to satisfy react-hooks/purity
This commit is contained in:
Anso
2026-04-16 15:03:36 -04:00
committed by GitHub
parent 4c5aa73196
commit c9cd6990d2
23 changed files with 3452 additions and 18 deletions
+2
View File
@@ -115,6 +115,7 @@
"features/audit-log",
"features/api-tokens",
"features/private-registries",
"features/vulnerability-scanning",
"features/auto-update-policies",
"features/scheduled-operations",
"features/sso",
@@ -140,6 +141,7 @@
"operations/backup",
"operations/upgrade",
"operations/self-hosting",
"operations/trivy-setup",
"operations/two-factor-admin"
]
}
+4
View File
@@ -103,6 +103,10 @@ Create point-in-time snapshots of every compose file and environment file across
Store credentials for private Docker registries: Docker Hub organizations, GHCR, AWS ECR, and self-hosted registries. Sencho injects them automatically during deploy and pull operations. ECR short-lived tokens are refreshed on every operation. Admiral only. [Learn more →](/features/private-registries)
## Vulnerability scanning
Scan container images for known CVEs with [Trivy](https://trivy.dev). On-demand scanning and severity badges are available on every tier; scheduled scans, scan policies that gate deploys, SBOM generation, and scan history are available on Skipper and Admiral. [Learn more →](/features/vulnerability-scanning)
## Audit log
Track every mutating action across your Sencho instance with a searchable audit trail. See who deployed, stopped, deleted, or changed settings, with timestamps, user attribution, and node context. Admiral only. [Learn more →](/features/audit-log)
+1
View File
@@ -23,6 +23,7 @@ Scheduled Operations lets you automate recurring maintenance tasks across your i
| **Restart Stack** | A specific stack (or specific services within it) on a specific node | Restarts all or selected containers in the stack |
| **Fleet Snapshot** | All nodes | Creates a fleet-wide backup of all compose files and `.env` files |
| **System Prune** | The default node | Prunes selected resources, optionally filtered by Docker label |
| **Vulnerability Scan** | All images on a specific node | Runs Trivy against every image on the target node and records the results. Requires Trivy to be installed — see [Installing Trivy](/operations/trivy-setup). Available on Skipper and Admiral. |
## Creating a Scheduled Task
+188
View File
@@ -0,0 +1,188 @@
---
title: "Vulnerability Scanning"
description: "Scan container images for known CVEs, surface severity badges in the Resources Hub, and alert on policy violations."
---
Sencho integrates with [Trivy](https://trivy.dev) to scan container images for known vulnerabilities (CVEs), surface severity badges next to your images, and alert when a scan result exceeds a configured threshold. On-demand scanning is available on every tier; automation, policies, and SBOM generation are Skipper and Admiral.
<Frame>
<img src="/images/vulnerability-scanning/resources-badges.png" alt="Resources Hub showing vulnerability severity badges next to image tags" />
</Frame>
## Prerequisites
The Trivy CLI must be available on the machine running Sencho. Trivy is not bundled with the Sencho Docker image; see [Installing Trivy](/operations/trivy-setup) for mount and installation options. Sencho checks for Trivy on startup and hides scanning UI when the binary is not available.
## Tier availability
| Feature | Community | Skipper | Admiral |
|---------|:---------:|:-------:|:-------:|
| On-demand image scanning | ✓ | ✓ | ✓ |
| Severity badges in the Resources Hub | ✓ | ✓ | ✓ |
| Scan results drawer with vulnerability table | ✓ | ✓ | ✓ |
| Post-deploy automated scanning | ✓ | ✓ | ✓ |
| Scheduled fleet scans (all images on a node) | | ✓ | ✓ |
| Scan policies (warning and critical alerts) | | ✓ | ✓ |
| SBOM generation (SPDX, CycloneDX) | | ✓ | ✓ |
| Scan history and comparison | | ✓ | ✓ |
## On-demand scanning
Navigate to the **Resources** tab and open the **Images** panel. When Trivy is available, every image row shows a shield icon alongside the delete action.
1. Click the shield icon on any image row to start a scan.
2. The row shows a loading spinner while Trivy runs. Most scans finish in 10 to 60 seconds depending on image size and whether the Trivy database is already cached.
3. When the scan completes, a severity badge appears next to the image status (e.g. `CRITICAL`, `HIGH`, `MEDIUM`, `LOW`, or `CLEAN`).
4. Click the badge to open the scan results drawer.
<Frame>
<img src="/images/vulnerability-scanning/scan-results.png" alt="Vulnerability scan results drawer showing CVE table with severity, package, and fix columns" />
</Frame>
### Reading severity badges
Hovering over a severity badge reveals the breakdown of vulnerabilities by severity and the timestamp of the last scan. The badge color reflects the highest severity found:
| Badge | Meaning |
|-------|---------|
| **CRITICAL** (red) | At least one vulnerability with a CVSS score of 9.0 or higher |
| **HIGH** (amber) | At least one high-severity vulnerability |
| **MEDIUM** (blue) | Only medium and lower vulnerabilities |
| **LOW** (muted) | Only low-severity vulnerabilities |
| **CLEAN** (green) | Scan completed with zero findings |
Scan results are cached by image digest. If the same digest is scanned again within 24 hours, Sencho returns the cached result instantly instead of re-running Trivy.
## The scan results drawer
The drawer shows a full breakdown of the most recent scan for an image:
- **Summary**: counts per severity (critical, high, medium, low), total vulnerabilities, how many have a fix available, the Trivy version used, and when the scan ran.
- **Filter tabs**: narrow the table to a specific severity.
- **Vulnerability table**: paginated list of every CVE found, including:
- **CVE ID** (linked to the upstream advisory)
- **Package** name and installed version
- **Severity** badge
- **Fixed version** with a green indicator if a fix is available
### Actions
From the drawer header you can:
- **Re-scan**: kick off a fresh scan, ignoring the digest cache.
- **Download SBOM**: export a Software Bill of Materials in SPDX JSON or CycloneDX format (Skipper and Admiral).
- **Export CSV**: export the full vulnerability list for offline review.
## Post-deploy automated scanning
When Trivy is available, Sencho automatically scans every deployed image in the background after a successful deploy. This applies to all deploy paths:
- Stack deploy and redeploy
- Stack update
- Template deploy from the App Store
- Git source apply
- Git source create
The deploy itself is never blocked by scanning; scans run asynchronously and surface their results via the severity badges in the Resources Hub. If high or critical vulnerabilities are found, an alert is dispatched through your configured [notification channels](/features/alerts-notifications).
### Opting out per deployment
The App Store deploy sheet includes an **Scan images for vulnerabilities after deploy** toggle (enabled by default). Uncheck it to skip the post-deploy scan for that single deployment. This does not disable scan policies globally; it simply opts this deployment out of scanning.
<Frame>
<img src="/images/vulnerability-scanning/app-store-toggle.png" alt="App Store deploy sheet showing the auto-scan checkbox enabled by default" />
</Frame>
## Scheduled fleet scans
<Note>
Scheduled scans require a **Skipper** or **Admiral** license.
</Note>
You can run recurring scans of every image on a node through the standard [Scheduled Operations](/features/scheduled-operations) system. Create a new scheduled task with action **Scan** and a cron expression. The scheduler iterates every image on the target node with a short delay between scans and records the result in the task's run history.
Use scheduled scans to keep CVE badges fresh even for images that are rarely redeployed: nightly (`0 3 * * *`) is a good default for most fleets.
## Scan policies
<Note>
Scan policies require a **Skipper** or **Admiral** license.
</Note>
Policies let you define severity thresholds that the post-deploy scanner evaluates against. When a deploy's scan exceeds a policy's threshold, Sencho dispatches an alert. The policy's **Block on deploy** toggle controls the alert severity: warning when off, critical when on.
<Frame>
<img src="/images/vulnerability-scanning/security-settings.png" alt="Security section of Settings showing the scan policies list with add policy button" />
</Frame>
### Creating a policy
Go to **Settings → Security** and click **Add Policy**.
| Field | Description |
|-------|-------------|
| **Name** | A descriptive label (e.g. "Production critical block"). |
| **Stack pattern** | Optional glob against stack names (e.g. `prod-*`). Leave empty to match every stack. |
| **Max severity** | The threshold. If a scan finds any vulnerability at or above this severity, the policy fires. |
| **Block on deploy** | When enabled, policy violations are dispatched as critical (error) alerts. When disabled, they are dispatched as warnings. |
| **Enabled** | Disabled policies are skipped during evaluation. |
### Policy scoping
When multiple policies match a deploy, Sencho picks the most specific one:
1. Policies scoped to a specific node win over global policies.
2. Policies with a stack pattern win over wildcard policies.
3. Disabled policies are never applied.
Only one policy is evaluated per deploy; use a single tight pattern rather than overlapping policies for clarity.
## SBOM generation
<Note>
SBOM generation requires a **Skipper** or **Admiral** license.
</Note>
A Software Bill of Materials (SBOM) is a machine-readable inventory of every package present in a container image. SBOMs are required by an increasing number of security frameworks (SLSA, Executive Order 14028, EU Cyber Resilience Act) and are useful for offline compliance reviews.
From the scan results drawer, click **Download SBOM** and choose a format:
| Format | Use case |
|--------|----------|
| **SPDX JSON** | Widely supported, best for tooling integration. |
| **CycloneDX** | Richer dependency metadata, better for supply-chain analysis. |
The download starts immediately and uses the image's digest (when available) in the filename.
## Scan history
Every scan Sencho runs is stored with its full vulnerability detail. Scan records are automatically pruned after 90 days to keep the database compact. The history is used to power:
- **Digest caching**: skip re-scanning an image that has already been scanned within 24 hours.
- **Trend badges**: surface whether the latest scan added or resolved vulnerabilities compared to the previous scan for the same image.
## How it works
1. On startup, Sencho looks for the `trivy` binary on `PATH` and caches its availability.
2. When a scan is triggered, Sencho resolves the image digest via Docker and checks the 24-hour cache. If a completed scan exists for that digest, it is returned instantly.
3. Otherwise, Sencho spawns `trivy image --format json --quiet <image-ref>` and parses the JSON output into the vulnerability database.
4. Private registry credentials are forwarded automatically by writing a temporary `DOCKER_CONFIG` for the Trivy subprocess, then deleting it when the scan completes.
5. Scan status, counts per severity, and the full vulnerability list are persisted for display in the drawer and for policy evaluation.
## Troubleshooting
### Scan button is not visible
Sencho hides scanning UI when the Trivy binary is not detected. Check **Settings → Support** for the Trivy availability status, then follow [Installing Trivy](/operations/trivy-setup) if it is missing.
### Scans time out
The default scan timeout is 5 minutes. Very large images (2+ GB) over a slow connection may exceed this; pre-pulling the image to the host speeds up the scan significantly because Trivy works against the local image store.
### Private registry images fail to scan
Sencho forwards the same registry credentials configured under **Settings → Registries** to Trivy during a scan. If a pull works in Sencho but a scan fails, make sure the image has been pulled at least once (Trivy can then work against the cached local image).
### Badge is out of date after an image update
Post-deploy scanning only runs on deploy actions. For long-running images that aren't redeployed, schedule a recurring scan (Skipper+) or click the shield icon in the Resources Hub to re-scan on demand.
+185
View File
@@ -0,0 +1,185 @@
---
title: Installing Trivy
description: Install and mount the Trivy CLI so Sencho can scan container images for vulnerabilities.
---
Sencho's [Vulnerability Scanning](/features/vulnerability-scanning) feature uses the [Trivy](https://trivy.dev) CLI. Trivy is not bundled with the Sencho Docker image; operators provide it via a bind mount or a custom image so Sencho can find it on `PATH`. Once Trivy is available, the scanning UI appears automatically.
## Why Trivy is not bundled
Trivy's vulnerability database updates multiple times per day and is around 100 MB. Bundling Trivy would force every Sencho instance to carry an out-of-date database in its image, then re-download on first scan. By keeping Trivy external, you can:
- Pick the Trivy version you want and upgrade it on your own schedule
- Persist the Trivy cache (the vulnerability DB) across Sencho container restarts
- Pre-seed an air-gapped cache for environments without internet access
## Installing Trivy on the host
### Linux (Debian / Ubuntu)
```bash
sudo apt-get install wget gnupg
wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | sudo apt-key add -
echo "deb https://aquasecurity.github.io/trivy-repo/deb generic main" | sudo tee /etc/apt/sources.list.d/trivy.list
sudo apt-get update
sudo apt-get install trivy
```
### Linux (RHEL / Fedora)
```bash
sudo rpm --import https://aquasecurity.github.io/trivy-repo/rpm/public.key
echo "[trivy]
name=Trivy repository
baseurl=https://aquasecurity.github.io/trivy-repo/rpm/releases/\$basearch/
gpgcheck=1
enabled=1" | sudo tee /etc/yum.repos.d/trivy.repo
sudo dnf install trivy
```
### macOS
```bash
brew install trivy
```
### Verify the install
```bash
trivy --version
```
The command should print a `Version: X.Y.Z` line. Note the path that `which trivy` (or `where trivy` on Windows) returns; you will mount that path into the Sencho container.
## Making Trivy available to Sencho
Sencho runs inside a container and looks for `trivy` on its own `PATH`. There are two approaches:
### Option 1: Bind mount the host binary
Mount the host's Trivy binary into the Sencho container. This is the simplest option when Sencho and Trivy share the same CPU architecture.
```yaml
services:
sencho:
image: sencho/sencho:latest
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./sencho-data:/app/data
- /opt/compose:/opt/compose
- /usr/local/bin/trivy:/usr/local/bin/trivy:ro
- trivy-cache:/root/.cache/trivy
environment:
- COMPOSE_DIR=/opt/compose
- TRIVY_CACHE_DIR=/root/.cache/trivy
volumes:
trivy-cache:
```
The `trivy-cache` volume persists the vulnerability database across Sencho container restarts so Trivy does not re-download it every time.
Adjust the first path if `which trivy` on the host prints something other than `/usr/local/bin/trivy` (for example `/usr/bin/trivy` on some distributions).
### Option 2: Build a custom Sencho image
If the host's Trivy binary is not ABI-compatible with the Sencho container (for example because you are running macOS host binaries or a different glibc version), install Trivy inside the image instead:
```dockerfile
FROM sencho/sencho:latest
RUN apk add --no-cache curl ca-certificates \
&& curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh \
| sh -s -- -b /usr/local/bin
ENV TRIVY_CACHE_DIR=/root/.cache/trivy
```
Build and run:
```bash
docker build -t sencho-with-trivy .
docker compose up -d
```
## Persisting the vulnerability database
Trivy downloads a ~100 MB vulnerability database on first run and refreshes it every six hours. Without a persistent cache, every Sencho restart re-downloads the database, wasting bandwidth and adding 1030 seconds to the first scan.
Set `TRIVY_CACHE_DIR` to a directory inside a named or bind-mounted volume (see Option 1 above). The directory must be writable by the Sencho process.
## Air-gapped environments
Trivy supports offline use through pre-built database bundles.
1. On a networked machine, download the latest DB bundle:
```bash
trivy image --download-db-only
cp -r ~/.cache/trivy /path/to/portable/cache
```
2. Transfer the cache directory to the air-gapped host.
3. Mount it into the Sencho container at `TRIVY_CACHE_DIR`:
```yaml
volumes:
- /path/to/portable/cache:/root/.cache/trivy:ro
environment:
- TRIVY_CACHE_DIR=/root/.cache/trivy
```
4. Set `TRIVY_SKIP_DB_UPDATE=true` to prevent Trivy from attempting a refresh:
```yaml
environment:
- TRIVY_CACHE_DIR=/root/.cache/trivy
- TRIVY_SKIP_DB_UPDATE=true
```
Plan to refresh the bundle on a schedule (weekly is typical) so CVE data stays current.
## Verifying Sencho detects Trivy
After restarting Sencho with the mount in place:
1. Open the **Resources** tab.
2. Look at the **Images** panel. If Trivy is detected, a shield icon appears in the Actions column next to the delete icon on every row.
3. You can also check **Settings → Support** for an explicit Trivy availability status.
If the shield icon is missing, see the troubleshooting section below.
## Troubleshooting
### Sencho does not detect Trivy
Sencho runs `trivy --version` on startup and caches the result. If you added the mount after Sencho started, restart the container so the check runs again.
Verify the binary is visible from inside the container:
```bash
docker exec sencho trivy --version
```
If the command returns "not found", the mount path inside the container is wrong. The binary must be on `PATH`. Both `/usr/local/bin/trivy` and `/usr/bin/trivy` work.
### Binary exists but reports an exec format error
This means the host binary is not ABI-compatible with the Sencho image. Use Option 2 (custom image) instead; the `install.sh` script pulls the right architecture-specific build.
### Scans take a long time on first run
The first scan after a Trivy install downloads the vulnerability database. Expect 1030 seconds of additional latency. Subsequent scans are near-instant once the cache is warm and `TRIVY_CACHE_DIR` is persisted.
### Private registry images fail to scan
Trivy scans the locally-cached image layers. If Sencho can pull the image but a scan fails, pull the image to the host first (a deploy will do this) and retry the scan.
### Permission denied writing to the Trivy cache
`TRIVY_CACHE_DIR` must be writable by the Sencho process. If you mounted the directory from the host with restrictive permissions, either loosen them (`chmod -R a+w /path/to/cache`) or use a named Docker volume which inherits the container user's permissions.
### Trivy version is reported as "unknown"
Sencho reads the version from `trivy --version` output. Very old Trivy releases (< 0.35) use a different output format. Upgrade to a recent version.
+6
View File
@@ -51,6 +51,10 @@ Every self-hosted instance includes the full security stack, with advanced featu
Long-lived JWT bearer tokens, encrypted at rest, injected automatically during proxied requests.
</Card>
<Card title="Vulnerability scanning" icon="shield-virus" href="/features/vulnerability-scanning">
Trivy-powered CVE scanning of container images, with severity badges, post-deploy automation, and policy gating.
</Card>
</CardGroup>
## Tier availability
@@ -66,7 +70,9 @@ Every Sencho instance includes the foundational security stack. Advanced access-
| Encryption at rest (AES-256-GCM) | ✓ | ✓ | ✓ |
| Rate limiting (auth + API) | ✓ | ✓ | ✓ |
| Node-to-node authentication | ✓ | ✓ | ✓ |
| Vulnerability scanning (on-demand + post-deploy) | ✓ | ✓ | ✓ |
| Multi-user with RBAC (Admin, Viewer) | | ✓ | ✓ |
| Scan policies, scheduled scans, SBOM generation | | ✓ | ✓ |
| Advanced RBAC (Deployer, Node Admin, Auditor) | | | ✓ |
| Scoped permissions (per-stack, per-node) | | | ✓ |
| API tokens (scoped, expiring) | | | ✓ |