sencho/docs/features/vulnerability-scanning.mdx

---
title: "Vulnerability Scanning"
description: "Scan container images for known CVEs, surface severity badges in the Resources Hub, and alert on policy violations."
---

Sencho integrates with [Trivy](https://trivy.dev) to scan container images for known vulnerabilities (CVEs), surface severity badges next to your images, and alert when a scan result exceeds a configured threshold. Manual scanning, secret and misconfiguration detection, scan comparison, and CVE suppressions are all available on every tier. Skipper and Admiral add automation, policy enforcement, and compliance exports.

<Frame>
  <img src="/images/vulnerability-scanning/resources-badges.png" alt="Resources Hub showing vulnerability severity badges next to image tags" />
</Frame>

## Prerequisites

The Trivy CLI must be available on the machine running Sencho. Trivy is not bundled with the Sencho Docker image; see [Installing Trivy](/operations/trivy-setup) for mount and installation options. Sencho checks for Trivy on startup and hides scanning UI when the binary is not available.

<Note>
  Trivy is installed independently on each Sencho instance. When you select a remote node, **Settings > Security** shows only the scanner status for that node; install, update, or uninstall Trivy from there to manage the remote's binary. Scan policies and CVE suppressions are managed on the control node and apply fleet-wide at read time.
</Note>

## Tier availability

| Feature | Community | Skipper | Admiral |
|---------|:---------:|:-------:|:-------:|
| Install / update / uninstall Trivy from Settings | ✓ | ✓ | ✓ |
| On-demand image scanning (vulnerabilities) | ✓ | ✓ | ✓ |
| Severity badges in the Resources Hub | ✓ | ✓ | ✓ |
| Scan results drawer with vulnerability table | ✓ | ✓ | ✓ |
| Post-deploy automated scanning | ✓ | ✓ | ✓ |
| Secret detection in image filesystems | ✓ | ✓ | ✓ |
| Compose file misconfiguration scanning | ✓ | ✓ | ✓ |
| Scan history and comparison | ✓ | ✓ | ✓ |
| CVE suppressions | ✓ | ✓ | ✓ |
| Scheduled fleet scans (all images on a node) | | ✓ | ✓ |
| Scan policies with `block_on_deploy` enforcement | | ✓ | ✓ |
| SBOM generation (SPDX, CycloneDX) | | ✓ | ✓ |
| SARIF export (code scanning integration) | | ✓ | ✓ |
| Auto-update of the managed Trivy binary | | | ✓ |

## On-demand scanning

Navigate to the **Resources** tab and open the **Images** panel. When Trivy is available, every image row shows a shield icon alongside the delete action.

1. Click the shield icon on any image row. A menu appears with two options:
   - **Scan (vulnerabilities)**: the default, fastest path. Trivy inspects package metadata only.
   - **Full scan (vulnerabilities + secrets)**: additionally walks the image filesystem for hardcoded credentials, tokens, and keys. This takes noticeably longer.
2. The row shows a loading spinner while Trivy runs. Most vulnerability scans finish in 10 to 60 seconds depending on image size and whether the Trivy database is already cached. Full scans add the time needed to read the filesystem.
3. When the scan completes, a severity badge appears next to the image status (e.g. `CRITICAL`, `HIGH`, `MEDIUM`, `LOW`, or `CLEAN`).
4. Click the badge to open the scan results drawer.

<Frame>
  <img src="/images/vulnerability-scanning/scan-details-sheet.png" alt="Vulnerability scan results drawer showing CVE table with severity, package, and fix columns alongside a policy violation banner" />
</Frame>

### Reading severity badges

Hovering over a severity badge reveals the breakdown of vulnerabilities by severity and the timestamp of the last scan. The badge color reflects the highest severity found:

| Badge | Meaning |
|-------|---------|
| **CRITICAL** (red) | At least one vulnerability with a CVSS score of 9.0 or higher |
| **HIGH** (amber) | At least one high-severity vulnerability |
| **MEDIUM** (blue) | Only medium and lower vulnerabilities |
| **LOW** (muted) | Only low-severity vulnerabilities |
| **CLEAN** (green) | Scan completed with zero findings |

Scan results are cached by image digest. If the same digest is scanned again within 24 hours, Sencho returns the cached result instantly instead of re-running Trivy.

## The scan results drawer

The drawer shows a full breakdown of the most recent scan for an image and groups findings across three tabs: **Vulnerabilities**, **Secrets**, and **Misconfigs**. The summary header shows counts across all three so you can see the full risk picture at a glance.

- **Summary**: counts per severity (critical, high, medium, low), total vulnerabilities, how many have a fix available, the Trivy version used, and when the scan ran.
- **Vulnerabilities tab**: severity filter pills narrow the table, paginated list of every CVE found, including:
  - **CVE ID** (CVE-prefixed identifiers link to [cve.org](https://www.cve.org); GHSA identifiers link to the GitHub Advisory Database)
  - **Package** name and installed version
  - **Severity** badge
  - **Fixed version** with a green indicator if a fix is available

Critical and high rows in the table are tinted with a left accent rail so the rows that need attention catch the eye even before you read the severity column.

If the scan was evaluated against a [scan policy](#scan-policies) and the highest severity meets or exceeds the policy threshold, a destructive **Policy violation** banner appears at the top of the drawer naming the policy and the threshold it crossed.
- **Secrets tab**: hardcoded credentials or keys detected in the image filesystem, with severity, rule, title, and the file/line location. Secret values are redacted: only the first eight characters of the match are stored.
- **Misconfigs tab**: misconfiguration findings with severity, check ID, title, target file, and a suggested resolution. For image scans this tab is empty; for stack config scans (see below) it is the primary view.

### Actions

From the drawer header you can:

- **Re-scan**: kick off a fresh scan, ignoring the digest cache.
- **Download SBOM**: export a Software Bill of Materials in SPDX JSON or CycloneDX format (Skipper and Admiral).
- **Export CSV**: export the full vulnerability list for offline review.
- **Export SARIF**: download the full scan (vulnerabilities, secrets, and misconfigs) as SARIF 2.1.0 for upload to GitHub code scanning or other SARIF-aware tooling (Skipper and Admiral).

## Post-deploy automated scanning

When Trivy is available, Sencho automatically scans every deployed image in the background after a successful deploy. This applies to all deploy paths:

- Stack deploy and redeploy
- Stack update
- Template deploy from the App Store
- Git source apply
- Git source create

The deploy itself is never blocked by scanning; scans run asynchronously and surface their results via the severity badges in the Resources Hub. If high or critical vulnerabilities are found, an alert is dispatched through your configured [notification channels](/features/alerts-notifications).

### Opting out per deployment

The App Store deploy sheet includes an **Scan images for vulnerabilities after deploy** toggle (enabled by default). Uncheck it to skip the post-deploy scan for that single deployment. This does not disable scan policies globally; it simply opts this deployment out of scanning.

<Frame>
  <img src="/images/vulnerability-scanning/app-store-toggle.png" alt="App Store deploy sheet showing the auto-scan checkbox enabled by default" />
</Frame>

## Scheduled fleet scans

<Note>
  Scheduled scans require a **Skipper** or **Admiral** license.
</Note>

You can run recurring scans of every image on a node through the standard [Scheduled Operations](/features/scheduled-operations) system. Create a new scheduled task with action **Scan** and a cron expression. The scheduler iterates every image on the target node with a short delay between scans and records the result in the task's run history.

Use scheduled scans to keep CVE badges fresh even for images that are rarely redeployed: nightly (`0 3 * * *`) is a good default for most fleets.

### Completion notifications

When a scheduled scan finishes, Sencho dispatches an alert through your configured [notification channels](/features/alerts-notifications). The alert includes the task name and a summary of scanned, cached, and failed images:

- **Info** when every image scanned successfully.
- **Warning** when one or more images in the run failed to scan.

Failures are typically transient (registry timeouts, missing credentials) and do not stop the rest of the run from completing. Check the task's run history for the detailed output.

## Scan policies

<Note>
  Scan policies require a **Skipper** or **Admiral** license.
</Note>

Policies let you define severity thresholds that govern whether a stack can deploy at all. A policy with **Block on deploy** enabled runs a pre-flight scan on every image in the stack before `docker compose up` executes; if any image meets or exceeds the threshold, the deploy is rejected with a dialog listing the offending images. Policies with **Block on deploy** disabled still evaluate every post-deploy and scheduled scan, and dispatch warning alerts when the threshold is exceeded.

See [Deploy Enforcement](/features/deploy-enforcement) for the full pre-flight flow, admin bypass path, and audit-log behavior.

<Frame>
  <img src="/images/vulnerability-scanning/security-settings.png" alt="Security section of Settings showing the scan policies list with add policy button" />
</Frame>

### Creating a policy

Go to **Settings → Security** and click **Add Policy**.

| Field | Description |
|-------|-------------|
| **Name** | A descriptive label (e.g. "Production critical block"). |
| **Stack pattern** | Optional glob against stack names (e.g. `prod-*`). Leave empty to match every stack. |
| **Max severity** | The threshold. If a scan finds any vulnerability at or above this severity, the policy fires. |
| **Block on deploy** | When enabled, deploys are rejected before `docker compose up` runs if any image violates the threshold. When disabled, the policy still evaluates post-deploy and scheduled scans and dispatches warning alerts on violations. |
| **Enabled** | Disabled policies are skipped during evaluation. |

### Policy scoping

When multiple policies match a deploy, Sencho picks the most specific one:

1. Policies scoped to a specific node win over global policies.
2. Policies with a stack pattern win over wildcard policies.
3. Disabled policies are never applied.

Only one policy is evaluated per deploy; use a single tight pattern rather than overlapping policies for clarity.

### Example policies

**Block criticals and highs in production.** A tight gate that keeps known vulnerable base images out of your production fleet.

- **Name:** `prod-high-gate`
- **Stack pattern:** `prod-*`
- **Max severity:** `HIGH`
- **Block on deploy:** On
- **Enabled:** On

**Alert on criticals in staging, never block.** Lets engineers iterate without friction while still surfacing critical findings for triage.

- **Name:** `staging-critical-alert`
- **Stack pattern:** `staging-*`
- **Max severity:** `CRITICAL`
- **Block on deploy:** Off
- **Enabled:** On

### Creating a policy via the API

Policy CRUD endpoints are documented in the [Security API reference](/api-reference/security). A typical create call from CI looks like this:

```bash
curl -X POST https://your-sencho-instance:1852/api/security/policies \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "prod-high-gate",
    "stack_pattern": "prod-*",
    "max_severity": "HIGH",
    "block_on_deploy": 1,
    "enabled": 1
  }'
```

## SBOM generation

<Note>
  SBOM generation requires a **Skipper** or **Admiral** license.
</Note>

A Software Bill of Materials (SBOM) is a machine-readable inventory of every package present in a container image. SBOMs are required by an increasing number of security frameworks (SLSA, Executive Order 14028, EU Cyber Resilience Act) and are useful for offline compliance reviews.

From the scan results drawer, click **Download SBOM** and choose a format:

| Format | Use case |
|--------|----------|
| **SPDX JSON** | Widely supported, best for tooling integration. |
| **CycloneDX** | Richer dependency metadata, better for supply-chain analysis. |

The download starts immediately and uses the image's digest (when available) in the filename.

## Secret detection

<Note>
  Secret detection requires a **Skipper** or **Admiral** license.
</Note>

Full scans ask Trivy to walk the image filesystem for patterns that look like hardcoded credentials, API tokens, cloud access keys, or private keys. Detection rules cover common providers (AWS, GCP, GitHub, Slack, Stripe) plus generic high-entropy strings.

To run a full scan, click the shield icon in the Resources Hub and pick **Full scan (vulnerabilities + secrets)**. Findings appear on the **Secrets** tab of the scan drawer:

| Column | Description |
|--------|-------------|
| **Severity** | Trivy-assigned severity for the rule that matched. |
| **Rule** | The detection rule identifier (e.g. `aws-access-key-id`). |
| **Title** | A short description of what was detected. The second line shows a redacted excerpt of the match. |
| **Target** | The file path inside the image filesystem, including the line number range when available. |

Only the first eight characters of any matched secret are stored, followed by an ellipsis. The full value is never written to the database, so exporting a scan drawer or comparing scans cannot leak the underlying credential.

Full scans take longer than vulnerability-only scans because Trivy reads every file in the image. If runtime is a concern, schedule full scans overnight and keep deploy-time scans on the default vulnerability-only setting.

## Compose misconfiguration scanning

<Note>
  Compose misconfiguration scanning requires a **Skipper** or **Admiral** license.
</Note>

Beyond package CVEs, Sencho can run `trivy config` against a stack's Compose file to flag insecure defaults before you deploy. Typical checks include containers running as root, missing resource limits, privileged mode, host network, and mounted Docker sockets.

From any stack page, click **Scan config** next to the Deploy controls. Sencho runs the scanner against the stack's working directory and opens the scan drawer on the **Misconfigs** tab:

| Column | Description |
|--------|-------------|
| **Severity** | Rule severity (CRITICAL/HIGH/MEDIUM/LOW). |
| **Check** | Rule ID or AVD identifier for the violated check. |
| **Title** | Short summary, linked to the upstream advisory when available. The second line shows Trivy's detailed message. |
| **Target** | The file that triggered the finding. |
| **Fix** | The recommended resolution. |

Config scans are stored in the same history as image scans with an `image_ref` of `stack:<name>`, so they appear on the Scan history page and can be exported as CSV.

## SARIF export

<Note>
  SARIF export requires a **Skipper** or **Admiral** license.
</Note>

SARIF (Static Analysis Results Interchange Format) is the standard format supported by GitHub code scanning, Microsoft Defender for Cloud, and most security dashboards. Sencho generates SARIF 2.1.0 documents directly from the stored scan results so the download matches what you see in the drawer (same findings, same suppression state) without re-running Trivy.

From the scan drawer header, click **SARIF** to download the report. The file is named after the image reference with a `.sarif.json` extension.

What the export contains:

- **Vulnerabilities**: one SARIF result per CVE, with `security-severity` scored 9.8 (CRITICAL), 7.5 (HIGH), 5.0 (MEDIUM), 2.5 (LOW), or 0.0 (UNKNOWN). The affected package appears as a logical location (`<pkg>@<version>`).
- **Secrets**: rule IDs are namespaced as `SECRET:<rule>`. Results point at the file and line number where the match was found.
- **Misconfigs**: rule IDs are namespaced as `MISCONFIG:<rule>`. Results point at the Compose file that triggered the check.
- **Suppressions**: CVEs you have suppressed in Sencho are emitted with a SARIF `suppressions` entry of kind `external` and status `accepted`, so code-scanning dashboards can dismiss them with the justification you recorded.

Typical upload flow for GitHub code scanning:

```yaml
- name: Upload SARIF to GitHub
  uses: github/codeql-action/upload-sarif@v3
  with:
    sarif_file: sencho-scan.sarif.json
```

## Scan history

Every scan Sencho runs is stored with its full vulnerability detail. Scan records are automatically pruned after 90 days to keep the database compact. The history is used to power:

- **Digest caching**: skip re-scanning an image that has already been scanned within 24 hours.
- **Trend badges**: surface whether the latest scan added or resolved vulnerabilities compared to the previous scan for the same image.

Click **Scan history** from the top of the Resources Hub to open a right-side sheet layered over the current page. The sheet lists completed scans grouped by image, lets you search by image reference, and lets you pick two scans to compare. Close the sheet by pressing Escape, clicking the overlay, or clicking the close button in the header.

<Frame>
  <img src="/images/vulnerability-scanning/scan-history-sheet.png" alt="Scan history sheet overlaid on the Resources Hub, with search box, pagination, and scan rows grouped by image" />
</Frame>

## Comparing scans <Badge>Skipper</Badge>

Compare any two completed scans for an image to see what changed between them.

**From the Scan history page**: select two scans via the checkboxes (one baseline, one newer) and click **Compare**. Selecting a third scan replaces the oldest selection.

**From an open scan**: click **Compare** in the drawer header, then pick a baseline scan from the dropdown. Only completed scans for the same image appear.

The comparison sheet shows:

- A **delta ribbon** summarizing the net change per severity (CRITICAL, HIGH, MEDIUM, LOW). Net-positive deltas on CRITICAL render in destructive red so a regression on the worst tier is immediately visible.
- Filter pills to switch between **Added** (new findings since the baseline), **Removed** (resolved findings), and **Unchanged** (findings present in both).
- A sorted table of CVEs with severity, affected package, and direct links to the upstream advisory. CVE-prefixed identifiers resolve to [cve.org](https://www.cve.org); GHSA identifiers resolve to the GitHub Advisory Database. Critical and high rows carry the same left-rail tint as the scan results drawer for visual continuity.

Comparisons are scoped to a single node; scans taken on different nodes cannot be compared against each other.

Cross-image comparisons (picking scans from two different image references) are allowed but flagged with a warning, since package-level changes may reflect image differences rather than CVE drift. In that mode, the **Unchanged** pill is relabeled to **Shared** to reflect that same CVE + package matches across different images are not necessarily the same finding.

Up to 1000 findings per scan are loaded for comparison. When a scan exceeds this limit, the sheet shows a banner indicating the comparison may be incomplete.

<Frame>
  <img src="/images/vulnerability-scanning/scan-compare-sheet.png" alt="Compare scans sheet with delta ribbon, Added/Removed/Unchanged filter pills, and a diff table tinted by severity" />
</Frame>

## How it works

1. On startup, Sencho looks for the `trivy` binary on `PATH` and caches its availability.
2. When a scan is triggered, Sencho resolves the image digest via Docker and checks the 24-hour cache. If a completed scan exists for that digest, it is returned instantly.
3. Otherwise, Sencho spawns `trivy image --format json --quiet <image-ref>` and parses the JSON output into the vulnerability database.
4. Private registry credentials are forwarded automatically by writing a temporary `DOCKER_CONFIG` for the Trivy subprocess, then deleting it when the scan completes.
5. Scan status, counts per severity, and the full vulnerability list are persisted for display in the drawer and for policy evaluation.

## Troubleshooting

### Scan button is not visible

Sencho hides scanning UI when the Trivy binary is not detected. Check **Settings → Support** for the Trivy availability status, then follow [Installing Trivy](/operations/trivy-setup) if it is missing.

### Scans time out

The default scan timeout is 5 minutes. Very large images (2+ GB) over a slow connection may exceed this; pre-pulling the image to the host speeds up the scan significantly because Trivy works against the local image store.

### Private registry images fail to scan

Sencho forwards the same registry credentials configured under **Settings → Registries** to Trivy during a scan. If a pull works in Sencho but a scan fails, make sure the image has been pulled at least once (Trivy can then work against the cached local image).

### Badge is out of date after an image update

Post-deploy scanning only runs on deploy actions. For long-running images that aren't redeployed, schedule a recurring scan (Skipper+) or click the shield icon in the Resources Hub to re-scan on demand.

### A scan shows "in progress" for a long time

Scans have a 5-minute internal timeout. The scheduler also sweeps every tick and marks any scan that has been `in progress` for more than 15 minutes as failed, so the UI always recovers on its own. Wait for the sweep to run, then click the shield icon again to start a fresh scan.

### Trivy is not detected after installing it

Sencho re-checks for the Trivy binary every ten minutes from the scheduler. Install Trivy on the host, wait for the next window, and the scanning UI lights up automatically.

### Collecting diagnostic logs for support

Enable **Developer Mode** under **Settings → Developer** and trigger the failing scan again. The backend logs verbose `[Trivy:diag]` entries covering detection timing, cache hits, Trivy invocation, and parse statistics. Attach these lines when filing a support issue. Turn Developer Mode off once you have captured the output.

### Post-deploy scan failure notifications

When a post-deploy scan fails for a specific image (for example because Trivy could not resolve a private registry pull), Sencho dispatches a warning-level alert through your configured notification channels. The deploy itself is never blocked by a scan failure.

### A deploy was blocked by a policy I did not expect

The block dialog names the policy that fired and lists every image that violated the threshold. Open **Settings → Security → Scan Policies** and review the matching policy: check the stack pattern glob and the max severity. If the policy should not apply, tighten the pattern (for example `staging-*` instead of `*`) or turn **Block on deploy** off to keep the evaluation in alert-only mode. Admins can also bypass a single deploy with the **Deploy anyway** button; every bypass is recorded in the [Audit Log](/features/audit-log) with the actor, policy, and violation list.

### Trivy is not installed and a deploy with a block policy went through

Sencho fails open when Trivy is not installed on the target node, so users are never locked out by tooling state. A warning alert is dispatched through your configured notification channels with the message `Pre-deploy scan for "<stack>" skipped: Trivy not installed on this node`. Install Trivy from **Settings → Security** to enforce the policy; see [Installing Trivy](/operations/trivy-setup) for options.

### Compare button is disabled

Scan comparison is a Skipper feature; on Community, the Compare button stays disabled with a tooltip explaining the upgrade path. If your license is Skipper or Admiral, make sure you have ticked exactly two completed scans: selecting zero, one, or three scans leaves the button disabled. If you have only one scan for an image, trigger a second scan from the Resources Hub (or wait for a scheduled scan), then return to the Scan history page and tick both.

### Comparison shows unexpected results

Two common causes:

- **Cross-image comparison.** If the Baseline and Current rows at the top of the sheet point at different image references, the warning banner is shown and the "Unchanged" pill is labeled "Shared". Items in that bucket match on CVE + package name but may not be the same finding across two distinct images. Stick to scans of the same image reference for apples-to-apples drift analysis.
- **Truncated scans.** When either scan has more than 1000 stored findings, the sheet shows a truncation banner. In that case, items past the 1000-row cap do not contribute to the Added / Removed / Unchanged buckets and the totals may be misleading. Re-run the scan with a tighter image (or scope the investigation to the most severe findings) to avoid truncation.

### An older scan is missing from Scan history

The Scan history page uses server-driven pagination. If you know the scan exists but cannot see it, use the search box to filter by image reference, or page forward with the arrows in the card header. Scans older than 90 days are pruned automatically to keep the database compact.

### Scan policies are missing on one of my nodes

Scan policies are managed from the control Sencho instance and replicate to every remote. When you view **Settings → Security** on a remote Sencho (a replica), you will see a banner explaining that rules are managed upstream. See [Fleet Sync](/features/fleet-sync) for how replication works and how to investigate push failures.

### I suppressed a CVE but the scan badge count is unchanged

Badge counts reflect the raw findings so alerting and policy evaluation stay accurate. Open the scan drawer to confirm the row is dimmed with a shield-off icon. See [CVE Suppressions](/features/cve-suppressions) for how the filter is applied across the drawer, compare sheet, and other read surfaces.

### The Secrets tab is empty on an image I expect to contain credentials

Secret detection matches against Trivy's built-in rule set, which focuses on well-known provider patterns. Plain text passwords, custom token formats, or values that do not match any published rule will not appear. Ensure you picked **Full scan (vulnerabilities + secrets)** from the shield-icon menu; a plain vulnerability scan does not walk the filesystem.

### Scan config button is disabled on a stack

The button is only shown when Trivy is available on the stack's node, the current user is an admin, and the license is Skipper or Admiral. If all three conditions are met but the button stays disabled, another stack action (deploy, update, rollback) is still in progress; wait for it to finish.

### Compose misconfiguration scan returns 404

The scanner needs to locate a Compose file in the stack directory. If the stack was created outside Sencho and the working directory does not contain a file named `compose.yml`, `compose.yaml`, `docker-compose.yml`, or `docker-compose.yaml`, the scan returns 404. Name the file accordingly or keep the stack under Sencho's managed compose directory.

### SARIF download returns 409 "Scan not complete"

SARIF export requires a completed scan. If a scan failed, timed out, or is still running, the button downloads nothing and the server returns a 409. Trigger a fresh scan from the Resources Hub or the stack page, wait for the drawer to populate, then export again.