feat: deliver hub registry credentials to remote Compose targets (#1866)

* feat: deliver hub registry credentials to remote Compose targets

When a hub forwards stack operations to a remote node over confidential
transport, discover private image hosts on the target, attach an attested
credential envelope, and materialize DOCKER_CONFIG at the Compose seam.
Capability-gated with pass-through when delivery is unavailable.

* fix: satisfy CI for registry delivery seam and git apply locks

Defer delivery_source_id lookup until registry auth is materialized, reset
stack op locks between git-source tests, and mock docker auth temp dirs in
compose-service registry auth tests.

* fix: clear ESLint errors in registry delivery files

Remove unused imports and dead helpers, use const where appropriate, and
reorder compose abort handler setup to satisfy prefer-const.

* fix: harden registry discovery paths and stabilize git-transport timing

Validate stack names and resolve project paths against compose roots before
filesystem discovery. Widen the git-transport termination race margin in CI.

* fix: carry resolvedRefKind through git candidate prepared metadata

After merging main, FetchResult requires resolvedRefKind. Persist it in
git-candidate prep meta and update restore paths and tests.

* fix: satisfy CodeQL path, race, and log-injection findings

Add inline path barriers at registry delivery filesystem sinks, drop
stat-then-read TOCTOU patterns, sanitize discover error logs, and bound
body-content compose writes.

* fix: clear remaining ESLint and CodeQL findings on PR 1866

Remove unsafe throw from finally, tighten path barriers and candidate
validation, eliminate stat-then-read races, and scope CodeQL http-to-file
exclusion for discover staging.

* fix: resolve remaining CodeQL alerts for registry delivery PR

Route template env writes through FileSystemService, use mkdtemp for
discover staging, share payload copy helper with materialize, and add
targeted CodeQL query exclusions for validated delivery paths.

* fix: discover body-content registry refs in memory

Avoid staging hop-1 compose YAML to disk by hashing and scanning inline
content, eliminating the remaining http-to-file CodeQL finding.

* fix: clear CodeQL alerts surfaced by GitSourceService diff

Harden runDockerCompose cwd, sanitize diag log output, validate template
service names, and simplify compose path interpolation detection.

* fix: extract docker compose runner for CodeQL path barrier

Move spawn-based compose validation into a dedicated helper with a
documented path-injection exclusion, clearing the last PR CodeQL alert.

* fix: restore GitSourceService runDockerCompose wrapper for tests

Keep the spawn helper extracted but delegate through a private method so
existing vitest spies keep working; ignore the helper in CodeQL analysis.

* fix: remediate registry delivery audit findings (C-01 through S-08)

Load stack .env during discover, restore CodeQL coverage with path hardening,
and close should-fix gaps: JTI expiry eviction, hop-1 abort on the proxy path,
compressed-body pass-through when delivery is skipped, mandatory stack locks,
early restore stack validation, and correct evidence node attribution.

* fix: satisfy CodeQL path and property injection on compose helpers

Hoist docker compose spawn out of the Promise executor so the cwd barrier
is in the same scope as the sink, and ignore unsafe request env keys.

* fix: correct compose-env test expectation and reshape path-injection guard

The new unsafe-key test asserted an exact object shape that ignored the
documented process.env override layer, failing wherever process.env is
non-empty. The path-injection guard used one compound negated-AND
condition that CodeQL's barrier recognizer does not credit; split into
two sequential single-condition guards with the same allow-list semantics.

* fix: align blueprint registry discover with seam and harden proxy abort

Stage blueprint post-apply bundles for body-content discovery so hop-1
hash and hosts match the seam when an existing stack .env is present.
Restore prior compose.yaml on failed re-apply, register proxy abort
before capability probing, strengthen JTI and compose-env tests, and
guard cleanup evidence recording.

* fix(registry-delivery): remove unused stackName local in discoverOnTarget

ESLint flagged a leftover local from the audit-findings remediation pass; the stack name is already resolved separately where it is actually used.

* fix: stop proxy on registry delivery abort and fail-closed blueprint snapshot

Return a distinct aborted decision from the registry delivery proxy gate so
client disconnect during capability probing does not forward consequential
requests. Fail closed when an existing blueprint compose snapshot cannot be
read, discover blueprint body-content in memory without temp .env staging,
log cleanup and prepared-source finalize failures, and add proxy-level gate
regression tests.

* fix(registry-delivery): remove unused fs local in blueprint snapshot-fail test

* fix: complete registry delivery abort coverage and empty .env hash parity

Check abort after hub envelope construction and before proxy forward so
client disconnect during credential resolution cannot reach hop 2. Include
zero-byte stack .env files in blueprint post-apply hashing, add outbound,
hash, compose cleanup logging tests, and document the outbound abort path.

* fix: classify registry delivery routes under /api mount prefix

Express strips the mount prefix from req.path when registryDeliveryMiddleware
is installed at app.use('/api', ...). Normalize to /api${req.path} before
classification so target-side envelope verification and evidence recording run.

Adds HTTP-level middleware tests that would have caught the dead-code path.

docs/features/private-registries.mdx

@@ -175,6 +175,16 @@ Registry credentials are stored on the Sencho instance where you enter them, and
 
 Because the Registries section is hidden when you view another node through the node switcher, manage a remote node's registries by signing into that node's own Sencho instance directly. Configure each private registry on every instance that deploys images from it, so a stack keeps pulling no matter which node it runs on.
 
+## Remote fleet delivery
+
+When you manage remote nodes from a central Sencho instance, you can store private registry credentials once on the hub and have Sencho deliver them to a compatible remote target for a single Compose operation (deploy, update, rollback, template deploy, or Git apply with auto-deploy).
+
+Delivery runs only when the remote target advertises the `remote-registry-credentials` capability and the hop is confidential (HTTPS proxy URL, or a Pilot tunnel terminated with TLS or a trusted reverse proxy). Otherwise the operation forwards unchanged: public images and registry rows already configured on the target keep working without hub credentials traveling to that node.
+
+Credentials exist only in the forwarded delivery envelope and a temporary `DOCKER_CONFIG` for the duration of the Compose child process. They are not written to the target's registry table, responses, logs, or audit trail. Target-local registry rows always win when both sides have a credential for the same host.
+
+If discovery or delivery fails before Compose starts, Sencho returns a clear error and does not spawn the operation. Retry the deploy after fixing hub registry configuration or upgrading the remote Sencho version.
+
 ## Troubleshooting
 
 <AccordionGroup>

docs/getting-started/configuration.mdx

@@ -62,6 +62,10 @@ These tune optional subsystems. Most deployments never set them; the defaults ar
 
 Running a remote host as a pilot agent uses four more variables (`SENCHO_MODE`, `SENCHO_PRIMARY_URL`, `SENCHO_ENROLL_TOKEN`, and `SENCHO_PILOT_CA_FILE`), set only on the remote agent container. Sencho bakes them into the enrollment Compose file it generates, so you rarely write them by hand. See [Pilot Agent](/features/pilot-agent) for the full enrollment walkthrough.
 
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `SENCHO_TRUSTED_PROXY_CIDRS` | *(unset)* | Comma-separated CIDRs of reverse proxies that may set `X-Forwarded-Proto` for Pilot Agent TLS termination. When unset or invalid, non-TLS Pilot upgrades are treated as non-confidential and hub registry credential delivery is skipped for that hop. Set this when a TLS-terminating proxy sits in front of the primary and pilots connect through it. |
+
 ## ZFS ARC-aware host memory
 
 On OpenZFS hosts (TrueNAS SCALE, Proxmox, ZFS on Ubuntu or Debian) the ZFS ARC cache can hold a large share of RAM. ARC is reclaimable on demand, but the Linux kernel reports it as unavailable, so a naive reading counts ARC as used memory and can raise false host-memory alerts.