feat: add per-stack project env file selection for Docker Compose (#1457)

* feat: add per-stack project env file selection for Docker Compose

Allow users to configure an ordered list of env files per stack that serve
as the project environment file(s) for Docker Compose ${VAR} interpolation.
The selected files are passed via repeated --env-file flags during all
compose commands.

Backend:
- Add stack_project_env_files table (node-scoped, ordered)
- Extend authoredComposeEnvFileArgs to emit --env-file for configured files
- Add GET/PUT /stacks/:name/project-env-files and /candidates endpoints
- Update resolveStackEnvSources to use configured files as interpolation source
- Update resolveAllEnvFilePaths to merge injection + interpolation sources
- Add discoverStackLocalEnvFiles for candidate discovery
- Extend backupStackFiles and snapshotStackFiles for project env files
- Add project-env-files capability to CapabilityRegistry

Frontend:
- Add project env file selector to EnvironmentPanel (capability-gated)
- Update EditorView banner to generic "project environment file" language
- Add project-env-files capability to capabilities.ts

Issue: #1454

* fix: add realpath validation, clear all stale backup files, reject nested paths

- authoredComposeEnvFileArgs: use fsPromises.realpath + isPathWithinBase
  for symlink escape defense at use time
- backupStackFiles: clear ALL non-marker files from backup slot before
  writing, not just PROTECTED_STACK_FILES (handles stale old.env)
- PUT project-env-files: reject paths containing / or \ (root-level
  only, matching Compose auto-discovery behavior)

* fix: add getStackProjectEnvFiles to compose-service mock

The new authoredComposeEnvFileArgs calls getStackProjectEnvFiles
on the DatabaseService singleton. The compose-service mesh-override
tests mock that singleton without the new method, causing 6 failures.
Add getStackProjectEnvFiles: () => [] (empty = fall back to legacy
behavior, which is what these tests exercise).

* fix: add getStackProjectEnvFiles to remaining service mocks

The new authoredComposeEnvFileArgs calls getStackProjectEnvFiles,
which is missing from the mock in compose-images.test.ts (6 failures)
and image-update-service.test.ts (proactive fix).

* fix: apply inline path-injection barrier at fs sink for CodeQL

The PUT project-env-files route resolved paths via isPathWithinBase
before calling fsp.stat, but CodeQL does not credit a containment check
separated from the sink. Apply the canonical inline barrier pattern
(path.resolve + startsWith at the sink) used throughout the codebase.

* fix: resolve stackDir from the same canonical root as safePath

Prevents a containment bypass when the compose base directory is
a symlink: stackDir was previously joined from the unresolved
baseDir while the inline barrier used path.resolve(baseDir),
which could differ for symlinked paths. Now both stackDir and
safePath are resolved from a single canonical root, then each is
containment-checked against it.

* fix: remove unused isPathWithinBase import

The inline path-injection barrier refactor replaced isPathWithinBase
with an inline startsWith check at the fs sink, so the import is now
unused and fails ESLint no-unused-vars.
This commit is contained in:
Anso
2026-06-25 18:03:05 -04:00
committed by GitHub
parent b7dd9dc1b0
commit a698aaa926
13 changed files with 550 additions and 95 deletions
@@ -39,6 +39,7 @@ export const CAPABILITIES = [
'update-guard',
'compose-networking',
'env-inventory',
'project-env-files',
'compose-storage',
] as const;
+43
View File
@@ -1875,6 +1875,22 @@ export class DatabaseService {
} catch (e) {
console.warn('[DatabaseService] mesh_stacks migration:', (e as Error).message);
}
try {
this.db.prepare(`
CREATE TABLE IF NOT EXISTS stack_project_env_files (
id INTEGER PRIMARY KEY AUTOINCREMENT,
node_id INTEGER NOT NULL,
stack_name TEXT NOT NULL,
env_file TEXT NOT NULL,
position INTEGER NOT NULL,
UNIQUE(node_id, stack_name, env_file),
FOREIGN KEY (node_id) REFERENCES nodes(id) ON DELETE CASCADE
)
`).run();
this.db.prepare('CREATE INDEX IF NOT EXISTS idx_stack_project_env_files_lookup ON stack_project_env_files(node_id, stack_name)').run();
} catch (e) {
console.warn('[DatabaseService] stack_project_env_files migration:', (e as Error).message);
}
// mesh_centrals was the peer-side cache of the reverse-callback JWT
// (central → peer bootstrap material). Peer→central traffic now
// multiplexes over the existing forward WS via `tcp_open_reverse`,
@@ -2025,6 +2041,33 @@ export class DatabaseService {
this.db.prepare('DELETE FROM mesh_stacks WHERE node_id = ? AND stack_name = ?').run(nodeId, stackName);
}
// --- Project env files ---
public getStackProjectEnvFiles(nodeId: number, stackName: string): string[] {
const rows = this.db.prepare(
'SELECT env_file FROM stack_project_env_files WHERE node_id = ? AND stack_name = ? ORDER BY position ASC'
).all(nodeId, stackName) as Array<{ env_file: string }>;
return rows.map(r => r.env_file);
}
public setStackProjectEnvFiles(nodeId: number, stackName: string, files: string[]): void {
const deleteStmt = this.db.prepare('DELETE FROM stack_project_env_files WHERE node_id = ? AND stack_name = ?');
const insertStmt = this.db.prepare(
'INSERT INTO stack_project_env_files (node_id, stack_name, env_file, position) VALUES (?, ?, ?, ?)'
);
const tx = this.db.transaction((ordered: string[]) => {
deleteStmt.run(nodeId, stackName);
ordered.forEach((file, idx) => {
insertStmt.run(nodeId, stackName, file, idx);
});
});
tx(files);
}
public deleteStackProjectEnvFiles(nodeId: number, stackName: string): void {
this.db.prepare('DELETE FROM stack_project_env_files WHERE node_id = ? AND stack_name = ?').run(nodeId, stackName);
}
public setNodeMeshEnabled(nodeId: number, enabled: boolean): void {
this.db.prepare('UPDATE nodes SET mesh_enabled = ? WHERE id = ?').run(enabled ? 1 : 0, nodeId);
}
+60 -24
View File
@@ -6,6 +6,7 @@ import type { Dirent } from 'fs';
import { Readable } from 'stream';
import { pipeline } from 'stream/promises';
import { NodeRegistry } from './NodeRegistry';
import { DatabaseService } from './DatabaseService';
import { isPathWithinBase, isValidStackName } from '../utils/validation';
import { isBinaryBuffer } from '../utils/binaryDetect';
import { sanitizeForLog } from '../utils/safeLog';
@@ -953,24 +954,32 @@ export class FileSystemService {
}
await fsPromises.mkdir(backupDir, { recursive: true });
// Clear stale managed files from the backup slot before writing the current
// ones. The slot is reused across runs, so a managed file removed from the
// stack since the last backup (e.g. a deleted .env or a switched compose
// variant) would otherwise linger here and a later restore would resurrect
// it, breaking the faithful-revert guarantee. Scope is the protected set
// Sencho writes; .timestamp is rewritten below. A clear failure is logged but
// not fatal: it only risks a stale future rollback, so it should not block an
// otherwise valid deploy.
for (const file of PROTECTED_STACK_FILES) {
const stale = path.resolve(backupRoot, path.join(backupDir, file));
if (!stale.startsWith(backupRoot + path.sep)) continue;
try {
await fsPromises.unlink(stale);
} catch (e: unknown) {
if ((e as NodeJS.ErrnoException)?.code !== 'ENOENT') {
console.warn(`[FileSystemService] Could not clear stale backup ${file}:`, (e as Error).message);
// Clear ALL non-marker files from the backup slot before writing the current
// set. The slot is reused across runs, so a file removed from the stack since
// the last backup (e.g. a deleted .env, a switched compose variant, or a
// removed project env file like old.env) would otherwise linger here and a
// later restore would resurrect it, breaking the faithful-revert guarantee.
// Marker files (.timestamp, .checksums) are preserved until rewritten below.
// A clear failure is logged but not fatal: it only risks a stale future
// rollback, so it should not block an otherwise valid deploy.
try {
const existing = await fsPromises.readdir(backupDir);
for (const item of existing) {
if (BACKUP_MARKER_FILES.has(item)) continue;
const stale = path.resolve(backupRoot, path.join(backupDir, item));
if (!stale.startsWith(backupRoot + path.sep)) continue;
try {
await fsPromises.unlink(stale);
} catch (e: unknown) {
if ((e as NodeJS.ErrnoException)?.code !== 'ENOENT') {
console.warn(`[FileSystemService] Could not clear stale backup ${item}:`, (e as Error).message);
}
}
}
} catch (e: unknown) {
if ((e as NodeJS.ErrnoException)?.code !== 'ENOENT') {
console.warn('[FileSystemService] Could not read backup directory for stale cleanup:', (e as Error).message);
}
}
// Copy each managed file by reading it into memory, writing it to the backup
@@ -1028,6 +1037,23 @@ export class FileSystemService {
}
await writeManagedBackupFile('.env', envSrc);
// Backup configured project env files (e.g. stack.env, .env.production).
// Dedup against .env (already in PROTECTED_STACK_FILES) so it is not
// duplicated. Stale dynamic files from a prior backup are cleared below so a
// previously-backed-up old.env does not linger.
let projectEnvFiles: string[] = [];
try {
projectEnvFiles = DatabaseService.getInstance().getStackProjectEnvFiles(this.nodeId, stackName);
} catch {
// DB read failure is not fatal to the deploy; skip project env file backup.
}
for (const file of projectEnvFiles) {
if (file === '.env') continue; // already handled above
const src = path.resolve(baseResolved, path.join(stackDir, file));
if (!src.startsWith(baseResolved + path.sep)) continue;
await writeManagedBackupFile(file, src);
}
// Write the integrity manifest before the timestamp marker, so a crash
// between the two leaves the checksums present (a backup that restore can
// verify) rather than a timestamp with no integrity data. Only files that
@@ -1142,9 +1168,10 @@ export class FileSystemService {
}
/**
* Capture the current managed stack files (PROTECTED_STACK_FILES) in memory and
* return a function that puts them back, faithfully (writing the captured
* contents and removing any managed file that did not exist when captured).
* Capture the current managed stack files (PROTECTED_STACK_FILES plus configured
* project env files) in memory and return a function that puts them back,
* faithfully (writing the captured contents and removing any managed file that
* did not exist when captured).
*
* Used by the rollback route to undo a restored backup when the policy gate
* blocks before the deploy commits: restoreStackFiles has already overwritten
@@ -1154,12 +1181,21 @@ export class FileSystemService {
async snapshotStackFiles(stackName: string): Promise<() => Promise<void>> {
const stackDir = this.resolveStackDir(stackName);
await this.assertRealWithinBase(stackDir);
// Canonical js/path-injection barrier inline with the read/write sinks, the
// same pattern restoreStackFiles uses: resolve against the base and confirm
// containment so static analysis credits the barrier.
const baseResolved = path.resolve(this.baseDir);
const snapshot = new Map<string, Buffer>();
for (const file of PROTECTED_STACK_FILES) {
// Collect the unified set: PROTECTED_STACK_FILES + configured project env files.
const files = new Set(PROTECTED_STACK_FILES);
try {
const projectEnvFiles = DatabaseService.getInstance().getStackProjectEnvFiles(this.nodeId, stackName);
for (const f of projectEnvFiles) {
if (f !== '.env') files.add(f); // .env already in PROTECTED_STACK_FILES
}
} catch {
// DB read failure: snapshot without project env files (safe fallback).
}
for (const file of files) {
const target = path.resolve(baseResolved, path.join(stackDir, file));
if (!target.startsWith(baseResolved + path.sep)) {
throw Object.assign(new Error('Path escapes compose directory'), { code: 'INVALID_PATH' });
@@ -1171,7 +1207,7 @@ export class FileSystemService {
}
}
return async () => {
for (const file of PROTECTED_STACK_FILES) {
for (const file of files) {
const target = path.resolve(baseResolved, path.join(stackDir, file));
if (!target.startsWith(baseResolved + path.sep)) continue;
const saved = snapshot.get(file);