feat: ordered multi-file Compose for Git sources (#1380)

* feat: ordered multi-file Compose for Git sources

Extend Git sources to deploy an ordered list of compose files merged with
docker compose -f base.yaml -f override.yaml ..., plus an optional project
directory.

- Pick and reorder compose files from the repository tree (drag to reorder on
  desktop, up/down arrows on phones); manual path entry is also supported.
- The ordered set drives every stack-scoped compose command (deploy, update,
  start/stop/restart/down, image scans, Compose Doctor) and the container
  lookup, so a service or image declared only in an override is handled too.
- Runtime keys off the materialized set, not the saved configuration: saving a
  source does not change deploy args until the pull is applied, and apply
  materializes from the pending snapshot rather than live config.
- The project directory is passed as --project-directory, with -p <stack>
  pinning the Compose project so container labels stay stable.
- The Mesh override is layered last; single-file sources are byte-identical to
  before, and existing rows keep working via the single-path fallback.

Docs cover the picker, ordering, project directory, and the new troubleshooting
and limitations (referenced files are not materialized; the dependency graph,
drift, and networking views read the primary file).

* fix: harden multi-file Git source (hash, unlink, collisions, node id)

- hashContent folds ordered file CONTENTS (not paths) so a clean multi-file
  stack is not flagged as locally edited: create/apply hash the fetched files
  (repo paths) while pull hashes the on-disk files (materialized paths), which
  previously disagreed and showed a false "local edits detected".
- Block unlinking a multi-file or project-directory Git source (409): the deploy
  spec lives on the source row, so removing it would silently revert deploys to
  root compose.yaml. Single-file sources still unlink.
- Reject materialized-path collisions in the selection validator: an additional
  file equal to or nested under compose.yaml, an ancestor/descendant overlap
  between selected files, and a project directory nested under a compose file
  (previously a 500 at materialization).
- DockerController.getContainersByStack uses the controller's node compose dir
  and passes its node id to the authored prefix, instead of the process default.

* fix: CI failures on multi-file Git source (test crash, aria query, path barrier)

- GitSourceFields no longer crashes when repoUrl/branch are falsy: the canBrowse
  trim() is optional-chained, so a reusable field component tolerates partial
  props. Fixes the apply-binding panel test, which feeds a minimal source object.
- GitSourcePanel tests query the footer Remove button by its exact name, so the
  picker's per-file "Remove <path>" buttons no longer collide with the broad
  /remove/i match (the test intent, footer Remove present/absent, is unchanged).
- validateCompose uses an inline resolve + startsWith barrier at the context-dir
  mkdir sink (CodeQL does not credit the wrapped isPathWithinBase helper),
  clearing the js/path-injection alert. The containment check is equivalent and
  contextDir is also validated upstream.

* test: update Git source E2E spec for the multi-file compose picker

The compose-file picker replaced the single #git-source-path input and added
per-file Remove buttons, so the E2E spec drove selectors that no longer exist:

- Drop the redundant compose.yaml fills (the picker defaults to compose.yaml).
- Select the footer Remove button by exact name so the picker's per-file
  "Remove <path>" buttons no longer make the locator ambiguous.
- Set a custom compose path through the picker (add via the manual input, press
  Enter, then remove the default compose.yaml).

* test: match the footer Remove button with an exact Playwright name

Playwright's getByRole name option is a substring match by default, so
{ name: 'Remove' } also matched the picker's "Remove <path>" buttons. Require an
exact match so only the footer Remove button is selected.
This commit is contained in:
Anso
2026-06-17 13:24:55 -04:00
committed by GitHub
parent 7ce045accb
commit f23b7e1bac
34 changed files with 2299 additions and 385 deletions
+37 -12
View File
@@ -18,7 +18,7 @@ import { lookupContainerIp } from '../mesh/containerLookup';
import { STREAM_PENDING_DATA_MAX_BYTES } from '../pilot/protocol';
import { redactSensitiveText, sanitizeForLog } from '../utils/safeLog';
import { isDebugEnabled } from '../utils/debug';
import { isPathWithinBase, isValidStackName } from '../utils/validation';
import { isPathWithinBase, isValidStackName, isValidRelativeStackPath } from '../utils/validation';
import { PORT as SENCHO_LISTEN_PORT } from '../helpers/constants';
import { assertPolicyGateAllows, buildSystemPolicyGateOptions } from '../helpers/policyGate';
@@ -2080,18 +2080,23 @@ export class MeshService extends EventEmitter implements MeshForwarderHost {
const targetNodeId = nodeId ?? NodeRegistry.getInstance().getDefaultNodeId();
try {
const fsSvc = FileSystemService.getInstance(targetNodeId);
const filename = await fsSvc.getComposeFilename(stackName);
const baseDir = fsSvc.getBaseDir();
// path.basename strips any directory component as defense-in-depth
// on top of isValidStackName + isPathWithinBase. Recognized by
// CodeQL's path-injection model.
const composePath = path.join(baseDir, path.basename(stackName), filename);
if (!isPathWithinBase(composePath, baseDir)) return [];
const content = await fs.readFile(composePath, 'utf8');
const parsed = YAML.parse(content) as { services?: Record<string, unknown> } | null;
const services = parsed?.services && typeof parsed.services === 'object' ? parsed.services : null;
if (!services) return [];
return Object.keys(services).filter((name) => /^[A-Za-z0-9_][A-Za-z0-9_.-]*$/.test(name));
// For a multi-file Git stack, read every materialized compose file and
// union their service names, so a service declared only in an override
// file is still attached to the mesh. Single-file stacks read the one
// resolved compose file, byte-identical to the prior behavior.
const spec = DatabaseService.getInstance().getGitSource(stackName)?.applied_deploy_spec;
const relFiles = spec && spec.files.length > 0
? spec.files
: [await fsSvc.getComposeFilename(stackName)];
const names = new Set<string>();
for (const relFile of relFiles) {
if (relFile === '' || !isValidRelativeStackPath(relFile)) continue;
for (const name of await this.readComposeServiceNames(baseDir, stackName, relFile)) {
names.add(name);
}
}
return Array.from(names);
} catch (err) {
console.warn(
'[MeshService] getDeclaredStackServiceNames failed:',
@@ -2101,6 +2106,26 @@ export class MeshService extends EventEmitter implements MeshForwarderHost {
}
}
/**
* Read one compose file under a stack directory and return its declared
* service names. The stack segment uses path.basename as defense-in-depth and
* the resolved path is re-checked against the base dir; the relative file is
* validated by the caller. Returns [] when the file is missing or unparseable.
*/
private async readComposeServiceNames(baseDir: string, stackName: string, relFile: string): Promise<string[]> {
const composePath = path.join(baseDir, path.basename(stackName), relFile);
if (!isPathWithinBase(composePath, baseDir)) return [];
try {
const content = await fs.readFile(composePath, 'utf8');
const parsed = YAML.parse(content) as { services?: Record<string, unknown> } | null;
const services = parsed?.services && typeof parsed.services === 'object' ? parsed.services : null;
if (!services) return [];
return Object.keys(services).filter((name) => /^[A-Za-z0-9_][A-Za-z0-9_.-]*$/.test(name));
} catch {
return [];
}
}
/**
* Parse an existing mesh override file and extract the service names
* it already lists. Used by the defensive fallback so a transient