mirror of
https://github.com/Studio-Saelix/sencho.git
synced 2026-08-09 18:32:52 +00:00
4a350e7a0ae2f27492f3f526fc3b59a9549f2ba0
17 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Anso
|
4a350e7a0a | feat: add Docker label audit across Fleet and Stack views (#1531) | ||
|
Anso
|
bb4ddde35a |
feat(sidebar): surface partial status for multi-container stacks (#1426)
Bulk stack-status aggregation collapsed a stack to "running" as soon as any container was up, so a multi-container stack with crashed containers showed a green UP pill and the degradation was invisible from the sidebar. Add a crash-aware "partial" state: a stack is partial when at least one container is running and at least one has genuinely failed (exited with a non-zero code, dead, or crash-looping). Cleanly finished one-shot containers (exit 0) and clean restart-policy cycling do not count, so an app with a completed init job stays UP. The exit code is read from the container Status string, so no extra inspect calls are needed. Render partial as an amber PT pill with a hover tooltip showing the running/total count, fold it into the Down filter (needs-attention), and treat it as running for context-menu lifecycle actions so operators keep stop/restart/update. The dashboard stack-health table, cross-node search rows, and the command palette all pick up the new state through the shared status surfaces. |
||
|
Anso
|
f23b7e1bac |
feat: ordered multi-file Compose for Git sources (#1380)
* feat: ordered multi-file Compose for Git sources
Extend Git sources to deploy an ordered list of compose files merged with
docker compose -f base.yaml -f override.yaml ..., plus an optional project
directory.
- Pick and reorder compose files from the repository tree (drag to reorder on
desktop, up/down arrows on phones); manual path entry is also supported.
- The ordered set drives every stack-scoped compose command (deploy, update,
start/stop/restart/down, image scans, Compose Doctor) and the container
lookup, so a service or image declared only in an override is handled too.
- Runtime keys off the materialized set, not the saved configuration: saving a
source does not change deploy args until the pull is applied, and apply
materializes from the pending snapshot rather than live config.
- The project directory is passed as --project-directory, with -p <stack>
pinning the Compose project so container labels stay stable.
- The Mesh override is layered last; single-file sources are byte-identical to
before, and existing rows keep working via the single-path fallback.
Docs cover the picker, ordering, project directory, and the new troubleshooting
and limitations (referenced files are not materialized; the dependency graph,
drift, and networking views read the primary file).
* fix: harden multi-file Git source (hash, unlink, collisions, node id)
- hashContent folds ordered file CONTENTS (not paths) so a clean multi-file
stack is not flagged as locally edited: create/apply hash the fetched files
(repo paths) while pull hashes the on-disk files (materialized paths), which
previously disagreed and showed a false "local edits detected".
- Block unlinking a multi-file or project-directory Git source (409): the deploy
spec lives on the source row, so removing it would silently revert deploys to
root compose.yaml. Single-file sources still unlink.
- Reject materialized-path collisions in the selection validator: an additional
file equal to or nested under compose.yaml, an ancestor/descendant overlap
between selected files, and a project directory nested under a compose file
(previously a 500 at materialization).
- DockerController.getContainersByStack uses the controller's node compose dir
and passes its node id to the authored prefix, instead of the process default.
* fix: CI failures on multi-file Git source (test crash, aria query, path barrier)
- GitSourceFields no longer crashes when repoUrl/branch are falsy: the canBrowse
trim() is optional-chained, so a reusable field component tolerates partial
props. Fixes the apply-binding panel test, which feeds a minimal source object.
- GitSourcePanel tests query the footer Remove button by its exact name, so the
picker's per-file "Remove <path>" buttons no longer collide with the broad
/remove/i match (the test intent, footer Remove present/absent, is unchanged).
- validateCompose uses an inline resolve + startsWith barrier at the context-dir
mkdir sink (CodeQL does not credit the wrapped isPathWithinBase helper),
clearing the js/path-injection alert. The containment check is equivalent and
contextDir is also validated upstream.
* test: update Git source E2E spec for the multi-file compose picker
The compose-file picker replaced the single #git-source-path input and added
per-file Remove buttons, so the E2E spec drove selectors that no longer exist:
- Drop the redundant compose.yaml fills (the picker defaults to compose.yaml).
- Select the footer Remove button by exact name so the picker's per-file
"Remove <path>" buttons no longer make the locator ambiguous.
- Set a custom compose path through the picker (add via the manual input, press
Enter, then remove the default compose.yaml).
* test: match the footer Remove button with an exact Playwright name
Playwright's getByRole name option is a substring match by default, so
{ name: 'Remove' } also matched the picker's "Remove <path>" buttons. Require an
exact match so only the footer Remove button is selected.
|
||
|
Anso
|
f253276303 |
fix: make published port links open reliably (#1359)
* fix: make published port links open reliably Container published-port links now render as real anchors that open on desktop and mobile, replacing ad hoc window.open calls. A shared service URL builder centralizes host resolution (configured host, remote node API host, or the browser host, with no browser fallback for unreachable remote nodes), protocol selection (HTTPS for port 443), and known app sub-paths (Plex opens its web path). The container port mapping itself is the link, with a Copy URL action beside it. The stack Open App menu and the anatomy panel footer use the same builder, and the menu only offers Open App when a reachable URL can be built. * fix: skip UDP ports and scope known-app paths to the container port Two follow-ups to the published-port links: - The known-app path (Plex web sub-path) was borrowed from the published host port even when the container port was known and unregistered, so a non-Plex service published on host port 32400 wrongly inherited it. The container-port lookup now wins when known; the published-port lookup stays a fallback for the menu and anatomy footer, which only have the host port. - UDP ports could surface as HTTP links. The backend now carries the port protocol through both container-mapping paths and skips UDP when choosing the main web port (extracted as selectMainWebPort), and the container card filters UDP before selecting a port to link. |
||
|
Anso
|
e7895c889d |
fix: base Stack health uptime on container start, not creation (#1341)
The dashboard Stack health UP column counted from each container's Created timestamp, which never moves on stop/start or restart, so a restarted container kept reporting its original age. Resolve uptime from State.StartedAt (via a briefly cached inspect with bounded concurrency, falling back to Created when inspect is unavailable) so it reflects the real time since last start. The current CPU and MEM columns separately summed the latest sample per container with no recency filter, letting a recently stopped container's final reading linger in the totals. Drop samples that trail the freshest sample by more than the stale window so stopped containers leave the sum. |
||
|
Anso
|
af4083175c |
feat(fleet): add read-only dependency map tab (#1324)
* feat(fleet): add read-only dependency map tab Add a fleet-wide Dependencies tab to Fleet view that maps how stacks, services, networks, volumes, and ports relate, with flags for missing dependencies, port conflicts, orphaned resources, and cross-stack shared resources. Read-only; filterable by stack, node, and flag; collapsed by default with a list-view fallback at scale. The graph is derived at request time from Docker and compose metadata, so no new table or persisted state is introduced. A per-node graph endpoint feeds a hub aggregation endpoint that fans out across the fleet and degrades gracefully, surfacing unreachable or unparseable nodes inline while the rest of the map still renders. * fix(fleet): harden dependency map flag detection and remote merge Address review findings on the dependency map: - Port-conflict detection now does pairwise host-scope overlap, so an unrelated bind on the same port and protocol but a different specific host IP is no longer flagged, and the flag lands on the exact scoped port node. - A running service's depends_on target is only considered satisfied when it is actually running, so a crashed (exited) dependency is surfaced while a deliberately stopped stack stays quiet. - Declared external networks and volumes are reported missing when they do not exist on the host instead of being assumed present. - The hub deep-validates each remote node-graph payload before merging, so a reachable-but-malformed remote degrades to a single node error rather than failing the whole fleet map, and the validation failure is logged. - Searching or filtering on a network, volume, or port now also reveals the services that claim it and their stacks. |
||
|
Anso
|
308949282c |
feat(resources): reclaim banner controls and accurate reclaim math (#1318)
* feat(resources): reclaim banner controls and accurate reclaim math Make the Resources Hub reclaim banner match what it advertises and give operators control over when it appears. - "Review & prune" now reclaims every category the banner lists (unused images, stopped containers, and dangling volumes) instead of images only, so the banner clears in one action. Pruning runs volumes first, while stopped containers still reference their named volumes, so a stopped stack's data is never cascaded into deletion. - Add a "Show reclaimable-space banner" toggle under Settings, System, Docker hygiene (on by default, per node) and a dismiss control on the banner that snoozes it until the reclaimable total grows again. - Fix the reclaimable-space math: count only containers a prune can actually remove (created, exited, dead) and size them by their writable layer, so a small, un-prunable remainder no longer keeps the banner up. * fix(resources): show the reclaim banner when the settings fetch fails A failed or empty /settings load left the banner's enabled flag at the previously active node's value, so switching from a node with the banner turned off to a node whose /settings errored kept the new node's banner hidden. Set the flag unconditionally after the staleness guard so a failed fetch falls back to the default-on state for the current node. |
||
|
Anso
|
716daf77d0 |
feat(updates): auto-prune dangling images after updates (#1316)
* feat(updates): auto-prune dangling images after updates Each update pulls a fresh image and recreates containers, leaving the replaced image behind as a dangling layer that previously had to be pruned by hand. A new "Prune dangling images after updates" toggle under Settings > System > Docker hygiene reclaims these automatically. The setting is on by default and opt-out. When enabled, a successful stack update (manual or scheduled) and a Sencho self-update each remove the dangling image layers they orphaned. Only untagged layers are touched; tagged images, volumes, and data are never removed. The toggle requires an admin account and is per node: each instance honors its own value, so a remote node self-update applies that node's own preference. A prune failure never affects the update result: on the stack path it is caught and logged after the update has already succeeded, and on the self-update path the helper-shell prune runs only after a clean recreate and cannot change the exit code or the recorded update error. * security(self-update): shell-quote label-derived values in helper command Address review feedback on the prune-on-update change: - The self-update helper command interpolated the compose service name and config-file paths (both read from Docker Compose labels) straight into a shell string. Shell-quote them via shQuote so a label carrying shell metacharacters stays inert data and cannot break the exit-code capture, error-file write, or prune guard. - Correct the settings copy and docs: the prune is a standard dangling-image prune, so it reclaims every untagged layer on the node, not only the one the current update orphaned. Tagged images, volumes, and data remain untouched. - Add tests: shell-metacharacter neutralization and prune-output suppression in the self-update command, and an atomic-update case asserting a prune failure does not trigger a rollback. * fix(updates): omit the reclaim figure when the daemon reports zero bytes End-to-end testing on a Docker daemon backed by the containerd image store showed the post-update prune removing a dangling image while the prune API returned SpaceReclaimed=0, so the stream printed "reclaimed 0.0 MB" even though an image was removed. Show the reclaimed figure only when the daemon reports a non-zero value; otherwise the line reads "=== Pruned dangling images ===". The overlay2 store still reports real figures and shows them. Add a test covering both branches. |
||
|
Anso
|
cd1cde2fd4 |
fix(resources): subtract shared layers when accounting managed prune bytes (#1155)
* fix(resources): subtract shared layers when accounting managed prune bytes Both `pruneManagedOnly` and `estimateManagedReclaim` walked the Sencho-managed prunable image set and summed `img.Size` per image. That counts shared base layers once per image, so a 1 GB base layer shared across N managed images was reported as N GB freed — the same shape of inflation we just fixed for the system-scope banner. Introduce `getImageSharedSizeMap()` which reads `df.Images[].SharedSize` once and lets both code paths subtract `SharedSize` per image when totalling: `+= max(0, Size - shared)`. If `df` fails, the helper returns an empty map and the accounting degrades to the prior sum-of-Size behavior rather than failing the prune. Verified against a live daemon: `/api/system/prune/estimate` with `scope: managed, target: images` now returns the layer-aware number; the older per-image-Size sum was roughly 1.8× larger on the same set. * fix(resources): use df-delta for destructive managed prune; label estimate as lower bound The first attempt at this PR subtracted SharedSize per prunable image on both paths. That formula undercounts when prunable images share a layer exclusively with each other: Docker frees the layer once, but the per-image subtraction removes it from every referrer. The reported total is then strictly less than the truth. Split the two paths: - pruneManagedOnly (destructive) now snapshots `docker df` before and after the parallel removes and reports `max(0, before.LayersSize - after.LayersSize)`. That is the honest measurement of bytes freed. Concurrent pulls during the prune can grow the after value; the clamp treats that as 0 reclaimed for the affected delta rather than attributing the new bytes to us. - estimateManagedReclaim keeps the per-image Σ(Size - SharedSize) formula but the JSDoc now calls it a "conservative lower bound" and documents the under-report mechanism. There is no cheap way to exactly price an arbitrary prune subset without per-layer enumeration. Fallback chain when df fails on the destructive path: - before-snapshot succeeded, after failed → per-image lower bound from before-snapshot (safe; SharedSize was known at start). - before-snapshot failed → report 0 with a warn log (after-only would build a SharedSize map missing the just-pruned images, which would over-report by treating them as having no sharing). Replaces the prior `getImageSharedSizeMap()` helper with two pieces: `safeDfSnapshot()` (I/O) and a private static `mapSharedSizesFromDf()` (pure parse), reused by both code paths. New invariant test asserts `prune.reclaimedBytes >= estimate.reclaimableBytes` on the same inputs so future changes to either formula cannot flip the direction. Addresses Codex audit blocker on PR #1155. |
||
|
Anso
|
a1caf6b0dd |
fix(resources): use daemon-reported reclaimable image bytes (#1154)
* fix(resources): use daemon-reported reclaimable image bytes The Reclaim banner summed per-image `VirtualSize` (or `Size`) across every image with no running container. That counts shared base layers once per image, so an unused base layer of 1 GB shared across ten builds showed up as 10 GB of "prunable" space. The actual prune frees the layer once and reports a much smaller `SpaceReclaimed`, leaving the banner and the post-prune toast badly out of step. Prefer Docker's own `ImageUsage.Reclaimable` (API v1.44+), which is the exact value `docker system df` displays. Older daemons fall back to the Docker CLI's internal formula: `LayersSize - sum(Size - SharedSize)` for in-use images, clamped to 0, skipping any image Docker flags with the -1 unknown-size sentinel. Verified against a live daemon: the banner now matches `docker system df`'s IMAGES RECLAIMABLE byte-for-byte. * fix(resources): treat SharedSize=-1 as 0 in fallback, don't drop in-use bytes The fallback formula is `LayersSize - used`. The previous version skipped any active image whose VirtualSize or SharedSize was -1 (Docker's "unknown" sentinel). Skipping leaves the image's bytes out of `used`, which reads back as reclaimable -- the exact inflation the PR set out to fix, just on older daemons. Treat SharedSize=-1 (or absent) as 0 so the image's full size counts as in-use, and only skip when no usable size is available at all (VirtualSize and Size both unknown). Under-reporting reclaimable is the safe direction; over-reporting was the original bug. Add a test for the SharedSize=-1 case with Size known, and rename the existing test so it reflects what is actually being asserted now. Addresses Codex audit blocker on PR #1154. |
||
|
Anso
|
23bbee4f45 |
feat(mesh): replace host-mode with shared sencho_mesh Docker network (#1009)
* feat(mesh): replace host-mode with shared sencho_mesh Docker network Phase D of the mesh redesign: drop the operator's `network_mode: host` requirement and the `host-gateway` extra_hosts pattern that did not work on cloud iptables-restrictive distros (OCI, etc.) or Docker Desktop. Each Sencho creates a shared `sencho_mesh` Docker bridge network on boot (default subnet 172.30.0.0/24, override via SENCHO_MESH_SUBNET), pins itself at `<network>+2`, and attaches every meshed user service to the same bridge. Compose overrides now emit IP-based `extra_hosts` plus a top-level `networks` block declaring `sencho_mesh` external. Override delivery: central renders for local stacks; for remote stacks it sends the fleet alias list to the remote's new `PUT /api/mesh/local- override/:stackName` endpoint, which renders against the remote's OWN local senchoIp and writes under its OWN DATA_DIR. Each node may use a different subnet without coordination beyond the env var. Opt-in / opt-out now trigger an automatic redeploy of the affected stack via the existing deploy code path (local: ComposeService; remote: HTTP POST through proxyFetch). The frontend opt-in sheet shows a confirmation modal (ConfirmModal) before the mutation. Failed redeploys emit both a mesh activity event and a durable audit-log row. Hardening: - Reserve port 1852 at opt-in (prevents user containers from racing the Sencho API listener). - ensureMeshNetwork refuses to continue if `sencho_mesh` exists with a mismatched subnet rather than silently routing to the wrong IP. - Idempotent network connect/disconnect helpers in DockerController. - optInStack rolls back the DB row if the just-inserted stack's override push fails (no half-states surviving across calls). - regenerateOverridesForNode runs in parallel and skips the just- pushed stack on opt-in. Operator template: drop `network_mode: host`, restore `ports: ["1852:1852"]`. Mesh now works identically on Linux LAN, OCI, and Docker Desktop without firewall changes. Docs: rewrite docs/features/sencho-mesh.mdx around the shared bridge network, document SENCHO_MESH_SUBNET, surface the host-network-service opt-in restriction, and cross-link with the Pilot Agent docs. BREAKING CHANGE: the operator's `docker-compose.yml` no longer uses `network_mode: host`. After upgrading, redeploy any meshed stacks once so they pick up the new IP-based override and join `sencho_mesh`. * fix(mesh): wrap stackName with path.basename in local-override fs ops CodeQL flagged js/path-injection on the new applyLocalOverride and removeLocalOverride methods because they are publicly reachable and its data-flow model does not recognize isValidStackName / isPathWithinBase as sanitizers. The validation IS sufficient (the allowlist regex blocks path separators, the path-prefix check blocks escape), but path.basename is a model CodeQL recognizes and is purely defensive: for any input that already passes isValidStackName, basename is the identity. |
||
|
Anso
|
7e5dc2d9ea |
feat(resources): add image details sheet with layer history (#925)
Adds a read-only inspect panel for Docker images. Click the eye icon on any image row to open a sheet showing: - Overview: ID (with copy), size, created date, arch/OS, author, tags - Config: Cmd, Entrypoint, WorkingDir, User, exposed ports, env (collapsible), labels (collapsible) - Layers: ordered history list with size, age, and build command per layer. Empty layers (metadata-only) are dimmed. Backend adds DockerController.inspectImage(id) which combines image.inspect() and image.history() in parallel, exposed via GET /api/system/images/:id. The route accepts both bare hex IDs and sha256-prefixed IDs, since the list endpoint surfaces the prefixed form. Returns 400 for malformed IDs and 404 for missing images. Documents the new panel in docs/features/resources.mdx under Images. |
||
|
Anso
|
279ec62dff |
perf(backend): replace docker system df shell-out with dockerode API (#818)
MonitorService.evaluate() forked the docker CLI every 30s and
walked the human-readable Reclaimable strings ("1.196GB", etc.)
with a regex to compute the janitor threshold check. The Docker
Engine API returns raw byte counts, and the existing
DockerController.getDiskUsage() already wraps it for images,
containers, and volumes. Extend that helper with reclaimable
build-cache bytes so MonitorService can sum the four categories
in one call.
Drops the child_process / promisify imports from MonitorService and
removes about 30 lines of stdout parsing. Also widens the explicit
return type of getDiskUsageClassified so the new fields aren't
silent runtime additions.
|
||
|
Anso
|
4909c35e50 |
fix(resources): harden Resource Explorer with auth, validation, design, and UX fixes (#527)
- Sanitize error messages in all delete/prune/create/inspect endpoints to prevent Docker internals from leaking to the frontend - Add CIDR, IPv4, and Docker resource ID input validation - Add requirePaid gate to network topology endpoint - Add invalidateNodeCaches after image/volume/network mutations - Fix design system violations: card borders, destructive button variant, visible DialogDescription, overflow-auto replaced with ScrollArea, hardcoded Tailwind colors replaced with tokens - Gate purge button behind isAdmin to prevent silent 403s - Fix shared inspect loading state to be per-network-row - Parse error response bodies for meaningful toast messages - Add clipboard API fallback for non-HTTPS contexts - Render Options section in network inspect sheet - Add operational and diagnostic logging for resource operations - Extend validation and DockerController test suites - Update docs with Options field in network inspect |
||
|
Anso
|
24299a0115 |
feat(resources): add network management with create, inspect, and topology (#338)
* feat(resources): add network management with create, inspect, and topology visualization
Add full Docker network CRUD: create networks with custom drivers, subnets,
and IPAM config; inspect network details including connected containers and
IP addresses; interactive topology graph visualization (Pro-gated to
Skipper/Admiral tiers). Includes backend routes, DockerController methods,
unit tests, documentation with screenshots, and updated changelog.
* refactor(resources): address code review findings for network management
- Add batch GET /api/system/networks/topology endpoint to eliminate N+1
HTTP calls from the topology view
- Export DockerNetwork type from ResourcesView, remove duplicate in
NetworkTopologyView
- Wire up isInspectLoading state to Eye button (spinner + disabled)
- Remove unnecessary wrapper div around FilterToggle
- NetworkTopologyView is now self-contained (fetches from batch endpoint,
no longer needs networks prop)
* fix(resources): align pre-existing UI with design system standards
- Replace hardcoded red-500 on image/volume delete buttons with
text-destructive/60 hover:bg-destructive tokens
- Replace shadow-sm with shadow-card-bevel on all cards (Disk Footprint,
Quick Clean, Resource Tabs, Unmanaged Containers)
- Add strokeWidth={1.5} to all action button Trash2 icons
- Type network drivers as union type instead of raw strings (backend
NetworkDriver type + frontend NETWORK_DRIVERS constant)
* fix(resources): add generic type args to useNodesState/useEdgesState
Fixes TS2345 build error in CI where tsc -b (strict mode via
tsconfig.app.json) infers never[] from untyped empty array literals
passed to React Flow hooks.
* fix(resources): replace explicit any in catch blocks with unknown narrowing
ESLint no-explicit-any errors in CI for three catch blocks added by the
network management feature.
* test(resources): add network management edge case tests and fix bugs
Fix topology route using unclassified networks (missing managedStatus),
fix inspect route returning 500 instead of 404 for missing networks,
add driver validation and array-type labels rejection on create route,
replace all any types with proper unknown narrowing, and add comprehensive
edge case tests for createNetwork and inspectNetwork.
* fix(tests): add nullish guard for Containers in inspectNetwork test
Dockerode types NetworkInspectInfo.Containers as potentially undefined,
causing TS2769 under strict mode when passed directly to Object.keys().
* refactor(resources): replace Select with Combobox for network driver picker
Use the existing reusable Combobox component (same as Auto-Update and
Scheduled Task modals) for the driver selection in Create Network dialog.
Provides inline search filtering and consistent UX across all modals.
|
||
|
Anso
|
4488637656 |
feat(resources): add network management with create, inspect, and topology visualization (#335)
* feat(resources): add network management with create, inspect, and topology visualization
Add full Docker network CRUD: create networks with custom drivers, subnets,
and IPAM config; inspect network details including connected containers and
IP addresses; interactive topology graph visualization (Pro-gated to
Skipper/Admiral tiers). Includes backend routes, DockerController methods,
unit tests, documentation with screenshots, and updated changelog.
* refactor(resources): address code review findings for network management
- Add batch GET /api/system/networks/topology endpoint to eliminate N+1
HTTP calls from the topology view
- Export DockerNetwork type from ResourcesView, remove duplicate in
NetworkTopologyView
- Wire up isInspectLoading state to Eye button (spinner + disabled)
- Remove unnecessary wrapper div around FilterToggle
- NetworkTopologyView is now self-contained (fetches from batch endpoint,
no longer needs networks prop)
* fix(resources): align pre-existing UI with design system standards
- Replace hardcoded red-500 on image/volume delete buttons with
text-destructive/60 hover:bg-destructive tokens
- Replace shadow-sm with shadow-card-bevel on all cards (Disk Footprint,
Quick Clean, Resource Tabs, Unmanaged Containers)
- Add strokeWidth={1.5} to all action button Trash2 icons
- Type network drivers as union type instead of raw strings (backend
NetworkDriver type + frontend NETWORK_DRIVERS constant)
* fix(resources): add generic type args to useNodesState/useEdgesState
Fixes TS2345 build error in CI where tsc -b (strict mode via
tsconfig.app.json) infers never[] from untyped empty array literals
passed to React Flow hooks.
* fix(resources): replace explicit any in catch blocks with unknown narrowing
ESLint no-explicit-any errors in CI for three catch blocks added by the
network management feature.
|
||
|
Anso
|
93ae147ec1 |
test: expand backend test coverage for stability gaps (#329)
Add 5 new test files (116 test cases) targeting previously untested service-layer logic: - database-metrics: metrics CRUD, minute-bucket aggregation, cleanup retention, notification auto-cap at 100, stack alerts CRUD, and stress tests with 1000+ metrics - monitor-service: CPU/memory/network calculation helpers, all 6 alert condition operators, breach state machine lifecycle, global crash detection, host limit thresholds, cleanup delegation, and isProcessing concurrency guard - docker-controller: validateApiData error detection, state-safe start/stop (304 handling), batch container removal with partial failures, disk usage calculation, resource classification (managed/unmanaged/system), orphan detection, and daemon unreachable error propagation - scheduler-service: cron parsing, license tier gating, concurrent task prevention via runningTasks Set, manual trigger, all 4 task types (restart/snapshot/prune/update), wildcard targets, error recording, recovery notifications, and cleanup - compose-service: subprocess spawn/exit handling, WebSocket output, deploy with health probe and atomic rollback, registry auth temp dir lifecycle, and downStack teardown |