sencho/CHANGELOG.md

# Changelog

All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]
- **Fixed:** Container stats WebSocket flooding React with up to 20+ `setState` calls per second in `EditorLayout` — each container's `onmessage` handler called `setContainerStats` independently, causing React to schedule a separate reconciliation pass per container per second. Replaced with a ref-buffer + 1.5 s flush interval pattern: incoming stats are written to `pendingStatsRef` (no re-render cost), snapshotted and cleared before a single batched `setContainerStats` call every 1.5 s. A separate `rawBytesRef` (never cleared) tracks raw rx/tx bytes for accurate rate calculation, avoiding the stale-closure bug that would have produced 0 B/s net I/O after every flush cycle. Buffer is also cleared on cleanup so stale entries from the previous stack don't flash in on stack switch.
- **Fixed:** Browser Out of Memory crash in `GlobalObservabilityView` (Logs view) — the component rendered all log entries (up to 1,859+) as real DOM nodes with no virtualization, causing ~9,600 DOM nodes and ~25 MB of GC pressure every 5-second poll cycle. On RAM-constrained hosts (host system was at 97% usage) the browser renderer process OOMed within minutes. Fixed by capping DOM rendering to the last 300 entries (`MAX_DISPLAY_ROWS`), reducing the in-memory SSE log cap from 10,000 to 2,000 entries (`MAX_LOG_ENTRIES`), switching auto-scroll from `behavior: 'smooth'` (stacked layout animations) to `behavior: 'instant'`, and reducing the backend polling response limit from 2,000 to 500 lines. Also replaced `key={idx}` (array index) with a monotonic `_id` counter stamped at ingestion time so the slice window shifting no longer forces O(n) DOM mutations per new log line — React now only reconciles the one entry that actually changed.
- **Fixed:** `HomeDashboard` create-stack error handling — HTTP error responses were thrown as hardcoded strings, discarding the server's actual error message (e.g. "Stack already exists", "Invalid stack name"). Now reads the JSON error body before throwing, and uses the defensive `error?.message || error?.error || fallback` toast pattern.
- **Fixed:** LogViewer (container SSE log stream) returning 404 on remote nodes — the `?nodeId=` query param was forwarded by `remoteNodeProxy` to the remote server, where `nodeContextMiddleware` rejected it with `Node X not found` because the gateway's node IDs don't exist on the remote instance. Fixed by stripping `nodeId` from `proxyReq.path` in `onProxyReq`, mirroring the existing `x-node-id` header removal.
- **Fixed:** Terminal logs and container stats WebSockets failing with "HTTP Authentication failed" on remote nodes — the WebSocket upgrade proxy was forwarding the browser's `cookie` header to the remote Sencho instance. The remote's `authMiddleware` picks `cookieToken` before `bearerToken`, and the cookie (signed with the local JWT secret) fails verification on the remote, returning 401. Fixed by deleting the `cookie` header before `wsProxyServer.ws()`, mirroring the `proxyReq.removeHeader('cookie')` already present in the HTTP proxy. Also strips the gateway's `nodeId` query param from the forwarded URL so the remote defaults cleanly to its own local node.
- **Fixed:** `streamStats` Docker stats stream leaking after WebSocket client disconnect — the Docker daemon stream was never destroyed when the WS closed, causing orphaned streams to poll the daemon indefinitely. Fixed by adding a `ws.on('close')` handler that calls `stats.destroy()`. Also guards all `ws.send()` calls with a `readyState === OPEN` check to prevent silent errors on a closed socket.
- **Fixed:** `streamStats` and `execContainer` called unawaited in the WS `connection` handler — unhandled promise rejections (e.g., invalid container ID, Docker daemon unavailable) would terminate the Node.js process in production (Node ≥ 15). Fixed by chaining `.catch()` on both calls, logging the error, and closing the WebSocket cleanly.
- **Fixed:** Per-connection `WebSocket.Server` instances for stack logs and host console never closed after upgrade — each WS connection allocated a persistent `ws.Server` object accumulating listeners. Fixed by calling `wss.close()` immediately after `handleUpgrade` completes.
- **Fixed:** `authMiddleware` and WS upgrade handler evaluated `cookieToken || bearerToken` — a browser cookie present alongside a valid Bearer token would shadow it, risking 401 on node-to-node proxy calls. Fixed by inverting precedence to `bearerToken || cookieToken` in both places.
- **Fixed:** `remoteNodeProxy` error handler unsafely cast `proxyRes` to `Response` — on WebSocket or TCP-level proxy errors `proxyRes` is a raw `Socket`, so calling `.status()` would throw. Fixed by type-narrowing before sending the 502 response.
- **Fixed:** Container stats (CPU/RAM/NET), bash exec terminal, and "Open App" button all broken for remote nodes — stats and exec WebSockets connected to the bare root (`ws://host`) with no `?nodeId=` query param, so the upgrade handler couldn't detect the remote node and skipped the WS proxy. Moved all generic WebSockets to `/ws?nodeId=` path; upgrade handler now correctly proxies them to the remote Sencho instance.
- **Fixed:** Bash exec and stats WebSockets not reaching the backend in `npm run dev` — Vite proxy config now includes `ws: true` on `/api` and a new `/ws` proxy entry so all WebSocket upgrades are forwarded to `localhost:3000`.
- **Fixed:** Backend WS message handler crashing when a proxied WebSocket arrives from a gateway and the forwarded `nodeId` doesn't exist in the remote instance's DB — now falls back to the default local node instead of throwing.
- **Fixed:** "Open App" button opening `http://localhost:{port}` for remote node containers — now resolves the hostname from the remote node's `api_url` so the correct remote host is used.
- **Fixed:** Remote node system stats, container stats, logs, and exec returning errors — `remoteNodeProxy` middleware was already positioned before API route definitions, but `/api/system/stats` contained a dead remote branch that called `NodeRegistry.getDocker()` for remote nodes, which throws since remote nodes have no direct Docker socket access. Removed the broken branch; remote requests are correctly intercepted by the proxy middleware before reaching any route handler.
- **Added:** Background image update checker — `ImageUpdateService` polls OCI-compliant registries (Docker Hub, GHCR, LSCR, etc.) every 6 hours using manifest digest comparison against local `RepoDigests`. Results cached in a new `stack_update_status` SQLite table. A pulsing blue dot badge appears in the stack list sidebar for stacks with available updates. Manual refresh available via `POST /api/image-updates/refresh` (rate-limited to once per 10 minutes).
- **Fixed:** `AppStoreView` and `GlobalObservabilityView` using raw `fetch()` instead of `apiFetch()` — all calls now inject the `x-node-id` header so templates, deploys, stacks, and logs are correctly proxied to the active remote node.
- **Fixed:** `HostConsole` WebSocket URL missing `?nodeId=` query parameter — the upgrade handler now receives the active node ID and routes the PTY session to the correct node.
- **Added:** Two-tier Option A scoped navigation UX — a context pill in the top header bar always shows the active node name (pulsing blue for remote, green for local).
- **Added:** Remote-aware headers in `HostConsole` ("Host Console — [Node Name]"), `ResourcesView` ("Resources Hub — [Node Name]"), `GlobalObservabilityView` (floating node badge), and `AppStoreView` deploy sheet ("Deploying to: [Node Name]").
- **Added:** `SettingsModal` now scopes its sidebar to the active node type — when a remote node is selected, global-only tabs (Account, Appearance, Notifications, Nodes) are hidden, and the header subtitle shows the remote node name.
- **Fixed:** A massive memory leak (browser Out of Memory crash) by throttling historical metrics polling down to 60s and downsampling SQLite metrics payload sizes by 12x.
- **Fixed:** A bug where the active node UI dropdown would desync from the actual API requests on initial page load by properly hydrating state from localStorage.
- **Fixed:** Remote node proxy forwarding the browser's `sencho_token` cookie to the remote Sencho instance — the remote's `authMiddleware` evaluates `cookieToken || bearerToken` and the cookie (signed with the local JWT secret) was validated before the valid Bearer token, causing 401 on all proxied API calls. Fixed by stripping the `cookie` header in `proxyReq` so only the Bearer token is used for remote authentication.
- **Fixed:** `nodeContextMiddleware` blocking `/api/nodes` when `x-node-id` references a deleted/non-existent node — the nodes list endpoint must always succeed so the frontend can re-sync a stale node ID in localStorage; exempted alongside `/api/auth/`.
- **Fixed:** Remote node proxy stripping the `/api` path prefix — `remoteNodeProxy` is mounted at `app.use('/api/', ...)` so Express strips that prefix from `req.url` before `http-proxy-middleware` sees it; added `pathRewrite: (path) => '/api' + path` to restore the full path when forwarding to the remote Sencho instance (e.g. `/stats` → `/api/stats`). This was the root cause of all remote API calls returning the remote's SPA HTML instead of JSON.
- **Fixed:** Dashboard cards (Active Containers, Host CPU, Host RAM, Docker Network) showing stale local-node data after switching to a remote node — `HomeDashboard` polling effects now depend on `activeNode?.id` and clear state immediately on node change.
- **Fixed:** `refreshStacks` crashing with `SyntaxError` or `TypeError` when the remote proxy returns a non-JSON response (e.g., connection refused to unreachable remote node) — now checks `res.ok` before calling `res.json()` and iterates a typed `fileList` instead of the raw parsed value.
- **Fixed:** Restored Local/Remote type selector and fixed state resets in the Add Node modal — form now resets to defaults every time the dialog opens, and the title reflects the chosen type dynamically.
- **Fixed:** Remote node connection details failing to display Containers, Images, and CPU metrics — `testRemoteConnection` now fires parallel requests to `/api/stats`, `/api/system/stats`, and `/api/system/images` after auth succeeds, mapping real values into the info panel.
- **Fixed:** Suppressed `[DEP0060] DeprecationWarning: util._extend` from `http-proxy@1.18.1` — override is applied to `process.emitWarning` before the proxy instances are created, cleanly intercepting the warning at its call site without suppressing other warnings.
- **Fixed:** Backend memory leak caused by improper proxy middleware instantiation - `createProxyMiddleware` was called inside the request handler on every API call, spawning a new `http-proxy` instance (and registering new server listeners) per request. Refactored to a single globally-instantiated proxy using the `router` option for dynamic per-request target resolution.
- **Fixed:** `[DEP0060] DeprecationWarning: util._extend` deprecation eliminated as a side-effect of the above fix (deprecation was triggered on every new `http-proxy` initialisation).
- **Fixed:** Remote node authentication failures - `authMiddleware` and WebSocket upgrade handler both accept `Authorization: Bearer` tokens (Sencho-to-Sencho proxy auth).
- **Fixed:** Node connection testing logic updated to perform authenticated HTTP pings to `/api/auth/check` on the remote instance.
- **Fixed:** Node switcher dropdown failing to trigger data refreshes - `EditorLayout` now reacts to `activeNode` changes, re-fetching stacks and clearing stale editor/container state when the user switches nodes.
- **Fixed:** API Token copy button failing silently on HTTP / non-localhost deployments where `navigator.clipboard` is unavailable - added `try/catch` with `document.execCommand('copy')` fallback.
- **Fixed:** Remote node authentication failures by updating middleware to support Bearer tokens in WebSocket upgrade handler (node-to-node WS proxy now authenticates correctly on the receiving instance).
- **Fixed:** Node connection testing logic updated to normalize `api_url` trailing slashes before constructing the authenticated HTTP ping URL.
- **Fixed:** Memory leak in `GlobalObservabilityView` SSE mode - log array now capped at 10,000 entries (`.slice(-10000)`) to prevent unbounded accumulation across long sessions.
- **Fixed:** Infinite re-fetch loop in `NodeContext` - `refreshNodes` useCallback no longer depends on `activeNode` state; replaced with a `useRef` to read current node inside the callback without being a reactive dependency.
- **Fixed:** Infinite page reload loop - `apiFetch` was calling `window.location.href = '/'` on every 401, causing a full browser reload before auth could complete. Replaced with a `sencho-unauthorized` custom event that `AuthContext` handles by setting `appStatus` to `notAuthenticated`.
- **Fixed:** `NodeProvider` was mounted outside the auth gate in `App.tsx`, causing `refreshNodes` to fire before authentication was established (hitting 401 immediately on boot). Moved `NodeProvider` inside the authenticated branch so it only mounts after login.
- **Removed:** SSH/SFTP file adapters and remote Docker TCP connections (net negative ~500 lines of code).
- **Added:** Distributed API proxying using http-proxy-middleware for HTTP and WebSockets.
- **Added:** Long-lived JWT generation for Sencho-to-Sencho API authentication (`POST /api/auth/generate-node-token`).
- **Changed:** Node Manager UI vastly simplified - remote nodes now only require an API URL and Token.
- **Fixed:** Critical port routing conflict - separated Docker API port (`port`) from SSH/SFTP port (`ssh_port`) in the `nodes` schema. Previously, a single `port` field served both protocols, causing ECONNREFUSED.
- **Fixed:** `FileSystemService` now reads the node's `compose_dir` from the database for remote nodes instead of always using the `COMPOSE_DIR` env var.
- **Fixed:** SSH/SFTP connections in `SSHFileAdapter`, `ComposeService.executeRemote()`, and `ComposeService.streamLogs()` now use `ssh_port` (default 22) instead of Docker API `port`.
- **Added:** Full SSH credential fields (SSH Port, Username, Password, Private Key) to the Node Manager Add/Edit forms.
- **Added:** `ssh_port` column to the `nodes` database table with migration support (default: 22).
- **Changed:** Global `FileSystemService` and `ComposeService` singletons refactored into node-aware instances.
- **Added:** `IFileAdapter`, `LocalFileAdapter`, and `SSHFileAdapter` to abstract all filesystem interactions for remote node support.
- **Changed:** `MonitorService` now evaluates limits, fetches metrics, and detects container crashes across all registered nodes concurrently.
- **Added:** Node Context Middleware in Express API to dynamically extract `x-node-id` headers and parse WebSocket query parameters.
- **Added:** Remote Nodes Foundation - `nodes` table in SQLite with auto-seeded default local node.
- **Added:** `NodeRegistry` service for managing multiple Docker daemon connections (local socket + TCP).
- **Added:** Node management API endpoints: list, get, create, update, delete, and test connection.
- **Added:** Settings Hub → Nodes tab with full CRUD UI, connection testing, and Docker info display.
- **Added:** Node switcher dropdown in sidebar (auto-visible when multiple nodes are configured).
- **Added:** `NodeContext` for frontend-wide active node state management.
- **Fixed:** Global logs false-positive error misclassifications caused by Docker containers writing INFO logs to STDERR. Replaced naive regex with a robust 3-tier classification engine supporting `level=info`, `[INFO]`, and ` INFO ` format standards.
- **Added:** Developer Mode setting to enable true Real-Time (SSE) global log streaming and infinite scroll.
- **Added:** Configurable polling rates for standard global logs monitoring.
- **Added:** React Throttle Buffer to prevent UI freezing during heavy real-time log ingestion.
- **Fixed:** Global Logs aggressive auto-scrolling preventing users from reading log history.
- **Fixed:** Quiet stacks missing from the Global Logs filter dropdown by fetching the definitive stack list independently.
- **Fixed:** Global logs misclassifying INFO messages as errors due to naive string matching.
- **Changed:** Global logs now display chronologically (newest at bottom) with smooth auto-scrolling.
- **Changed:** Renamed Observability navigation tab to Logs.
- **Fixed:** TTY container log streams failing to parse globally.
- **Fixed:** Global logs displaying in UTC instead of local browser timezone.
- **Changed:** Global Logs UI revamped to use a floating, hover-based action bar to maximize terminal space.
- **Fixed:** Docker raw byte multiplex headers leaking into global logs stream.
- **Changed:** Relocated historical CPU/RAM charts to the Home Dashboard and normalized data values (CPU relative to host cores, RAM to GB).
- **Added:** Dozzle-style Action Bar to Global Logs with multi-select stack filtering, search, and STDOUT/STDERR toggles.
- **Added:** Centralized observability dashboard tracking 24-hour historical metrics and aggregating global tail logs across all running containers.
- **Added:** Live Container Logs viewer using Server-Sent Events (SSE) for real-time terminal output.
- **Added:** Pre-deploy folder collision check to prevent silent configuration overwrites in the App Store.
- **Added:** UI subtitle during deployment to reassure users during long image downloads.
- **Changed:** Standardized manual stack deletion to use the Two-Stage Teardown (Compose Down -> File Wipe) to prevent ghost networks.
- **Fixed:** Atomic Rollback failure where non-empty directories caused silent file system errors.
- **Added:** Two-Stage Teardown mechanism to ensure `docker compose down` sweeps up ghost networks before deployment files are deleted.
### Deprecated
- **(Planned)** Port 2375 (TCP) fallback support; future releases may require SSH-only for Node config.

### Fixed
- Fixed backend MonitorService crash (`Cannot read properties of undefined (reading 'cpu_usage')`) occurring when Docker containers lacked CPU telemetry during transition states.
- Handled UI deleted nodes ghost API calls by intercepting 404 errors globally in API and forcing the UI to resync to the default Node context.
- Hardened `nodeContextMiddleware` in Express to intercept queries to invalid or deleted Node IDs gracefully instead of bubbling to Docker API 500 crashes.
- Hardened Remote Node connection testing (`docker.info()`) to explicitly validate expected Docker API daemon properties instead of merely checking string length.
- Caught Unhandled SFTP Promise Rejections in Node registry gracefully returning empty arrays to prevent frontend loading UI stalls.
- Fixed horizontal UI overflowing in Node Manager settings on smaller resolutions.
- **Fixed:** Docker API parsing bug where HTML string responses from misconfigured ports were counted as containers.
- **Fixed:** Stack list crashing when SFTP connections fail by gracefully catching SSH errors and returning empty arrays.

### Changed
- **Changed:** Expanded Node Manager UI width and added horizontal scrollbars for better data visibility.
- **Added:** Smart Error Parser with telemetry-ready rule IDs to translate cryptic Docker output.
- **Added:** Post-Deploy Health Probe to catch immediate container crashes that slip past Compose.
- **Changed:** Rollback engine respects a `canSilentlyRollback` flag to protect user-authored configurations.
- **Changed:** Removed rigid volume sanitization, allowing full user control over bind paths.
- **Added:** Editable Host Volumes in the deployment UI.
- **Added:** Custom Environment Variable injection tool.
- **Fixed:** ScrollArea UI height rendering and dynamic browser timezone detection.
- **Changed:** Rebranded "Templates" to "App Store" across the UI.
- **Added:** Advanced deployment configuration panel (Editable Ports and Environment Variables) with smart defaults.
- **Fixed:** Implemented smart image fallbacks for broken registry logos and added expandable descriptions.
- **Added:** Atomic Deployments: Failed App Store deployments now automatically roll back and delete their orphaned folders.
- **Fixed:** Global dark mode scrollbar styling to eliminate blinding white native scrollbars.
- **Fixed:** Input overlap UI bug in the App Store deployment panel.
### Added
- **Added:** Official LinuxServer.io API integration as the default Template Registry.
- **Added:** Rich metadata display in the App Store (Architectures, Documentation links, GitHub repository links).
- **Added:** Dynamic Template Registry URL support via global settings, defaulting to LinuxServer.io templates.
- **Fixed:** Smart Volume Sanitizer to automatically rewrite messy Portainer bind mounts into clean, relative paths (Sencho 1:1 path rule).
- Git Flow branching strategy and branch protection.
- GitHub Actions CI pipeline for automated TypeScript build verification.
- **Added:** Automated Docker Hub CI/CD pipeline for the `dev` and `latest` tags.
- **Added:** App Templates (App Store) with One-Click deployment, utilizing Portainer v2 JSON registries and auto-generating compose files.