gsadmin/sencho
1
0
Fork 0
mirror of https://github.com/Studio-Saelix/sencho.git synced 2026-10-26 13:40:26 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
3ecb954f282d778b72da01bf67ee331c4ca3fc28
sencho/docs/features/multi-node.mdx
T
Anso ab6e4940e7 feat(ui): add resizable sidebar and Anatomy pane (#1902) 
* feat(db): per-user preference store with versioned revision contract

Add the user_preferences table (per user and domain, with a strictly
advancing integer revision and schema_version) plus the atomic
persistence methods behind it:

- getUserPreferenceDomain returns a transport envelope with corrupt-row
  reporting (the revision stays intact so clients can repair)
- mutateUserPreferenceDomain applies a document mutation inside a single
  transaction under an expectedRevision or create-if-absent
  precondition; two competing writers cannot both succeed
- resetUserPreferenceDomain writes a tombstone (schema_version 0) under
  the same precondition contract, so a post-reset edit can
  conditionally un-tombstone while a stale pre-reset edit conflicts
- deleteUserPreferenceDomains removes every row on account deletion
  (foreign keys are declarative only in this codebase, so deleteUser
  runs the cleanup explicitly in its transaction)

* feat(preferences): per-user preference API with conditional writes

Add the /api/user-preferences router: collection GET, per-domain PUT
and DELETE (reset to tombstone), and a migrate endpoint for the
create-if-absent first-run race.

- Every operation is identity-guarded: the x-sencho-pref-user header
  must match the authenticated user before any persistence access, so
  a stale tab can never read or write another account's rows. A
  mismatch is a 409 IDENTITY_CHANGED with no current envelope, kept
  distinct from the revision-conflict 409 CONFLICT which carries one.
- Writes are conditional (expectedRevision or absent: true); there is
  no unguarded write path. Stale preconditions return the fresh
  envelope so clients reconcile instead of retrying blind.
- Documents are validated server-side with strict Zod schemas mirroring
  the frontend field guards; legacy navigation mode values are rejected
- Machine identities (node proxy, pilot tunnel) are rejected and API
  tokens lack the required scope
- The route is hub-only: a remote x-node-id targeting it gets an
  explicit 403, since account-owned state lives on the hub
- Audit summaries cover migrate, update, reset, and failed attempts

* feat(preferences): parity script pinning server enums to frontend registries

The server-side enum tables duplicate the frontend's field guards, and
nothing in the TypeScript compilation boundary links the two (backend
and frontend compile separately). Add a standalone check script that
extracts both value sets from source text and fails on drift, wired as
the pretest step for backend and frontend so a mismatch surfaces in
the ordinary test run instead of as a 400 at write time.

* feat(preferences): frontend preference domains, sync bus, and identity guards

Introduce the client half of server-backed per-user preferences: the
existing localStorage hooks remain the owners of state and DOM
application; a thin sync layer carries their documents to and from the
per-user preference API.

- preferenceEvents: the leaf module with write notifications (dirty
  field attribution), the identity-generation counter, the
  eligibility-readiness store with ownership-captured publications,
  and the unsaved-episode failure surface
- preferencesDocuments: build and per-field sanitize of appearance and
  navigation documents, tombstone-aware hydration, and the owner
  marker that wipes cached values when another account claims the
  browser
- syncBus: the single write choke point. Per-domain serialization,
  same-kind coalescing, reset-cancels-prior-puts chronology, debounced
  trailing writes, conditional PUTs with 409 reconciliation (reset
  conflicts re-queue the DELETE once on the adopted revision),
  keepalive flush on page hide, and pump-level identity guards that
  drop operations captured by a superseded account or generation
- useUserPreferencesSync: the always-mounted sync owner. Hydrates on
  identity resolution, reconciles late GETs against dirty fields
  (server wins untouched fields), adopts remote resets by observable
  revision, repairs corrupt rows with a conditional PUT, and surfaces
  failures as a single unsaved episode per domain
- AuthContext: sequenced identity transitions and a cross-tab storage
  signal so a login or logout in one tab re-checks auth in the others
- the localStorage hooks gain apply/current exports for the hydration
  path; user-facing setters notify the bus, hydration and derived
  seeding paths deliberately do not

* feat(preferences): reset actions, account-scope copy, and frontend tests

Wire the per-domain reset actions into the settings surface: the
appearance section footer and navigation section each offer a
complete-domain reset, and the recovery section's interface reset
runs both domains through the shared handler with a reload only after
both tombstones settle successfully.

- Settings registry: appearance scope moves from browser to account,
  with copy that states where preferences now live
- ThemeQuickSwitch: the save-location line reflects account syncing
- Tests: dirty-field merge against a late hydration GET, identity
  guards (cache ownership, queue invalidation on account switch,
  loading-state no-op), corrupt-row repair through the hook path,
  remote-reset precedence, reset settle/failure handling, registry
  scope assertions, and the eligibility readiness bridge

* test(e2e): cross-browser preference convergence suite

Add the end-to-end spec covering the promise the feature makes: two
browser contexts on the same account converge (appearance fields,
density, navigation mode and pin order), a reset in one context
reaches the other on reload, an account switch renders the new
account's state rather than the previous account's cache, and a
pending write captured before the switch is rejected server-side
without touching either account's rows.

The existing navigation, motion, and effects suites seed a known
server-side preference baseline per suite account so the new
hydration layer cannot overwrite their localStorage-only setups.

* docs: appearance and navigation preferences are account-scoped

Rewrite the user-facing pages that described browser-only
personalization: the appearance feature page (including the quick
switcher, troubleshooting entries, and screenshot description), the
settings reference scope legend, and the multi-node and overview
pages that stated preferences stay per browser. The recovery page's
emergency-reset entry now explains that clearing site data only
drops the local cache, not the saved preferences.

* test(preferences): mint API tokens through the shared test helper

Replace the duplicated token-minting code with the existing
createTestApiToken helper so the file leans on the repo-standard
scaffolding and CodeQL stops flagging the inline sha256 hashing
as new alerts on this diff.

* feat(preferences): sidebar mode and width in the appearance document

Extend the appearance preference domain with sidebarMode (fixed or
resizable) and sidebarWidth (224 to 440 px, default 256), the persisted
contract for the upcoming desktop sidebar resize feature.

Frontend: a use-sidebar-layout hook modeled on the density hook (guarded
localStorage cache, settings-changed broadcast, per-field dirty-field
notification), the new fields in the appearance document builder,
hydration with default fallbacks for older documents, both keys in the
identity-clear cache key list, and a targeted sidebar-layout reset that
puts defaults without tombstoning the domain. The theme cache document
excludes the new fields so pre-paint cache rewrites stay consistent.

Backend: the strict appearance schema gains both fields with defaults, so
an older writer's 16-field document parses and is stored normalized
instead of being rejected; out-of-bounds, fractional, and unknown mode
values still fail validation. The preference parity script now compares
the mode enum across backend and frontend.

Tests: backend normalization (PUT and migrate) plus boundary rejection
cases, and a frontend hook suite covering sanitization, cached reads,
write-path attribution, cross-tab sync, and cache key registration.

* feat(shell): resizable stacks sidebar

Add a desktop resize boundary for the stacks sidebar. When the sidebar
mode preference is resizable, the shell wraps the sidebar in a width-
owned pane with a draggable separator: pointer drag updates the pane
live and persists one sanitized width on release; pointercancel, lost
pointer capture, and unmount clean up without committing. The separator
is keyboard accessible (arrows, Home, End) and exposes role=separator
ARIA values that track the effective bounds.

The effective width clamps to the viewport through a ResizeObserver on
the shell flex row (sidebar plus workspace), reserving a minimum
workspace width. Narrowing the window shrinks the pane live without
rewriting the stored preference; widening restores the preferred width.
Fixed mode (the default) renders the original shell DOM unchanged, and
mobile is untouched.

* fix(shell): harden sidebar resize drag teardown

The resize separator now ignores a second pointerdown while a drag is
in progress, so an extra touch or mouse button cannot steal or corrupt
an active drag. The commit on pointerup releases the drag state in a
finally block, so teardown (body cursor, user-select, capture release)
always runs even if a consumer's commit handler throws. The drag
teardown helper drops its unused commit flag: cancellation paths never
commit, and the single pointerup commit now owns that decision
explicitly.

Covered by three new pane tests: keyboard End clamping to the
viewport-limited max, sub-minimum drags clamping to the minimum
without a sub-minimum commit, and interleaved second-pointer
pointerdowns being ignored.

* feat(settings): sidebar layout preference

Add a Sidebar layout section to Settings > Appearance with a
Fixed/Resizable mode control, a width slider (224 to 440 px, step 4),
and a targeted reset that restores the defaults without touching other
appearance values.

The sidebar layout hook now broadcasts the settings-changed event when
one of its setter writes actually changes localStorage, so every mounted
instance (the shell separator, the Settings slider, other tabs) converges
on the commit instead of holding a stale draft.

Settings search keywords gain sidebar/resize/layout aliases, and the
preference sync tests cover the targeted reset path: local defaults apply
immediately, one bus op carries both dirty fields, and the write degrades
to a migrate POST when no preference row exists yet.

* fix(shell): review cleanup for the resizable sidebar

Restore the Fixed-mode main column class exactly as before (min-w-0 is
now applied only in Resizable mode, so the desktop default DOM stays
byte-identical), drop the unused committed drag flag, remove a stray
blank line and the redundant width-commit wrapper, and align the
separator keyboard step on the 8 px used by the file explorer pane.

* test(e2e): sidebar resize journeys

Cover the resizable stacks sidebar end to end: Fixed default with no
separator, the Settings opt-in writing one preference PUT, drag and
keyboard resize with release as the persistence boundary, viewport
clamping that never persists the clamped width, mode-toggle retention,
a targeted reset that leaves other appearance values untouched,
per-account scoping, and an unchanged phone layout. The shared
preference GET helper now retries briefly on rate-limit responses so a
burst of suite traffic cannot fail an assertion read.

* docs(preferences): sidebar layout docs and E2E polling hardening

User-facing docs for the sidebar layout preference: a Display bullet and
new Sidebar section in the Appearance feature page (with screenshot), a
Display row in the settings reference, and an appearance row update on
the multi-node page stating that sidebar layout is account-scoped.

E2E: every server-state assertion now polls (expectStoredAppearance
helper over GET user-preferences) instead of relying on a fixed sleep,
the dead flushPreferenceWrite/DEBOUNCE_MS pair is removed, the
targeted-reset seed uses non-default theme and density so the
"untouched" assertion is meaningful, and serial hand-off points gained
storage boundary proofs. Helper retry comment now states the 429 retry
window explicitly.

* feat: add resizable Anatomy pane

* fix: expand Anatomy resize range

* test: build storage events without the two-arg constructor

Static analysis models StorageEvent as one-argument only. The tests
still need key and newValue, so those fields are defined on a
generic storage Event.

* ci: give Playwright E2E thirty minutes on PRs

The pane-layout journeys pushed the chromium suite past the old
twenty-minute job limit, so the run was cancelled mid-suite.
2026-09-18 01:19:15 -04:00

338 lines
24 KiB
Plaintext
Raw Blame History

---
title: Multi-Node Management
sidebarTitle: Multi-node
description: Connect multiple Sencho instances and manage every server from one console with no central server, no SSH, and no shared Docker sockets.
---
Sencho's multi-node feature lets you operate Docker Compose stacks on every server you run from a single browser tab. Each server runs its own Sencho instance and the control instance acts as a transparent proxy to the others.
<Frame>
<img src="/images/multi-node/node-manager.png" alt="Settings · Infrastructure · Nodes panel showing the masthead breadcrumb, SCOPE/NODES/REMOTE stat strip, Add node button, Generate Node Token card, and a populated table of one Local row plus three Proxy-mode remote rows." />
</Frame>
## How it works
There is no central server. Each Sencho instance manages its own host independently. When you select a remote node from the switcher, your browser keeps talking to your control instance; the control instance forwards the call to the remote one, authenticated by a per-node bearer token in Distributed API Proxy mode or by the enrolled outbound tunnel session in Pilot Agent mode. No SSH. No shared Docker sockets. No remote Docker TCP exposure.
Remote nodes connect in one of two modes. Pick the one that matches your network topology before you add the node, because the add-node flow branches on the choice.
<Note>
Adding, editing, removing, and testing nodes, and generating a node token, require an administrator account.
</Note>
## The local node
Only one local node can exist per Sencho instance. The local node cannot be deleted because each instance must retain its own Docker engine identity. All operations on the local node run directly against the host's Docker socket.
If multiple local nodes exist from an older version, the extra rows show a Delete action so you can clean them up. Deleting a local node removes its schedules, labels, dossiers, findings, and other node-scoped data; containers and compose files on the host are not affected. The last remaining local row hides Delete because it cannot be removed.
## Choose a remote mode
When you add a remote node, the Mode picker offers two options. The form tells you the difference inline:
> Pilot Agent requires only outbound HTTPS from the remote host. Distributed API Proxy requires the remote host to expose an inbound port. Sencho Mesh works with both modes.
| | Pilot Agent (default) | Distributed API Proxy |
|---|---|---|
| **Direction** | Remote dials the control plane (outbound only) | Control plane dials the remote (inbound to remote) |
| **Network requirement** | Outbound HTTPS to the control instance | Inbound TCP port reachable from the control instance |
| **What you run on the remote host** | A generated Compose file, brought up with `docker compose up -d` | A full Sencho install reachable on a routable URL |
| **Token model** | One-shot enrollment token, 15-minute expiry | Long-lived bearer token, rotates on regeneration |
| **Best for** | Hosts behind NAT, dynamic IPs, restrictive ingress, edge boxes | Servers you already expose on a stable URL with TLS termination |
If you are not sure, start with **Pilot Agent**. It is the default because it works in more network shapes and the enrollment is a single Compose file.
For a deep look at how the pilot tunnel works under the hood (credential lifecycle, security model, resource limits, environment variables, and pilot-specific troubleshooting), see [Pilot Agent](/features/pilot-agent). Cross-node container networking (reaching a meshed service on any node by hostname) rides the same authenticated channel and works in either mode; see [Sencho Mesh](/features/sencho-mesh).
## Add a remote node: Pilot Agent
### Step 1. Open the Add node form
On the control instance, click your avatar in the top-right and choose **Settings**. In the sidebar pick **Infrastructure → Nodes**, then click **Add node**.
<Frame>
<img src="/images/multi-node/add-node-pilot.png" alt="Add remote node modal with Type set to Remote and Mode set to Pilot Agent. The form shows Name, Type, Mode, Mode helper text, and Compose Directory fields. URL and token fields are hidden because Pilot Agent does not need them." />
</Frame>
| Field | Notes |
|-------|-------|
| **Name** | A display name (e.g. `staging-vps`, `media-box`). Visible in the switcher and across the UI. |
| **Type** | Set to **Remote**. |
| **Mode** | Set to **Pilot Agent**. |
| **Compose Directory** | The absolute directory where compose stack folders live on the remote host. Pilot Agent defaults to `/opt/docker/sencho` and mounts the directory at the same path inside the agent so relative stack bind mounts remain host-correct. |
Click **Add node**.
### Step 2. Deploy the agent on the remote host
A modal opens with a generated Compose file and a start command.
<Frame>
<img src="/images/multi-node/pilot-enroll.png" alt="Enroll the pilot agent modal with two steps. Step 1 shows a generated compose.yaml with SENCHO_MODE=pilot, SENCHO_PRIMARY_URL pointing to the control instance, and a SENCHO_ENROLL_TOKEN. Step 2 shows the docker compose up -d start command with a copy button. The footer reads Expires 14m from now and offers a Copy compose file button." />
</Frame>
Save Step 1's contents as `compose.yaml` on the remote host, then run Step 2's command (`docker compose up -d`) in the same directory as the user that owns Docker. The generated file pulls the `saelix/sencho:latest` image, mounts the host Docker socket, attaches a named volume for the agent's own state so re-enrollment survives a container restart, and starts a container named `sencho-agent` that opens an outbound tunnel back to the control plane.
The enrollment token embedded in the file is **valid for 15 minutes and can only be used once**. If you wait too long, follow the regeneration steps below.
### Step 3. Verify the tunnel
Back on the Nodes table, the new row shows **Status: Online** once the agent dials home, and the Endpoint column reads `tunnel (seen Xs ago)`. Until the agent connects for the first time, the Endpoint reads `tunnel (waiting)`.
### Re-enrollment
If the enrollment token expires, the container is removed, or the host is rebuilt, click the **Edit Node** icon on the pilot row. The Edit modal shows a **Regenerate enrollment token** card.
<Frame>
<img src="/images/multi-node/edit-pilot-regenerate.png" alt="Edit node modal for a Pilot Agent node. Below the Compose Directory field a bordered card explains: 'Re-enroll the agent if the container was lost or the enrollment token expired. The previous tunnel is disconnected automatically.' A Regenerate enrollment token button sits below the explanation." />
</Frame>
Click the button to mint a fresh enrollment Compose file. The previous tunnel is disconnected automatically; save and deploy the new file on the remote host the same way (`compose.yaml`, then `docker compose up -d`) to reconnect.
## Add a remote node: Distributed API Proxy
### Step 1. Generate a long-lived token on the remote
On the **remote** Sencho instance (the server you want to add), open **Settings → Infrastructure → Nodes**. Click **Generate Token** in the **Generate Node Token** card and copy the token that appears.
<Frame>
<img src="/images/multi-node/generate-token.png" alt="Generate Node Token card showing the explanation text and a freshly issued JWT-style token displayed in a monospace block with a copy button." />
</Frame>
<Note>
The token is a long-lived credential that grants full control over that Sencho instance. Treat it like a password. Generating a new token invalidates the previous one.
</Note>
### Step 2. Add the node on your control instance
Open **Settings → Infrastructure → Nodes** on the control instance and click **Add node**. Set Type to **Remote** and Mode to **Distributed API Proxy** to reveal the URL and token fields.
<Frame>
<img src="/images/multi-node/add-node-proxy.png" alt="Add remote node modal in Distributed API Proxy mode. The form shows Name, Type, Mode, Sencho API URL with a plain-HTTP warning banner underneath, API Token (masked), and Compose Directory. The HTTP warning recommends HTTPS or a VPN when the node is reachable over the public internet." />
</Frame>
| Field | Notes |
|-------|-------|
| **Name** | A display name shown in the switcher. |
| **Type** | **Remote**. |
| **Mode** | **Distributed API Proxy**. |
| **Sencho API URL** | The full URL of the remote Sencho instance, including scheme and port (e.g. `https://sencho.example.com` or `http://10.0.1.20:1852`). |
| **API Token** | The token you copied in Step 1. |
| **Compose Directory** | The root directory where compose stack folders live on the remote host. |
If the URL starts with plain `http://`, the form shows an inline warning. Plain HTTP is fine on a private network or VPN, but it leaks the bearer token over any path you do not control. See [Transport encryption](#transport-encryption) below.
Click **Add node**. Sencho immediately tests the connection and shows the result.
### Step 3. Verify connectivity
Click the **Test Connection** icon on any row to re-check status. A successful test shows a Connection Details panel below the table with the remote's OS, architecture, container count, and Sencho version.
If the row shows **Offline** or **Unknown**, see [Troubleshooting](#troubleshooting).
## Switching between nodes
The **node switcher** sits at the top of the sidebar and acts as a persistent identity anchor. It shows the active node's status dot, a `Node · LOCAL`, `Node · REMOTE`, or `Node · AGENT` kicker, and the node name, so you always know which node you are operating on.
When two or more nodes are registered, clicking the switcher opens a popover listing every connected node. Each row shows a status dot, the name, and a metadata line that joins the type chip, the last-seen timestamp (pilot agents only), and the node version with middle-dot separators (for example, `AGENT · SEEN 13H AGO · v0.76.3`). The active node is marked with a brand-coloured rail, and a filled star calls out the default node.
<Frame>
<img src="/images/multi-node/node-switcher-dropdown.png" alt="Node switcher popover with a Connected header above a 4 NODES count. The Local row shows LOCAL · v0.95.0 with a star icon. Three Remote rows show the REMOTE chip. A Manage nodes link sits at the bottom." />
</Frame>
Click any row to switch. All views (dashboard, stack list, editor, resources, logs) immediately reflect the selected node. The **Manage nodes** link at the bottom of the popover opens the Settings → Nodes panel to add, edit, or remove nodes.
## What top-level views show when a remote node is active
Top-level views that manage fleet-wide state are scoped to the local hub and are hidden from the nav strip when a remote node is the active selection. They reappear automatically when you switch back to your local node.
| View | Why it is hub-only |
|------|--------------------|
| **Fleet** | Manages your node registry and cross-node topology; only meaningful from the hub. |
| **Schedules** | Schedules are dispatched from the hub to target nodes. |
| **Audit** | Audit history is centralized on the hub. |
| **Logs** | Aggregates logs across the fleet. |
| **Auto-Update** | Compares image versions across all nodes in your fleet. |
Node-level views (Home, Resources, Networking, App Store, Console, the editor) work as expected against whichever node you have selected.
## The Nodes table
The Nodes table surfaces routing, status, and per-node automation at a glance for every node:
<Frame>
<img src="/images/multi-node/nodes-table-overview.png" alt="Nodes table close-up showing the ten columns: a default-marker column with a star on the Local row, Name, Type, Mode, Endpoint, Status, Labels, Schedules, Updates, and Actions. The Local row shows Endpoint 'docker.sock' and an Auto updates badge. Three Proxy-mode remote rows show full URLs in the Endpoint column." />
</Frame>
| Column | What it shows |
|--------|---------------|
| **Default** | A filled star marks the default (Local) node. The column has no header. |
| **Name** | A globe or monitor icon followed by the display name. |
| **Type** | `Local` or `Remote` badge. |
| **Mode** | `-` for the local node; `Proxy` or `Pilot Agent` badge for remotes, with an icon matching the mode. |
| **Endpoint** | `docker.sock` for local; the full Sencho API URL for proxy nodes; `tunnel (seen X ago)` or `tunnel (waiting)` for pilot agents. |
| **Status** | `Online`, `Offline`, or `Unknown` badge. |
| **Labels** | Per-node label palette. The cell shows the label picker; an empty cell reads `No labels` with an Add label control. |
| **Schedules** | Number of active scheduled tasks targeting this node, plus a `next X` countdown to the next run. Click the count or the calendar icon in the Actions column to filter the Schedules view to that node. |
| **Updates** | `Auto` if at least one enabled `Auto-update Stack` or `Auto-update All Stacks on Node` schedule targets the node; `Off` otherwise. A pulsing dot and count appear when stacks have pending image updates. |
| **Actions** | **View Schedules**, **Test Connection**, **Edit Node**, and **Delete Node** icon buttons. The last local row hides Delete because the local node cannot be removed. When multiple local nodes exist from an older version, the extra rows show Delete so you can clean them up. |
Clicking the schedules-link icon opens the Schedules view filtered to the selected node. From there you can create, edit, or manage scheduled tasks scoped to that node. The filter bar shows which node you are viewing, with a Clear filter control to return to the full list.
When a node is deleted, all scheduled tasks and update status data associated with it are cleaned up automatically.
## What Settings apply per node
When you select a remote node in the switcher, the Settings hub filters to the panels that control that specific instance. Values saved here never cross over to other nodes.
| Panel | Scope | Notes |
|-------|:-----:|-------|
| Appearance | Account | Theme, density, navigation, sidebar layout, and Anatomy panel layout preferences are saved to your account, not to the node. Changing the active node does not change your look. |
| Stacks | Per node | Stack editor, lifecycle workflow preferences, and deploy guardrails for the selected node. |
| Fleet (Settings panel) | Per node | Fleet-snapshot documentation capture for Dossier notes on the selected node. Administrator only. Distinct from the hub-only **Fleet** nav view described below. |
| App Store | Per node | Template registry URL and featured-catalog source for the selected node's catalog. |
| Host Alerts | Per node | Host CPU, RAM, and disk alert thresholds for the selected node. |
| Container Alerts | Per node | Crash, OOM, and healthcheck alert behavior for containers on the selected node. |
| Docker & Storage | Per node | Reclaimable-space alerts and image cleanup after updates on the selected node. |
| Channels | Per node | Discord, Slack, Webhook, and Apprise destinations that fire from whichever node detects the event. |
| Image update checks | Per node | How often the selected node polls registries for available image updates. |
| Labels | Per node | Stack and container label palettes for the selected node. |
| Data Retention | Per node | How long the selected node keeps container metrics, notification logs, scan history, and audit entries. |
| Developer Diagnostics | Per node | Developer mode for real-time metrics streams and verbose debug diagnostics on the selected node. |
| Support | Same everywhere | Diagnostics bundle, docs links, and contact channels; identical regardless of the active node. |
| About | Same everywhere | Build metadata, release notes, and licence attributions; identical regardless of the active node. |
Panels that manage control-plane concerns (Account, Admiral Account, Users, SSO, API Tokens, Nodes, Registries, Recovery Vault, Routing, Mute Rules, Webhooks, Recovery) are hub-only and are hidden when a remote node is active.
## License enforcement across nodes
When the control instance has an Admiral license, all remote nodes inherit that license tier for proxied requests. You do not activate a license on each remote node separately.
### How it works
The control instance asserts its license tier on every proxied request. Remote nodes trust the assertion because it arrives alongside the valid bearer token you configured when adding the node.
This means:
- **Admiral control plane → remote nodes**: Admiral features work on every remote node, governed by the control instance's license.
- **Community control plane → remote nodes**: Admiral features are blocked on remote nodes, even if a remote node has its own Admiral license. The control instance's tier is authoritative for proxied requests.
- **Direct access to a node**: If you load a remote Sencho instance in your browser directly (not through the control plane), it uses its own local license tier.
<Note>
Remote nodes do not need their own license keys. A single license on the control instance covers every node managed through it.
</Note>
## Editing and deleting nodes
Click the pencil icon on any row to edit its name, URL, token, or compose directory. The API Token field opens blank for security: leave it blank to keep the current token, or paste a new one to rotate it. For a Pilot Agent row, the Edit modal also surfaces the **Regenerate enrollment token** card described earlier.
Click the trash icon to remove a remote node. The local row hides this icon when it is the only local node. When multiple local nodes exist from an older version, the extra rows expose Delete so you can clean them up. Deleting a local node removes its schedules, labels, dossiers, findings, and other node-scoped data; containers and compose files on the host are not affected.
## Security
### Token security
Bearer tokens grant full control over the remote Sencho instance. Treat them like passwords:
- **Rotate immediately** if a token is compromised: open the remote instance's **Settings → Infrastructure → Nodes** and click **Generate Token** to mint a new one. The previous token is invalidated instantly.
- Tokens are **encrypted at rest** in the local SQLite database.
- Tokens cannot be used to open interactive terminals (Host Console or container exec). Interactive shell access requires a signed-in browser session. From the control plane you can open Host Console on a compatible Distributed API Proxy remote; Pilot Agent remotes do not offer Host Console.
### Transport encryption
Sencho delegates transport encryption to your infrastructure rather than implementing TLS at the application layer. This is the same approach used by other self-hosted tools; it avoids certificate management burden while letting you use the encryption layer that best fits your environment.
<Warning>
**Never send bearer tokens over plain HTTP across the public internet.** A token intercepted in transit grants full control of the remote instance. Use one of the approaches below when nodes communicate over untrusted networks.
</Warning>
There are three recommended approaches.
#### Private network (LAN or VPC)
If all your Sencho instances are on the same local network, VPC, or subnet, HTTP is fine. The token never leaves the private network, so there is no interception risk.
```
http://192.168.1.50:1852 ← safe on a private LAN
http://10.0.1.20:1852 ← safe inside a VPC
```
#### VPN tunnel (WireGuard, Tailscale)
<Tip>
This is the simplest approach for connecting nodes across the internet. A VPN encrypts all traffic between your servers at the network layer, with no certificate management required.
</Tip>
With a mesh VPN like [Tailscale](https://tailscale.com) or [WireGuard](https://www.wireguard.com/), each server gets a private IP on the VPN. Use those IPs as your Sencho API URLs:
```
http://100.64.0.2:1852 ← Tailscale IP, encrypted by the VPN tunnel
```
All traffic between nodes is encrypted by the VPN. Sencho does not need to do anything additional.
Remote node URLs may use private LAN, VPC, or VPN addresses. Sencho refuses targets that resolve to loopback, link-local, multicast, or selected special-use addresses.
#### Reverse proxy (Caddy, Nginx, Traefik)
If you prefer TLS termination at each node, place a reverse proxy in front of each Sencho instance. [Caddy](https://caddyserver.com/) is the simplest option; it auto-provisions HTTPS certificates from Let's Encrypt with zero configuration:
<CodeGroup>
```text Caddyfile
sencho.example.com {
reverse_proxy localhost:1852
}
```
```nginx nginx.conf
server {
listen 443 ssl;
server_name sencho.example.com;
ssl_certificate /etc/letsencrypt/live/sencho.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/sencho.example.com/privkey.pem;
location / {
proxy_pass http://localhost:1852;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
```
</CodeGroup>
Then use the HTTPS URL when adding the remote node:
```
https://sencho.example.com ← TLS terminated by Caddy/Nginx
```
### Why Sencho doesn't implement application-layer TLS
Some tools auto-generate self-signed TLS certificates between their server and agents. In practice that provides minimal security benefit because the certificates are self-signed with no verification, meaning they do not protect against man-in-the-middle attacks.
Sencho takes the opposite approach: infrastructure-level encryption (VPN, reverse proxy, or private networking) is more robust, easier to manage, and does not impose certificate rotation burden on you. This is the same model used by other self-hosted orchestrators.
## Troubleshooting
<AccordionGroup>
<Accordion title="A remote node shows Offline">
Click **Test connection** on the row to re-probe and read the toast. The most common causes are: a wrong URL (missing scheme, missing port, trailing slash typo), a token that was rotated on the remote (any new **Generate Token** invalidates the previous one, so re-issue and update the saved row), and a firewall or reverse proxy that is not forwarding to port 1852 on the remote. Distributed API Proxy mode requires the control instance to reach the remote on the URL you saved; Pilot Agent mode requires the remote to reach the control instance on outbound HTTPS.
</Accordion>
<Accordion title="A pilot agent stays on `tunnel (waiting)`">
The agent has not dialled home yet. Three things can cause this. First, the enrollment token expired (15 minutes from generation): open **Edit node**, click **Regenerate enrollment token**, save the new `compose.yaml`, and run `docker compose up -d` again on the remote host. Second, the `sencho-agent` container exited or never started; `docker ps -a` on the remote will show the exit code. Third, outbound HTTPS to the control instance is blocked, so check the remote's egress firewall or proxy.
</Accordion>
<Accordion title="Test Connection fails with 401 Unauthorized">
The bearer token saved for the row no longer matches what the remote will accept. This usually means somebody clicked **Generate Token** on the remote (which invalidates the previous token) or the row was saved with a typo. Generate a fresh token on the remote, click the pencil icon on the row in the control instance, paste the new token into the API Token field, and save.
</Accordion>
<Accordion title="A paid feature is blocked when I switch to a remote node">
The control instance's license tier is authoritative for proxied requests, so a Community control plane gates Admiral features on every remote, even if the remote itself has its own Admiral license. Activate an Admiral license on the control instance to lift the gate fleet-wide. The reverse case (Admiral control plane, Community remote) works automatically because the control plane's tier is what the remote trusts.
</Accordion>
<Accordion title="Settings panels disappear when I switch to a remote">
That is intentional. Account, Admiral Account, Users, SSO, API Tokens, Nodes, Registries, Recovery Vault, Routing, Mute Rules, Webhooks, and Recovery are control-plane concerns and are hidden while a remote node is selected. Switch back to **Local** from the node switcher to manage them. The full list of which panels are per-node, per-browser, and control-plane-only lives in the [What Settings apply per node](#what-settings-apply-per-node) table above.
</Accordion>
</AccordionGroup>
Reference in New Issue View Git Blame Copy Permalink