209 Commits

Author SHA256 Message Date
Alphaeus Mote dfd0cf36bf Merge branch 'feat/snippets' [skip ci] 2026-11-18 17:46:57 -05:00
Alphaeus Mote 4f689ba4be feat(scripts): Snippets library for the editors and the remote shell [skip ci]
A shared library of code to insert, under Automation > Snippets. 55 built-in
snippets ship with the server (PowerShell for Windows admins, Bash for Linux,
and ExecuteR's own: exit 3010/194, $Secrets, $Files, $ExecuteRJobDir,
$Trigger); operators add their own. Built-ins come from code with fixed ids:
they can be read and copied, never changed or deleted.

- /api/snippets CRUD plus /api/snippets/{id}/copy: Scripts View to read,
  Manage Scripts to change, audited under Scripts.
- Script and function editors: snippets for the editor's language appear in
  completions (CompletionItemKind.Snippet) and in a searchable Snippets picker
  that inserts at the cursor with live tab stops.
- Remote shell: the Snippets picker types one at the prompt, tab stops filled
  with their defaults, no Enter.
- Tab stops are ${1:default}, ${1}, ${1|a,b|} and ${0}; everything else is
  code as written (Bash's ${1:-x} included), converted to Monaco's syntax at
  insertion.

Migration for all three providers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 17:42:41 -05:00
Alphaeus Mote 244032f7a9 feat(devices): Restart and Shut Down for devices and groups [skip ci]
Restart… and Shut Down… in the device, selection and group menus start a
run of a built-in script (never stored) with Restart After Run on, so the
delay, the warning, cancel-while-scheduled, AllowRestart and the run's
targets, exclusions, Only Devices Online Now (on by default) and stagger
spread all come from the existing pipeline. POST /api/power, permission
devices.power (Admin, Operator, Help Desk).

Install Pending Updates (Windows, off by default) passes
SHUTDOWN_INSTALL_UPDATES only when asked; off, updates waiting for
"Update and restart" stay waiting. Shutting down is SHUTDOWN_POWEROFF on
Windows (wpeutil shutdown on PE) and logind PowerOff / ScheduleShutdown
("poweroff") / poweroff.target on Linux. Console actions give Windows a
planned Other: Maintenance reason. Shut down and updates need agent
1.14.0: older agents get their job failed instead of restarting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 17:24:11 -05:00
Alphaeus Mote e1fcdf4cea feat(scripts): Run As Each Signed-In User [skip ci]
The script editor's Run As is now a choice: Agent Account, Signed-In User, or
Each Signed-In User. Each runs the script once per signed-in user, in turn,
under one timeout; any failure fails the job and names who. Nobody signed in
runs nothing and succeeds. The server refuses it for agents before 1.14.0.
Migration for all three providers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 12:28:54 -05:00
Alphaeus Mote b05837d51f feat(shell): choose which signed-in user a Run As shell opens as [skip ci]
The agent lists who is signed in with each shell state; the console's Run As
menu offers each name, and the agent opens as that user only while they are
signed in. Several users with nobody at the console is no longer a dead end.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 12:04:50 -05:00
Alphaeus Mote 3832790484 Merge branch 'feat/install-repair' [skip ci] 2026-11-18 09:29:49 -05:00
Alphaeus Mote 21e9bad12f feat(groups): script groups, joined and left by what a script reports [skip ci]
- Every script gets built-ins: Set-ExecuteRGroupMembership (member when the
  value is true, above 0, a non-empty collection or yes-like text; pipe results
  in), Add-/Remove-ExecuteRGroupMembership, and executer_group in Bash. They
  need no credentials: the agent takes their lines out of the output and sends
  the answers with the job's final state.
- The server adds or removes the device in that script group, creating it (top
  level, kind Script) the first time a device joins. Each run decides again.
- Only script groups change: a curated group by that name is ignored, so a
  device can never add itself to one. Last answer per group wins; 25 a job.
- Console: Script kind in the group editor, tree and lists (reported devices,
  no hand-listing), editor completions and snippets for the built-ins.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 09:29:48 -05:00
Alphaeus Mote 0b85bd5e8a feat(install): idempotent install scripts that upgrade or repair a failed agent [skip ci]
Every served install script (Windows/WinPE PowerShell 5.1, Linux and macOS
bash) now looks at the device before acting: a healthy, current agent
enrolled with this server is left alone (Already Installed), an older one is
upgraded in place with its enrollment kept, a failed one (service missing,
disabled, stopped or crash-looping, watchdog down, binary missing or broken,
Windows Installer registration or cache gone) is restarted through the
agent's own initialize and, if that does not help, reinstalled (MSI repair
REINSTALL=ALL REINSTALLMODE=vamus, cache restored from the kept package,
uninstall+install with ProgramData and the registry key set aside as a last
resort; Linux/macOS move files in one by one and re-register the unit or
daemon). The bootstrap token is used only when there is no usable credential
or the agent points at another server, never printed, and masked out of
failure messages. One setup at a time, leftovers of interrupted runs cleared.
Each run ends with one outcome line and exits 1 on failure.

The agent's status verb now prints a JSON line with version, server URL,
credential state (valid/missing/unreadable/other-device) and whether a
bootstrap token is pending, which the scripts read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 09:20:40 -05:00
Alphaeus Mote 505e9849eb Merge branch 'feat/log-format' [skip ci] 2026-11-18 09:03:20 -05:00
Alphaeus Mote 3e994102ed feat(agent): shells and scripts can run as the signed-in user [skip ci]
- Remote Shell gets Run As: Agent Account (default) or Signed-In User. Windows
  starts the shell with the user's own token, environment and profile folder on
  their desktop, headless; Linux and macOS start their login shell through su -l.
- Scripts get Run As Signed-In User. Every engine then runs out of process as the
  user (PowerShell 7 in the agent's built-in host, 5.1, Bash; su on unix), with
  no window, and the user is given that job folder only.
- The user is the console's, else the only one signed in; several and nobody at
  the console is refused rather than guessed.
- The server never sends either to agents older than 1.13.0 (they would ignore it
  and run as the agent's account): such jobs are created failed, the shell is
  refused. Migration for all three providers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 08:59:56 -05:00
Alphaeus Mote 28b2a53c29 feat(logging): one log line format everywhere, repeated warnings bounded [skip ci]
Every text line ExecuteR writes now reads
[2026-11-16T17:53:40.653Z] - [Category] - [LEVEL] - message key=value:
UTC time to the millisecond, a short category (the logging type without its
namespace, or Agent/Server), TRACE..FATAL, the rendered message and the
properties it did not use as key=value (correlation fields kept, enrichment
noise left out). Serilog writes it through LogLineFormatter (console, file
and the live views' message text); LOG_JSON keeps compact JSON. The watchdog
and its MSI/removal runners write the same format, and the Rust screen host
too (a tracing layer), with the agent taking each host line apart and logging
it once at the host's level and category, without a second timestamp.

LogLineParser reads the format back and the old ones still in device files
(services' [yyyy-MM-dd HH:mm:ss.fff LVL] Source: message, the watchdog's
...Z [LVL] message, tracing's default); the watchdog's live view uses it.

A warning, error or fatal that repeats (same level, SourceContext and
template) is written once, then counted for 60 s; the window's end writes one
line with repeated=N window=60s, a quiet window forgets it, at most 1,000 keys
are tracked and disposing writes open counts. The screen host does the same.

The console's log viewer shows the new level names and categories and copies
lines in the file format.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 08:55:11 -05:00
Alphaeus Mote d5274dfbea test(backend): presigned link expiry tolerates slow runners [skip ci]
CI #181 measured 298 s left on a 300 s link; allow 10 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 14:57:02 -05:00
Alphaeus Mote b4202e9794 feat(enrollment): stable agent download links, optional public downloads, new Add Device dialog
CI / build-test (push) Failing after 11m51s
CI / images (push) Has been skipped
Release / release (push) Successful in 24m35s
Release / images (push) Successful in 5m13s
- Stable links per package (/api/downloads/agent/Agent.msi, Agent.zip,
  Agent-<os>-<arch>.zip) always serve the newest build (GET and HEAD,
  X-Agent-Version); /api/downloads/agent lists version, SHA-256, size and
  link as JSON. With offload on they redirect to a fresh presigned link
  (S3 presigned links cannot be permanent).
- Public Downloads setting (off by default): on, anyone; off, install
  scripts (their package URLs carry the bootstrap token) and signed-in
  users with devices.view.
- No MSI/ZIP choice: the Windows script installs the MSI, or the ZIP in
  Windows PE. Add Device is redesigned: platform cards, token step with
  summary chips, install step, and per-platform downloads.
- Settings > Enrollment Tokens gains an Agent Downloads card.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 21:25:58 -05:00
Alphaeus Mote 890bab034b fix: devices follow the real server address; static screen host
CI / build-test (push) Successful in 38m1s
CI / images (push) Has been skipped
Release / release (push) Successful in 19m58s
Release / images (push) Successful in 12m49s
- PUBLIC_SERVER_URL and TRUSTED_HOSTS left at .env.example's sample
  (control.example.com, any RFC 2606 name) are ignored with a startup
  warning: links follow the forwarded headers and devices download from
  the server URL they already use. .env.example ships them empty and
  deploy.sh clears sample values from an existing .env.
- The screen host is built from its own folder so .cargo/config.toml's
  +crt-static applies; the shipped exe imported VCRUNTIME140.dll and died
  without output on devices without the Visual C++ runtime ("the screen
  host gave no answer"). The build now refuses an exe with CRT imports.
- A failed screen host probe is retried with backoff (30 s up to 10 min)
  and the inventory is republished once it answers; the warning and the
  reason carry the exit code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 20:02:03 -05:00
Alphaeus Mote af7fd25a69 feat(webui): Folder field in the script and sequence editors
CI / build-test (push) Failing after 7m22s
CI / images (push) Has been skipped
Release / release (push) Successful in 20m28s
Release / images (push) Successful in 4m34s
Pick a folder or type a path (a new one is created on save); new items
start in the folder they were created from and copies stay with the
draft's folder. A folder changed from the editor is audited as a move
(moved / sequence-moved), like a move from the list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 18:02:56 -05:00
Alphaeus Mote cd0378a2fc feat(sequences): folders for sequences in the shared library-folder system [skip ci]
- Sequences carry a folder ("/" or "/a/b") with the scripts' rules:
  normalized, compared ignoring case through an upper-cased key, adopting
  the casing of folders a path passes through
- "sequences" is a library of the shared folder service and endpoints:
  GET/POST/DELETE /api/sequences/folders, POST /api/sequences/folders/move
  (rename or move with everything in it), POST /api/sequences/move (one or
  many); delete only while empty
- Saves take an optional folder: new sequences land in it, updates without
  one keep the sequence where it is; bad paths are refused as for scripts
- Scripts View to read, Manage Scripts to change; audited in the scripts
  category as sequence-folder-created/moved/deleted and sequence-moved
- SequenceFolders migration for SQLite, PostgreSQL and SQL Server: existing
  sequences go to "/"
- Tests: folder CRUD, rename and move, bulk move, validation, Viewer 403,
  migration default, and a run under way unaffected by a move

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 15:21:39 -05:00
Alphaeus Mote 88aed64038 Merge main (Script Sequences, agent-held restart timer) into the storm and upgrade fan-out work [skip ci] 2026-11-07 11:04:34 -05:00
Alphaeus Mote 23ddf5c665 test(loadbench): the connect storm's settings answers and ready events, and an upgrade fan-out [skip ci]
- Connect storm: settings answers counted by first try and retries, with
  their reply times (p50, p95, max); the phase also waits for every
  device's ready event to be stored. --settings-retry retries a missed
  pull as the agent does (default: one try, what reaches a device at once).
- --upgrade: an MSI made up for the catalog, every device queued for it,
  the scheduler running as with SCHEDULE_WORKER on and StartWithinMinutes
  from --start-within (default 1); the simulated agents count upgrade
  commands and a second one for the same version, and each pass's sends are
  timed from the pass's start to the last publish returned.
- --burst false skips the job burst.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 11:02:40 -05:00
Alphaeus Mote 7bd3cdfd14 fix(agent): a settings pull without an answer is asked again with backoff and jitter [skip ci]
The pull on connect tried four times (now, then after 15 s, 60 s and 5 min,
the same for every device) and then left the device on what it held until
the next reconnect. A fleet whose pulls timed out together asked again
together.

It now asks again after a pause that doubles from 10 s up to 10 minutes, a
random point in the upper half of each, for as long as the session lasts;
it stops once answered, or when a pushed set newer than the one it started
with has arrived meanwhile. After four unanswered tries it says so once in
the log and goes on asking. The request and its sealing are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 10:57:19 -05:00
Alphaeus Mote bfbd9b1d4f fix(messaging): a fleet reconnecting at once gets every settings answer [skip ci]
A 10,000-device connect storm answered 1,526 settings pulls on SQLite and
4,365 on PostgreSQL; the rest came after the agent's 20 s. Measured (a
probe in the lanes): every pull queued in the 8 agent-event lanes behind
every device's ready event, and each event's replay id was one database
insert (19 to 36 ms under the storm's write contention) before anything
else; the lanes' wait reached 61 s on SQLite (41 s on PostgreSQL), 8,312
of 9,845 SQLite answers left after the agent had stopped waiting, and 155
more failed to publish. Nothing was dropped by a subscription.

- Requests on the event subject (anything with an inbox: settings pulls,
  CLI runs, uploads, restarts) leave the lanes for a request path of their
  own: up to 256 at a time opened side by side, their ids remembered for
  every replica in one insert (ServerReceipt.EventBatch, RememberEventsAsync;
  a replay is refused as before), settings pulls answered together.
- AgentSettingsAnswers: the enabled profiles (and every device's groups when
  a profile names groups) kept as a snapshot until AgentSettingsSignal fires
  (profile, group, member, tag changes) or 30 s; a batch's devices and
  stored sets in one query each. A stored set that is what the snapshot
  decides, not pending and not behind the agent, is the answer with no
  write; everything else goes through ReconcileDeviceAsync, read afresh.
- The lanes take what has queued (up to 64) and write it together:
  inventory reports in one transaction (one at a time when that fails),
  ready and logon events' device rows in one, event ids in one insert;
  order per device is kept. These writes, settings rows and replay ids
  take SqliteWriteGate on SQLite.
- Presence, inventory, events and requests read the keys of devices not
  cached a query per 500 (DeviceKeyCache.LoadAsync) and open side by side.
- The server's ECDH key is held in up to 8 instances: one instance under
  one lock made every first message of every device (after a server start)
  wait its turn, which starved the thread pool once opening ran in
  parallel (833 answers in that run).
- Subscriptions hold 65,536 messages (NATS.Net's default is 16,384); what
  a full one drops is counted by kind and logged every 10 s at most. An
  answer sent after the agent's 20 s is counted and logged too.
- An upgrade's key check reads the keys the command was just sealed with.

After: 10,000/10,000 first-try answers on SQLite (p50 0.5 s, p95 1.8 s,
max 2.1 s) and on PostgreSQL (p50 0.1 s, p95 0.3 s, max 0.9 s); every
inventory, presence and ready event stored by the end of the 20 s ramp.

Replies are sealed and signed for the asking device and bound to its inbox
as before; pulls and other requests are still replay-checked in the
database for every replica.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 10:56:23 -05:00
Alphaeus Mote 67bee0fffa perf(upgrades): a paced batch is sent side by side, its rows marked in batches [skip ci]
The scheduler sent a batch one device at a time, each with its own save
over every open upgrade row in memory, a fresh key read and a second
device read: a 2,500-device batch (10,000 devices, StartWithinMinutes 1)
took 507 s on SQLite and 556 s on PostgreSQL, so StartWithinMinutes meant
nothing past a few hundred devices.

- Rows go Queued -> Sent 500 per statement, only from Queued and only when
  due, with a new concurrency stamp; the pass keeps the rows it moved and
  lets go of the others (cancelled or taken meanwhile), so no row is handed
  to JetStream twice, also with two schedulers at once.
- Commands are sealed and published JOB_DISPATCH_PARALLELISM at a time (the
  job runner's setting, default 16); packages and rollback MSIs are read
  once per batch. A send that fails puts that device alone back in the
  queue (one statement for all of them), tried by a later pass; the others
  go on.
- The upgrade id is the command's JetStream message id, so a resend of an
  unconfirmed publish is dropped by the stream (and by the agent, which
  dedupes on the upgrade id).
- Pacing is unchanged: batch sizes still come from StartWithinMinutes and
  the at-least-ten rule, counted from when each row was first ready.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 10:55:02 -05:00
Alphaeus Mote 8b4d64289e docs(restart): Restart After Run with the agent's delay, the warning and Cancel Restart [skip ci]
A sequence step's test carries the warning to the agent and waits through Scheduled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 10:52:55 -05:00
Alphaeus Mote 6ff8665ab3 feat(restart): migrations for the warning and due time, sequences carry the warning, tests [skip ci]
RestartWarningSeconds on Scripts (0 for existing ones) and RestartDueUtc on Jobs, for SQLite,
PostgreSQL and SQL Server. A sequence step's Restart On Completion carries the warning to the
agent, and a Scheduled step restart waits like Requested and Restarting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 10:25:02 -05:00
Alphaeus Mote 408a960f42 feat(sequences): server-orchestrated script sequences that continue across restarts [skip ci]
A sequence is an ordered list of steps, each an existing script with its own
arguments, extra Functions, own S3 files, condition (previous outcome,
platform, full OS / Windows PE, groups), Restart On Completion (one
RestartSettings value over the script's own), Continue On Error and
Success/Warning/Failure exit-code lists (codes, ranges, hex, *; defaults
0 / 3010, 1641 / *; validated server side, no overlaps).

A start records the sequence run, one ordinary run per step with that
step's sealed snapshot, and each device's run with its first step's job.
Every step is a normal job sent by the JobDispatcher, so history, output,
run pages, dashboards and audit are unchanged. A SequenceWorker (on the
SCHEDULE_WORKER node, every 2 s) classifies a finished step, waits while its
restart is under way (Restarted, or a newer inventory boot time, which also
covers Windows PE), checks the next step's condition and queues it, with one
update conditioned on the device run's version and the next job in the same
transaction: duplicate or concurrent advances move a device once. The
batched job-state writer is untouched.

Sequences run from Run Now (POST /api/jobs with sequenceId), schedules and
event triggers (sequenceId on /api/triggers), and On Demand with the device
CLI, which returns at once for a sequence. Runs can be cancelled
(POST /api/sequence-runs/{id}/cancel); steps not yet sent are cancelled.
Jobs, runs and summaries carry a sequence step reference. Migrations for
SQLite, PostgreSQL and SQL Server.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 09:54:26 -05:00
Alphaeus Mote 4cb34ce7e5 Merge parallel dispatch and batched job-state writes [skip ci]
Batched final states keep Restart After Run (Requested/Refused) in the same conditional update, without replacing one the restart request stored; the restart request keeps the in-memory ApplyJobState. Dispatch test send delay raised so overlap holds on a loaded runner.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 07:18:28 -05:00
Alphaeus Mote 1520bafe41 perf(messaging): job states stored in batches, output keeps up with a burst [skip ci]
Job states (JetStream, sealed) are read ahead into a channel and stored up to
256 at a time, waiting 5 ms for more once one has arrived: opened and checked
side by side (arrival order kept), a replay acked and a refused one terminated
at once, the rest applied in one transaction and acked only after it commits,
then remembered (a redelivery is a replay) and relayed to browsers. When the
transaction fails nothing of it is kept and each state is stored on its own;
one that still fails is nak'd (offered again in 2 s), never acked.

Each state is one conditional update that only moves a job forward: Running
from Queued, Dispatched or Running (the first start kept), a final state from
any state that is not final, never over a final one. The check is the update's
own, so a cancel or another replica writing meanwhile cannot be overwritten.
Activity and job events go to browsers only for a state that was applied.
Run counts, last-run outcomes and the dashboards read the job rows as before.

Job output, which a burst now produces as fast as the states: lines are opened
side by side and stored by a second stage while the next are opened (up to
2,000 a transaction, through the prepared insert), and the subscription holds
131,072 lines meanwhile (NATS drops past that). Without this, a 5,000-device
burst on SQLite lost a quarter of its output lines.

Presence, output and state writes take the SQLite write gate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 06:49:02 -05:00
Alphaeus Mote 084f569c31 perf(jobs): parallel dispatch for large runs, job rows written in batches [skip ci]
A run without a stagger now writes its jobs 500 per transaction and seals and
publishes each batch's commands JOB_DISPATCH_PARALLELISM at a time (default 16,
at most 128) while the next batch is written and the last one's Dispatched
states and history pruning are stored. A batch's rows commit before any of its
commands goes out, and a batch is sent only once the one before has been, so an
agent's first state always finds its job and a device's jobs leave in order.
The job dispatcher (staggered slots, retries) sends a pass's due jobs the same
way: only what is due, so Spread Over and Devices at a Time hold as before.

- A device's failed send stays with that device: it is left queued for the
  dispatcher, and its later jobs in the batch wait behind it.
- A script command carries its job id as the JetStream message id, so a resend
  inside the stream's duplicate window (a publish that was not confirmed, a run
  stopped part way) is dropped by the stream: a job is never delivered twice.
- A run stopped part way hands what was written but not sent to the dispatcher.
- Staggered runs write their rows a batch per transaction after the snapshot.
- History pruning for a batch: one grouped count, deletes in one transaction.
- On SQLite, new rows go through one prepared INSERT (BulkInsert): SaveChanges
  cost about 0.4 ms a row there. PostgreSQL and SQL Server keep SaveChanges.
- SqliteWriteGate: the batch writers (a run's rows and states, and the
  messaging workers' in the next commit) take SQLite's one writer in turn,
  awaited, instead of meeting at its lock, where Microsoft.Data.Sqlite retries
  after 150 ms sleeps. It does nothing on PostgreSQL and SQL Server.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 06:49:02 -05:00
Alphaeus Mote f32d79fd1c Merge S3 pre-run downloads: file/folder picks, Download To root, progress and cleanup [skip ci]
# Conflicts:
#	src/Agent/PowerShell/PowerShellJobHandler.cs
#	src/WebUI/src/sections/runs/RunScriptDialog.tsx
2026-11-07 06:40:18 -05:00
Alphaeus Mote eb7d4f8c18 Merge Restart After Run and plain Automatic agent start [skip ci] 2026-11-07 06:19:38 -05:00
Alphaeus Mote ea60dc7848 feat(scripts): pick S3 files and folders, Download To root, live progress and cleanup [skip ci]
- Script editor: Add From S3 opens the S3 browser in multi-select mode
  (files and folders, search, breadcrumb); each pick gets a unique
  identifier name. Download To (default $ExecuteRJobDir\Files) with the
  variables on hover, per-binding Path inside it, Clean Up Downloads (on).
- Run Script: Review > Files > Change for This Run overrides the selection
  for one run (needs scripts.manage or files.send), sealed into the
  run's snapshot.
- Storage: FileBindingsJson becomes {root, cleanUp, bindings}; the older
  array form still reads (absolute destinations, nothing removed). No
  schema change.
- Server: only the selected keys are listed and presigned; single files
  carry their listed size; a folder over 2,000 objects is refused.
- Agent: paths expand on the device ($ExecuteRJobDir, $ExecuteRAgentDir,
  $ExecuteRAgentPath, %VAR%, $env:VAR, $VAR/${VAR} on Unix, $(...));
  undefined variables and ".." fail before the script runs. Missing
  folders are created (inheriting the parent's ACL). Four downloads at
  once, streamed, retried, checked against size and plain-MD5 ETags,
  timeout/cancel aware. Information lines and progress records while it
  runs, and a summary. $Files (FileInfo/DirectoryInfo, ordered,
  case-insensitive) and $FilesRoot in PowerShell 7 and 5.1;
  EXECUTER_FILE_<NAME> and EXECUTER_FILES_ROOT in Bash. Clean Up
  Downloads removes only what the job wrote, on success, failure,
  timeout or cancel.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 06:17:41 -05:00
Alphaeus Mote b8f4aced81 feat(agent): Restart After Run through the platform's restart API, plain Automatic start [skip ci]
Restart After Run (Off / Always / When the Script Asks, with a delay up to
60 minutes) is a script option, versioned in the run snapshot and carried
in ScriptCommandPayload like the other run options, and Run Script can
override it for one run. Asking means exit 3010 or 1641, and on linux and
macOS also 194 (what exit 3010 becomes in an 8-bit status); with the option
on, an asking exit is a success.

The agent restarts only after the server holds the job's result: the
result is kept in the local store first, then a restart request on the
device's event subject is answered once the server has stored the final
state (applied from the request if its JetStream copy lags) and every
output line (missing lines are resent). If the server is unreachable, the
kept result is sent after the restart. The job shows Restart Requested,
Restarting, then Restarted when the device's inventory reports a newer
boot time; Refused, Cancelled and Failed are reported too, and every step
is audited. Pending restarts carry an opaque continuation for script
sequences.

Mechanisms, all behind ISystemRestart: Windows InitiateShutdownW
(SHUTDOWN_RESTART | SHUTDOWN_FORCE_OTHERS, the delay as grace period,
planned Application: Installation reason) after enabling SeShutdownPrivilege;
Windows PE tries the same API, then wpeutil reboot; linux uses logind
Reboot/ScheduleShutdown over D-Bus (Tmds.DBus.Protocol), falling back to
systemd StartUnit(reboot.target); macOS reports it is unsupported.

AllowRestart (default on) is a managed agent setting with ADMX/ADML, an
MSI property and registry/agent.json support. The agent service is now
plain Automatic: install sets it, and the agent switches an earlier
Delayed Start when it starts as a service. Migration RestartAfterRun for
SQLite, PostgreSQL and SQL Server.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 06:17:37 -05:00
Alphaeus Mote b242509675 feat(deploy): bundled Postgres on the internal network by default [skip ci]
docker-compose.yml gains a postgres service (EXECUTER-DB-00001, official
image pinned by major and digest like NATS, INTERNAL network only, no host
port, pg_isready health check, shm_size, runs as PUID:PGID, data in
${STACK_BINDMOUNTROOT}/${STACK_NAME}/Postgres/Data). It is in the "postgres"
Compose profile; COMPOSE_PROFILES in .env turns it on, and the backend waits
for it healthy (depends_on required: false, so SQLite and external databases
start without it). The backend defaults to DATABASE_PROVIDER=postgres with
DATABASE_CONNECTION_STRING and DATABASE_PASSWORD from .env.
docker-compose.postgres-port.yml publishes a host port, opt-in through
POSTGRES_BIND.

deploy.sh decides the database: what .env names when the operator set it;
else SQLite when API/Data/app.db exists (written to .env, with a note on how
to move); else the bundled Postgres, its password generated into .env (and
exported), never regenerated over an existing database. COMPOSE_PROFILES
follows. The decisions are functions that scripts/tests/Test-DeployDatabase.sh
runs on fixtures without Docker. Compose 2.20.2+ is checked; a new .env is
mode 600. New-LocalSecrets.ps1 generates the password for a new .env and
keeps an existing .env's database settings on -Force.

README: deploy, secrets, Compose, database providers, Postgres backups and
pg_restore, and Moving from SQLite to Postgres (migrate-database).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 06:01:01 -05:00
Alphaeus Mote 74399e7fc6 feat(database): Postgres backups with pg_dump and a migrate-database verb [skip ci]
Settings › Database backs up Postgres: a pg_dump custom-format archive
(executer-postgres-<stamp>.dump), checked with pg_restore --list, to the
server folder and/or the S3 store with the same retention as SQLite
(pruning only this provider's backups). pg_dump comes from PG_DUMP_PATH or
PATH; without it the console says so, as before. The backend image installs
postgresql-client-18 from apt.postgresql.org and sets PG_DUMP_PATH.

migrate-database copies a SQLite database into the configured Postgres one
through EF (both at the same migration), in foreign-key order, in one
transaction, keeping keys, identity values (sequences moved past them),
concurrency stamps and values equal to a column default, and checks every
table's row count on both sides before committing. --overwrite empties a
target a server already started on; a non-empty -wal is refused without
--force.

Tests run the server on a throwaway Postgres database (insight, VACUUM
(ANALYZE), backups restored with pg_restore, retention) and the copy end to
end; both skip without PG* variables.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 05:07:40 -05:00
Alphaeus Mote dead6683e2 perf(bench): load benchmark switches to PostgreSQL and times the burst to the second [skip ci]
--provider postgres runs on a throwaway database (PG* variables), dropped afterwards. The job burst is sampled every second and reports when every job's final state was stored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-07 04:30:37 -05:00
Alphaeus Mote 8a55e05547 Merge branch 'main' into worktree-agent-a84ffb902581d66a2 2026-11-07 00:21:11 -05:00
Alphaeus Mote 6920217403 perf: load benchmark keeps the shared server's streams in memory storage
Switching the streams to file storage for a run had a concurrent test
run delete them under the benchmark (its agents then got no commands).
The benchmark now keeps memory-storage streams, as the integration tests
do, and makes the backend's job event consumer itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 23:09:38 -05:00
Alphaeus Mote 94bccc1235 Merge main; sealed tests keep the shared NATS in memory storage
Merges local main (its job pipeline test starts the command stream over
in memory storage when the server holds an incompatible one; taken as
it is). The sealed tests' fixture now makes the command and job event
streams itself in memory storage, as the pipeline test does, with the
backend's job event consumer, starting a stream over when another run
left it with settings that cannot be updated, and again before each test
agent (another run may have reset them). The backend's own file-storage
provisioning then only logs that it cannot change them. Each test agent
deletes its consumer when it stops.

The load benchmark switches the streams to file storage for a run (as in
production) and its cleanup, run after each run, puts memory-storage
streams back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 22:53:07 -05:00
Alphaeus Mote 844c044d08 test(screen): wait for the recorded session outcome and audit rows
CI / build-test (push) Failing after 11m47s
CI / images (push) Has been skipped
Release / release (push) Failing after 1m24s
Release / images (push) Has been skipped
The socket learns an outcome a moment before the server records it; under
CI load the tests read the session too early.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 22:52:18 -05:00
Alphaeus Mote c277010007 test(integration): start the command stream over when a test server holds an incompatible one
CI / build-test (push) Failing after 6m58s
CI / images (push) Has been skipped
Release / release (push) Failing after 1m21s
Release / images (push) Has been skipped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 22:39:36 -05:00
Alphaeus Mote b04bf93a70 perf: load benchmark for server sizing (tests/LoadBench)
A console program, not a test project and not run by CI. "run" hosts the
real backend (its messaging workers, job runner, message security and
enrollment, SQLite) against the NATS server at NATS_URL, enrolls N
devices through the real bootstrap endpoint (sealed or v1), and starts a
child process of simulated agents: each with the agent's own message
security and publisher, multiplexed over a few NATS connections, each
pulling its own durable command consumer. It measures a connect storm
(ready, settings pull, inventory, presence from every device within
20 s), steady state (presence every 30 s, inventory every 15 min) and a
job burst (a script to every device through the job runner, ~20 output
lines and two states each): CPU and memory of the server, the agents and
nats-server, messages per second, drops, and how long each takes.
"micro" times sealing and opening per message, and the job runner's
fan-out per device; "cleanup" removes consumers a stopped run left.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 22:33:52 -05:00
Alphaeus Mote 19f6b48a79 fix(files): a cancelled transfer's slot is not refused as busy; close after answers
ACancelledDownloadReleasesTheDevice failed most runs on unmodified main
over a real NATS server (5 of 6 here), two ways:

- "busy": a download stopped from the server frees its slot only once the
  agent hears the abort, and a request arriving just before that was
  refused at once. A request now waits up to 3 s for a slot before it is
  answered busy.
- the listener's session ended while the requests it had handed to the
  thread pool were still writing frames, and the connection was closed
  under them. It now waits (up to 10 s) for them to stop first.

Left: NATS.Net now and then times out writing an unsubscribe on close
after a large publish was cancelled part way; the test's teardown
tolerates that one exception (the connection closes anyway).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 22:33:10 -05:00
Alphaeus Mote bb707b9b33 test: hold on the CI runner (SYSTEM account, path case, presign second)
CI / build-test (push) Failing after 9m36s
CI / images (push) Has been skipped
Release / release (push) Failing after 1m25s
Release / images (push) Has been skipped
The runner runs the suite as SYSTEM: a pipe it owns is trusted by design,
and its temp path's case varies. A presigned link signed a moment after its
expiry was set reads one second shorter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 22:26:39 -05:00
Alphaeus Mote c1a5200ca3 test(security): sealed messaging end to end over a real NATS server
The real backend (messaging workers, command dispatcher, message security,
enrollment) hosted against the live NATS with its own SQLite database, and
agents enrolled through the real bootstrap endpoint running the agent's own
command processor, publisher, file listener and request link. A raw client
watches the wire and replays, tampers with and injects messages:

- a queued script sealed and signed by the server runs; its output and
  states come back sealed and signed by the device and are stored
- a replayed command runs once, an expired one and four tampered copies are
  refused (the untampered copy then runs)
- tampered, forged, foreign and replayed job states and output are refused
  by the server
- the CLI run and the Agent Settings pull are sealed both ways; a captured
  request sent again gets no answer
- a sealed file transfer round-trips intact while duplicated, reordered and
  forged frames are injected both ways
- a key rotation reaches the agent as a queued command; both keys work
- Require Sealed Agents refuses a v1 agent's presence, job states and
  requests, and sends it nothing
- a script at the library's limit: sent sealed, or refused with the sizes
  when the NATS server is at its 1 MB default

The unsealed pipeline test now makes the command stream as the backend does
(file storage), so both can share one server. All skip without NATS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 20:55:39 -05:00
Alphaeus Mote 779958c5c1 Merge Agent Settings profiles [skip ci]
# Conflicts:
#	src/Agent/Service/AgentSession.cs
#	src/Backend/Messaging/MessagingWorkers.cs
#	src/Shared.Contracts/Messages/MessageTypes.cs
2026-11-06 20:03:35 -05:00
Alphaeus Mote 69666d5c4e feat(settings): Agent Settings profiles on the server, pushed on every change
Profiles (name, priority, enabled, the settings they set, Include/Exclude
targets as on a schedule) under /api/agent-profiles with settings.manage,
validated against the catalog (ranges, JobRoot path rules), device scope
enforced through the target checks now shared with schedules (TargetScope),
every change audited. Per setting the lowest priority number wins.

Each device's resolved set is kept with a version that only grows; a profile
edit, a group or tag change, an enrollment or an inventory report recomputes
it and pushes a changed set as a sealed durable command, a five-minute sweep
retries the rest, and the agent's pull on connect is answered (newer than the
version it holds). A device no profile reaches any more gets the empty set.
GET /api/devices/{id}/agent-settings shows the decision beside the agent's
report. Migrations AgentSettingsProfiles for SQLite, PostgreSQL and SQL Server.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 19:59:21 -05:00
Alphaeus Mote cbf08a192a feat(settings): agent takes Agent Settings profile values below Group Policy
Agent settings now resolve as Group Policy > Agent Settings profile > device
registry (agent.json on linux/macOS) > default, per setting, for the settings
profiles manage (AgentSettingCatalog): the access switches, Remote View, the
inventory interval, log level and job folder root. ServerURL, the bootstrap
token, TlsSkipVerify, ProxyUrl, the reconnect delays and MessageClockSkew are
never taken from a profile.

The agent pulls its set on connect (agent.settings.request, sealed) and takes
pushes (command.agent.settings, durable), ignores a set older than the one it
holds or for another client id, keeps the last set under Runtime\ManagedSettings
so a restart without the server still applies it, applies it to the running
session (permission checks, Remote View policy, inventory interval, closing a
remote shell switched off) and reports every setting's value, source and
pending restart with its inventory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 19:58:50 -05:00
Alphaeus Mote 5775f290fb feat(s3): device uploads under uploads/devices/<hostname>/
- DeviceUploadService answers the agent's upload steps: limits and the
  on/off setting checked server-side, keys built from the device's identity
  and cleaned (no '..', '.', control characters, drives, rooted names), a
  grant sealed by the server binding device, upload and folder, presigned
  PUT / UploadPart URLs for exactly one key, multipart completion and a
  size check that removes what does not match; audited at begin and finish.
- The built-in store's gateway takes a signed PUT of one object or one part
  (nothing else of the store's API).
- Settings > Storage > Device Uploads (on by default, 1,000 files, 5 GB per
  file, 20 GB per upload); console Upload to S3 on the Files and Logs tabs,
  Uploads link to the device's folder, and the S3 page opens ?prefix=.
- README: the variables, the upload command, S3 layout, limits, settings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 19:10:01 -05:00
Alphaeus Mote b83cdc0fe4 feat(agent): upload to S3 from the device, and the agent's path for scripts
- $ExecuteRAgentDir / $ExecuteRAgentPath in both PowerShell engines (every
  mode), and EXECUTER_AGENT_DIR / EXECUTER_AGENT_PATH for PowerShell 5 and
  Bash (PowerShell 7 shares the agent's environment, so variables only),
  resolved from the running binary (links followed) for MSI, portable,
  Windows PE and Linux/macOS layouts.
- ExecuteR-Agent upload <path|mask>... [--recurse] [--flatten|--preserve]
  [--dest] [--zip [name.zip]] [--overwrite] [--quiet]: the CLI plans paths
  and masks where it runs, the service reads the files (sharing with writers
  and deleters) and sends them over the local channel's caller checks; exit
  codes 0 all, 1 refused/none, 2 unreachable, 3 partial.
- The upload steps (begin, presign, complete, abort, finish) go on the same
  request path as `run` (AgentLink.RequestAsync, sealed both ways); bytes go
  straight to the store over HTTPS, whole or in parts, with retries, size
  checks and progress.
- The console's Upload to S3 (files/folders, and the agent's logs as a zip)
  takes the same path from the file listener.
- The upload runs as the agent, not under the pipe client's identification
  token, which leaked into pool threads and made file opens fail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 19:10:00 -05:00
Alphaeus Mote 58fe02e70c Merge message security: sealing, signing and replay protection [skip ci]
# Conflicts:
#	README.md
2026-11-06 18:15:10 -05:00
Alphaeus Mote f0268c899d feat(security): bound the unsealed-before-key-exchange window to the stream's age
The stream time a queued v1 message is judged by comes from the NATS server,
which is inside the threat model; once nothing from before the key exchange
can still be in the stream, unsealed messages are refused outright.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-06 18:13:41 -05:00