buckit/docs/manager/backend-design.md

Buckit Manager (bm) — Backend design

Companion to phase1-web-ui.md and ui-architecture.md. The UI prototype is fully clickable on top of an in-memory mock layer; this document describes the backend that replaces that mock, the REST surface it exposes, and how bm reuses MinIO's admin client and (selectively) mc source to keep parity with the mc CLI.

Goals

1. One binary, two surfaces. bm is a single static binary that serves both a CLI (bm cluster ls, bm cluster restart …) and a local web UI (bm web → 127.0.0.1:9443). Both call into the same internal packages.
2. Personal-tool framing. Default listener is loopback-only. No multi-tenant auth, no RBAC, no audit retention plumbing. Optional remote-access mode adds a passcode and TLS — see ui-architecture.md § Optional remote access.
3. Parity with mc for cluster operations. Every cluster Action, bulk-host action, and per-node Action the UI exposes maps to an admin API call (or SSH per-host loop) that mc operators would recognise. The CLI should accept the same mc admin service … verbs where it makes sense.
4. The UI contract drives the API. Domain types defined in web/src/mock/data.ts are the wire format. The backend doesn't get to invent new shapes.

Non-goals

• Becoming a fork of MinIO or a re-implementation of mc. bm calls out to running clusters via the admin API; it doesn't reimplement erasure coding or object storage primitives.
• Background polling. The on-demand cache loop refreshes a cluster's facts when a UI consumer asks. No timers fanning out per-cluster probes every N seconds (see ui-architecture.md).
• Multi-host coordination. The manager runs on one operator's machine at a time. If two operators each run bm, they have two independent databases.

Big picture

+-------------------------------------------------------------+
|                          bm binary                          |
+-------------------------------------------------------------+
|  cmd/bm/                                                    |
|  ├── bm-native verbs        (urfave/cli registration)       |
|  │     web, cluster, manager, migrate, rolling, node, …     |
|  └── github.com/buckit-io/bm-cli  (vendored mc fork)        |
|        cp, ls, mb, admin *, alias, share, event, ilm, …     |
+-------------------------------------------------------------+
|  internal/                                                  |
|  ├── app/      Process lifecycle, lockfile, signal handling |
|  ├── api/      chi router, REST handlers, SSE plumbing      |
|  ├── store/    bbolt wrapper + KEK-encrypted secrets bucket |
|  ├── tasks/    Operation orchestration + history finalize   |
|  ├── ssh/      Per-cluster SSH client cache + run helpers   |
|  ├── admin/    madmin-go wrapper exposing the calls we use  |
|  ├── deploy/   New-cluster install + MinIO→Buckit cutover   |
|  ├── cluster/  Cluster repo (load, save, refresh, health)   |
|  ├── alias/    Bridge: store → ~/.config/bm/config.json     |
|  ├── auth/     Optional remote-access passcode + TLS        |
|  ├── config/   $XDG_CONFIG_HOME/bm/, KEK material, perms    |
|  └── health/   On-demand cache loop, admin-info probes      |
+-------------------------------------------------------------+
|  forked deps under buckit-io/* (own repos, go.mod-pinned)   |
|  ├── buckit-io/bm-cli       (forked mc — Buckit CLI)        |
|  ├── buckit-io/madmin-go    (forked admin API SDK)          |
|  ├── buckit-io/minio-go     (forked S3 client SDK)          |
|  ├── buckit-io/pkg          (forked shared utilities)       |
|  ├── buckit-io/cli          (forked urfave/cli)             |
|  └── buckit-io/selfupdate   (forked, points at Buckit rel.) |
+-------------------------------------------------------------+
|  web/dist/   (embed.FS in the release binary)               |
+-------------------------------------------------------------+

Two CLI surfaces share one binary:

• bm-native commands (web, cluster ls, cluster import, cluster migrate, cluster deploy, manager *) are written fresh in cmd/bm/ using the same urfave/cli framework the vendored CLI uses.
• Buckit CLI commands (cp, ls, mb, admin service restart, admin trace, alias, share, event, ilm, replicate, …) come from buckit-io/bm-cli — a fork of minio/mc we own outright. Full feature parity with mc from day one.

The CLI calls straight into internal packages — no HTTP hop. bm web runs the same internal packages behind an HTTP router. The two surfaces share state via bbolt; concurrent access is serialised by bbolt's short-lived write locks.

The HTTP server's admin-API path (POST /operations for restart/stop/freeze/heal) deliberately does not go through the forked CLI commands. CLI commands call os.Exit, write to os.Stdout via a CLI printer, and look up clusters via the alias config — none of which fits an HTTP handler. The handlers call internal/admin/ (the madmin-go wrapper) directly. CLI and HTTP share the same Go SDK underneath; they're separate consumers of it.

Persistence layout

bbolt file at ${XDG_CONFIG_HOME:-~/.config}/bm/bm.db, mode 0600. One file per operator. Buckets:

Bucket	Key	Value	Notes
clusters	<clusterId>	Cluster JSON	Mirrors mock/data.ts Cluster
nodes	<clusterId>:<nodeId>	Node JSON	Includes drives, NIC, RAM, kernel
node_facts	<clusterId>:<nodeId>	last admin-info + SSH fact blob	Cache for refresh
cluster_ssh	<clusterId>	ClusterSshConfig JSON, AES-GCM encrypted	Per-host overrides included
cluster_admin	<clusterId>	{ user, password }, AES-GCM encrypted	Root creds for admin API
history	<ULID>	HistoryEntry JSON incl. result	Newest-first reads
settings	app	manager settings	Remote access state, version pin

Encryption: AES-GCM with a 32-byte data key. The key itself sits in ${XDG_CONFIG_HOME}/bm/data.key, mode 0600. First-launch generates it. Real backend later may swap to OS keychain (macOS Keychain, Windows DPAPI) but the in-tree default is the file. Pulling a row decrypts in the store/ package; callers never see ciphertext.

History bucket is bounded — tasks.Finalize triggers a sweep that deletes the oldest entries past 1000 rows (configurable). No timers; sweep happens on next write.

REST API

Base path /api/v1. Content-type application/json. Errors follow ui-architecture.md § Errors.

Sessions (optional remote access)

Method	Path	Purpose
POST	/sessions/login	Body { passcode }. Sets the session cookie.
POST	/sessions/logout	Drops the cookie.
GET	/sessions/me	Returns current user (always admin in default mode).

Loopback default skips auth entirely; the handlers no-op when auth.RemoteEnabled == false.

Clusters

Method	Path	Purpose
GET	/clusters	List clusters with Cluster.healthSummary populated.
GET	/clusters/:id	One cluster (same shape as the list row).
POST	/clusters/refresh	Synchronous re-fetch admin info + cluster-healthy probe across every cluster. Returns the updated list. Powers the Refresh button.
POST	/clusters/:id/refresh	Same, scoped to one cluster.
DELETE	/clusters/:id	Drops the cluster definition. Equivalent to the Remove cluster definition operation; data on hosts is not touched.

Cluster import

Two-step flow — discover (read-only) then commit (persist).

Method	Path	Purpose
POST	/clusters/import/discover	Body { url, username, password }. Calls /minio/admin/v3/info, returns ImportCandidate. Streams progress lines via SSE on the same response (Content-Type: text/event-stream).
POST	/clusters/import/commit	Body { candidate, chosenName }. Persists, returns { clusterId }.

Cluster deployment (new-cluster wizard)

Method	Path	Purpose
POST	/clusters/new/preflight	Body NewClusterDraft. Runs SSH probes, drive uniformity check, hostname pattern check. Returns PreflightResult[].
POST	/clusters/new/deploy	Body NewClusterDraft. Returns { taskId }. Progress via SSE on /operations/:taskId/events.

Migration (MinIO → Buckit)

Method	Path	Purpose
POST	/clusters/:id/migrate/snapshot	Captures MinIO state (buckets, users, lifecycle, …) into MinioSnapshot.
POST	/clusters/:id/migrate/preflight	SSH probe + minio detection + drive prep checks.
POST	/clusters/:id/migrate/cutover	Returns { taskId }. Install Buckit, swap systemd unit, verify per host.
POST	/clusters/:id/migrate/rollback	Rollback completed nodes to MinIO.

Nodes

Method	Path	Purpose
GET	/clusters/:id/nodes	List nodes for one cluster.
GET	/clusters/:id/nodes/:nodeId	One node detail (drives, NIC, kernel, …).

Operations (the unified dispatch path)

Every cluster Action, bulk-host action, and per-node Action goes through this surface. The frontend's dispatchOperation() mock maps to a single endpoint.

Method	Path	Purpose
POST	/operations	Body { clusterId, kind, params?, targetHostIds? }. Returns { taskId }.
GET	/operations/:taskId	Snapshot of current OperationProgress.
GET	/operations/:taskId/events	SSE stream of progress updates until terminal.
POST	/operations/:taskId/cancel	Cancel an in-flight op (orchestrated only).

kind is the OpKind union from the UI catalog (restart_cluster, rolling_restart, systemctl_restart, redeploy_software, reboot_host, …). The orchestrator selects the executor by kind.

Streaming

Method	Path	Purpose
GET	/clusters/:id/nodes/:nodeId/logs	SSE proxying admin journalctl/logs API to the browser.
GET	/clusters/:id/nodes/:nodeId/trace	SSE proxying admin trace API.

History

Method	Path	Purpose
GET	/history	List history rows. Query: ?status=&clusterId=&since=&until=.
DELETE	/history	Clear all (optional ?before=<ts>).

Settings

Method	Path	Purpose
GET	/settings	Manager preferences (remote access state, version, etc.).
PATCH	/settings	Update remote access on/off, passcode, TLS paths.
GET	/clusters/:id/ssh	ClusterSshConfig.
PUT	/clusters/:id/ssh	Update SSH config. Returns 204 on success.

Operation orchestration

The dispatch path is the heart of the backend. One pipeline handles all 21 ops the UI knows about. Pseudo-flow:

POST /operations
   ↓
1. Validate kind + params + targetHostIds.
2. Resolve cluster + admin creds + SSH creds.
3. Write a `running` history row (returns historyId).
4. Allocate a taskId.
5. Pick the executor by kind:
     - Signal (freeze/unfreeze/remove_cluster): one admin API call.
     - Admin w/ progression (restart/stop/heal): admin call + poll back
       to terminal; emit per-host or summary updates as we go.
     - Orchestrated (rolling_restart, redeploy, reboot, systemctl_*):
       per-host SSH loop with health-wait between hosts.
6. Stream OperationProgress updates to subscribers (in-memory pub/sub;
   SSE consumers attach via /operations/:taskId/events).
7. On terminal:
     a. Snapshot the OperationProgress into an OperationResult.
        Drop live-only fields (events stream, progress counters).
     b. Update the history row: status, durationSec, failureNote,
        result.
     c. Close the SSE channel and free in-memory progress state after
        a short grace period.

Cancellation: an in-flight orchestrated op checks ctx.Done() between hosts. The in-flight host finishes naturally; the loop halts before the next host. The history row is finalized as canceled.

Concurrency rules:

• At most one mutating operation per cluster at a time. The dispatcher takes a per-cluster lock from store.AcquireOpLock(clusterId).
• Refresh and read-only fetches run in parallel with operations.
• Two operations on different clusters run in parallel.

In-memory state for in-flight ops: a map[taskId]*OperationProgress guarded by a mutex. Lost on restart — any in-flight op at restart is marked failed with a "process restarted mid-op" note when the history sweep runs.

Forking the MinIO Go ecosystem

bm aims for full mc parity at the CLI — every verb an operator already knows (cp, ls, mb, admin service restart, admin trace, admin heal, event add, replicate, ilm, tag, share, …) should work in bm. Rewriting ~150 mc command files in cobra is not the path; we fork mc wholesale and own it from there.

This is a hard fork — no upstream rebase obligation. From the fork point on, Buckit's CLI evolves independently from mc. That same posture extends to the MinIO Go libraries mc depends on: if we want the freedom to change an admin API request shape or extend the S3 client, we need to own those too.

Repos to fork

License is uniformly AGPLv3 — Buckit, mc, and all of MinIO's Go libraries — so the fork itself raises no licensing concerns.

Tier 1 — fork in one coordinated pass

These are mc's load-bearing dependencies. Forking partially leaves you in go.mod replace hell (one fork still imports an upstream sibling that hasn't been forked), so do them together:

Fork target	Upstream	Why
buckit-io/bm-cli	minio/mc	The CLI itself. Vendored at the top level of bm/cmd/bm/.
buckit-io/madmin-go	minio/madmin-go/v3	Admin API client SDK. Every admin * verb uses it; the HTTP backend uses it too. Already in buckit/go.mod at v3.0.109 against upstream. Must fork to evolve the Buckit admin protocol.
buckit-io/minio-go	minio/minio-go/v7	S3 client SDK. Powers cp, ls, mirror, cat, … and anything in bm that needs to talk S3 to a Buckit cluster.
buckit-io/pkg	minio/pkg	Shared utilities: console, ellipses, env, words, sync, wildcard, … mc imports dozens of subpackages. The other Tier-1 forks also import this; forking it last would break their builds.
buckit-io/cli	minio/cli	MinIO's fork of urfave/cli v1.x. mc uses it for command registration. Small surface; fork now to control flag-parsing behaviour.

Tier 2 — fork at the same time for cohesion

Lower change frequency, but coupled enough that owning them avoids surprises:

Fork target	Upstream	Why
buckit-io/selfupdate	minio/selfupdate	Powers bm update. Hard-coded to MinIO release URLs upstream — must repoint to Buckit's release feed.
buckit-io/colorjson	minio/colorjson	--json output formatter. Small.
buckit-io/kms-go	minio/kms-go	Used by admin kms * commands. Only matters if you ship those verbs.

Tier 3 — leave as upstream deps

Small specialized libraries that almost never need to change. Keeping them upstream saves maintenance bandwidth:

minio/highwayhash, minio/sha256-simd, minio/crc64nvme, minio/filepath, minio/csvparser, minio/dnscache, minio/dperf.

Promote individual repos to Tier 2 if a specific need arises.

Repo layout decision: bm-cli is standalone

Two structural options were considered for the mc fork:

1. Vendor mc directly under buckit-io/bm/cmd/bm/mc/. Simpler dependency graph; only one binary ever ships the CLI.
2. (chosen) Standalone buckit-io/bm-cli repo, imported by bm via go.mod as a library (the upstream package main was removed at fork time; bm is the only binary). Keeps the option open of a standalone buckit-cli build in the future without forcing the repo split later.

The CLI has independent value outside bm. Standalone forces a clean public API boundary and keeps the manager-specific code (web, cluster import, manager *) out of the CLI fork.

bm-native verbs vs. forked mc verbs

Both live in cmd/bm/main.go's command registration. No top-level naming conflicts — mc's verbs (cp, ls, mb, rb, admin, alias, share, event, replicate, mirror, cat, stat, legalhold, retention, encrypt, tag, ilm, quota, tier, license, support, batch, idp, …) don't overlap with bm-native verbs (web, cluster, manager, migrate, rolling, node, history, settings).

bm-native commands use the same buckit-io/cli (urfave/cli fork) framework that the vendored mc commands use, so registration is uniform.

Cluster references — the alias bridge

mc commands take a cluster reference as a named alias (bm admin service restart prod-east expects prod-east to be listed in mc's config). bm's source of truth for clusters is bbolt, not the alias file. Bridge approach:

• internal/alias/ watches the cluster store. Every cluster create / update / delete also writes to ${XDG_CONFIG_HOME}/bm/config.json (same filename and JSON shape as ~/.mc/config.json, just a different directory).
• Forked mc commands are patched to read aliases from the bm path instead of ~/.mc/. Single-line change in the forked buckit-io/bm-cli (the alias-resolution helper).
• If an operator has an existing ~/.mc/config.json, bm doesn't touch it. The two configs stay independent. Operators who want unified aliases can cp ~/.mc/config.json ~/.config/bm/config.json once.

The alias file contains admin URL + access/secret pairs per cluster. On disk it's the operator's responsibility to mode it 0600; the bm writer enforces 0600 on every save.

The HTTP path still uses madmin-go directly

CLI commands call os.Exit, write to os.Stdout via a CLI printer, and surface errors as user prompts — none of which fits an HTTP handler. internal/admin/ is a thin Go wrapper around buckit-io/madmin-go that the HTTP backend calls directly. It:

1. Keeps a per-cluster madmin.AdminClient cached (created lazily on first call, evicted on credential rotation).
2. Exposes only the methods we use: ServiceRestart, ServiceStop, ServiceFreeze, ServiceUnfreeze, Heal, ServerInfo, AccountInfo, LogStream, Trace.
3. Applies a short default timeout (5s for one-shot calls, no timeout for streams) and surfaces errors as typed OpError values the orchestrator can map to history failureNote.

Both surfaces — CLI and HTTP — share the same forked madmin-go underneath. They're independent consumers of one Go SDK, not callers of each other.

CLI command tree

Every mc verb works in bm from day one — they come from the vendored buckit-io/bm-cli fork. bm-native verbs sit alongside them under the same top-level dispatch.

bm-native verbs

Things mc doesn't do — manager state, deployment, migration:

bm web                              Start the local web UI + API.
bm version

bm cluster ls                       List clusters this manager knows.
bm cluster info <c>                 Show health, pools, node count.
bm cluster import <url> --name <n>  Two-step import (discover + commit).
bm cluster deploy <draft.yaml>      Drive the new-cluster wizard headless.
bm cluster migrate <c>              Run the MinIO → Buckit cutover.
bm cluster rm <c>                   Drop the definition (no data touched).

bm rolling restart <c>              SSH-orchestrated rolling restart.
bm rolling upgrade <c> --version v  SSH-orchestrated rolling upgrade.
bm node restart <c> <hostnames>     systemctl restart on selected hosts.
bm node reboot <c> <hostnames>      systemctl reboot, sequential.

bm history                          List recent ops.
bm settings                         Show / edit manager settings.

These are written fresh in cmd/bm/ against the same urfave/cli framework the vendored CLI uses.

Inherited from buckit-io/bm-cli (full mc parity)

The full mc verb set, identical flag surface and output:

bm cp / mv / ls / rm / mb / rb / cat / head / tail / pipe / find / du / stat / tree / mirror / diff / sql / version / share

bm admin service restart / stop / freeze / unfreeze
bm admin heal
bm admin trace
bm admin info
bm admin config / cluster / decommission / logs / prometheus / replicate / kms / idp / accesskey / policy / user / group / bucket / tier / scanner / speedtest / lock / rebalance / top / inspect

bm alias set / list / ls / remove / rm
bm event add / list / remove
bm replicate / ilm / tag / legalhold / retention / encrypt / quota / anonymous / version / undo
bm batch / license / support / idp / tier

These come unchanged from the fork; nothing for us to maintain on the command-implementation side unless we deliberately diverge.

Only operations dispatched through the HTTP POST /operations path (i.e. the web UI's unified operation modal) write rows into the history bucket via internal/tasks/. Inherited bm-cli verbs (cp, ls, admin service restart, ...) behave exactly like mc — they print to stdout, return an exit code, and do not touch the history bucket. Operators who run a verb from a shell and want a record of it can rely on their shell history, the same way they do with mc. --json and --quiet flags inherit their mc semantics.

Security model

• Default mode. Listener 127.0.0.1:9443. No authentication on the HTTP API — the listener restricts access to the loopback interface on the operator's machine. The CLI uses local Unix domain socket (${XDG_CONFIG_HOME}/bm/bm.sock) or short-lived bbolt locks when no bm web is running.
• Remote-access mode. Operator toggles it in Settings → Remote access. The listener moves to 0.0.0.0:9443, the handlers require a passcode-signed session cookie, and TLS is mandatory (self-signed or operator-supplied cert).
• Credentials at rest. SSH creds and admin root creds are AES-GCM encrypted using a 32-byte data key stored beside the database. Real follow-up: integrate OS keychain.
• No CLI history capture. History rows only come from UI dispatch and the bm CLI's own write-side commands. We don't intercept the user's shell history or mc invocations.

SSH layer

internal/ssh/:

• Per-cluster *ssh.Client cache, keyed by clusterId. Idle timeout 5 min. Re-dialed automatically on disconnect.
• Per-host override semantics: when a HostRow carries sshOverride, the client uses those credentials for that host only (different user, different key, different password).
• A Run(ctx, host, command) (stdout, stderr, exitCode, error) helper used by every orchestrated op's executor.
• RunStream(ctx, host, command, lineCh) streams stdout/stderr lines to the caller — used by the long-running rolling-upgrade install step so logs appear live in the modal.
• No SSH agent forwarding. No pubkey distribution. bm is a personal tool; the operator owns the keys.

Health & refresh

• No background polling. Cluster health, healthSummary, lastFetchedAt, unreachableSince come from the last completed refresh.
• On-demand cache loop. The Clusters page calls POST /clusters/refresh on first load and when the operator clicks Refresh. Cluster detail calls POST /clusters/:id/refresh. Both run admin-info + cluster health probe per cluster in parallel; the response is the updated records.
• Staleness display. The UI shows "Fetched Ns ago" using lastFetchedAt — see ui-architecture.md § Staleness display.

Phasing — mapping to existing milestones

phase1-implementation.md already defines M0–M9. This doc fills in the design for those, plus adds a prerequisite fork pass before M1.

Milestone	Scope informed by this doc
M0 — Module bootstrap	Done.
M0.5 — Fork the MinIO Go ecosystem	Done. All seven forks live under buckit-io/* with rebranded module paths, all CI workflows green: cli, selfupdate, minio-go, pkg, madmin-go, colorjson, bm-cli. bm-cli is library-only (no root package main); user-facing mc/MinIO strings rebranded; upstream copyright headers preserved for AGPL attribution. bm/go.mod is currently bare — pins materialise automatically once M1 imports the forks. See § Forking the MinIO Go ecosystem.
M1 — Storage + server shell	bbolt setup; internal/{app,store,api,config,alias}/; bm web starts a chi server with shape-correct empty handlers for the UI's read paths. Plus: import buckit-io/bm-cli and wire bm-native verbs alongside; alias bridge writes ${XDG_CONFIG_HOME}/bm/config.json on cluster save. See § M1 — Storage + server shell for the punch list.
M2 — Task engine + SSE	internal/tasks/; the orchestrator pipeline; pub/sub for OperationProgress; SSE endpoint.
M3 — SSH layer + node CRUD	internal/ssh/; node bucket; per-cluster client cache.
M4 — Discovery	/clusters/import/discover + commit. madmin-go AccountInfo + ServerInfo.
M5 — Topology + preflight	New-cluster + migrate preflight checks.
M6 — New-cluster deploy	internal/deploy/ install loop; SCP + dnf + daemon-reload + systemctl.
M7 — Cluster operations	The operation catalog under internal/operations/: 3 signal ops (freeze/unfreeze/stop), 2 admin-with-progression (restart_cluster, start_heal), 4 SSH-orchestrated (start_cluster parallel, rolling_restart sequential, rolling_upgrade + redeploy_software both Buckit-only), 5 host-scoped (3 systemctl verbs + reboot + shutdown). Most ops support both Buckit and MinIO clusters; rolling_upgrade and redeploy_software are Buckit-only and reject MinIO at dispatch with engine_mismatch. start_cluster is NOT a madmin call (no ServiceStart exists) — it's parallel SSH systemctl start followed by a cluster-wide health-wait. rotate_root_creds and add_pool deferred to M7.5; remove_cluster dropped (use DELETE /clusters/:id).
M8 — MinIO migration	Done. Snapshot writer captures buckets / users / groups / canned policies / service accounts / lifecycle / notifications / per-bucket versioning + object-lock + tags into ~/.config/bm/snapshots/<clusterId>-<ts>.json mode 0600 (wire-stable across versions, soft per-field failures recorded in Warnings). Cutover executor runs the install pipeline sequentially (stopping_minio → uploading_pkg → installing → switching_unit → waiting_health → waiting_cluster → done), backing up /etc/default/minio to /etc/default/minio.bm-bak per host and waiting for cluster-wide health between hosts. Rollback reverses the unit swap on hosts where buckit.service is currently active; hosts already on MinIO are skipped cleanly. Engine flips at commit time on success and back on rollback; MigratedFrom is stamped/cleared accordingly. Post-cutover verify pass populates the wizard's audit table; failures surface as a warning on the history row, not an auto-rollback. See § M8 — MinIO migration for the punch list.
M9 — Packaging + installers + embed	nfpm, install.sh, install.ps1; embed web/dist/.

Three adjustments to call out vs. the original M-plan:

• New M0.5: the fork pass. Done as of 2026-05-18. Five Tier-1 repos (bm-cli ex mc, madmin-go, minio-go, pkg, cli) and two Tier-2 (selfupdate, colorjson) live under buckit-io/*. Every cross-import in the fork tree points at buckit-io/* paths — no go.mod replace directives, no upstream sync work. bm-cli is library-only (the root main.go was removed; bm is the binary entry point and will consume the package in M1).
• Drop the separate Tasks page work. The UI consolidates onto the History page (every op writes a result snapshot; History's View modal renders it). No /tasks/:id page; no Task records distinct from history. internal/tasks/ still exists — it's the in-flight orchestrator — but it doesn't persist a task table.
• mc is vendored, not selectively ported. Original plan was silent on this. This doc lands on "fork the whole CLI tree and own it" — see the "Forking the MinIO Go ecosystem" section above.

M1 — Storage + server shell

Goal: bm web boots, opens bbolt, serves a chi router with shape-correct empty responses for the read paths the UI calls, and the urfave/cli dispatch mounts both bm-native verbs and the forked bm-cli verbs. No real cluster operations yet — every mutating endpoint stays 501 with a milestone tag.

Each bullet is work item → acceptance check.

Dependencies

• Import buckit-io/bm-cli, buckit-io/madmin-go/v3, buckit-io/cli, go-chi/chi/v5, go.etcd.io/bbolt into go.mod → go build ./... succeeds and the binary still fits the ~10–12 MB target.

internal/config/

• Resolve XDG paths (~/.config/bm/ on unix, %APPDATA%\bm\ on Windows) → config.Dir() returns the right path per-OS.
• KEK bootstrap chain: BM_DATA_KEY env → data.key file → auto-generate 32-byte key at ~/.config/bm/data.key mode 0600 → config.DataKey() returns 32 bytes; auto-generation logs the path once on first launch.
• Settings struct (remote access off, version pin nil) loaded from the settings/app bucket → defaults applied on first launch; PATCH /settings persists across restarts.

internal/store/

• bbolt opened at ~/.config/bm/bm.db mode 0600 with 5s lock timeout → second bm web fails fast with a wrapped, friendly "another bm process is using ~/.config/bm/bm.db" error (bbolt's flock is kernel-held and released on crash, so there is no stale-lock case).
• View(fn) / Update(fn) helpers wrap db.View / db.Update with a 5s per-txn timeout → both helpers covered by unit tests.
• Buckets auto-created on first open: clusters, nodes, node_facts, cluster_ssh, cluster_admin, history, settings → bucket-list assertion test passes.
• PutEncrypted / GetEncrypted AES-GCM helpers for cluster_ssh and cluster_admin buckets → round-trip test on a random 1 KiB payload passes; callers never see ciphertext.
• History bucket sweep deferred — a code comment notes M2's tasks.Finalize owns it.

internal/app/

• SIGINT / SIGTERM handler triggers graceful shutdown of the chi server with a 5s deadline → Ctrl-C on a running bm web exits 0 within 5s.
• Single-instance enforcement piggy-backs on bbolt's file lock (above). No separate bm.lock.

internal/api/

• chi router mounted at /api/v1, listener default 127.0.0.1:9443, refuses non-loopback bind in M1 → bm web --addr 0.0.0.0:9443 errors out pointing at the (future) remote-access milestone.
• Middleware: recoverer, request logger, JSON content-type → a panic in a handler returns 500 + structured JSON, not a stack trace.
• GET /api/v1/healthz returns {"status":"ok","version":...} → curl returns 200.
• Shape-correct read stubs so the UI loads against the real backend with empty state:
  • GET /clusters → []
  • GET /clusters/:id → 404
  • GET /history → []
  • GET /settings → the settings struct from bbolt
  • GET /sessions/me → {"username":"admin"} (no auth in loopback default)
• All other endpoints listed in § REST API → 501 with {"error":"not implemented","milestone":"Mn"} and the owning milestone tag.
• Static-asset handler: if web/dist/ exists on disk, serve it; else return 404 with a hint to run npm run build → manual npm run build + bm web renders the Clusters page in a browser.

internal/alias/

• Write-through helper alias.Sync(store) that snapshots all clusters into ~/.config/bm/config.json mode 0600 in mc-compatible JSON shape → on-demand call produces a file bm-cli verbs can read; no clusters yet so the output is the empty mc config skeleton.
• Patch bm-cli at startup to read its config from ~/.config/bm/ (the fork already has setMcConfigDir at cmd/config.go:40 — promote to exported if needed) → bm alias list reads from the bm path, not ~/.mc/.

cmd/bm/

• Replace the hand-rolled switch in cmd/bm/main.go with buckit-io/cli (urfave/cli fork) dispatch → existing bm version and bm help keep working; bm with no args prints the unified help.
• Register only web and version as bm-native commands in M1 — the other bm-native verbs (cluster, manager, migrate, rolling, node, history, settings) land in their owning milestones, not as M1 stubs.
• Mount bm-cli's appCmds slice alongside the bm-native commands → bm alias list, bm admin info --help, and at least one cp/ls smoke run dispatch correctly.
• New cmd/bm/web.go with the web action: flags --addr (default 127.0.0.1:9443), --no-browser, --data-dir → starting bm web opens the default browser (unless --no-browser) and binds the listener.

Milestone exit criteria

1. bm web starts, serves /api/v1/healthz, exits cleanly on SIGINT.
2. ~/.config/bm/{bm.db,data.key,config.json} all exist mode 0600 after a first run.
3. The Clusters page in the UI loads against the real backend and shows the empty state.
4. A second bm web started while the first is running exits non-zero with the wrapped bbolt-timeout message.
5. bm version, bm help, and bm alias list all dispatch under the unified urfave/cli tree.
6. make build produces a binary in the ~10–12 MB range; the cross-compile sanity loop in CLAUDE.md still passes.

M1-local open questions

• appCmds export. Is bm-cli's appCmds slice exported today, or does it need a one-line fork patch (Cmds or Commands())? Decide before the urfave/cli wiring lands.
• Dev fallback for the static handler. When web/dist/ is missing, should bm web 404 strictly, or redirect to http://localhost:5173 so vite dev is the natural inner loop? Strict 404 is simpler; redirect is friendlier.
• Browser auto-open on headless hosts. open / xdg-open fails silently on macOS without a GUI session and on bare Linux. Log a warning vs. surface an error?
• Default listener port. The doc commits to 127.0.0.1:9443. 9443 is IANA-registered as tungsten-https and is Portainer's default HTTPS port — a real conflict for operators running both tools on one machine. Decide before M1 merge: keep 9443 and rely on --addr overrides, or move to a less-crowded port (9543 and 9445 are the cleanest nearby options; 8443 collides with Tomcat / K8s).

M8 — MinIO migration

Goal: an operator on the Migrate wizard can capture a MinIO cluster's state, run a sequential per-host cutover that swaps minio.service for buckit.service, and roll the change back if needed. All of it goes through the M2 orchestrator, so progress streams over SSE and history rows record Result snapshots like every other op.

The cutover is roll-forward only inside one task — verify failures surface as warnings on the history row, not an auto-rollback. The operator triggers rollback explicitly. That keeps the failure mode visible instead of silently masking real bugs.

Each bullet is work item → acceptance check.

Snapshot capture (internal/migration/, internal/admin/)

• Extend internal/admin/Client with ListUsers, ListGroups (+ per-group GetGroupDescription), ListCannedPolicies, ListServiceAccounts(users) → admin client returns the typed slices the snapshot writer consumes.
• Add internal/admin/S3Client (minio-go wrapper) for bucket-level reads: ListBuckets, EnrichBucket (versioning + object-lock + tags), BucketLifecycle, BucketNotifications → 404/NotImplemented on older MinIO versions becomes a snapshot warning, not a fetch failure.
• migration.Snapshot(ctx, dir, clusterID, creds) populates the full domain.MinioSnapshot and writes it to ~/.config/bm/snapshots/<clusterId>-<ts>.json mode 0600 → file written, in-memory snapshot returned, soft per-field errors collected in snap.Warnings.
• migration.Summarize(snap) derives domain.MinioSnapshotSummary (counts + largestBucket) for the wizard's Review step → counts match what the wizard's MinioSnapshot interface in state.ts expects.
• migration.ReadSnapshot(path) reloads a snapshot for the cutover/rollback executors → wire-stable round trip; new optional fields don't break older files.

Cutover executor (internal/migration/)

• CutoverParams + MigrationBody (wire shape) + Stage enum (mirrors UI's CutoverNodeState.state byte-for-byte) → Validate() rejects empty hosts, missing snapshot, unsupported version.
• Installer.Install(ctx, host, params, emit) per-host pipeline: backup /etc/default/minio → stop minio → curl rpm → dnf/yum/apt install → disable minio.service + enable --now buckit.service → curl /minio/health/live → done. Reuses deploy.PickInstallCmd, deploy.SudoWrap, deploy.ShellEscape, deploy.RunStep (exported from internal/deploy/install.go) → emits a StepEvent per stage.
• CutoverExecutor (sequential, no parallel knob): per-host loop with waitClusterHealthy between hosts via admin.Pool ServerInfo (default 120s timeout) → halts on first host failure with FailureNote listing the failed host; remaining hosts stay on MinIO.
• After all hosts done: commitEngineFlip updates the persisted domain.Cluster: Engine: minio→buckit, sets Version, stamps MigratedFrom{Product:"minio", Version: snap.Version, FinalizedAt: now} → cluster row reflects the new engine; UI banner shows the migration source.
• Cancellation: markCanceled records the in-flight host's stage in FailureNote ("cutover canceled at (stage: )"), keeps earlier hosts as HostSucceeded, leaves later hosts as HostPending → operator reads exactly which hosts are on Buckit and which are still on MinIO from the History result modal.

Verify pass (internal/migration/verify.go)

• After commit, Verify(ctx, pool, params) reads back the migrated cluster: ServerInfo for clusterHealthy + nodesReporting, AccountInfo for bucket count + smoke check (every snapshot bucket still exists), ListUsers/ListGroups/ListCannedPolicies/ListServiceAccounts for IAM counts → result populates the wizard's VerifyResult shape.
• Verify failures land in OperationResult.FailureNote and surface as a warning on the history row. No auto-rollback → operator decides; bm doesn't second-guess.
• Time-boxed at 60s so a wedged cluster doesn't push the cutover history row past the operator's expectation.

Rollback executor (internal/migration/rollback.go)

• RollbackExecutor validates, then per host: systemctl is-active buckit.service → if inactive, mark HostSucceeded with detail "Already on MinIO" and skip; otherwise run Installer.Rollback: stop buckit.service → restore env-file backup → enable --now minio.service → wait /minio/health/live → if at least one host actually rolled back, flip Cluster.Engine back to EngineMinio and clear MigratedFrom.
• Pure no-op rollback (every host already on MinIO) leaves the cluster row alone → idempotent; safe to call twice.

Preflight check

• bak_writable (blocking, per host) — sudo touch + rm /etc/default/.bm-bak-probe → catches sudo-required hosts where /etc/default isn't writable before the cutover hits the same step on host #2.

REST surface

• POST /clusters/:id/migrate/snapshot — returns {snapshot, summary, path}. Already mounted in M5; M8 fills in the body.
• POST /clusters/:id/migrate/preflight — already mounted in M5; M8 adds bak_writable.
• POST /clusters/:id/migrate/cutover — body is the wizard's MigrationBody, dispatches migrate_cutover. 404 on missing cluster, 404 on missing admin creds, 400 on validation, 409 cluster_busy.
• POST /clusters/:id/migrate/rollback — same body shape minus the snapshot requirement, dispatches migrate_rollback. 404/400/409 as above.
• The wire stays compatible with the wizard's existing MigrationDraft shape — FromMigrationBody picks the executor-relevant fields.

Wiring (cmd/bm/web.go)

• migration.Register(deps) wires CutoverExecutor + RollbackExecutor into the tasks registry. Added alongside operations.RegisterAll(...) after the existing deploy.Register(...) block. New OpKind constants OpMigrateCutover / OpMigrateRollback live in internal/tasks/types.go.

Tests

• internal/migration/snapshot_test.go — write/read round-trip + 0600 mode + Summarize counts + end-to-end against fake httptest admin/S3 endpoints.
• internal/migration/cutover_test.go — 1-host happy path, snapshot-missing rejection, 2-host with cluster-healthy wait between hosts. Uses the in-memory internal/sshtest server.
• internal/migration/rollback_test.go — full rollback (engine flips back) + no-op when buckit isn't active (uses a new sshtest.Server.CmdOverride hook to simulate the inactive probe).
• internal/api/m8_integration_test.go — full HTTP path: dispatch cutover → poll terminal → assert engine, then dispatch rollback → assert engine flips back. Includes 404 (missing cluster) and 400 (validation) cases.

Milestone exit criteria

1. make build + make test clean (-race -count=1 ./...).
2. The wizard's Migrate step renders against the real backend: snapshot endpoint returns counts the Review step renders, cutover dispatch streams stage events over SSE.
3. Cluster row's Engine flips deterministically on success and on rollback. MigratedFrom is stamped on cutover, cleared on rollback.
4. The 5-platform cross-compile sanity loop in CLAUDE.md still passes.

M8-local open questions

• Snapshot file format versioning. The on-disk JSON has no schema version field today — adding fields with omitempty keeps older files decodable, but a future breaking change would need an explicit schemaVersion int. Defer until a real schema break is needed.
• Per-bucket smoke read. Verify doesn't yet HEAD the largest bucket to confirm reads work post-cutover — ObjectsSampled is 0/0 today. Cheap to add when minio-go's anonymous read is already wired in for bm cp parity.
• Parallel cutover. Sequential is the M8 contract because the UI's per-node state machine assumes one host at a time. If operators want a parallel knob (à la BM_DEPLOY_CONCURRENCY), it lands as a future M8.5 alongside the corresponding wizard change.

Open questions

1. Where does Buckit console live for the deep-link button? Today the UI assumes cluster.url/cluster.consoleUrl. Backend needs to capture and persist whatever the admin-info reports as the console address (and handle the case where it's empty).
2. TLS at the admin API layer. Many real clusters terminate TLS at a load balancer. bm needs to handle both http:// and https:// endpoints; for the latter, decide whether to require cert verification or allow --insecure per cluster.
3. Cluster import discovery latency. Discovery streams progress lines today (per-host detail). For clusters with many hosts the total time can be tens of seconds. Confirm SSE is the right transport vs. WebSocket vs. one-shot poll-and-return.
4. bm ↔ bm coordination. Two bm processes on the same machine contending for bbolt is fine for short-lived CLI calls, but a bm web server plus a long-running CLI op (e.g. bm cluster migrate) probably wants explicit coordination through a manager socket — not just bbolt locks. To revisit when M3 lands.
5. Alias-file coexistence with an existing mc install. If the operator already runs mc and has ~/.mc/config.json populated, does ~/.config/bm/config.json shadow it, supplement it, or stay isolated? Current design says isolated — bm reads its own file only. Revisit if user feedback says otherwise.
6. Rebrand depth in the forked CLI. Forked mc still says "mc" in error messages, help text, and the binary name. Decide once how aggressively to rebrand (all strings → "bm", or leave as-is with only the binary entry renamed). Affects upstream-pull friction — which we no longer pay anyway — and operator clarity.