buckit/docs/manager

Buckit Manager (bm) High-Level Design

Overview

This document describes the long-term high-level design for bm, the Buckit Manager. The intent is for bm to become the primary operational entrypoint for Buckit deployments:

• In the near term, bm provides cluster lifecycle management for Buckit deployments.
• In the long term, bm replaces mc for Buckit-specific administration and, over time, for a broader set of object and bucket operations.

The guiding principle is to ship one product with multiple frontends:

• a CLI for operators and automation
• an HTTP API for browser and programmatic use
• a web UI for interactive cluster management

bm should be distributed as a single binary in production.

Goals

• Simplify creation and deployment of multi-node Buckit clusters.
• Reduce operational friction compared to manually managing distributed startup arguments, config files, and service units.
• Provide a clear path from "cluster manager" to "Buckit operational client" without building unrelated products.
• Keep production deployment simple: one manager binary, minimal moving parts.
• Preserve a good developer experience for the web UI and CLI.

Non-Goals

• Replacing all mc functionality in the first release.
• Building a general-purpose infrastructure orchestrator.
• Requiring a node-side agent in the initial design.
• Making Kubernetes the primary deployment target for the first version.

Product Model

bm is a single binary with two execution modes:

1. bm as a direct CLI
2. bm server as a long-running manager service

Examples:

bm alias set prod https://s3.example.com AK SK
bm ls prod/mybucket
bm admin info prod

bm server --listen :9443

The central requirement is that CLI and server mode share the same application core. The web UI is a frontend on top of the server API, not a separate product.

Architecture

High-Level Components

bm
  - shared application core
  - CLI frontend
  - HTTP API server
  - web UI static asset server
  - SSH orchestration layer
  - task/job engine
  - cluster state store

Shared Core

The shared core owns real business logic:

• cluster inventory
• topology planning
• Buckit install/deploy/upgrade workflows
• node discovery
• state persistence
• audit and task tracking

This logic must not be duplicated across CLI handlers and HTTP handlers.

Frontends

CLI frontend

The CLI parses flags and arguments, then invokes shared application services. It should be thin and mostly free of cluster logic.

HTTP API frontend

The HTTP API exposes the manager for the web UI and future automation. REST+JSON is the default interface for the first version.

Web UI frontend

The web UI talks only to the HTTP API. It should not implement deployment or cluster logic itself.

Packaging Model

bm should be released as a single binary that embeds the web UI static assets.

Recommended model:

• Development:
  • web UI built as a conventional frontend app using Node tooling
  • frontend dev server talks to local bm API during development
• Release:
  • frontend compiled to static assets
  • static assets embedded into the bm binary using Go embed

This keeps local frontend development sane while preserving a simple production artifact.

Distribution Model

The first distribution target should optimize for low-friction installation and predictable release mechanics, not package-manager breadth.

Primary Distribution

The initial release model should be:

• versioned release binaries for supported OS/architecture pairs
• checksums and, ideally, signatures for release artifacts
• thin install scripts that download and install the correct binary

Recommended installation entrypoints:

curl -fsSL https://get.buckit.io/bm/install.sh | sh

iwr https://get.buckit.io/bm/install.ps1 -useb | iex

These convenience commands should be backed by a safer documented path that downloads the script before execution.

Safer Unix path:

curl -fsSL https://get.buckit.io/bm/install.sh -o install.sh
sh install.sh

Safer Windows path:

iwr https://get.buckit.io/bm/install.ps1 -OutFile install.ps1
powershell -ExecutionPolicy Bypass -File .\install.ps1

Platform Coverage

• install.sh
  • primary install path for Linux, macOS, and WSL
• install.ps1
  • primary install path for native Windows
• direct binary download
  • available on all supported platforms as the lowest-level install path

The Unix shell installer is not the primary path for native Windows. Windows should be treated as a first-class install target with its own script and release artifact flow.

Installer Responsibilities

Install scripts should stay thin and predictable. They should:

• detect OS and architecture
• select the correct release artifact
• download the binary
• verify checksum and, ideally, signature
• install to a reasonable prefix
• print the installed version

Install scripts should not:

• build from source
• silently mutate shell startup files
• silently require privileged installation when a user-local install works
• auto-install a service unless explicitly requested

Packaging Roadmap

Distribution should expand in stages:

1. release binaries + install.sh + install.ps1
2. signed and checksummed releases as the standard trust model
3. Homebrew tap for macOS/Linux users
4. additional package-manager integration as needed, such as winget, scoop, or Linux package repositories

This keeps early release engineering proportional to product maturity while still supporting an easy installation story.

Cluster Management Model

Agentless by Default

The initial design uses SSH-based orchestration from the manager instead of a required node-side agent.

Manager responsibilities:

• discover hosts and disks
• install or upgrade Buckit binaries
• write config and environment files
• create and manage service units
• start, stop, and restart Buckit
• run preflight validation
• perform rolling upgrades
• collect logs for deployment tasks

Rationale:

• fewer moving parts
• easier recovery when something fails
• easier adoption in VM and bare-metal environments
• no extra daemon to install, upgrade, or debug

Desired State

The manager should own desired state, not just execute ad hoc commands.

Examples of desired state:

• known nodes and credentials
• selected disks
• Buckit version
• cluster topology
• service state

The manager computes plans and executes them, rather than storing only a history of shell commands.

Phased Delivery Plan

Phase 1: Manager foundation

Deliver the manager as a server-oriented control plane:

• bm server
• embedded web UI
• node inventory
• SSH-based discovery
• cluster planning
• Buckit deployment
• service management
• health/status views

This phase establishes the control-plane model and the shared core.

Phase 2: Manager CLI

Expand the CLI around the same shared services:

• CLI commands that drive cluster lifecycle workflows directly
• task inspection and operational workflows from the CLI

This phase makes the manager usable without the UI.

Phase 3: Selected mc admin replacement

Move Buckit-specific admin capabilities into bm:

• admin info
• policy/user/admin flows
• alias/profile management
• cluster-aware administration

This phase should target administrative parity before broad object-operation parity.

Phase 4: Broader object client replacement

Add more mc-style data-path commands where it makes sense:

• ls
• cp
• mirror
• bucket/object inspection helpers

This phase should happen only after the internal command model and client abstractions are stable.

Open Questions

• Should manager state start on SQLite only, or should Postgres be supported from the first release?
• Which subset of mc admin commands should move first into bm?
• How should credentials be stored and rotated for SSH, S3 aliases, and Buckit admin access?
• What level of multi-user/RBAC support is needed in the first manager release?

Summary

bm should be designed as a single Buckit operational product with:

• one shared application core
• one binary for release
• multiple frontends: CLI and web/API
• agentless SSH orchestration for initial cluster management
• a phased path from manager-first workflows to gradual mc replacement

The first release should prioritize cluster lifecycle management and stable application boundaries over broad command-surface expansion.

Appendix — Implementation Notes

The sections below capture implementation-level details that an engineer will need when building bm, but which are not necessary to understand the high-level design above. They are directional, not contractual.

Backend Design

The web UI backend is the bm server process itself, not a separate Node.js backend.

Recommended backend subsystems:

• api — REST handlers, request validation, response models
• app — service wiring and shared use cases
• store — persistent state for clusters, nodes, tasks, credentials, audit events
• tasks — long-running asynchronous workflows
• ssh — SSH execution and file transfer
• deploy — install, upgrade, config, systemd workflows
• cluster — domain models and topology planning
• health — polling and status aggregation

Task Model

Long-running operations must be task-backed. Examples: node discovery, cluster planning, cluster deployment, rolling restart, rolling upgrade.

Preferred flow:

CLI or HTTP request
  -> create task
  -> task runner executes workflow
  -> progress and logs stored
  -> caller polls or subscribes for updates

This applies especially to server mode and any long-running orchestration workflow.

Command and API Modes

bm has two execution styles.

Direct operations — short-lived request/response operations that run directly in-process:

• bm ls
• bm cp
• bm admin info
• bm user list

Manager-backed operations — long-running orchestration flows that are task-backed and own durable workflow state in the manager:

• bm cluster add
• bm node discover
• bm cluster plan
• bm cluster deploy
• bm cluster upgrade

This split keeps direct CLI commands responsive while letting the manager own durable workflow state for everything that takes time.

Storage and Persistence

The manager needs persistent state for:

• clusters
• nodes
• node facts
• cluster specs
• tasks
• task logs
• deployments
• SSH credentials or credential references
• audit events

Initial recommendation: start with SQLite for single-node deployment simplicity, and design the storage layer so Postgres can be added later without changing the higher-level application model.

Repository Shape

One possible layout:

cmd/bm/
internal/app/
internal/api/
internal/auth/
internal/store/
internal/tasks/
internal/ssh/
internal/deploy/
internal/cluster/
internal/health/
web/

This is intended as a directional layout, not a strict commitment.