Commit Graph

12637 Commits

Author SHA1 Message Date
abuckit 4aafa83e11 Document Docker OFF and ON replay results 2026-06-14 22:33:14 -04:00
abuckit 15b5ea3f03 Add FastOpen test rigs and run docs 2026-06-14 19:30:01 -04:00
abuckit 0740a461f7 Add FastOpen timing metrics 2026-06-13 21:05:23 -04:00
abuckit 9f6f18c1b1 test: cover deferred FastOpen decode failures 2026-06-11 09:46:13 -04:00
abuckit 4cf193ff7c feat: profile FastOpen body-path timing 2026-06-11 09:46:09 -04:00
abuckit d52d9d5865 chore: remove FastGet prototype path 2026-06-10 11:37:21 -04:00
abuckit b09d64fe30 feat: add FastOpen observability 2026-06-10 10:44:21 -04:00
abuckit 91526a9788 test: expand FastOpen GET golden coverage 2026-06-10 09:55:15 -04:00
abuckit 6344ff1289 feat: add FastOpen lazy replacement 2026-06-09 23:42:18 -04:00
abuckit dbd22c0b7d feat: add FastOpen GET landing path 2026-06-09 21:13:42 -04:00
abuckit e3b3246c6d feat: add FastOpen storage transport 2026-06-09 19:20:13 -04:00
abuckit 831b5e8b5b feat: add disk-local FastOpen part stream 2026-06-09 17:18:22 -04:00
abuckit f5939780b7 feat: add FastOpen frame foundation 2026-06-09 16:57:37 -04:00
abuckit 5d77f36c3c docs: add FastGet phase 1 design plan 2026-06-09 15:48:26 -04:00
abuckit 5d28a361c4 docs: add single-trip benchmark handoff 2026-06-08 14:04:35 -04:00
abuckit e5b0a6f47d perf: stabilize single-trip fast get 2026-06-08 14:04:14 -04:00
abuckit bb8b663c0f docs: record single-trip phase-1 benchmark results
Document the container A/B on the corrected single-pool rig (one 16-drive
EC:4 set, 2 MiB non-inlined objects):

- single-trip proven directly: with xl.meta and the canonical data dir
  deleted on all drives, FAST_GET=1 still serves byte-exact from the shadow
  (multi-pool returned 404).
- cold single-stream TTFB ~26% lower (server-side trace) / ~21% (curl),
  captured paired on the same requests; saturated throughput flat on the
  seek-free loopback medium.
- record the two rig traps (multi-pool xl.meta pre-read; inline cutoff is
  per-shard, so EC:4 inlines everything below ~1.5 MiB).

Design note: the on-disk shadow header should be variable-length with a
self-describing payload so fields can evolve without lockstep; the fixed
1024-byte positional header is a phase-1 shortcut.
2026-06-04 16:00:26 -04:00
abuckit 6d6579cb47 test: wire single-pool cluster for single-trip benchmark
cluster.sh emitted one endpoint arg per node, which brought the rig up as
four independent server pools. Multi-pool GET resolves the owning pool via
getLatestObjectInfoWithIdx (a per-pool xl.meta read) before the set-level
fast path runs, so BUCKIT_FAST_GET=1 still read xl.meta and the single-trip
path was never exercised.

Emit a single pool spanning all nodes (http://node{1...4}:9000/data/...)
so SinglePool() is true and GET dispatches straight to the set, letting the
fast path bypass xl.meta. Regenerated docker-compose.yml reflects the change.
2026-06-04 16:00:26 -04:00
abuckit e0e3955742 test: wire cluster rig for single-trip benchmark 2026-06-04 11:52:04 -04:00
abuckit 87ca27b872 docs: record single-trip smoke validation 2026-06-04 11:09:59 -04:00
abuckit 2807ace9f0 test: isolate single-trip counter assertions 2026-06-04 09:42:22 -04:00
abuckit eed84c8dbe test: allow body in tracking response writer test 2026-06-04 09:33:37 -04:00
abuckit 29982574ef feat: add single-trip GET phase 1 prototype 2026-06-04 09:10:40 -04:00
abuckit f4b729273c docs(kms): pin epoch-key derivation, split MVP vs phase-2 security
Address security review: specify the exact epoch-key derivation scheme
(deterministic KMS MAC, or random-and-wrap) since cloud KMS GenerateDataKey
is random by default and would otherwise make keys unrecoverable. Promote
authenticated metadata binding, basic token scoping, a minimal revocation
path, and bounded historical cache into MVP requirements; keep full replay
protection and coordinated revocation as phase-2. Add the accepted-residual-
risk note and a multi-instance proxy section.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 15:53:44 -04:00
abuckit 88210b94ee docs(kms): revise KES replacement design to keep master key in KMS
Switch the design to a true-HSM model where the master key never leaves
the KMS, unlike KES/MinKMS which load it into memory. The KMS produces an
epoch key per window; bucket and object keys derive locally. Adds the
two-window model (12h epoch + ~15min Buckit cache TTL), corrected near-zero
cost analysis, the bounded-compromise security framing, and clarifies that
only the bucket key is cached.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 14:39:25 -04:00
abuckit c03f2e5bb8 Merge docs/kes-replacement-design: KES replacement design doc 2026-06-01 22:54:36 -04:00
abuckit 822d5d374d docs(kms): add KES replacement design
Design for replacing deprecated MinIO KES with an open-source approach:
a cached per-cluster encryption key in Buckit (L1) plus a stateless
KMS-auth proxy (Fargate, L2) that holds the cloud credentials so they
never live in Buckit. Covers cost analysis, the two-tier cache, security
boundary (credential isolation vs. key-material exposure), and open
decisions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 22:52:51 -04:00
abuckit 764a577ab8 add test container rig for migration 2026-05-31 13:51:53 -04:00
abuckit f9e878aa5e ci(release): restyle gh-pages release & archives pages to match bm landing design RELEASE.2026-05-31T17-41-53Z 2026-05-31 13:22:52 -04:00
abuckit d26fd31215 ci(gh-pages): keep hand-written styled root index out of regeneration RELEASE.2026-05-31T17-04-46Z 2026-05-31 12:43:17 -04:00
abuckit 42f49f3cc3 docs(manager): mark M1-M9 landed; document shipped installer approach
M9 reflects what shipped: embedded UI bundle, signed multi-platform
release pipeline + gh-pages download site, bm update self-update, and
the install.sh/install.ps1 per-user installers (Homebrew/Scoop dropped).
Refresh the Progress and resuming notes (mock layer removed; TS refs
ported to Go).
2026-05-31 12:00:21 -04:00
abuckit 3f9e6c8a6d deps: bump go-jose to v4.1.4 to fix govulncheck
Resolves GO-2026-4945 (JWE decryption panic in github.com/go-jose/go-jose/v4),
reachable via the OIDC init path (cmd/utils.go -> oidc.init -> jose.init).
Fixed in go-jose/v4 v4.1.4.
2026-05-29 23:09:58 -04:00
abuckit f0c2589239 deps: bump x/crypto to v0.52.0 and x/net to v0.55.0 to fix govulncheck
Resolves govulncheck failures flagged in CI:
- GO-2026-5013..5023 (golang.org/x/crypto/ssh) — fixed in v0.52.0
- GO-2026-5026 (golang.org/x/net/idna) — fixed in v0.55.0

x/sys bumped to v0.45.0 as a transitive dependency.
2026-05-29 23:01:51 -04:00
abuckit 27a73eae7e ci: remove SonarQube workflow 2026-05-29 22:54:51 -04:00
abuckit 371e426469 Create sonarqube.yml 2026-05-29 22:50:58 -04:00
abuckit b4011734af ci: remove Lock Threads workflow 2026-05-29 22:50:17 -04:00
abuckit 6cc3cf1a0e add local test setup 2026-05-26 10:18:46 -04:00
abuckit eb9bd45e13 Bump console dependency to pick up self-hosted Montserrat font
Pulls in buckit-io/console@f8ce35f8, which replaces the
CSP-blocked Google Fonts <link> with a 6.7 KB embedded
Montserrat subset for the BuckitLogo wordmark.
2026-05-23 22:07:03 -04:00
abuckit 019a533215 docs: rebrand package paths and update backend design milestones 2026-05-22 21:53:01 -04:00
abuckit b2b1237a8c ci: replace gh-pages binary storage with HTML index pages linking to GitHub Releases
- update-gh-pages: remove zip_binary/publish_package/trim_archives; only write
  4 buckit.sha256sum files for self-update; generate release/index.html with
  per-platform download links and archives/index.html from GitHub API listing
  all stable releases; add force_orphan to prevent git history bloat
- gh-pages-index.yml: skip server/buckit/release and server/buckit/archives
  so the auto-indexer does not overwrite the custom HTML pages
2026-05-22 17:33:05 -04:00
abuckit e9ee3eb59a feat: keep only last 30 releases in gh-pages archives/ 2026-05-22 17:03:56 -04:00
abuckit 4e8e603b63 feat: archive versioned packages (deb/rpm/apk) in gh-pages archives/ 2026-05-22 16:54:20 -04:00
abuckit 316d2707b4 feat: zip binaries into gh-pages release/ and archives/ directories RELEASE.2026-05-22T20-48-27Z 2026-05-22 16:47:50 -04:00
abuckit ce852535fa admin: log update progress and version comparisons 2026-05-22 11:20:20 -04:00
abuckit 9df885036d admin: expose server os and arch in server info RELEASE.2026-05-22T02-20-39Z 2026-05-21 22:05:22 -04:00
abuckit c4d5c58014 deps: switch buckit to buckit madmin-go and minio-go forks 2026-05-21 22:05:21 -04:00
abuckit f68ca79dfc fix: download binaries from GitHub Releases instead of gh-pages
Binaries exceed GitHub's 100MB git push limit so they cannot be stored
in the gh-pages branch. Two changes to fix this:

- release.yml: stop copying binaries/minisig to gh-pages; write
  buckit.sha256sum files with the release tag embedded in the filename
  field (e.g. 'buckit.RELEASE.xxx') so the Go code can construct the
  versioned GitHub Releases URL.

- update.go / admin-handlers.go: getBinaryURL now constructs a
  github.com/releases/download URL when the checksum source is github.io
  and the sha256sum filename contains a release tag. Fixes two dead-code
  bugs in both admin handlers where 'if updateURL == ""' was always
  false after updateURL had already been set.
RELEASE.2026-05-22T01-47-42Z
2026-05-21 21:47:16 -04:00
abuckit 338973577f fix: add dist/ prefix to package globs in update-gh-pages step RELEASE.2026-05-22T01-14-55Z 2026-05-21 21:12:50 -04:00
abuckit df9f18ef61 ci: generate directory indexes on gh-pages after each deploy 2026-05-21 20:42:30 -04:00
abuckit 6f17ed7db3 packaging: remove rpm preremove service stop 2026-05-21 18:34:29 -04:00