Commit Graph

7595 Commits

Author SHA1 Message Date
Chris 902c97ce05 fix: auto-repair breaking changes from acd26afd8 (#8483) 2026-10-11 20:09:55 -07:00
dependabot[bot] 7ad4fa439e build(deps): bump the dependencies group across 1 directory with 3 updates (#8481)
Bumps the dependencies group with 3 updates in the / directory: [datafusion](https://github.com/apache/datafusion), rust-embed and [hotpath](https://github.com/pawurb/hotpath-rs).


Updates `datafusion` from 55.1.0 to 55.2.0
- [Changelog](https://github.com/apache/datafusion/blob/main/CHANGELOG.md)
- [Commits](https://github.com/apache/datafusion/compare/55.1.0...55.2.0)

Updates `rust-embed` from 8.12.0 to 8.13.0

Updates `hotpath` from 0.28.5 to 0.28.6
- [Release notes](https://github.com/pawurb/hotpath-rs/releases)
- [Changelog](https://github.com/pawurb/hotpath-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pawurb/hotpath-rs/compare/v0.28.5...v0.28.6)

---
updated-dependencies:
- dependency-name: datafusion
  dependency-version: 55.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: hotpath
  dependency-version: 0.28.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: rust-embed
  dependency-version: 8.13.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-11 19:09:19 -07:00
Hauser 1266d4c270 feat(tls): reload and publish external certificate generations (#8467)
* feat(tls): reload external certificate snapshots coherently

Pin projected certificate generations, validate complete TLS material, coordinate server consumers, and invalidate outbound RPC caches coherently. Add externally managed TLS Secret support and rotation guidance.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* feat(tls): publish server certificates through a fenced admin API

Add opt-in server-leaf publication with dedicated IAM actions, actual TLS
transport checks, immutable generations, durable captured-token CAS and
idempotent receipts. Preserve independent trust and client identity sources
and distinguish visible publication state from the full serving chain.

Validate 64 TLS tests, 41 TLS-runtime tests, the focused IAM policy test,
and eight live local API scenarios including least privilege and restart.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(admin): route certificate handler types through S3 facade

Use the existing admin S3 facade to keep direct s3s imports within the footprint ratchet.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(admin): include TLS certificate routes in route matrix

Account for the registered certificate-state GET and certificate publication POST without changing route behavior or weakening matrix assertions.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(tls): reconcile startup trust before skipping reloads

Compare the initialized listener bundle with coherently published outbound material, retain startup trust and client identity when sources disappear, and cover independent startup rotations through real reload passes.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* refactor(tls): reuse pinned generations and prepared listener state

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(tls): preserve startup trust source requirements

---------

Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
Co-authored-by: overtrue <anzhengchao@gmail.com>
2026-10-12 08:53:11 +08:00
overtrue 9286442f87 feat(capture): add rebuildable committed event pagination 2026-10-12 08:31:29 +08:00
Anton acd26afd8b fix(scanner): skip child prefixes removed before they are opened (#8479)
A child prefix can be deleted after its parent listed it and before scan_folder opens it. read_dir then returned NotFound, which propagated through the compacted subtree and dropped the whole top-level child (child_scan_failed), so the cycle never produced a durable usage snapshot under continuous deletes.

Treat NotFound on open like a directory that vanished during iteration for non-root folders. A missing bucket root still reports NotFound so namespace removal handling is unchanged.

Refs #8478

Co-authored-by: Anton <5527364+basic-system@users.noreply.github.com>
2026-10-11 15:25:14 -07:00
Rafael Kansy 5924a55931 fix(obs): honor RUSTFS_OBS_USE_STDOUT with per-signal endpoints (#8476)
use_stdout was forced to true whenever the root RUSTFS_OBS_ENDPOINT was
empty, so a deployment that sets only a per-signal endpoint such as
RUSTFS_OBS_TRACE_ENDPOINT could not turn the stdout exporters off: every
exported span was also printed to stdout as a multi-line text block.

Count a non-empty trace, metric or log endpoint as a configured endpoint.
With no endpoint at all the stdout default is still forced on. Add a test
covering each per-signal endpoint and an empty value.
2026-10-11 13:00:24 -07:00
GatewayJ 2d2679a710 refactor(oidc): isolate config updates and replication view (#8446) 2026-10-11 08:53:29 -07:00
overtrue b8aad1c168 fix(capture): validate checkpoint cut against captured committed log 2026-10-11 23:48:08 +08:00
overtrue 5fcb229760 feat(ecstore): observe bounded config under retained write owner 2026-10-11 22:59:16 +08:00
Chris 257c9f11ae docs: refresh security advisory lessons (#8472) 2026-10-11 22:43:28 +08:00
Hauser 1324e296f8 test(scanner): bound expansion fixture preparation I/O (#8473)
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
2026-10-11 22:43:08 +08:00
overtrue 94baef7462 feat(capture): add explicit schema-bound checkpoint capabilities 2026-10-11 21:30:55 +08:00
Hauser 38263ba896 fix(log-analyzer): match structured heal writer failures (#8469)
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
2026-10-11 20:46:44 +08:00
overtrue 60d90f085a feat(ecstore): add bounded persisted config observations 2026-10-11 20:29:27 +08:00
overtrue dfa5d50565 feat(capture): persist first-decision provenance 2026-10-11 18:58:12 +08:00
overtrue a902886737 feat(capture): atomically migrate legacy Created history 2026-10-11 16:39:10 +08:00
overtrue 8e53b87421 test(capture): specify atomic populated native migration 2026-10-11 16:39:10 +08:00
overtrue f7631e91b1 feat(capture): stream an owned coherent snapshot view 2026-10-11 16:30:12 +08:00
overtrue f9bc4e921f test(capture): specify coherent snapshot view ownership 2026-10-11 16:30:12 +08:00
overtrue 781584307a feat(capture): apply committed native entries on the ordered worker 2026-10-11 15:35:02 +08:00
overtrue 59d641a5fe docs(capture): define recoverable checkpoint and purge contracts 2026-10-11 15:00:38 +08:00
overtrue c618b57870 feat(capture): stream snapshot rows with a permanent failure fence 2026-10-11 14:44:05 +08:00
overtrue bbe1d54098 feat(capture): add bounded streaming snapshot codec 2026-10-11 14:21:30 +08:00
Chris ba2047b184 ci: reuse workspace test builds for serial ILM checks (#8465) 2026-10-11 13:55:48 +08:00
overtrue 9e8ea4dcdb feat(capture-authority): serialize native async writes 2026-10-11 13:35:15 +08:00
GatewayJ da42266bfb refactor(iam): organize OIDC runtime modules (#8445) 2026-10-11 13:27:46 +08:00
Chris 367f13f3f9 ci: reduce repeated PR runner work (#8463) 2026-10-10 22:19:25 -07:00
overtrue 63cf8e2f2f feat(capture-authority): add native OpenRaft log reader 2026-10-11 13:01:43 +08:00
Hauser 0263d5fcb9 fix(storage): preserve heal failures and timeout responses (#8462)
* fix(storage): preserve heal failures and timeout responses

Stop repairs with no writable target before opening source readers. Consolidate target failure aggregation, preserve typed causes, and distinguish all-ignored diagnostics from actual successful nil entries. Map typed disk timeouts to retryable S3 responses without changing quorum decisions.

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* refactor(heal): unify staging failure cleanup

Route data-stage failures through one local result boundary after temporary
readers and writers are released. Preserve typed report failures, outer
reconstruction errors, partial repair, and the existing commit fences.

Exercise actual source loss after second-part writer admission using the
existing test pause registry. Assert all staged parts are removed, committed
metadata stays unchanged, and restoring the source allows a full-byte retry.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: zhi22915 <qiuzgang@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: Chris <anzhengchao@gmail.com>
2026-10-11 12:41:13 +08:00
Chris 93a64fdca3 fix(dev): restore VS Code debug configurations (#8450) 2026-10-11 12:34:41 +08:00
overtrue b7fe40f4b3 feat(capture): support complete native log range bounds 2026-10-11 12:28:23 +08:00
Hauser 9ecb00d388 fix(scanner): avoid placeholder retention and resume checkpoints (#8460)
* fix(scanner): bound old-cache lookups and recover checkpoint saves

Keep missing child lookups read-only and preserve the checkpoint receiver
after ordinary persistence errors. Reload on stale CAS or indeterminate
revision refresh, and report confirmed periodic saves separately.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(scanner): remove redundant checkpoint error conversion

Return the existing StorageError directly so strict Clippy passes.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* docs(scanner): clarify checkpoint persistence guarantees

Distinguish a confirmed periodic save from validated restart coverage.
Keep adoption and subtree-skip requirements explicit.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(scanner): preserve resumable periodic coverage

Materialize active ancestor accounting in bounded periodic snapshots and
seal completed coverage independently of raw enumeration state. Preserve
durable coverage during early resumed enumeration and exclude unfinished
out-of-order heal work from periodic and cancellation receipts.

Reuse ancestor frame slots to avoid growing child-set copies on descent,
retain compacted accumulator safety, and report checkpoint adoption resets.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(scanner): reject covered active ancestors in checkpoints

Check every suspended ancestor alongside the current folder before
certifying completed coverage. An unfinished covered ancestor must not
become a resumable subtree merely because its descendant sorts later.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* fix(scanner): retain an incomplete root for missing namespaces

Create the empty bucket root explicitly at the missing-namespace boundary.
Preserve nonempty durable caches and keep namespace results unpublished.

Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
2026-10-10 20:54:12 -07:00
overtrue 5b872b8e56 feat(capture): apply native entries atomically with durable replay 2026-10-11 11:50:32 +08:00
cxymds b8889fccc9 fix(ecstore): allow cold-start recovery at EC read quorum (#8458)
* fix(ecstore): load existing formats at EC read quorum

* fix(ecstore): complete startup at EC read quorum
2026-10-11 11:50:24 +08:00
overtrue ca5be1ef95 feat(capture): persist native Raft entries in explicitly enabled stores 2026-10-11 05:49:17 +08:00
overtrue 9373e8fa3e Bind capture commands and responses to native Raft entries 2026-10-11 03:45:17 +08:00
overtrue cb32965522 feat(capture): preserve exact raft values and membership codec 2026-10-11 02:04:47 +08:00
overtrue fb6cf49a67 feat(capture): validate peer metadata and CNOD codec 2026-10-11 01:25:21 +08:00
overtrue 8c97049e68 build(capture): pin OpenRaft and guard resolved features 2026-10-11 00:56:30 +08:00
overtrue 17d4fe27cc feat(capture): apply committed Created decisions atomically 2026-10-10 23:35:52 +08:00
overtrue fb0c7972b2 feat(capture): persist committed log boundary 2026-10-10 23:10:28 +08:00
overtrue eba7967755 feat(capture): persist atomic log suffix truncation 2026-10-10 22:59:26 +08:00
overtrue 451c1c31ad feat(capture): add durable local authority storage 2026-10-10 22:33:44 +08:00
overtrue 8780dd670e test(ecstore): verify publication across process kills 2026-10-10 20:19:46 +08:00
GatewayJ a424d10fa0 refactor(iam): construct OIDC runtime directly (#8436)
* refactor(iam): remove legacy OIDC federation paths

* refactor(iam): construct OIDC runtime directly
2026-10-10 04:01:40 -07:00
overtrue f29bf2c902 fix(iam): preserve empty identity read failures 2026-10-10 11:37:07 +08:00
overtrue fd03436090 feat(ecstore): expose unbounded preserve-empty config reads 2026-10-10 11:10:27 +08:00
overtrue 1f1e7b1c97 fix(iam): preserve identity decryption errors 2026-10-10 10:37:15 +08:00
overtrue 45a952609e test(iam): add real identity storage fixtures 2026-10-10 10:01:03 +08:00
overtrue 323190a95a test(test-utils): support caller-owned instance contexts 2026-10-10 09:43:01 +08:00