034efa8dac
Merge branch 'main' into test/distributed-e2e-hardening
2026-09-05 22:21:31 +08:00
188f380b3b
feat(ecstore): add native azure blob and gcs migration sources ( #7211 )
...
* feat(ecstore): add a native azure blob odm source backend
* feat(ecstore): add a native gcs odm source backend and one backend contract
* fix(ecstore): refuse an empty azure account key at client build
* fix(ecstore): probe gcs sources with the listing permission
* fix(app): drop a redundant match guard on the sse config lookup
* fix(ecstore): drop stale rename commit duplicates from local.rs
* test(ecstore): use the sanctioned placeholder key in the gcs fixture
2026-09-05 22:06:30 +08:00
e2a921bc16
fix(storage): harden ODM and scanner publication ( #7187 )
...
* fix(storage): harden ODM and scanner publication
* fix(app): simplify absent SSE configuration matching
* test(heal): settle PUT rename tails before disk-wipe fixtures
* fix(ecstore): remove duplicate local rename implementation
Keep the canonical commit module after concurrent storage changes merged.
The control-write and rollback changes are already present there.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(ci): satisfy new clippy lints
* style(scanner): order merged test imports
* fix(scanner): invalidate bucket work after namespace completion
* fix(scanner): fence cached snapshots by scan execution
---------
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: zhi22915 <qiuzgang@gmail.com >
2026-09-05 13:47:12 +00:00
447f3c704b
feat(heal): add explicit committed MRF snapshot reader ( #7179 )
...
* chore(deps): refresh SDKs and pin clock skew regression coverage
Refresh compatible dependencies for Scanner/Heal V2 batch 1 and verify
the production S3 retry/signing path with a deterministic clock.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* feat(heal): add explicit committed MRF snapshot reader
Refs rustfs/backlog#2263 and rustfs/backlog#2240 .
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* docs(heal): register legacy MRF inspection cleanup
State the compatibility removal condition on the source marker and in
the architecture cleanup register.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
---------
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: zhi22915 <qiuzgang@gmail.com >
2026-09-05 13:33:41 +00:00
d915f9565e
fix(ci): preserve reported functional suite failures ( #7199 )
2026-09-05 13:26:23 +00:00
55ad7508b9
fix(tier): persist coordinator intent before waking refresh ( #7171 )
2026-09-05 13:24:00 +00:00
33fd056000
fix(ecstore): release heal disk snapshot before nested reads ( #7189 )
...
* fix(ecstore): release heal disk snapshot before nested reads
* fix(ecstore): remove duplicate local rename implementation
Keep the canonical commit module after concurrent storage changes merged.
The control-write and rollback changes are already present there.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(app): simplify absent SSE configuration matching
* fix(tests): satisfy new clippy lints
---------
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: zhi22915 <qiuzgang@gmail.com >
2026-09-05 12:49:40 +00:00
af2e9df821
fix(lifecycle): correct expiration and transition evaluation ( #7169 )
2026-09-05 12:42:22 +00:00
0a92a7d98c
fix(tier): bound remote transition requests ( #7147 )
...
Co-authored-by: Zhengchao An <anzhengchao@gmail.com >
2026-09-05 12:41:11 +00:00
c589fd2439
fix(dev): install a lightweight formatting commit hook ( #7198 )
2026-09-05 12:22:56 +00:00
3e5d4ebb09
fix(ecstore): release multipart disk snapshot before nested reads ( #7184 )
...
* fix(ecstore): release multipart disk snapshot before nested reads
* fix(ecstore): remove duplicate local rename implementation
Keep the canonical commit module after concurrent storage changes merged.
The control-write and rollback changes are already present there.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(app): simplify absent SSE configuration matching
* fix(tests): satisfy new clippy lints
---------
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: zhi22915 <qiuzgang@gmail.com >
2026-09-05 12:20:50 +00:00
3677871468
chore(deps): bump zstd to 0.14 ( #7173 )
...
Signed-off-by: houseme <housemecn@gmail.com >
Co-authored-by: zhi22915 <qiuzgang@gmail.com >
2026-09-05 12:16:46 +00:00
cc1ec6b992
fix(ci): share quick checks and lint workflows ( #7194 )
...
* fix(ci): share quick checks and lint workflows
* fix(ci): install actionlint from its verified release
* fix(ci): reject dependencies on required quick checks
2026-09-05 12:03:59 +00:00
9e2545244c
fix(odm): bound empty pagination chains with staged tokens ( #7197 )
...
* fix(odm): add staged cross-request pagination progress budgets
* fix(ecstore): remove duplicate local rename implementation
Keep the canonical commit module after concurrent storage changes merged.
The control-write and rollback changes are already present there.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(app): simplify absent SSE configuration matching
* fix(tests): satisfy new clippy lints
---------
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: zhi22915 <qiuzgang@gmail.com >
2026-09-05 11:44:49 +00:00
f053862aad
docs: request concrete behavior evidence in pull requests ( #7196 )
2026-09-05 18:56:26 +08:00
e8a7f4bc4a
fix(ecstore): remove duplicate local rename implementation ( #7190 )
...
* fix(ecstore): remove duplicate local rename implementation
Keep the canonical commit module after concurrent storage changes merged.
The control-write and rollback changes are already present there.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(ci): satisfy new clippy lints
---------
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: zhi22915 <qiuzgang@gmail.com >
Co-authored-by: Zhengchao An <anzhengchao@gmail.com >
2026-09-05 10:44:01 +00:00
7ba5cd6888
chore(deps): refresh SDKs and verify clock skew behavior ( #7174 )
...
chore(deps): refresh SDKs and pin clock skew regression coverage
Refresh compatible dependencies for Scanner/Heal V2 batch 1 and verify
the production S3 retry/signing path with a deterministic clock.
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: zhi22915 <qiuzgang@gmail.com >
2026-09-05 09:06:52 +00:00
acfeef55ab
feat(scanner): add bounded incarnation-scoped ACK receiver ( #7182 )
...
* chore(deps): refresh SDKs and pin clock skew regression coverage
Refresh compatible dependencies for Scanner/Heal V2 batch 1 and verify
the production S3 retry/signing path with a deterministic clock.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* feat(scanner): add bounded incarnation-scoped ACK receiver
Refs rustfs/backlog#2265 and rustfs/backlog#2240 .
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
---------
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: zhi22915 <qiuzgang@gmail.com >
2026-09-05 08:54:55 +00:00
0d1b312673
fix(ci): require fresh successful scheduled validations ( #7192 )
2026-09-05 16:49:58 +08:00
42c32381b6
fix(scanner): make reset cleanup safely reentrant ( #7180 )
...
* chore(deps): refresh SDKs and pin clock skew regression coverage
Refresh compatible dependencies for Scanner/Heal V2 batch 1 and verify
the production S3 retry/signing path with a deterministic clock.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(scanner): make reset cleanup safely reentrant
Refs rustfs/backlog#2264 and rustfs/backlog#2240 .
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
---------
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: zhi22915 <qiuzgang@gmail.com >
2026-09-05 08:45:53 +00:00
e6bf2a4646
fix(admin): report partial background heal coverage ( #7178 )
...
* chore(deps): refresh SDKs and pin clock skew regression coverage
Refresh compatible dependencies for Scanner/Heal V2 batch 1 and verify
the production S3 retry/signing path with a deterministic clock.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(admin): report partial background heal coverage
Refs rustfs/backlog#2035 and rustfs/backlog#2240 .
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
---------
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: zhi22915 <qiuzgang@gmail.com >
2026-09-05 08:45:45 +00:00
cf9688898d
fix(heal): retain completed task progress ( #7177 )
...
* chore(deps): refresh SDKs and pin clock skew regression coverage
Refresh compatible dependencies for Scanner/Heal V2 batch 1 and verify
the production S3 retry/signing path with a deterministic clock.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* fix(heal): retain completed task progress
Refs rustfs/backlog#2262 and rustfs/backlog#2240 .
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
---------
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: zhi22915 <qiuzgang@gmail.com >
2026-09-05 08:39:26 +00:00
2d159635ed
feat(scanner): plan dirty bucket cache refreshes ( #7146 )
...
* feat(scanner): plan dirty bucket cache refreshes
* fix(scanner): route peer snapshot through storage boundary
---------
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com >
Co-authored-by: cxymds <cxymds@gmail.com >
Co-authored-by: Zhengchao An <anzhengchao@gmail.com >
2026-09-05 16:32:02 +08:00
2f02d1d2d8
fix(ci): preserve security suite failures and isolate reports ( #7188 )
2026-09-05 16:26:13 +08:00
8ae8fb7eea
fix(ecstore): drain control writes and preserve uncertain rollback ( #7163 )
...
* fix(ecstore): drain durable control-plane write tails
* fix(ecstore): retain PUT staging after incomplete rollback
* fix(ecstore): drain backfill checkpoint before confirmation
* fix(ecstore): retain per-disk rename rollback outcomes
* fix(ecstore): retain indeterminate rename recovery evidence
* test(ecstore): mark rollback fixtures as inline data
* test(ecstore): match sealed context fixture map type
* fix(ecstore): preserve known preflight rename rejections
* test(ecstore): cover observed rename outer failures
* test(ecstore): count decommission faults across retry restarts
2026-09-05 08:16:53 +00:00
bbd7b9ef17
fix(site-replication): bound and order outage recovery ( #7148 )
...
* fix(site-replication): wake retry drain after peer recovery
* fix(site-replication): replay configure after bucket make
* fix(site-replication): serialize retry replay state
* fix(site-replication): persist destructive retry intents
* fix(site-replication): bound retry recovery rounds
* fix(site-replication): keep recovery replay live
* fix(site-replication): preserve retry ordering
* fix(site-replication): bound retry coordination
* fix(site-replication): serialize topology replay
* fix(site-replication): fence distributed retry state
* fix(site-replication): bound outage retry drain
* fix(site-replication): drop unsafe delete retry intents
* fix(site-replication): order bucket mutation replay
* fix(site-replication): harden outage retry replay
* fix(site-replication): fence destructive peer delivery
* fix(site-replication): avoid peer edit retry deadlock
* fix(site-replication): fence retry error classification
* fix(site-replication): classify connect timeouts
* fix(site-replication): close recovery review races
* test(site-replication): cover timeout endpoint text
* fix(site-replication): close destructive recovery gaps
* fix(site-replication): fence recovery revisions
* fix(site-replication): replay bucket metadata on recovery
* fix(site-replication): preserve s3gate boundary
---------
Co-authored-by: overtrue <anzhengchao@gmail.com >
2026-09-05 15:55:54 +08:00
15e9bc5ed0
test(ecstore): count injected faults across retry restarts ( #7170 )
...
test(ecstore): count decommission faults across retry restarts
2026-09-05 07:54:56 +00:00
882d9ca8a4
refactor(ecstore): isolate metadata quorum decisions ( #7165 )
...
* refactor(ecstore): isolate metadata quorum decisions
* test(ecstore): match sealed context fixture map type
* test(ecstore): count decommission faults across retry restarts
2026-09-05 07:46:13 +00:00
19a29a7027
test(s3): add Snowball tar-codec compatibility fixtures ( #7157 )
...
Co-authored-by: Zhengchao An <anzhengchao@gmail.com >
2026-09-05 07:41:48 +00:00
2e4ab045b6
test(scanner): add durable checkpoint diagnostics ( #7175 )
...
* chore(deps): refresh SDKs and pin clock skew regression coverage
Refresh compatible dependencies for Scanner/Heal V2 batch 1 and verify
the production S3 retry/signing path with a deterministic clock.
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
* test(scanner): add durable checkpoint diagnostics
Refs rustfs/backlog#2260 and rustfs/backlog#2240 .
Co-Authored-By: heihutu <heihutu@gmail.com >
Co-Authored-By: zhi22915 <qiuzgang@gmail.com >
---------
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: zhi22915 <qiuzgang@gmail.com >
2026-09-05 07:34:29 +00:00
cbfd5b92f4
refactor(ecstore): isolate local object rename commit ( #7166 )
...
* fix(ecstore): drain durable control-plane write tails
* fix(ecstore): retain PUT staging after incomplete rollback
* fix(ecstore): drain backfill checkpoint before confirmation
* refactor(ecstore): isolate local object rename commit
* refactor(ecstore): remove moved quota fence import
* fix(ecstore): retain per-disk rename rollback outcomes
* fix(ecstore): retain indeterminate rename recovery evidence
* test(ecstore): mark rollback fixtures as inline data
* test(ecstore): match sealed context fixture map type
* test(ecstore): match sealed context fixture map type
* fix(ecstore): preserve known preflight rename rejections
* test(ecstore): cover observed rename outer failures
* test(ecstore): count decommission faults across retry restarts
2026-09-05 07:19:29 +00:00
971f9acdf4
fix(ecstore): reject stalled ODM pagination before merging pages ( #7164 )
...
* fix(odm): reject non-progressing listing cursors
* docs(odm): clarify folded source probe pagination
* test(odm): match SDK bucket-root listing requests
* test(ecstore): match sealed context fixture map type
* test(odm): use app facade for listing wire types
* fix(odm): resolve pagination Clippy failures
2026-09-05 07:17:25 +00:00
a6589c19e3
chore(tier): remove stage-a blanket lint allowances ( #7153 )
...
Co-authored-by: Zhengchao An <anzhengchao@gmail.com >
2026-09-05 06:43:05 +00:00
0885c721fe
ci(upgrade): support manual runs between any two release versions ( #7145 )
...
The workflow_dispatch inputs already accept arbitrary release tags, but
the run failed late and unclearly when a tag had no .deb asset, and the
from_version default pointed at 1.0.0-rc.4-preview.1, whose release
ships no .deb at all - so scheduled runs died on a 404 while installing
the old package.
- Add a fail-fast preflight that resolves each requested tag via the
GitHub release API and verifies the rustfs_<tag>_amd64.deb asset
exists before the suite starts, with an actionable error message
otherwise (e.g. 1.0.0-rc.4 ships only zip/sbom assets).
- Change the from_version default to 1.0.0-rc.3, the newest release
that actually ships a .deb asset.
- Reword the from_version/to_version descriptions so manual triggers
state the .deb-asset requirement and the nightly fallback.
- Pass PF_TESTING_GH_TOKEN as GH_TOKEN to the suite step for the gh api
release lookups, matching the other functional workflows.
Co-authored-by: Zhengchao An <anzhengchao@gmail.com >
2026-09-05 06:39:47 +00:00
eaf5159d0f
ci: refresh Linux full E2E membership after test additions ( #7156 )
...
Co-authored-by: cxymds <cxymds@gmail.com >
2026-09-05 06:27:59 +00:00
2477e31059
test(ecstore): require core regressions in the existing CI lane ( #7162 )
...
* test(ecstore): require core invariant tests in existing CI lane
* test(ci): require a fresh core JUnit report
* test(ecstore): match sealed context fixture map type
2026-09-05 06:22:48 +00:00
d8c3b1bb26
fix(app): fail closed on an unreadable bucket encryption config ( #7183 )
...
The object write path read the bucket default encryption configuration
with `.ok()`, which made "this bucket has no default encryption" and "the
encryption configuration cannot be read" the same value. A bucket whose
encryption blob is damaged therefore stored plaintext objects the
operator had mandated be encrypted, with nothing returned to the client
and nothing in the object to tell those writes apart afterwards.
PUT, COPY and the snowball extract path now share one resolver: an
absent configuration still writes plaintext exactly as before, and every
other outcome refuses the write, carrying the accessor's typed error so
a damaged blob surfaces as a deterministic InternalError while a
transient metadata read failure surfaces as the retryable
ServiceUnavailable. A missing bucket and a cold metadata cache both
still resolve to "no configuration", so neither becomes a refusal. This
matches `prepare_sse_configuration` in `storage::sse`, the resolver the
multipart writer has always used, which fails closed on this lookup.
2026-09-05 14:13:59 +08:00
a3b8183be9
test(ecstore): narrow barrier re-export cfgs ( #7152 )
...
Co-authored-by: Zhengchao An <anzhengchao@gmail.com >
2026-09-05 06:12:45 +00:00
4dbc58887a
fix(tier): probe legacy transition version state ( #7138 )
2026-09-05 06:00:14 +00:00
123967e729
fix(ecstore): fail closed on an unreadable bucket-targets blob ( #7172 )
...
* fix(ecstore): correct sealed-credential test helper parameter type
The helper took a HashMap that nothing imports, so the ecstore test target did not compile.
* fix(ecstore): fail closed on an unreadable bucket-targets blob
An undecodable bucket-targets.json was replaced by an empty BucketTargets,
so every replication target of that bucket disappeared, replication stopped,
and no caller saw an error. A missing secretKey alone triggers it, because
Credentials has no struct-level serde(default).
parse_all_configs now retains the failure instead: the raw bytes stay and the
typed field stays None, which BucketMetadata::bucket_targets_unreadable reads
as "exists but cannot be read" — the same distinction the fabricated marker
draws for bucket metadata as a whole. One corrupt sub-config still never fails
the metadata load, so an unreadable bucket cannot take down its neighbours or
the node.
BucketTargetSys records such buckets and answers every targets query with the
new BucketRemoteTargetsUnreadable, leaving any snapshot from an earlier
readable load in place so in-flight replication is not torn down. The
replication heal queue reports Missed rather than scheduling against an empty
target set, and the admin listing surfaces the fault instead of an empty list.
Refs: rustfs/backlog#2282
* fix(ecstore): report corrupt permissive bucket configs as invalid
Audit of the remaining parse_all_configs branches. Policy, versioning, object
lock and replication already fail closed at their accessors; encryption,
public access block and quota did not, and for those three "absent" is exactly
the state that grants something — plaintext storage, anonymous access,
unbounded capacity. They now report a stored-but-undecodable payload as
invalid rather than as ConfigNotFound, matching the guard the versioning and
object-lock accessors already use. The quota enforcement path already refused
such a payload; only the metadata read path was misreporting it.
The branches left degrading, and the concrete reason each is safe, are
recorded in the table on parse_all_configs.
Refs: rustfs/backlog#2282
2026-09-05 13:02:23 +08:00
1277c11f04
Merge branch 'main' into test/distributed-e2e-hardening
2026-09-05 11:57:33 +08:00
4b0d597d4d
test(ecstore): fix sealed context fixture map type ( #7161 )
...
test(ecstore): match sealed context fixture map type
2026-09-05 11:54:24 +08:00
13e6424e99
docs(architecture): settle remote credential sealing threat model ( #7168 )
...
* docs(architecture): settle remote credential sealing threat model
* docs(architecture): index sealing ADR threat-model scope
2026-09-05 11:20:58 +08:00
b33693fc19
feat(tier): fence legacy state reconciliation ( #7144 )
2026-09-05 01:42:24 +00:00
bdb069c0fd
Merge branch 'main' into test/distributed-e2e-hardening
2026-09-05 09:29:28 +08:00
9ed1d46090
docs(ecstore): define generation authority and recovery boundary ( #7159 )
2026-09-05 09:29:00 +08:00
b65ab747d3
Merge branch 'main' into test/distributed-e2e-hardening
2026-09-05 09:17:48 +08:00
6eb60f8e72
feat(tier): add durable probe intent protocol ( #7151 )
...
* feat(tier): add durable probe intent protocol
* test(tier): remove redundant intent clones
2026-09-05 09:06:05 +08:00
8dd3cabd41
test(ecstore): stabilize transition generation fixture ( #7140 )
2026-09-05 00:46:19 +00:00
e648f683bf
fix: use BTreeMap for deterministic encryption context serialization ( #7154 )
...
SealScope::encryption_context() returned a HashMap whose key order is
non-deterministic. The FakeSealer test round-trips the context through
JSON serialization, and HashMap's random iteration order caused the
prefix comparison to intermittently fail with 'encryption context mismatch'.
Switch to BTreeMap which guarantees stable key ordering.
2026-09-05 07:21:25 +08:00
Zhengchao An