gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-09-01 17:58:22 +00:00
Code Issues Packages Projects Releases Wiki Activity
6,162 Commits 48 Branches 130 Tags
896781a52bb98d562c9426d685117c3e9380e746
Commit Graph

6162 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
hector 896781a52b feat(ci): add upgrade compatibility suite and reorder functional chain (#6950) 
New RustFS Upgrade Test workflow (SUITE: upgrade) runs first in the
nightly functional chain:

- Nightly GNU Build -> Upgrade -> S3 -> KMS -> Tier -> Pool/Heal -> Security
- S3 compatibility now triggers on "RustFS Upgrade Test" completion, so an
  upgrade regression gates the rest of the chain.
- Security suite moves to the end, after pool/heal, on the shared VMs.
- The upgrade suite drives auto-testing's rustfs-upgrade-test.sh
  (UPG-101..402): seed golden data/identity/config on the OLD deb, upgrade
  in place to the NEW deb, verify byte-identical preservation, and publish
  functional-reports/upgrade/<date>.md.
- Add the Upgrade tab to every dashboard index writer so the shared
  functional/index.html stays consistent.
 2026-08-31 19:32:07 +08:00
hector 612dd38fea ci(kms): add enforcement/frame/config-secret lane inputs (#6946) 
New backlog#2024 KMS supplements (KMS-106..502) are gated behind node env
flags. Add workflow_dispatch inputs that append the corresponding
KEY=VALUE lines to /etc/default/rustfs via the suite's --extra-env option:

- enforce_sse_key_policy -> RUSTFS_KMS_ENFORCE_SSE_KEY_POLICY (KMS-401/402)
- frame_v2               -> RUSTFS_ENCRYPTION_FRAME_V2 (KMS-318)
- config_secret          -> RUSTFS_KMS_CONFIG_SECRET (KMS-107)

Nightly runs keep the default local+vault-kv2 lane unchanged.
 2026-08-31 19:31:51 +08:00
cxymds ff28b79088 fix(s3): harden Snowball archive extraction (#6942) 
* fix(s3): harden Snowball extract error boundaries

* fix(s3): close Snowball extract compatibility gaps

* fix(s3): verify Snowball request body completion

* test(s3): reject forged Snowball streaming signatures

* build(deps): pin Snowball archive parser limits

* fix(s3): preserve Snowball trailer and member errors

* docs(architecture): register Snowball tar fork cleanup

* refactor(s3): route Snowball errors through object boundary

* ci(deps): allow pinned tokio-tar source

* ci(e2e): refresh Snowball smoke selection
 2026-08-31 11:18:09 +00:00
唐小鸭 35456bcede test(scanner): serialize tests sharing process-global scanner state (#6940) 
Under the cargo test fallback (threads in one process), tests that touch
the process-global scanner cycle recovery status or the global usage-save
metrics raced each other and failed randomly in full-suite runs.

Mark all touchers with #[serial] per docs/testing/README.md:
- 22 tests reading or writing scanner_cycle_recovery_status() via
  load_scanner_cycle_state_for_startup / reset_scanner_cycle_recovery
- 24 tests mutating global_metrics() usage-save counters via
  store_data_usage_in_backend*, which raced the existing serial
  test_deferred_usage_save_keeps_last_real_save_metric

No-op under nextest, which isolates each test in its own process.
 2026-08-31 18:20:25 +08:00
Zhengchao An 9d4ccb7884 fix(ecstore): finalize decommission capacity recovery (#6955) 2026-08-31 18:09:09 +08:00
Zhengchao An 9a22cb85f3 fix(ecstore): complete decommission capacity recovery (#6949) 2026-08-31 16:53:14 +08:00
Zhengchao An 6c67086d0b fix(ecstore): reserve decommission capacity safely (#6917) 2026-08-31 15:20:09 +08:00
hector ea01cd339c fix(ci): continue functional chain and publish heal/pool reports (#6932) 2026-08-31 15:19:46 +08:00
houseme bb37841362 chore(capacity): update default refresh tuning (#6938) 
Align object-capacity refresh defaults with the production-oriented environment values and keep docs, script examples, and tests in sync.

Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-31 14:43:59 +08:00
GatewayJ 59a7194d7f feat(s3select): schedule streaming progress events (#6913) 
* feat(s3select): schedule streaming progress events

* test(s3select): poll permit release until timeout
 2026-08-31 13:36:47 +08:00
GatewayJ f647ada320 feat(table-catalog): update object namespace properties (#6815) 
* feat(table-catalog): update object namespace properties

* test(table-catalog): cover object namespace properties

* fix(ci): use admin storage contract in catalog test

* fix(table-catalog): reject corrupt namespace properties
 2026-08-31 13:36:23 +08:00
GatewayJ 589a954478 feat(s3select): support compressed CSV and JSON input (#6915) 2026-08-31 13:35:59 +08:00
houseme 1d606e1cf6 perf(ecstore): retry degraded GET with late parity (#6933) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-31 13:32:46 +08:00
Zhengchao An dc2e25b48c fix(admin): preserve raw XML in metadata backups (#6936) 2026-08-31 13:15:35 +08:00
houseme d690f5d60d test(ecstore): stabilize tier recovery cursor fixture (#6935) 2026-08-31 12:09:19 +08:00
houseme 3eca80e37d test(ecstore): make heal rename fixture deterministic (#6934) 2026-08-31 12:09:01 +08:00
houseme 45a2ccb734 fix(ecstore): recover late parity after exact quorum (#6927) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-31 03:26:20 +00:00
houseme c876df53f5 fix(ecstore): fence snapshot stream polls on lock loss (#6930) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-31 02:26:24 +00:00
Zhengchao An 769da6d81f test(admin): prove backup import rollback compatibility (#6929) 2026-08-31 02:19:37 +00:00
Zhengchao An ca46ae9e56 test(ecstore): pin bucket metadata rollback reads (#6928) 2026-08-31 01:48:03 +00:00
Zhengchao An 7df0920c80 test(replication): bind writable paths to DTO fields (#6923) 2026-08-31 00:48:10 +00:00
Zhengchao An c4ac11d22e fix(scanner): persist decommission catch-up debt (#6922) 2026-08-31 08:45:36 +08:00
houseme 602ed2cbcd test(ecstore): add targeted refresh-loss harness (#6924) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-31 08:45:04 +08:00
hector b6c3108e53 fix(ci): keep functional workflow chain running after failures (#6926) 
* fix(ci): isolate s3 compat temp file paths

* fix(ci): use rooted auto-testing s3 temp fix

* fix(ci): follow auto-testing main after temp-path merge

* fix(ci): stabilize tier mqtt bootstrap on shared runner

* feat(ci): publish functional reports and keep workflows non-blocking

* fix(ci): keep functional chain running after failures

* fix(ci): standardize functional workflow cleanup steps
 2026-08-31 08:43:28 +08:00
houseme 8ecd8f2520 fix(scanner): preserve cache cycle during usage recovery (#6921) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-31 00:19:54 +00:00
Zhengchao An e6234d3714 test(ecstore): pin default bucket config bytes (#6920) 2026-08-31 00:03:07 +00:00
Zhengchao An 042a0c3014 docs: register persisted XML compatibility cleanup (#6918) 
docs: register persisted XML compatibility
 2026-08-30 23:53:02 +00:00
houseme 87333f7b24 test(e2e): exercise cluster volume fault proxy (#6919) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 23:37:34 +00:00
Zhengchao An 9945c67f7e fix(ecstore): supervise decommission worker recovery (#6908) 2026-08-31 06:18:00 +08:00
houseme fca1514aac fix(scanner): recover legacy empty usage floor (#6914) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-31 06:17:26 +08:00
houseme 47ad69b691 fix(ecstore): fail closed on unverifiable data quorum (#6903) 
fix(ecstore): require verification source for degraded GET

Fail closed when reconstruction has only an exact decode quorum, because no surplus source remains to validate the rebuilt data. Cover both erasure engines and the data-shards-only rollout gate.

Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 21:07:47 +00:00
houseme 489408c0b0 perf(ecstore): reuse prepared Select metadata (#6911) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 20:41:24 +00:00
houseme 1b3744a1da test(perf): align GET attribution harness with backlog 2093 (#6912) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 20:28:28 +00:00
houseme 9244eb36ed test(e2e): route cluster volume endpoints through fault proxy (#6909) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 20:17:16 +00:00
houseme 442298d5f7 test(ecstore): prove in-flight prefetch cancellation (#6904) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 19:17:44 +00:00
houseme be7d35d441 perf(get): release disk permits for buffered bodies (#6906) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 19:10:26 +00:00
唐小鸭 ec1cd606d3 fix(replication): surface object-lock denied purges and back off heal retries (#6900) 2026-08-30 18:59:55 +00:00
houseme 16af688a7a fix(rpc): reject unsigned v2 control mutations (#6905) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 18:17:43 +00:00
唐小鸭 37b23a16da fix(replication): verify replica integrity and default to plain signed payloads (#6895) 2026-08-31 01:43:45 +08:00
houseme 006e9b7d28 test(e2e): cover four-node four-drive cluster topology (#6902) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 17:32:36 +00:00
houseme d214c27583 perf(ecstore): consolidate non-inline read planning (#6892) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 17:15:48 +00:00
GatewayJ 8fd364a99c feat(s3-tables): support object-backed table rename (#6899) 2026-08-31 00:30:49 +08:00
houseme c2d8488728 docs(architecture): reconcile generation contract (#6901) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-31 00:20:39 +08:00
唐小鸭 1370434f3a fix(scanner): unblock quota usage baseline on never-converged sites (#6896) 2026-08-31 00:20:04 +08:00
唐小鸭 5dde2c188c fix(replication): retry failed multipart aborts on bounded backoff (#6897) 2026-08-31 00:19:49 +08:00
houseme 2f9c75d04f perf(ecstore): reuse prepared metadata across pools (#6889) 
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 16:15:10 +00:00
唐小鸭 9ee7b1221d fix(admin): replicate user secret-key rotation to peer sites (#6893) 2026-08-30 23:32:07 +08:00
Zhengchao An fcc3c7fb6b test(s3): promote passing compatibility cases (#6891) 2026-08-30 21:56:17 +08:00
Zhengchao An 01dc55ee5b docs(security): add unsigned presign header lesson (#6894) 2026-08-30 21:55:47 +08:00
houseme 3d24526704 fix(ecstore): preserve parity reserves for data-only GET (#6888) 
fix(ecstore): hedge data-only GET with parity

Route the opt-in data-shards-only lockstep path through the bounded parity race and preserve deferred parity reserves across canceled hedges.

Co-authored-by: heihutu <heihutu@gmail.com>
 2026-08-30 20:16:32 +08:00