gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-09-06 20:19:14 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(odm): preserve progress when list-through is disabled (#7278)

Browse Source 
* test(odm): cover disabled list-through continuation progress

* test(odm): cover literal cache tags in disabled list cursors

* fix(odm): preserve progress when list-through is disabled
This commit is contained in:
Zhengchao An
2026-09-06 15:24:59 +08:00
committed by GitHub
parent c99efd9477
commit a40ec8a6f3
3 changed files with 385 additions and 174 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/operations/on-demand-migration.md
+2
View File
@@ -28,6 +28,8 @@ The two rollout switches have different defaults. Unset or invalid boolean value
- `RUSTFS_ON_DEMAND_MIGRATION_LIST_V2_TOKENS` defaults to `false` and allows a v1 listing to first issue a v2 token after an empty truncated merged page. Existing v2 tokens keep their budget even on reader-only nodes. Consuming an object/common prefix or reaching a new EOF resets the budget to v1 without changing the chain's framing.
- `RUSTFS_ON_DEMAND_MIGRATION_LIST_FRAMED_TOKENS` defaults to `true`, preserving the framed output of the #7187 / `e1608fbd9` generation. It allows a bare/new merged listing to first issue a NUL-prefixed JSON envelope inside the existing base64 encoding. Existing framed chains stay framed even with this switch off, including a reset to v1 and local continuation after list-through is disabled. With framing issuance off, new bare v1 output keeps its historical bytes; ordinary local listings remain unchanged. This switch does not enable the v2 budget.
When list-through or the global module is disabled, existing merged continuations retain their last emitted key and original bare/framed format, filter already returned objects and common prefixes, and make no source requests. If the local-only response remains truncated, its continuation can resume both sides after re-enabling list-through.
Choose the upgrade path from the binaries currently serving LIST requests, including every load-balancer route. This build reads complete historical bare envelopes and framed v1/v2 envelopes with the same strict version/count validation. There is no single writer format understood by both bare-only and framed-only readers. A bare-v1-only binary rejects bare v2 with `400 InvalidArgument`; a bare-only reader mistakes framed input for a local marker, while a framed-only reader mistakes bare input for one. These framing mismatches can restart a merged scan and lose its budget without returning an error. For example, with local keys `b,d` and source keys `a,c`, a new node issuing bare after `a` followed by an `e1608fbd9` reader can return `a` again. That old reader then emits framing, so the symptom need not be an infinite loop.
- **Upgrading from the framed-only #7187 / `e1608fbd9` generation:** before deploying the first new node, set `RUSTFS_ON_DEMAND_MIGRATION_LIST_FRAMED_TOKENS=true` in the new nodes' deployment environment, or leave it unset to use this build's `true` default. Remove any previous explicit `false` override. Existing framed-only nodes ignore this new variable and already emit framing. Keep framing enabled while any of those nodes serves continuations. While list-through remains active, new and existing framed v1/v2 chains then retain their cursors and any active budget in both directions. Only after all serving nodes are dual readers may you choose `false`; existing framed chains still stay framed, while newly started bare chains must stay on dual readers.