SSE and compression wrap the payload so its length is unknown and
advertise HashReader::SIZE_PRESERVE_LAYER (-1). Every layer preserved
that sentinel except create_bitrot_writer, which clamped it to 0 before
calling DiskAPI::create_file. RemoteDisk forwards that size verbatim in
the put_file_stream query, so remote peers were told the body was empty.
Since the authenticated put-file trailer (#5868) the receiver used the
declared size to split body from trailer, turning the clamp into a fatal
"auth trailer has trailing data" failure for every SSE PUT on multi-node
deployments (rc.2). #6320 relaxed the receiver to only trust size > 0;
this change fixes the sender so the sentinel survives end to end and the
wire no longer conflates empty objects with unknown-length streams.
Refs #6331
Decoded tier free versions always carry both the on-disk
free-version suffix and the in-memory tier marker, and real
decommission targets already contain the bucket volume. Mirror both
invariants in the migration regression fixtures so write-path metadata
validation and rename publication behave exactly as in production.
* refactor(e2e/kms): replace fixed startup sleeps with KMS readiness probe
Replace 33 hard-coded sleep(3s) / sleep(2s) startup waits in KMS e2e tests
with an active readiness probe (wait_for_kms_ready) that polls the KMS
status endpoint with exponential backoff (200ms→1s, 5s budget).
This cuts per-test startup latency from a fixed 3s to ~200-500ms while
remaining robust against slow CI machines.
Non-startup sleeps (ILM polling loops, fault-recovery detection delays,
test-runner inter-test pauses) are left untouched.
* style: cargo fmt
* fix(kms): use .expect() instead of ? in test functions that return ()
7 call sites of wait_for_kms_ready() used ? in async test functions
that return () instead of Result. Changed to .expect("KMS ready").
* fix(kms): enforce readiness probe deadline
* fix(kms): validate readiness backend status
The inline_block threshold used floor division (DEFAULT_INLINE_OBJECT_BUDGET
/ data_shards) while shard_file_size uses ceiling division (div_ceil). For
EC 12:4 with 256KiB objects, this caused a 1-byte discrepancy:
- inline_block = 262144 / 12 = 21845 (floor)
- shard_file_size = 262144.div_ceil(12) = 21846 (ceil)
- should_inline(21846, 12, false) = false (wrong!)
Fix by using div_ceil for the inline_block calculation, so both sides
use the same rounding and the inline path is correctly triggered.
Co-authored-by: heihutu <heihutu@gmail.com>
* refactor(data-usage): ReplicationStats -> ReplicationTargetUsage
Rename the data-usage crate's ReplicationStats to ReplicationTargetUsage.
Serde field names are byte-identical (only the Rust type name changed;
field identifiers that rmp encodes are untouched). An rmp round-trip test
guards against future drift.
Scanner test imports updated to match.
* style: cargo fmt
The read-proxy selector already honors a target's disable_proxy flag
(PR #6172), but the admin API still rejected the field, so the only way
to set it was importing a MinIO-written bucket-targets.json.
- move disableProxy from REMOTE_TARGET_UNSUPPORTED_FIELDS to
REMOTE_TARGET_WRITABLE_FIELDS (set-remote-target create accepts it)
- add TargetUpdateOp::Proxy so set-remote-target?update=true&proxy=true
overlays only the proxy group (MinIO TargetUpdateType parity)
- bump REMOTE_TARGET_CAPABILITY_CONTRACT_VERSION 1 -> 2 and update the
runtime capability pin tests
- keep edge/edgeSyncBeforeExpiry rejected (no implementation behind them)
- pin that a published TargetClient carries disable_proxy, the field the
proxy-target selector consults
Refs rustfs/backlog#1950
Tier free versions (xl.meta cleanup records for deleted transitioned
versions) are not migrated as free versions during decommission: the
exact inventory keeps them inline in versions and the migration loop
routes them through the generic delete-marker path, dropping the flag
and remote-tier identity. Reference audit across GET, heal, ILM,
transition, replication, and restore found no cluster-local consumer
that resolves a free version after decommission; on user-facing delete
paths the remote-delete obligation is also carried by a committed
tier-journal entry, leaving only journal-less records (transition
state unknown) exposed to remote orphaning.
- count and log skipped free versions per decommission entry with
disposition reason tier_free_version_not_migrated instead of
omitting them silently
- document free-version lifecycle, non-migration invariant, allowed
physical-delete timing, and the reference-audit result in
docs/architecture/decommission-compatibility.md
- state the invariant in doc comments at the filemeta free-version
sites
- guard the accounting with
decommission_free_version_accounting_reports_skipped_records
Closesrustfs/backlog#1923
chore(madmin): remove dead trace structs, keep TraceType bitflag helper only
TraceInfo, TraceInfoLegacy, TraceHTTPStats, TraceCallStats, TraceRequestInfo,
TraceResponseInfo, StorageStats, and OSStats are unreferenced outside trace.rs.
Trim to TraceType + its bitflag operations which are actively used by
service_commands.rs and profile_admin.rs.
-139 lines (215 -> 76 lines)
* chore(obs): ReplicationStats -> ReplicationMetricsSnapshot, BucketReplicationStats -> BucketReplicationMetricsSnapshot
Rename in-obs-crate ReplicationStats and BucketReplicationStats to
ReplicationMetricsSnapshot and BucketReplicationMetricsSnapshot respectively.
No serde impact (these types are Prometheus metric collectors, not serialized).
No external consumers found outside the obs crate.
* cleanup: remove #[serial] annotations from e2e_test, scanner, lifecycle, and object-capacity crates
Remove no-op #[serial] attributes (nextest ignores serial_test) and the
serial_test dependency from four crates. All tests already use temp_env
for env-var isolation, making #[serial] purely redundant.
Crates cleaned:
- e2e_test (37 annotations, 9 imports, removed serial_test dep)
- rustfs-scanner (115 annotations across 7 files, removed serial_test dep)
- rustfs-lifecycle (46 annotations, removed serial_test dep)
- rustfs-object-capacity (38 annotations, removed serial_test dep)
Also converted scanner/tests/lifecycle_integration_test.rs
with_forced_immediate_enqueue_timeout helper from unsafe raw
env::set_var/remove_var to temp_env::async_with_vars for proper
isolation, and added async_closure feature to scanner's temp-env dep.
* fix(lifecycle): restore #[serial] on 2 tests that read env vars without temp_env
eval_inner_expires_latest_object_after_days_due and
eval_inner_does_not_panic_on_many_equal_due_events call eval_inner()
which reads ENV_ILM_PROCESS_TIME via std::env::var(). Without #[serial]
they race with other tests that set these vars via temp_env.
* style: cargo fmt
* fix(scanner): defer usage publication during pool recovery
* fix(scanner): preserve metrics when publication is deferred
* fix(scanner): route test types through storage boundary
* fix(scanner): keep cache floor deferred during movement
houseme