Merging stored site-replication rules into a PutBucketReplication body
renumbered every rule 1..n in list order, rewriting the submitted policy:
overlapping same-target rules submitted as priority 5 then 1 became 1
then 2, so the delete-marker-disabled rule won the replication decision.
The reconciler and the peer-removal prune renumbered the same way.
Operator priorities now stay verbatim everywhere; only the reconciler's
derived rules move, to the lowest priorities no operator rule uses, via
one pure helper shared by the S3 edit merge, the peer ingestion merge,
the reconciler pass and the prune. Being a pure function of the rule
list it is idempotent, so the reconciler's no-op check still holds after
a merged write, and an on-disk config in the historical layout (operator
rules 1..k, site rules k+1..n) yields the same bytes, so nothing is
rewritten on upgrade.
The `site-repl-*` prefix alone classified any rule as site-owned, so on a
bucket outside site replication an owner's `site-repl-user` rule survived
DeleteBucketReplication (rule and target kept, success returned). Rule ids
do not reserve that namespace.
A rule is reconciler-owned only when it matches what the reconciler
derives: id `site-repl-<deployment id>` for a current remote site
replication peer and a destination ARN naming that same deployment id.
The S3 put/delete path reads the remote peer set (empty when site
replication is disabled) and keeps exactly those rules; everything else
is operator state the request replaces or deletes. An incoming rule that
claims a current peer's id is dropped so the reconciler rule's id stays
unique. The peer ingestion path and the reconciler keep their prefix
predicate unchanged.
* refactor(e2e/kms): replace fixed startup sleeps with KMS readiness probe
Replace 33 hard-coded sleep(3s) / sleep(2s) startup waits in KMS e2e tests
with an active readiness probe (wait_for_kms_ready) that polls the KMS
status endpoint with exponential backoff (200ms→1s, 5s budget).
This cuts per-test startup latency from a fixed 3s to ~200-500ms while
remaining robust against slow CI machines.
Non-startup sleeps (ILM polling loops, fault-recovery detection delays,
test-runner inter-test pauses) are left untouched.
* style: cargo fmt
* fix(kms): use .expect() instead of ? in test functions that return ()
7 call sites of wait_for_kms_ready() used ? in async test functions
that return () instead of Result. Changed to .expect("KMS ready").
* fix(kms): enforce readiness probe deadline
* fix(kms): validate readiness backend status
The inline_block threshold used floor division (DEFAULT_INLINE_OBJECT_BUDGET
/ data_shards) while shard_file_size uses ceiling division (div_ceil). For
EC 12:4 with 256KiB objects, this caused a 1-byte discrepancy:
- inline_block = 262144 / 12 = 21845 (floor)
- shard_file_size = 262144.div_ceil(12) = 21846 (ceil)
- should_inline(21846, 12, false) = false (wrong!)
Fix by using div_ceil for the inline_block calculation, so both sides
use the same rounding and the inline path is correctly triggered.
Co-authored-by: heihutu <heihutu@gmail.com>
* refactor(data-usage): ReplicationStats -> ReplicationTargetUsage
Rename the data-usage crate's ReplicationStats to ReplicationTargetUsage.
Serde field names are byte-identical (only the Rust type name changed;
field identifiers that rmp encodes are untouched). An rmp round-trip test
guards against future drift.
Scanner test imports updated to match.
* style: cargo fmt
The read-proxy selector already honors a target's disable_proxy flag
(PR #6172), but the admin API still rejected the field, so the only way
to set it was importing a MinIO-written bucket-targets.json.
- move disableProxy from REMOTE_TARGET_UNSUPPORTED_FIELDS to
REMOTE_TARGET_WRITABLE_FIELDS (set-remote-target create accepts it)
- add TargetUpdateOp::Proxy so set-remote-target?update=true&proxy=true
overlays only the proxy group (MinIO TargetUpdateType parity)
- bump REMOTE_TARGET_CAPABILITY_CONTRACT_VERSION 1 -> 2 and update the
runtime capability pin tests
- keep edge/edgeSyncBeforeExpiry rejected (no implementation behind them)
- pin that a published TargetClient carries disable_proxy, the field the
proxy-target selector consults
Refs rustfs/backlog#1950
chore(madmin): remove dead trace structs, keep TraceType bitflag helper only
TraceInfo, TraceInfoLegacy, TraceHTTPStats, TraceCallStats, TraceRequestInfo,
TraceResponseInfo, StorageStats, and OSStats are unreferenced outside trace.rs.
Trim to TraceType + its bitflag operations which are actively used by
service_commands.rs and profile_admin.rs.
-139 lines (215 -> 76 lines)
* chore(obs): ReplicationStats -> ReplicationMetricsSnapshot, BucketReplicationStats -> BucketReplicationMetricsSnapshot
Rename in-obs-crate ReplicationStats and BucketReplicationStats to
ReplicationMetricsSnapshot and BucketReplicationMetricsSnapshot respectively.
No serde impact (these types are Prometheus metric collectors, not serialized).
No external consumers found outside the obs crate.
* cleanup: remove #[serial] annotations from e2e_test, scanner, lifecycle, and object-capacity crates
Remove no-op #[serial] attributes (nextest ignores serial_test) and the
serial_test dependency from four crates. All tests already use temp_env
for env-var isolation, making #[serial] purely redundant.
Crates cleaned:
- e2e_test (37 annotations, 9 imports, removed serial_test dep)
- rustfs-scanner (115 annotations across 7 files, removed serial_test dep)
- rustfs-lifecycle (46 annotations, removed serial_test dep)
- rustfs-object-capacity (38 annotations, removed serial_test dep)
Also converted scanner/tests/lifecycle_integration_test.rs
with_forced_immediate_enqueue_timeout helper from unsafe raw
env::set_var/remove_var to temp_env::async_with_vars for proper
isolation, and added async_closure feature to scanner's temp-env dep.
* fix(lifecycle): restore #[serial] on 2 tests that read env vars without temp_env
eval_inner_expires_latest_object_after_days_due and
eval_inner_does_not_panic_on_many_equal_due_events call eval_inner()
which reads ENV_ILM_PROCESS_TIME via std::env::var(). Without #[serial]
they race with other tests that set these vars via temp_env.
* style: cargo fmt
Under site replication a user holding only bucket-scoped
s3:PutReplicationConfiguration could rewrite or erase the operator-managed
site-repl-* rules, with the change broadcast to every peer (backlog#1948,
audit A1/P2-17).
- Gate PutBucketReplication/DeleteBucketReplication in the S3 handlers:
when site replication is enabled and the requester is not the owner,
return MinIO-parity XMinioReplicationDenyEdit (HTTP 400). The gate runs
after policy authorization and only on the external S3 path; the
reconciler and peer bucket-meta ingestion are unaffected.
- Defense in depth in the bucket usecase: PUT merges the incoming config
with the stored site-repl-* rules (same merge as peer ingestion) instead
of overwriting verbatim; DELETE keeps the site-repl-* rules and never
garbage-collects a bucket target a surviving site-replication rule still
references.
- Move is_site_replication_rule / merge_incoming_replication_config /
replication_target_arn_deployment_id from the admin site-replication
handler down to rustfs-replication so the app layer can reuse them
without new layering violations.
* fix(scanner): defer usage publication during pool recovery
* fix(scanner): preserve metrics when publication is deferred
* fix(scanner): route test types through storage boundary
* fix(scanner): keep cache floor deferred during movement
houseme