mirror of https://github.com/rustfs/rustfs.git synced 2026-09-07 12:35:54 +00:00

Files

T

Zhengchao An 6655272c90 fix(ilm): reject invalid retention counts and validate lifecycle filters (#7132 )

* fix(ilm): reject invalid retention counts and validate lifecycle filters

`NewerNoncurrentVersions` had no lower bound at PUT, and evaluation read a
negative count through `usize::try_from(...).unwrap_or(usize::MAX)`. An
HTTP-accepted rule therefore retained (almost) everything and silently
stopped expiring versions — the one outcome a retention rule must never
produce by accident.

Reject a negative count during validation, and stop reading one as
"retain everything" anywhere it can still arrive from older persistence
or an import: evaluation takes no action for such a rule and says so in a
diagnostic, the batch limit path yields no event, and `Evaluator::eval`
reports a typed corruption error to callers that can surface one.

A count-only noncurrent expiration is a MinIO extension, not an AWS form.
It used to be rejected as an actionless rule and was never executed. It
is now accepted and honoured with the semantics MinIO gives it: the
newest N noncurrent versions are kept and every older one is due as soon
as it became noncurrent. Zero keeps the meaning the batch limit path has
always given it — no count constraint — so a zero-count rule with no age
condition still has no action.

`LifecycleRuleFilter` is an all-`Option` DTO, so the schema constraints
were not checked anywhere: validate at most one top-level predicate, an
`And` that combines at least two, no repeated tag key, tag key/value
limits, non-negative sizes, and `ObjectSizeGreaterThan <
ObjectSizeLessThan`. An empty filter stays valid — AWS documents it as
"every object in the bucket".

Schema-shape violations are reported with a distinct `ErrorKind` so the
S3 boundary answers them with `MalformedXML`; rejected values keep the
`InvalidArgument` this path has always returned.

backlog#2201

* fix(ilm): satisfy lifecycle clippy checks

* fix(ilm): fail closed on invalid lifecycle rules

* fix: initialize optional migration source fields

---------

Co-authored-by: cxymds <cxymds@gmail.com>

2026-09-06 10:11:05 +08:00

bucket_lifecycle_audit.rs

fix(ilm): notify on batch noncurrent version expiry (#7116 )

2026-09-04 18:25:10 +08:00

bucket_lifecycle_ops.rs

fix(tier): drain cleanup before tier removal (#7213 )

2026-09-06 10:10:48 +08:00

config_boundary.rs

fix(ecstore): retry manual ILM job CAS updates (#6012 )

2026-08-12 18:40:10 +00:00

core.rs

fix(ilm): reject invalid retention counts and validate lifecycle filters (#7132 )

2026-09-06 10:11:05 +08:00

durable_namespace.rs

fix(ilm): persist bounded transition recovery controls (#7200 )

2026-09-06 10:10:24 +08:00

evaluator.rs

refactor(lifecycle): extract core rule contracts (#4258 )

2026-07-04 20:05:56 +08:00

manual_transition_job.rs

refactor(ecstore): isolate local object rename commit (#7166 )

2026-09-05 07:19:29 +00:00

metadata_boundary.rs

fix(table-catalog): harden commit publication (#5779 )

2026-08-11 19:36:46 +08:00

mod.rs

fix(ilm): persist bounded transition recovery controls (#7200 )

2026-09-06 10:10:24 +08:00

object_handlers_common.rs

fix(ilm): notify on batch noncurrent version expiry (#7116 )

2026-09-04 18:25:10 +08:00

object_lock_boundary.rs

refactor(ecstore): move object_lock WORM evaluation onto storage-level types (#6666 )

2026-08-26 21:25:35 +08:00

README.md

refactor(runtime): batch owner boundary guards (#4092 )

2026-06-30 04:34:54 +08:00

recovery_control.rs

fix(ilm): persist bounded transition recovery controls (#7200 )

2026-09-06 10:10:24 +08:00

replication_sink.rs

feat(scanner): coordinate usage and workload boundaries (#7093 )

2026-09-03 17:02:43 +08:00

rule.rs

refactor(lifecycle): extract core rule contracts (#4258 )

2026-07-04 20:05:56 +08:00

runtime_boundary.rs

refactor(ecstore): migrate the background-services cancel token into InstanceContext (Phase 5 Slice 13) (#4586 )

2026-07-08 22:06:31 +00:00

tier_delete_journal.rs

refactor(ecstore): isolate local object rename commit (#7166 )

2026-09-05 07:19:29 +00:00

tier_free_version_recovery.rs

fix: bound tier free-version recovery sweeps (#7125 )

2026-09-04 08:12:01 +00:00

tier_last_day_stats.rs

feat(tier): report cluster tier stats and count tier requests (#7110 )

2026-09-04 09:40:03 +08:00

tier_sweeper.rs

chore(tier): remove stage-a blanket lint allowances (#7153 )

2026-09-05 06:43:05 +00:00

transition_transaction.rs

fix(ilm): persist bounded transition recovery controls (#7200 )

2026-09-06 10:10:24 +08:00

README.md

ECStore Lifecycle Split Inventory

This directory still belongs to ECStore. It is not ready to become a standalone crate because lifecycle workers currently depend on ECStore runtime state, object IO, bucket metadata, replication scheduling, notification/audit sinks, and tier services.

Current Modules

Module	Current role	Split blocker
`core.rs`	Lifecycle rule model, action evaluation, object options, and transition/expiry decisions.	Uses ECStore object metadata types and compatibility DTO re-exports.
`bucket_lifecycle_ops.rs`	Worker orchestration, expiry, transition, stale multipart cleanup, audit, replication delete scheduling, and queue state.	Depends on `ECStore`, `SetDisks`, runtime globals, bucket metadata/versioning, disk internals, event notification, tier services, and lifecycle-local object-lock/replication boundaries.
`evaluator.rs`	Bucket lifecycle evaluation wrapper.	Uses lifecycle-local object-lock boundary and replication state through lifecycle-local replication sink.
`rule.rs`	Lifecycle rule filter helpers.	Uses lifecycle-local tagging boundary.
`tier_delete_journal.rs`	Remote tier delete journal persistence and recovery.	Uses lifecycle-local config persistence boundary, object IO contracts, metadata bucket paths, and `ECStore`.
`tier_free_version_recovery.rs`	Free-version recovery queue and object restoration path.	Depends on `ECStore`, object metadata, storage-api contracts, and lifecycle queue callbacks.
`tier_last_day_stats.rs`	Tier statistics helpers.	Pure data/stat logic, but still part of lifecycle worker reporting.
`tier_sweeper.rs`	Remote tier deletion worker and transition cleanup.	Depends on runtime sources, `ECStore`, tier journal persistence, signer-error handling, and lifecycle object options.
`bucket_lifecycle_audit.rs`	Lifecycle audit event source labels.	Must remain wired to lifecycle audit and notification sinks.

Required Contracts

Contract	Responsibility	Current dependency to remove
`LifecycleObjectStore`	Object stat, delete, transition, restore, multipart cleanup, and version-aware metadata operations.	Direct `ECStore`, `SetDisks`, disk, and object API access in worker paths.
`LifecycleMetadataStore`	Lifecycle, object-lock, replication, bucket versioning, and stale multipart metadata reads.	Direct bucket metadata/versioning imports; object-lock and replication are still backed by local ECStore boundaries.
`LifecycleRuntime`	Expiry state, transition state, tier config, deployment ID, local node name, queue metrics, cancellation, and worker sizing.	Direct runtime source/global access and process environment reads inside worker code.
`LifecycleConfigStore`	Persist, read, and remove lifecycle-owned journal/config objects.	Direct ECStore config persistence helper imports from worker paths.
`LifecycleTagFilter`	Decode object tag strings for lifecycle rule matching.	Direct bucket tagging helper imports from lifecycle rule paths.
`LifecycleObjectLockStore`	Object-lock retention and deletion checks used by lifecycle evaluation and worker deletion paths.	Direct object-lock module imports from lifecycle code.
`LifecycleReplicationSink`	Lifecycle-originated delete and version-purge replication scheduling.	Boundary is local, but still backed by ECStore bucket replication internals.
`LifecycleAuditSink`	Lifecycle audit and notification event emission.	Direct event notification service calls and audit-side effects from worker code.

Migration Rules

Do not move bucket/lifecycle into a new crate while any worker imports crate::store::ECStore, crate::set_disk::SetDisks, runtime globals, or bucket replication internals directly.
Extract pure contracts before runtime movement. The first code-bearing PR should introduce one contract boundary and keep the old ECStore call path.
Preserve lifecycle queue behavior, transition/expiry state, replication delete scheduling, notification/audit events, scanner-visible metrics, and stale multipart cleanup semantics.
Keep rustfs_ecstore::api::bucket::lifecycle compatibility until scanner, OBS, and test boundary files compile through replacement paths.
Verify each code-bearing step with focused lifecycle tests before attempting broad gates.

First Code-Bearing Step

Start with LifecycleRuntime or LifecycleAuditSink. Both can be introduced as narrow internal contracts while keeping the current ECStore worker behavior unchanged. Do not start with a crate move.

Current first boundary: runtime_boundary.rs centralizes lifecycle access to runtime state while preserving the existing ECStore-backed implementations. config_boundary.rs centralizes lifecycle-owned config object persistence for tier delete journal recovery while preserving the existing ECStore config store. tagging_boundary.rs centralizes lifecycle tag decoding while preserving the existing ECStore bucket tagging implementation. object_lock_boundary.rs centralizes lifecycle object-lock checks while preserving the existing ECStore object-lock implementation. replication_sink.rs centralizes lifecycle-originated replication config checks and delete scheduling while preserving the existing ECStore replication worker path.