The RPM build step fails because fpm's --config-files flag requires
/etc/default/rustfs to exist in the staging area, but unlike the DEB
build (which creates it in its package directory structure), the fpm
command has no prior step creating this file.
Create the config file in a temporary directory and pass it to fpm
via a source=dest mapping, matching the DEB build's behavior.
Track accepted replay cache records by gRPC operation and split Lock/Unlock and ReadVersion methods out of grpc_other so hotpath validation can attribute nonce pressure without changing replay protection semantics.
Co-authored-by: heihutu <heihutu@gmail.com>
Add a replacement recovery peer RPC so Admin v4 can distinguish definitive cluster proofs from unsupported, unavailable, or conflicting peer state without extending the existing background heal v3/v1 status protocol.
Co-authored-by: heihutu <heihutu@gmail.com>
The DEB version substitution used ${VERSION/-/~} which caused bash
to expand ~ to $HOME (e.g. /home/runner), producing an invalid
version string like '1.0.0/home/runnerrc.1'.
Store ~ in a variable first to prevent tilde expansion.
Add a v4 admin status endpoint for local durable automatic replacement recovery records without changing the v3 background heal status or peer v1 payloads.
Co-authored-by: heihutu <heihutu@gmail.com>
Three Kubernetes endpoint-identity tests read the real kernel hostname
and panicked when it is an IP literal (e.g. macOS without a static
HostName, where DHCP/reverse-DNS sets the kernel hostname to an address
like 192.168.1.11).
Add a cfg(test) override seam (force_kernel_hostname_for_test, mirroring
the existing force_local_host_resolution_timeout_for_test pattern) and
route the production read through kernel_hostname_for_endpoint_identity()
so the tests inject deterministic hostnames instead of depending on the
host environment. Production behavior is unchanged.
Complete the encrypted-object replication series (backlog#1783, PR-C of
3, after #5872 and #5885): SSE-C objects replicate as ciphertext
passthrough — the source holds no customer key, so the stored bytes and
their encryption metadata travel verbatim and the replica decrypts only
with the original customer key, single-part and multipart.
- Sender: SSE-C objects read raw (raw_data_movement_read), transfer at
ciphertext size, and range multipart parts over stored part sizes.
- Receiver: authorized replication PUTs restore the stored SSE-C keys
from the transport headers (exact lowercase forms - the read-path
check is case-sensitive), set ObjectOptions.preserve_ciphertext, and
skip compression, bucket-default SSE, and sse_encryption behind one
restore-derived gate. Multipart uses an internal session marker to
store parts verbatim and strips it on complete.
- Convergence: the replication HEAD sends
x-rustfs-source-replication-check; the target authorizes it as
ReplicateObjectAction and skips SSE-C read validation for that
request only, so keyless convergence HEADs see etag/size/mtime
instead of 400 and SSE-C replicas stop re-driving forever.
- e2e: SSE-C contract flips to a key-gated readable replica (no-key and
wrong-key GETs fail - the direct silent-plaintext detector); new
multipart passthrough contract with ETag/marker/stability assertions.
* fix(ecstore): anchor Windows rename publication
* fix(ecstore): complete Windows rename confinement
* test(ecstore): retain Windows retry assertion path
* fix(ecstore): accept configured Windows root paths
* fix(ecstore): size Windows rename buffers correctly
* fix(ecstore): use native relative rename on Windows
* fix(ecstore): preserve Windows rename parent guards
* fix(ecstore): reuse guarded Windows rename trees
* fix(ecstore): compile Windows publication helpers
* fix(ecstore): preserve configured Windows disk roots
* fix(ecstore): flush Windows shards with write access
* fix(ecstore): stage Windows rollback backup replacement
* fix(ecstore): defer Windows staged file cleanup
* fix(ecstore): type Windows staged write result
* fix(ecstore): retry Windows sharing violations
* fix(ecstore): share Windows staged deletes
* fix(ecstore): split Windows staged publication handles
* fix(ecstore): close Windows staged writer before rename
* fix(ecstore): share Windows staged publication deletes
* fix(ecstore): allow guarded Windows child publication
---------
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: cxymds <cxymds@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Install tzdata in both published runtime variants and verify IANA timezone resolution during image builds.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Open the managed-SSE replication gate (backlog#1783, PR-B of 3, after
#5872): the replication reader already decrypts through the injected
object-encryption resolver, so the source sends plaintext plus an
encryption intent header (AES256 / aws:kms, never the source key id) and
the target re-encrypts on its normal PUT path with its own KMS. No DEK
crosses sites.
- replication_put_object_options: fail closed only on Unsupported;
insert the SSE intent after the strip loop.
- TargetClient::create_multipart_upload sends the full opts.header()
set, fixing multipart replicas losing content-type/user metadata
(plaintext included).
- Preserve source ETag and mtime on replicas (authorized replication
only): receiver wires x-rustfs-source-etag into preserve_etag for PUT
and CompleteMultipartUpload, resolve_complete_etag consumes it, and
complete options carry source_etag/source_mtime (absent mtime
degrades to epoch, not now_utc). Without this every replication HEAD
comparison re-drives re-encrypted objects forever.
- e2e: managed SSE contracts flip to success on an independent-KMS
dual-process pair (byte-identical plain GET proves target-owned
envelopes; ETag/mtime preserved; version stable across scanner
cycles; resync converges; multipart keeps structure and metadata);
new target-without-KMS fail-closed contract; SSE-C stays FAILED.
Co-authored-by: houseme <housemecn@gmail.com>
Groundwork for encrypted-object replication (backlog#1783, PR-A of 3):
- classify_replication_source_encryption: accept the AES256 marker that
every stored SSE-C object carries; the SseC arm was unreachable.
- Fail closed on sealed material without an SSE marker (MinIO-written
objects) instead of replicating ciphertext as plaintext.
- Replace the dead VALID_SSE_REPLICATION_HEADERS table with a transport
map keyed by the metadata keys the SSE writer actually persists, shared
via the new rustfs_utils::http::object_encryption_keys module.
- Structurally strip all encryption metadata from outbound replication
(x-rustfs-encryption-* envelopes previously passed the filters).
- Skip decrypt_checksums for encrypted objects at the boundary so its
is_multipart=false (a response-path contract) cannot misroute
encrypted multipart objects once managed replication opens.
- Redact X-Rustfs-Replication-* SSE transport values in FileInfo Debug.
A reconciliation test pins that every key encryption_material_to_metadata
produces is either transport-mapped or stripped. All four SSE replication
e2e contracts still assert FAILED unchanged.
* fix(build): support non-Linux Unix targets (illumos/Solaris/*BSD)
Two independent build-infrastructure blockers kept RustFS from building on
non-Linux Unix platforms. Neither touches runtime logic.
1. pulsar regenerates its protobuf bindings in build.rs on every build, which
needs `protoc`. Platforms without a packaged protoc (illumos/Solaris/*BSD)
now enable pulsar's `protobuf-src` feature via a cfg-gated dependency, which
builds a vendored protoc from C++ sources. Mainstream targets keep the lean
dependency and their existing system/CI protoc.
2. clocksource 0.8.3 (pulled in transitively by ratelimit 0.10) used the
Linux-only `CLOCK_MONOTONIC_COARSE`. ratelimit 2.0 dropped the clocksource
dependency entirely, so upgrading removes the portability problem at the
root rather than patching clocksource. The bandwidth throttle's bulk
`consume()` is rewritten onto ratelimit 2.0's `try_wait_n`, preserving the
best-effort partial-consumption semantics.
Verified: cargo check + bandwidth monitor unit tests pass; cargo tree confirms
protobuf-src is enabled only for illumos/Solaris/*BSD and clocksource is gone
from the graph. The final illumos build must be confirmed on-platform.
Closes#3195
* fix(ecstore): guard ratelimit v2 capacity overflow
Co-Authored-By: heihutu <heihutu@gmail.com>
* test(ecstore): avoid slow bandwidth reader timeout
Co-Authored-By: heihutu <heihutu@gmail.com>
* fix(targets): drop vendored pulsar protobuf build
Co-Authored-By: heihutu <heihutu@gmail.com>
---------
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
cxymds