#7847 routed schedule-failure alerting to rustfs/backlog by switching the
alert jobs' token to secrets.BACKLOG_ISSUE_TOKEN, but check_test_wiring.py
still asserts the literal 'github-token: ${{ secrets.GITHUB_TOKEN }}', so
Quick Checks fails on every open PR. Update the wiring assertions and the
self-test fixtures to the new literal.
Both `--self-test` (52 tests) and the full `validate()` pass locally.
* chore(deps): bump ed25519-dalek in the dependencies group
Bumps the dependencies group with 1 update: [ed25519-dalek](https://github.com/dalek-cryptography/curve25519-dalek).
Updates `ed25519-dalek` from 2.2.0 to 3.0.0
- [Release notes](https://github.com/dalek-cryptography/curve25519-dalek/releases)
- [Changelog](https://github.com/dalek-cryptography/curve25519-dalek/blob/3.0.0/CHANGELOG.md)
- [Commits](https://github.com/dalek-cryptography/curve25519-dalek/compare/ed25519-2.2.0...3.0.0)
---
updated-dependencies:
- dependency-name: ed25519-dalek
dependency-version: 3.0.0
dependency-type: direct:production
update-type: version-update:semver-major
dependency-group: dependencies
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore(deps): bump jiff, aws-sdk-s3, datafusion and smallvec
Bump the dependencies group:
- jiff 0.2.35 -> 0.2.37
- aws-sdk-s3 1.146.0 -> 1.146.1
- datafusion 55.0.0 -> 55.1.0
- smallvec 1.16.0 -> 1.16.1
Update Cargo.lock for the matching transitive bumps (datafusion
subcrates, hickory, cc, crc32fast, tinyvec, toml_edit and others);
tinyvec_macros is no longer required.
* chore(deps): use s3s 0.16.0 release and bump rust-version to 1.98.1
* fix: preserve ready reader errors during erasure write failures
When erasure writers fail after the producer has already surfaced a reader error, return the reader error instead of aborting the producer and masking it with the write-side failure.
This keeps UploadPart body read timeouts classified as RequestTimeout under loaded pipeline timing.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* test: use current checksum API in multipart regressions
* fix: stabilize connect dependency update tests
Classify Connect proxy failures only when an explicit proxy is configured, preserve TLS classification across request error shapes, keep short object performance windows from spending too much budget on cleanup, and update multipart checksum tests for the current HashReader API.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Hauser <housemecn@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Hauser <housemecn@gmail.com>
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
Co-authored-by: overtrue <anzhengchao@gmail.com>
* fix(auth): reject unsigned x-amz headers on header-signed SigV4 requests
A SigV4 request authenticated with an Authorization header only binds the headers named in its SignedHeaders list, but RustFS acted on every x-amz-* header that arrived, so a replayed header-signed PutObject carrying an unsigned x-amz-copy-source became a CopyObject run as the signer that could copy any object the signer can read (GHSA-xm99-m3gq-83g8). The presigned form was already closed by GHSA-g8w9-qw9q-fghr.
reject_unsigned_amz_headers_on_sigv4_request now guards S3Access::check and S3Router::check_access: every SigV4 signed-header list the request carries must cover every x-amz-* header, the Authorization header is parsed with the verifier's own s3s-sigv4 parser and compared case-insensitively, the algorithm token is pinned to AWS4-HMAC-SHA256 because the upstream header path accepts any token, and the exempt set mirrors the upstream s3s fix (x-amz-content-sha256, x-amz-decoded-content-length, x-amz-trailer, x-amz-checksum-algorithm) plus x-amz-cf-id. Adds ghsa_xm99 unit, router and e2e regressions, raises the security smoke floor to 28, and records the advisory in docs/testing/security-regressions.md and CHANGELOG.md.
* chore(deps): switch s3s and s3s-sigv4 to crates.io 0.16.0
* fix(server): enforce the SigV4 header guard ahead of s3s dispatch
s3s 0.16 verifies the claimed algorithm as the first step of its own
signature flow, so a request whose Authorization header swaps the
AWS4-HMAC-SHA256 token was answered with 501 NotImplemented before
RustFS's access layer could rule on the unsigned x-amz-copy-source
(GHSA-xm99-m3gq-83g8). Add the SigV4HeaderGuardLayer as the innermost
external stack layer, running reject_unsigned_amz_headers_on_sigv4_request
in front of s3s and serializing its rejections as the same AccessDenied
S3 error document the access layer produces.
---------
Co-authored-by: Hauser <housemecn@gmail.com>
The gateway key inventory handler added in #7785 imported s3s directly and built four errors with s3_error!, pushing the s3s footprint ratchet to 210 files and 1592 s3_error! lines on main. Route its S3 types and error construction through the crate::admin::storage_api::s3 facade like sibling admin handlers; the codes, HTTP statuses and messages are unchanged, and a new unit test pins them.
Co-authored-by: cxymds <cxymds@gmail.com>
Co-authored-by: Hauser <housemecn@gmail.com>
Scheduled pipeline failures currently open [scheduled-failure] tracking
issues in rustfs/rustfs itself, adding bot noise to the code repository's
issue tracker. Route that alerting to rustfs/backlog instead.
- schedule-failure-issue gains target-repository (default rustfs/backlog);
dedupe lookup, comment appends and issue creation target that repository.
- New source-token input (default ${{ github.token }}) reads the failed-job
list from the reported run; the run still lives in and links to
rustfs/rustfs. The issue body now carries a '- Repository:' line.
- All consumer workflows pass secrets.BACKLOG_ISSUE_TOKEN; the
workflow-scoped GITHUB_TOKEN cannot open issues cross-repo. Alert job
permissions blocks are unchanged.
* fix: resume get body after short eof
Treat mid-stream short EOF errors as resumable when the S3 GET body has an attached resume context. This lets peer-kill reads reopen the same object version at the emitted offset instead of aborting the 200 response body.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
* fix(connect): surface TLS peer bootstrap failures
Map Connect registration TLS peer validation failures to the dedicated bootstrap error so wrong CA inputs fail closed with the sanitized TLS variant instead of the generic exchange error.
Co-Authored-By: heihutu <heihutu@gmail.com>
Co-Authored-By: zhi22915 <qiuzgang@gmail.com>
---------
Co-authored-by: zhi22915 <qiuzgang@gmail.com>
Only the Local backend refused a key identifier containing a path
separator. On Vault KV2 a create with the name bad/name succeeded and
produced a nested KV2 path that the listing then reported as a
directory rather than a key, and an identifier containing .. addressed
a record outside the configured key prefix once the HTTP client
normalised the URL; Vault Transit built its transit key name and its
metadata path from the same unchecked identifier.
Lift the Local backend's containment rule into a shared segment check
(empty, /, backslash, NUL, . and ..) and apply it at the single point
where each backend turns the identifier into a path or a Transit key
name, so create, describe, encrypt, delete and the metadata writes all
refuse with InvalidKey before any request reaches Vault. The admin API
already maps that to 400. The AWS backend is untouched: it addresses
keys by ARN and alias, both of which contain /.
Refs rustfs/backlog#2474 (KMS-213 CreateNegatives on vault-kv2).
Co-authored-by: Hauser <housemecn@gmail.com>
* fix(replication): accept a bodiless 405 as a replicated delete marker during resync
A resync verifies each delete marker with `HEAD ?versionId=<marker>` on the
target. S3 targets (RustFS, MinIO, AWS) answer that with 405 and no body,
and the SDK only synthesizes an error code for 404, so the error arrived
with `code() == None`. `is_retryable_delete_replication_head_error` then
treated it as an ambiguous failure: every delete marker counted as a failed
object with `target service error`, and a site resync over any bucket that
holds a delete marker reported the whole bucket as failed
(rustfs/backlog#2479, SITE-105 on rc.6 and nightly).
Use the raw HTTP status the way the 404 path already does: a 405 without a
code confirms the marker propagated. Ambiguous statuses still fail.
- Unit tests drive `verify_resync_head_result` against a scripted target
answering 405 (accepted) and 503 (still failed).
- e2e `test_site_replication_resync_replicates_delete_marker` joins two
sites, converges a live object and a delete marker, and requires the site
resync to complete with zero failed objects; the repl-nightly selection
digest is refreshed for the new case.
* fix(replication): verify marker-version purges by absence during resync
A delete-marker resync entry with a pending or failed version purge asks
the target to remove the marker, so the bodiless 405 that proves a
created marker propagated proves the purge did not happen. Accept the
405-as-success mapping only for marker creation (empty
version_purge_status); a purge counts as replicated only when the target
answers not found, and any other HEAD outcome stays failed. Unit tests
drive both purge statuses against the 405 fixture and the 404 fixture.
* fix(io-metrics): drop the rustfs-common edge by injecting the S3 telemetry observer
dc700d366 (#7775) made the io-metrics leaf crate depend on rustfs-common
to publish typed telemetry trace events from S3HttpRequestGuard, which
breaks the backlog#1834 leaf-crate rule (io-metrics may only depend on
rustfs-s3-ops).
S3HttpRequestGuard now takes an optional fn-pointer completion observer
(operation, latency, 2xx?) and knows nothing about the trace bus. The
server builds guards through rustfs::server::s3_http_request_guard, which
attaches the observer only while a telemetry subscriber exists and owns
the S3Operation -> TelemetryTraceOperation mapping.
* chore(guard): admit #7785 s3s ratchet growth (+1 file, +4 lines)
#7785 landed the gateway key inventory admin handler after the
baselines were verified, leaving check_s3s_footprint.sh red on main
and every branch cut from it (measured 210 files / 1592 lines vs
209/1588 baselines).
The handler follows the house admin convention whose Operation::call
signature is s3s-typed (S3Request/S3Result), so it cannot route
through a non-s3s seam until the s3gate admin migration replaces the
admin router (rustfs/backlog#1677 F1); no local refactor can shed the
file-level import. Record the measured growth with rationale:
files 209 -> 210, s3_error! lines 1588 -> 1592.
* fix
* fix: resolve clippy gate failures in connect and CLI surfaces
- Box the two large CommandResult performance variants and
LicenseRenewalOutcome::Installed (clippy::large_enum_variant)
- Return RelayError instead of () from RelayTransport::deliver and the
test destination (clippy::result_unit_err)
- Drop an unused mut on the protected relay file handle (unused_mut)
- Pass the digest by value to base64 encoding and drop a redundant
result_json clone (clippy::needless_borrows_for_generic_args,
clippy::redundant_clone)
* fix(connect): resolve the remaining site-replication clippy errors
- Group the endpoint credential triple into SiteReplicationCredentials
so SiteReplicationEndpoint::new takes 6 arguments
(clippy::too_many_arguments)
- Take the build-feature validator by &str (clippy::ptr_arg); the
validator closure stays because &String needs the deref coercion
* style: apply rustfmt to the site-replication credential grouping
* fix(connect): keep the CPU profile fixture off the ecstore import scan
#7811 used "rustfs_ecstore::disk::read_object" as the raw-symbol fixture
of the profile redaction test, which the architecture guard counts as a
direct rustfs_ecstore reference outside the compatibility boundaries
(its scan deliberately includes inline test modules). The accumulator
treats symbols opaquely, so use the internal rustfs::storage:: path that
keeps the fixture realistic without matching the import scanner.
* fix
* fix
* fix
* fix
---------
Co-authored-by: Hauser <housemecn@gmail.com>
Chris