Commit Graph

3322 Commits

Author SHA1 Message Date
Henry Guo 74604201bb test(table-catalog): add client conformance baseline (#3400)
* test(table-catalog): add client conformance baseline

* docs(table-catalog): remove internal roadmap labels

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-13 13:54:07 +08:00
安正超 8f8064df80 feat(runtime): add memory observability status snapshot (#3397)
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-13 10:02:16 +08:00
houseme 0d68851f7a refactor(logging): standardize rustfs runtime events (#3396)
* fix(rpc): adjust log levels and reduce noise in storage RPC layer

Issue #682: pool rebalance normal flow logs were using warn! instead
of info!. Additionally, several high-frequency RPC entry points (ping,
write_stream, read_at, walk_dir) were logging at info! level, causing
unnecessary log noise in production.

Changes:
- load_rebalance_meta: warn! → info! for normal flow (3 occurrences)
- ping body decode: info! → debug! (fires on every health check)
- write_stream/read_at/walk_dir entry: info! → debug!
- metrics.rs: lowercase error messages, add structured error field
- http_service.rs: add structured error field to walk_dir failure
- Add tracing::instrument fields for start_rebalance context

* refactor(logging): standardize rustfs runtime events

* build(deps): bump workspace dependency versions

* build(deps): pin time to 0.3.47

* build(deps): update postgres client versions
2026-06-13 08:39:47 +08:00
Henry Guo b38e71fac3 feat(table-catalog): retain protected ref metadata (#3394) 2026-06-12 23:55:30 +08:00
escapecode b6973636b6 docs(sftp): document server operations (#3391)
Adds an operator guide for the SFTP server: recommended
configuration, the path model, host keys on Unix and Windows, the
environment variable reference, session cleanup behaviour, IAM
policy requirements per SFTP operation, client compatibility
notes, multipart upload sizing and cleanup, and the log lines
worth alerting on.

Corrects documentation the platform change left stale. The module
overview and the UnsupportedPlatform error text still described
Windows as unsupported. A source comment referenced a document
that does not exist in the repository and now points at the new
guide. The changelog adds the two host-key reload variables
missing from its environment list, describes the banner variable
as the SSH identification string, and corrects the upload size
cap and compliance case count.
2026-06-12 22:52:00 +08:00
Henry Guo f1a35bb9e2 fix(heal): require disk RPC before remote recovery (#3392)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-12 22:45:58 +08:00
安正超 f7bf4fd3e1 feat(targets): gate external plugin flow (#3393) 2026-06-12 22:17:54 +08:00
唐小鸭 8afb963d21 feat(ecstore): optimize the triggering conditions of the compression module (#3387)
feat. Optimize the triggering conditions of the compression module

Co-authored-by: houseme <housemecn@gmail.com>
2026-06-12 21:18:44 +08:00
安正超 5f7c991359 feat(targets): add extension hook diagnostics contracts (#3390)
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-12 21:17:34 +08:00
escapecode 7a8514bdfa feat(sftp): add macOS and Windows platform support (#3372)
The session watchdog now selects its detection method per
platform. It previously probed kernel TCP state through a
Linux-only procfs path, so on macOS every healthy idle session
was killed within a minute, and on Windows the watchdog never
spawned at all, leaving wedged sessions with no cleanup. Linux
keeps its fast-kill watchdog unchanged. Other platforms get a
silence-only backstop that kills a session only at the
documented 30-minute idle ceiling.

The host-key loader now has a Windows arm. It loads OpenSSH
format host keys from the configured directory and logs a
one-time warning to restrict NTFS ACLs on the key directory,
the same operator-managed approach FTPS, WebDAV, KMS, and IAM
already use on Windows. Startup previously aborted with
UnsupportedPlatform because the Unix mode-bit permission check
has no Windows equivalent. tokio's io-uring feature is now
enabled only in Linux builds. io-uring is a Linux kernel
interface and enabling it unconditionally broke the Windows
build.

Co-authored-by: houseme <housemecn@gmail.com>
2026-06-12 18:08:23 +08:00
安正超 e80b72ae79 feat(targets): gate sidecar runtime policy (#3388)
* feat(targets): gate sidecar runtime policy

* fix(admin): add extension route policy specs
2026-06-12 17:42:12 +08:00
安正超 434b71e569 feat(extension): add admin views (#3386) 2026-06-12 15:54:42 +08:00
安正超 1e7c376586 feat(targets): add extension schema adapter (#3385) 2026-06-12 15:27:55 +08:00
安正超 7d6d56a547 feat(extension): add schema contracts (#3384) 2026-06-12 15:07:17 +08:00
安正超 249a0b56ea feat(targets): require plugin artifact attestations (#3383) 2026-06-12 14:13:41 +08:00
安正超 8df38ea12c ci(docker): add release image scan report (#3382) 2026-06-12 13:11:59 +08:00
安正超 212a0913be ci(release): emit sbom and provenance assets (#3381) 2026-06-12 12:56:02 +08:00
cxymds 4f954397d7 fix: support regular users in access key info (#3285)
* fix: support regular users in access key info

* refactor: extract access key identity helpers

* refactor: add typed access key identity model

* test: cover info access key identity contract

* test: cover derived access key response contracts

* fix: remove needless borrow in access key test

* fix: block fallback on explicit admin deny

* fix: restrict service account access key fallback

* fix: correct info access key identity handling
2026-06-12 12:02:43 +08:00
安正超 99e68f82a2 ci(audit): report unpinned workflow actions (#3379) 2026-06-12 11:33:23 +08:00
安正超 fc32b76c0e fix: avoid mutating signed request headers (#3378) 2026-06-12 11:13:34 +08:00
安正超 559bf9d9d7 ci(audit): wire cargo deny supply-chain gate (#3377) 2026-06-12 11:11:40 +08:00
安正超 08c586d843 security(admin): reject unknown JSON ingress fields (#3376) 2026-06-12 10:21:50 +08:00
wood 9904a4f9eb fix(obs): fix grafana layout and wrong version of tempo (#3368)
* fix(obs): fix grafana dashboard layout errors

Signed-off-by: w0od <dingboning02@163.com>

* fix(docker): align the correct tempo version

Signed-off-by: w0od <dingboning02@163.com>

---------

Signed-off-by: w0od <dingboning02@163.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-12 09:27:32 +08:00
Henry Guo 19ff378e9b fix(scanner): preserve newer usage snapshots (#3370)
* fix(scanner): preserve newer usage snapshots

* fix(table-catalog): require namespace locking backend

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-12 07:06:08 +08:00
Henry Guo 51ef87ed19 fix(heal): recover renewed disk health checks (#3366)
* fix(heal): recover renewed disk health checks

* test(heal): cover replaced remote disk rebuild

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-12 07:05:52 +08:00
安正超 c3055f9335 security(kms): require explicit dev defaults opt-in (#3369)
* security(kms): require explicit dev defaults opt-in

* test(kms): satisfy clippy dev defaults checks
2026-06-12 07:05:20 +08:00
安正超 a85cc0354c refactor(storage-api): remove namespace lock from StorageAPI (#3365)
* refactor(storage-api): remove namespace lock from StorageAPI

* test(scanner): wait for runtime budget cancellation

---------

Co-authored-by: loverustfs <hello@rustfs.com>
2026-06-11 22:42:12 +08:00
houseme 82af181dcf refactor(logging): unify governance runtime events (#3367) 2026-06-11 22:26:02 +08:00
GatewayJ b5676dcc8e fix(table-catalog): support PyIceberg REST commits (#3342)
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-11 22:22:13 +08:00
安正超 7146c893cb refactor(storage-api): remove legacy bucket DTO re-export (#3364) 2026-06-11 21:24:56 +08:00
houseme 0a987d870b refactor(logging): reduce runtime noise (#3363) 2026-06-11 19:49:01 +08:00
安正超 a0b6636b61 refactor(config): remove legacy accessor re-export (#3362) 2026-06-11 19:32:57 +08:00
Henry Guo 2e3c777307 fix(heal): skip transient usage cache heal errors (#3359) 2026-06-11 19:16:32 +08:00
安正超 b4524033e3 refactor(config): move global config accessors (#3360) 2026-06-11 19:16:02 +08:00
安正超 ed3851782c refactor(config): remove legacy model re-export (#3357) 2026-06-11 18:09:01 +08:00
houseme f6137f9f77 refactor(auth): tighten startup log redaction (#3358) 2026-06-11 18:08:41 +08:00
安正超 69549634ea refactor(config): migrate scanner config consumer (#3356) 2026-06-11 17:30:26 +08:00
houseme 704c95a22d refactor(server): consolidate request transport logs (#3354) 2026-06-11 17:07:02 +08:00
安正超 ca58d7f0ec refactor(config): migrate server config consumers (#3353) 2026-06-11 17:04:50 +08:00
安正超 2205991180 refactor(config): extract server config model (#3351) 2026-06-11 16:11:17 +08:00
houseme 7d38b0cf90 refactor(obs): unify local and otlp log output (#3352)
wip(obs): start lg-002 output unification
2026-06-11 16:10:54 +08:00
houseme 8d337c4e87 feat(obs): add logging redaction guard rails (#3349)
* feat(obs): add logging redaction guard rails

* chore(docs): keep logging plans out of lg-001 pr

* test(obs): guard against unmasked access key logs

* test(obs): enforce masked access key log patterns
2026-06-11 15:24:32 +08:00
安正超 dd50359161 refactor(storage): narrow table catalog bounds (#3350) 2026-06-11 14:32:28 +08:00
安正超 2ead90d31b fix(security): add authorization check to ListRemoteTargetHandler (GHSA-796f-j7xp-hwf4) (#3346)
fix(security): add authorization check to ListRemoteTargetHandler

ListRemoteTargetHandler only verified that request credentials existed
but did not check whether the caller has replication or administrator
permissions. This allowed any authenticated user to list remote
replication target configuration, potentially leaking remote target
credentials (accessKey, secretKey, sessionToken).

Add validate_replication_admin_request() call with
GetBucketTargetAction, consistent with the other replication admin
handlers that already perform this check.

Fixes: GHSA-796f-j7xp-hwf4

Co-authored-by: houseme <housemecn@gmail.com>
2026-06-11 05:50:14 +00:00
安正超 d60deba9b6 fix(security): add IAM authorization to FTP read/metadata/cwd handlers (GHSA-3g29-xff2-92vp) (#3347)
fix(security): add IAM authorization to FTP read/metadata/cwd handlers

The FTP frontend's get() (RETR), metadata() (SIZE/MDTM), and cwd()
(CWD) handlers dispatched directly to the storage backend without
calling authorize_operation(). This allowed any authenticated FTP user,
including those with explicit Deny policies, to read arbitrary objects
and probe bucket existence regardless of IAM policy.

Add authorize_operation() calls matching the pattern used by the
write-path handlers (put, del, list, rmd) and the WebDAV driver:
- get(): S3Action::GetObject
- metadata() for files: S3Action::HeadObject
- metadata() for directories: S3Action::HeadBucket
- cwd(): S3Action::HeadBucket

Fixes: GHSA-3g29-xff2-92vp

Co-authored-by: houseme <housemecn@gmail.com>
2026-06-11 05:48:53 +00:00
安正超 d4ee44f49f refactor(scanner): narrow cache storage bounds (#3348) 2026-06-11 13:52:35 +08:00
安正超 2f7cc7cb9e refactor(storage): narrow resync metadata bounds (#3345) 2026-06-11 04:41:15 +00:00
houseme ee96e194a3 build(deps): bump s3s for header parsing fix (#3344) 2026-06-11 04:28:36 +00:00
houseme a49c6b4c2e fix(logging): clarify recovery and metacache warn messages (#3341)
* fix(logging): clarify recovery and metacache warn messages

Clarify cached gRPC connection eviction messages so they describe automatic reconnection instead of implying a persistent cluster fault.

Retitle metacache resolution logs to remove the misleading decommission_pool prefix and demote routine reconciliation traces to debug while preserving actionable warning paths.

* fix(logging): avoid overpromising reconnect success
2026-06-11 04:19:22 +00:00
Henry Guo 7191a3abae docs(scanner): document runtime scanner controls (#3339)
* docs(scanner): document runtime scanner controls

* docs(scanner): split English and Chinese README

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: majinghe <42570491+majinghe@users.noreply.github.com>
2026-06-11 12:04:34 +08:00