Commit Graph

1152 Commits

Author SHA1 Message Date
Henry Guo 49a893f010 feat(table-catalog): add credential endpoint boundary (#3407)
* feat(table-catalog): add credential scope boundary

* feat(table-catalog): add credential endpoint boundary

* fix(table-catalog): remove redundant scope clone

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-13 21:26:59 +08:00
安正超 6b1e114b39 feat(obs): add metrics runtime controller status (#3408) 2026-06-13 20:48:34 +08:00
GatewayJ 42d9d5247d fix(policy): normalize IAM policy readback (#3402) 2026-06-13 18:47:08 +08:00
houseme 0d68851f7a refactor(logging): standardize rustfs runtime events (#3396)
* fix(rpc): adjust log levels and reduce noise in storage RPC layer

Issue #682: pool rebalance normal flow logs were using warn! instead
of info!. Additionally, several high-frequency RPC entry points (ping,
write_stream, read_at, walk_dir) were logging at info! level, causing
unnecessary log noise in production.

Changes:
- load_rebalance_meta: warn! → info! for normal flow (3 occurrences)
- ping body decode: info! → debug! (fires on every health check)
- write_stream/read_at/walk_dir entry: info! → debug!
- metrics.rs: lowercase error messages, add structured error field
- http_service.rs: add structured error field to walk_dir failure
- Add tracing::instrument fields for start_rebalance context

* refactor(logging): standardize rustfs runtime events

* build(deps): bump workspace dependency versions

* build(deps): pin time to 0.3.47

* build(deps): update postgres client versions
2026-06-13 08:39:47 +08:00
escapecode b6973636b6 docs(sftp): document server operations (#3391)
Adds an operator guide for the SFTP server: recommended
configuration, the path model, host keys on Unix and Windows, the
environment variable reference, session cleanup behaviour, IAM
policy requirements per SFTP operation, client compatibility
notes, multipart upload sizing and cleanup, and the log lines
worth alerting on.

Corrects documentation the platform change left stale. The module
overview and the UnsupportedPlatform error text still described
Windows as unsupported. A source comment referenced a document
that does not exist in the repository and now points at the new
guide. The changelog adds the two host-key reload variables
missing from its environment list, describes the banner variable
as the SSH identification string, and corrects the upload size
cap and compliance case count.
2026-06-12 22:52:00 +08:00
Henry Guo f1a35bb9e2 fix(heal): require disk RPC before remote recovery (#3392)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-12 22:45:58 +08:00
安正超 f7bf4fd3e1 feat(targets): gate external plugin flow (#3393) 2026-06-12 22:17:54 +08:00
唐小鸭 8afb963d21 feat(ecstore): optimize the triggering conditions of the compression module (#3387)
feat. Optimize the triggering conditions of the compression module

Co-authored-by: houseme <housemecn@gmail.com>
2026-06-12 21:18:44 +08:00
安正超 5f7c991359 feat(targets): add extension hook diagnostics contracts (#3390)
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-12 21:17:34 +08:00
escapecode 7a8514bdfa feat(sftp): add macOS and Windows platform support (#3372)
The session watchdog now selects its detection method per
platform. It previously probed kernel TCP state through a
Linux-only procfs path, so on macOS every healthy idle session
was killed within a minute, and on Windows the watchdog never
spawned at all, leaving wedged sessions with no cleanup. Linux
keeps its fast-kill watchdog unchanged. Other platforms get a
silence-only backstop that kills a session only at the
documented 30-minute idle ceiling.

The host-key loader now has a Windows arm. It loads OpenSSH
format host keys from the configured directory and logs a
one-time warning to restrict NTFS ACLs on the key directory,
the same operator-managed approach FTPS, WebDAV, KMS, and IAM
already use on Windows. Startup previously aborted with
UnsupportedPlatform because the Unix mode-bit permission check
has no Windows equivalent. tokio's io-uring feature is now
enabled only in Linux builds. io-uring is a Linux kernel
interface and enabling it unconditionally broke the Windows
build.

Co-authored-by: houseme <housemecn@gmail.com>
2026-06-12 18:08:23 +08:00
安正超 e80b72ae79 feat(targets): gate sidecar runtime policy (#3388)
* feat(targets): gate sidecar runtime policy

* fix(admin): add extension route policy specs
2026-06-12 17:42:12 +08:00
安正超 1e7c376586 feat(targets): add extension schema adapter (#3385) 2026-06-12 15:27:55 +08:00
安正超 7d6d56a547 feat(extension): add schema contracts (#3384) 2026-06-12 15:07:17 +08:00
安正超 249a0b56ea feat(targets): require plugin artifact attestations (#3383) 2026-06-12 14:13:41 +08:00
安正超 08c586d843 security(admin): reject unknown JSON ingress fields (#3376) 2026-06-12 10:21:50 +08:00
Henry Guo 19ff378e9b fix(scanner): preserve newer usage snapshots (#3370)
* fix(scanner): preserve newer usage snapshots

* fix(table-catalog): require namespace locking backend

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-12 07:06:08 +08:00
Henry Guo 51ef87ed19 fix(heal): recover renewed disk health checks (#3366)
* fix(heal): recover renewed disk health checks

* test(heal): cover replaced remote disk rebuild

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-12 07:05:52 +08:00
安正超 c3055f9335 security(kms): require explicit dev defaults opt-in (#3369)
* security(kms): require explicit dev defaults opt-in

* test(kms): satisfy clippy dev defaults checks
2026-06-12 07:05:20 +08:00
安正超 a85cc0354c refactor(storage-api): remove namespace lock from StorageAPI (#3365)
* refactor(storage-api): remove namespace lock from StorageAPI

* test(scanner): wait for runtime budget cancellation

---------

Co-authored-by: loverustfs <hello@rustfs.com>
2026-06-11 22:42:12 +08:00
houseme 82af181dcf refactor(logging): unify governance runtime events (#3367) 2026-06-11 22:26:02 +08:00
安正超 7146c893cb refactor(storage-api): remove legacy bucket DTO re-export (#3364) 2026-06-11 21:24:56 +08:00
houseme 0a987d870b refactor(logging): reduce runtime noise (#3363) 2026-06-11 19:49:01 +08:00
安正超 a0b6636b61 refactor(config): remove legacy accessor re-export (#3362) 2026-06-11 19:32:57 +08:00
Henry Guo 2e3c777307 fix(heal): skip transient usage cache heal errors (#3359) 2026-06-11 19:16:32 +08:00
安正超 b4524033e3 refactor(config): move global config accessors (#3360) 2026-06-11 19:16:02 +08:00
安正超 ed3851782c refactor(config): remove legacy model re-export (#3357) 2026-06-11 18:09:01 +08:00
安正超 69549634ea refactor(config): migrate scanner config consumer (#3356) 2026-06-11 17:30:26 +08:00
安正超 ca58d7f0ec refactor(config): migrate server config consumers (#3353) 2026-06-11 17:04:50 +08:00
安正超 2205991180 refactor(config): extract server config model (#3351) 2026-06-11 16:11:17 +08:00
houseme 7d38b0cf90 refactor(obs): unify local and otlp log output (#3352)
wip(obs): start lg-002 output unification
2026-06-11 16:10:54 +08:00
houseme 8d337c4e87 feat(obs): add logging redaction guard rails (#3349)
* feat(obs): add logging redaction guard rails

* chore(docs): keep logging plans out of lg-001 pr

* test(obs): guard against unmasked access key logs

* test(obs): enforce masked access key log patterns
2026-06-11 15:24:32 +08:00
安正超 dd50359161 refactor(storage): narrow table catalog bounds (#3350) 2026-06-11 14:32:28 +08:00
安正超 d60deba9b6 fix(security): add IAM authorization to FTP read/metadata/cwd handlers (GHSA-3g29-xff2-92vp) (#3347)
fix(security): add IAM authorization to FTP read/metadata/cwd handlers

The FTP frontend's get() (RETR), metadata() (SIZE/MDTM), and cwd()
(CWD) handlers dispatched directly to the storage backend without
calling authorize_operation(). This allowed any authenticated FTP user,
including those with explicit Deny policies, to read arbitrary objects
and probe bucket existence regardless of IAM policy.

Add authorize_operation() calls matching the pattern used by the
write-path handlers (put, del, list, rmd) and the WebDAV driver:
- get(): S3Action::GetObject
- metadata() for files: S3Action::HeadObject
- metadata() for directories: S3Action::HeadBucket
- cwd(): S3Action::HeadBucket

Fixes: GHSA-3g29-xff2-92vp

Co-authored-by: houseme <housemecn@gmail.com>
2026-06-11 05:48:53 +00:00
安正超 d4ee44f49f refactor(scanner): narrow cache storage bounds (#3348) 2026-06-11 13:52:35 +08:00
安正超 2f7cc7cb9e refactor(storage): narrow resync metadata bounds (#3345) 2026-06-11 04:41:15 +00:00
houseme a49c6b4c2e fix(logging): clarify recovery and metacache warn messages (#3341)
* fix(logging): clarify recovery and metacache warn messages

Clarify cached gRPC connection eviction messages so they describe automatic reconnection instead of implying a persistent cluster fault.

Retitle metacache resolution logs to remove the misleading decommission_pool prefix and demote routine reconciliation traces to debug while preserving actionable warning paths.

* fix(logging): avoid overpromising reconnect success
2026-06-11 04:19:22 +00:00
Henry Guo 7191a3abae docs(scanner): document runtime scanner controls (#3339)
* docs(scanner): document runtime scanner controls

* docs(scanner): split English and Chinese README

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: majinghe <42570491+majinghe@users.noreply.github.com>
2026-06-11 12:04:34 +08:00
安正超 686068d8bd refactor(storage): narrow metadata bounds (#3343) 2026-06-11 12:04:02 +08:00
安正超 6b86e3875c refactor(storage): remove old admin surfaces (#3340) 2026-06-11 11:28:09 +08:00
安正超 87968275a7 refactor(storage): route maintenance inventory reads (#3337) 2026-06-11 10:32:48 +08:00
安正超 f325b9f714 refactor(ecstore): route admin read internals (#3336) 2026-06-11 08:43:42 +08:00
安正超 94c53af264 refactor(storage): use admin API for observability reads (#3335) 2026-06-11 08:13:15 +08:00
安正超 4b1714438e feat(storage-api): bind ecstore admin contract (#3331) 2026-06-11 05:49:14 +08:00
Henry Guo 91f80a5585 feat(scanner): expose lifecycle transition status (#3326)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-10 16:45:02 +00:00
安正超 d9ddd1bedc feat(storage-api): add disk inventory contract (#3330) 2026-06-10 16:33:31 +00:00
houseme b1339a98ed fix(scanner): delay deep verify for fresh objects (#3329)
Gate scanner-triggered deep heal behind a short new-object cooldown and retry transient bitrot shard size mismatches during local verification.

Add targeted tests for cooldown mode selection and size-mismatch classification, and log when deep scans are downgraded or size-mismatch retries occur.
2026-06-10 15:50:44 +00:00
houseme 3fed21c68a fix(startup): add disk init diagnostics (#3324)
* ci(build): remove macOS x86_64 target

* ci(build): reduce workflow timeout

* fix(startup): add disk init diagnostics
2026-06-10 14:24:12 +00:00
Henry Guo 66fd55a8e0 feat(scanner): expose pacing pressure status (#3319)
* feat(scanner): expose pacing pressure status

* fix(scanner): preserve merged pause pressure

* fix(scanner): default missing primary pressure

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-10 07:33:41 +00:00
安正超 bb5d9565a6 feat(storage-api): add bucket DTO contract (#3314)
* feat(storage-api): add bucket DTO contract

* ci(build): increase workflow timeout to 90 minutes

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-06-10 07:16:14 +00:00
Henry Guo 62e9c5b94f feat(scanner): add adaptive pacing controls (#3315)
* feat(scanner): add adaptive pacing controls

* fix(scanner): bound pacing delay status precision

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-06-10 03:46:02 +00:00