cf89291898
fix(notify): strip rustfs/minio internal metadata from event userMetadata ( #4419 )
...
fix(notify): strip rustfs/minio internal metadata from event userMetadata (backlog#964)
Notification event construction only stripped keys prefixed with
`x-amz-meta-internal-`, which is not a prefix RustFS actually uses. Real
internal metadata lives under `x-rustfs-internal-*` / `x-minio-internal-*`
(xl.meta compat, incl. `x-minio-internal-server-side-encryption-*`) and
server-side-encryption details under `x-rustfs-encryption-*` /
`x-minio-encryption-*`. None of these were filtered, so they leaked into
`s3.object.userMetadata` sent to downstream notification targets
(webhook/MQ) — an information disclosure of SSE/replication/healing
internal state.
Filter via the shared classifiers `rustfs_utils::http::is_internal_key`
and `is_encryption_metadata_key` (case-insensitive, covering both key
flavors), while retaining the legacy `x-amz-meta-internal-*` strip for
backward compat. Genuine user metadata (`x-amz-meta-*`, content-type, ...)
is preserved unchanged.
Adds a regression test asserting both internal prefixes, the SSE internal
key, both encryption prefixes (incl. mixed-case) and the legacy prefix are
stripped while user keys survive.
Refs: https://github.com/rustfs/backlog/issues/964
2026-07-08 15:01:49 +08:00
fb0e2d6d8f
refactor(plugins): harden target-plugin system and admin plugin contract ( #4217 )
2026-07-03 10:18:21 +08:00
1b3dea012e
refactor: route ecstore runtime globals through facade ( #3941 )
2026-06-27 12:27:03 +08:00
6efbfff2c5
refactor: segment residual storage api boundaries ( #3905 )
2026-06-26 15:56:38 +08:00
e37e367390
refactor: route remaining external storage boundaries ( #3889 )
2026-06-26 06:22:51 +08:00
7499dd085d
refactor: collapse app notify thin compat boundaries ( #3768 )
...
* refactor: collapse app notify thin compat boundaries
* refactor: route app runtime consumers through context (#3769 )
2026-06-23 08:13:58 +08:00
198fd4f150
refactor(runtime): route RustFS runtime consumers through storage owner ( #3756 )
2026-06-23 05:17:56 +08:00
db7ff8f513
refactor: expose external ECStore owner symbols ( #3748 )
2026-06-22 21:42:54 +08:00
418b5d04f9
refactor: route external ECStore imports through owners ( #3747 )
2026-06-22 20:57:10 +08:00
88a87b3e8f
refactor: centralize external ECStore facade aliases ( #3746 )
2026-06-22 20:37:13 +08:00
539c68778d
refactor: remove standalone compat bridge modules ( #3740 )
2026-06-22 17:35:58 +08:00
7bfa80cb0b
fix(notify): normalize event time precision ( #3725 )
2026-06-22 13:53:56 +08:00
de158743c3
refactor: split compat facade imports ( #3716 )
2026-06-22 09:53:22 +08:00
d39815470e
refactor: prune consumer storage compat paths ( #3704 )
2026-06-22 02:31:30 +08:00
30b46640b4
refactor: prune edge compat aliases ( #3692 )
2026-06-21 17:41:07 +08:00
0985225448
refactor: consolidate ecstore public facade ( #3678 )
...
* refactor: expand ecstore compatibility facade
* test: enforce ecstore api facade imports
* refactor: hide legacy ecstore layout modules
* refactor: hide ecstore facade root modules
2026-06-21 09:09:11 +08:00
08b8f58e64
refactor: prune notify ecstore object alias ( #3620 )
2026-06-19 20:11:56 +08:00
54ffbedbc8
refactor: remove ecstore operation compat facades ( #3608 )
2026-06-19 17:01:12 +08:00
b1c6578df1
refactor: narrow test harness compatibility surfaces ( #3592 )
2026-06-19 07:10:52 +08:00
a860f2b40c
refactor: narrow storage compatibility surfaces ( #3585 )
...
* refactor: narrow storage compatibility surfaces
* refactor: narrow observability compatibility surfaces
2026-06-19 00:56:14 +08:00
205f964dc5
refactor: tighten storage compat store_api aliases ( #3582 )
...
* refactor: collapse storage compat store_api modules
* chore: guard storage compat store_api aliases
2026-06-18 22:50:02 +08:00
c28fee0013
refactor: continue storage api contract cleanup ( #3580 )
...
* refactor: move delete object contracts to storage api
* refactor: narrow store api compatibility exports
* refactor: route table catalog test through storage compat
2026-06-18 22:42:02 +08:00
7b0cb9e725
refactor: prune storage compatibility re-export allowances ( #3579 )
...
* refactor: prune storage compatibility re-export allowances
* fix: reject whitespace-padded dot path segments
2026-06-18 21:14:24 +08:00
08371a6e09
refactor: narrow storage compatibility exports ( #3576 )
2026-06-18 18:31:14 +08:00
c098184c49
refactor: clean runtime and test storage boundaries ( #3573 )
...
* refactor: clean runtime observability select boundaries
* refactor: clean test harness fuzz storage boundaries
2026-06-18 18:09:25 +08:00
0e7e2660bb
refactor: clean remaining storage DTO imports ( #3568 )
2026-06-18 14:44:46 +08:00
acdf439371
refactor: clean external DTO consumers ( #3566 )
...
* refactor: clean external DTO consumers
* fix: handle empty erasure shard recovery
2026-06-18 14:22:44 +08:00
84f7027e90
refactor: add object store resolver for standalone crates ( #3437 )
2026-06-14 18:02:39 +08:00
e8012bd1ba
refactor(logging): normalize admin telemetry and error messages ( #3430 )
2026-06-14 13:27:10 +08:00
82af181dcf
refactor(logging): unify governance runtime events ( #3367 )
2026-06-11 22:26:02 +08:00
0a987d870b
refactor(logging): reduce runtime noise ( #3363 )
2026-06-11 19:49:01 +08:00
ca58d7f0ec
refactor(config): migrate server config consumers ( #3353 )
2026-06-11 17:04:50 +08:00
a63aaf933d
fix(notify): normalize lock ordering to prevent deadlock ( #3199 )
...
* fix(notify): normalize lock ordering to prevent deadlock
replace_targets() and shutdown() acquired target_list before replay_workers,
opposite to runtime_view.rs order. This is an ABBA deadlock pattern.
- Swap to replay_workers -> target_list order (canonical)
- Add lock order comments
- 82 notify tests pass
* docs(notify): update runtime facade lock order
* docs(notify): clarify runtime facade lock order
---------
Co-authored-by: houseme <housemecn@gmail.com >
2026-06-03 15:37:59 +00:00
1838922f07
fix(security): add deny_unknown_fields to deserialization structs ( #3198 )
...
* fix(security): add deny_unknown_fields to deserialization structs
Prevent silent acceptance of malformed or adversarial payloads.
- policy: Policy, BucketPolicy, Statement, BPStatement, PrincipalObject
- notify: S3KeyFilter custom deserializer rejects unknown child elements
- update: VersionInfo (remote HTTP response)
26 policy + 82 notify tests pass.
* fix(update): keep version response forward compatible
---------
Co-authored-by: houseme <housemecn@gmail.com >
2026-06-03 15:33:28 +00:00
73bde843d6
refactor(s3): consolidate semantic boundaries and remove s3-common ( #3012 )
...
* refactor(common): introduce rustfs-data-usage core crate
* refactor(concurrency): migrate workers crate into concurrency
* refactor(crypto): migrate appauth token APIs into crypto
* fix docs urls
* remove unused crate
* refactor(data-usage): switch consumers to rustfs-data-usage
* chore(fmt): apply cargo fmt and lockfile sync
* refactor(common): remove data_usage compatibility re-export
* refactor(capacity): move capacity_scope to object-capacity
* refactor(io-metrics): relocate internode metrics from common
* refactor(common): decouple scanner report from madmin
* chore(fmt): normalize import ordering after pre-commit
* refactor(s3): split s3 types and ops crates
* refactor(s3): centralize event version and safe parsing
* refactor(s3): add op-event compatibility guardrails
* refactor(s3): add runtime op-event mismatch observability
* refactor(s3): extract delete event mapping helper
* refactor(s3): extract put event mapping helper
* refactor(s3): consolidate remaining event semantic helpers
* refactor(s3): add op-event coverage checks and observability alerts
* refactor(s3-ops): consolidate op-event semantic mapping
* refactor(scanner): remove last_minute wrapper module
* refactor(scanner): consolidate duplicated data usage models
2026-05-19 12:50:25 +00:00
17ae9f34c7
fix(notify): accept case-insensitive filter rule names ( #2990 )
2026-05-18 03:41:48 +00:00
6e12289339
fix(runtime): finalize issue 2941 profiling cleanup ( #2983 )
...
* perf(runtime): narrow profiling support and upgrade starshard
* style(notify): normalize starshard imports
* perf(ecstore): reduce list_path_raw coordination overhead
* docs(scripts): add issue 2941 perf capture workflow
* fix(runtime): finalize issue 2941 profiling cleanup
* build(deps): bump quick-xml to 0.40.0
* chore(scripts): untrack local perf capture guide
* fix(scripts): honor label in perf capture output
2026-05-16 11:09:04 +00:00
11be278c40
test(notify): cover prefix suffix event dispatch ( #2960 )
2026-05-14 05:38:38 +00:00
81754d80b3
refactor(targets): unify queue/connectivity handling and coverage ( #2953 )
...
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: marshawcoco <marshawcoco@gmail.com >
2026-05-14 04:31:23 +00:00
c1bcee327c
fix(notify): keep live listen events active when disabled ( #2952 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-13 17:10:14 +00:00
28c9358482
test(notify): cover encoded key target union ( #2934 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-12 15:20:06 +00:00
07a01a1123
fix(notify): match filters against decoded event keys ( #2921 )
...
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-11 15:01:37 +00:00
bafff6d207
feat(targets): add check_mysql_server_available probe function ( #2884 )
...
Signed-off-by: JaySon-Huang <tshent@qq.com >
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-10 11:53:45 +00:00
81ad48dac2
feat(targets): add AMQP support for notify and audit ( #2879 )
...
Co-authored-by: Hyesook Yun <74169420+suk13574@users.noreply.github.com >
2026-05-09 01:56:26 +00:00
5431b9273d
feat(targets): complete redis mysql postgres target wiring ( #2842 )
...
Signed-off-by: jaehanbyun <awbrg789@naver.com >
Signed-off-by: houseme <housemecn@gmail.com >
Signed-off-by: Gunther Xing <jiengup@gmail.com >
Signed-off-by: JaySon-Huang <tshent@qq.com >
Co-authored-by: jaehanbyun <awbrg789@naver.com >
Co-authored-by: Gunther Xing <jiengup@gmail.com >
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com >
Co-authored-by: cxymds <Cxymds@qq.com >
Co-authored-by: JaySon <tshent@qq.com >
Co-authored-by: 安正超 <anzhengchao@gmail.com >
2026-05-07 10:00:59 +00:00
e19b0dd997
fix(notify): improve webhook target diagnostics for env setup ( #2810 )
2026-05-06 00:28:46 +00:00
368ef0f16c
refactor(targets): unify endpoint source/merge logic and bump rustfs-kafka-async to v1.2.0 ( #2654 )
...
Co-authored-by: Filipe Monteiro <a22407332@alunos.ulht.pt >
Co-authored-by: cxymds <Cxymds@qq.com >
Co-authored-by: weisd <im@weisd.in >
Co-authored-by: loverustfs <hello@rustfs.com >
2026-04-23 09:14:36 +00:00
3ac1d2ab0b
fix(security): redact target debug logs and remove eval-based bench hook ( #2637 )
2026-04-21 21:21:01 +00:00
3796b684f0
feat(targets): add NATS and Pulsar target support ( #2618 )
2026-04-21 00:28:19 +00:00
1cbf156559
refactor(obs): migrate metrics runtime/schema and tighten migration guards ( #2584 )
2026-04-18 07:51:15 +00:00
Zhengchao An