gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-08-05 12:57:42 +00:00
Code Issues Packages Projects Releases Wiki Activity
4,168 Commits 28 Branches 119 Tags
df9cbc4ed16efce0d5155b507d0d34afc428c170
Commit Graph

60 Commits

Author SHA1 Message Date
Zhengchao An 2adf33fcd2 fix(targets): surface truncated store batches instead of returning partial entries (#4423) 
fix(targets): surface truncated store batches instead of silently returning partial entries (backlog#967)

get_multiple deserializes a persisted batch by pulling item_count items
from a concatenated JSON stream. When the stream ended early (a truncated
or corrupted batch file), the partial-read branch only logged a warn! and
broke out of the loop, returning Ok(partial). Callers then treated the
batch as fully delivered, deleted the store entry, and permanently lost
the remaining events with no error.

Return StoreError::Deserialization as soon as the stream ends before
item_count items are read, so the caller keeps the entry on disk and can
retry rather than silently dropping events. The empty-file case already
returned an error and remains covered by this path.

Add a regression test that writes a 3-item batch, truncates the file at a
clean boundary after two items, and asserts get_multiple returns Err and
leaves the batch file in place.

Refs: https://github.com/rustfs/backlog/issues/967
 2026-07-08 15:02:02 +08:00
Zhengchao An fb0e2d6d8f refactor(plugins): harden target-plugin system and admin plugin contract (#4217) 2026-07-03 10:18:21 +08:00
houseme 70e1d79dfd feat: replace jemalloc with mimalloc (#4174) 
* feat: replace jemalloc with mimalloc

* docs: record allocator rounds5 retest

* fix(replication): satisfy clippy unwrap lints

* docs: keep allocator migration plan local only

* feat(profiling): rely on pyroscope cpu profiling

* refactor(profiling): centralize unsupported pprof responses

* chore(deps): update s3s revision
 2026-07-02 19:03:38 +08:00
Zhengchao An 710ae74cde perf: add S3 operations benchmark framework (#738) (#4005) 2026-06-28 18:02:41 +08:00
Zhengchao An f0ab812213 fix: replace unwrap() with expect() in remaining files (#729 batch 12) (#3992) 2026-06-28 11:45:03 +08:00
houseme 583a23bdf2 fix(ecstore): replace panic-driven pool and set stubs (#3753) 
* fix(ecstore): replace panic-driven pool and set stubs

* test(runtime): tolerate restricted local bind checks

* fix(ecstore): remove remaining trait stub placeholders

* fix(ecstore): tighten trait stub follow-up semantics

* chore: ignore local worktrees

* chore: update layer dependency baseline for resolve_* context entries

Add 7 accepted infra->app dependency entries introduced by recent
refactoring PRs (#3770, #3771, #3772) that route global state lookups
through app::context::resolve_* functions.

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
 2026-06-23 12:31:17 +08:00
安正超 5ad47d1354 refactor: publish ops profiler runtime contract (#3643) 2026-06-20 09:42:13 +08:00
houseme 8cf11af649 fix(security): add shared outbound egress guard (#3567) 
* fix(security): block unsafe outbound webhook targets

* chore: keep issue-3557 plan local only
 2026-06-18 15:30:09 +08:00
houseme e8012bd1ba refactor(logging): normalize admin telemetry and error messages (#3430) 2026-06-14 13:27:10 +08:00
安正超 f7bf4fd3e1 feat(targets): gate external plugin flow (#3393) 2026-06-12 22:17:54 +08:00
安正超 5f7c991359 feat(targets): add extension hook diagnostics contracts (#3390) 
Co-authored-by: houseme <housemecn@gmail.com>
 2026-06-12 21:17:34 +08:00
安正超 e80b72ae79 feat(targets): gate sidecar runtime policy (#3388) 
* feat(targets): gate sidecar runtime policy

* fix(admin): add extension route policy specs
 2026-06-12 17:42:12 +08:00
安正超 1e7c376586 feat(targets): add extension schema adapter (#3385) 2026-06-12 15:27:55 +08:00
安正超 249a0b56ea feat(targets): require plugin artifact attestations (#3383) 2026-06-12 14:13:41 +08:00
安正超 ca58d7f0ec refactor(config): migrate server config consumers (#3353) 2026-06-11 17:04:50 +08:00
安正超 a73c90c811 security: redact IAM and target debug secrets (#3306) 2026-06-10 09:03:50 +08:00
安正超 e12d63234a test(targets): cover Kafka check SASL validation (#3136) 
test(targets): cover kafka check sasl validation

Co-authored-by: houseme <housemecn@gmail.com>
 2026-05-31 04:47:21 +00:00
Henry Guo 5bdbd66d09 feat(targets): support Kafka SASL auth (#3128) 
* feat(targets): support Kafka SASL auth

* fix(targets): infer Kafka SASL when enable is blank

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
 2026-05-30 11:01:17 +00:00
安正超 ac97ceb744 fix(config): restore default credential startup (#3114) 
* fix(config): restore default credential startup

* fix: align e2e credentials with server env

* fix(config): restore default credential consistency

---------

Co-authored-by: houseme <housemecn@gmail.com>
 2026-05-29 11:52:46 +00:00
houseme d74e6eb042 refactor(tls): centralize runtime foundation (#3065) 
* refactor(targets): move notify net helpers from utils

* refactor(tls): centralize runtime foundation

* refactor(targets): move notify net helpers from utils

* refactor(tls): centralize runtime foundation

* feat(tls-runtime): add TLS debug state and admin handler

* refactor(tls-runtime): unify TLS debug consumer status view

* fix(tls): address PR3065 review feedback

* refactor(tls): align debug status payload types

* refactor(targets): harden TLS hot reload paths

* fix(targets): resolve review-4348251652 findings

* fix(targets): finalize tls runtime review follow-ups

* fix(targets): harden tls reload and review follow-ups

* fix(targets): align tls reload handling across targets

* fix(targets): finalize tls reload state and metrics updates

* chore(deps): trim unused TLS deps

* style(targets): normalize TLS reload formatting

* refactor(targets): introduce tls runtime adapter path

* chore: update workspace manifests for tls refactor

* fix(tls): stabilize material reload and audit workflow

* fix(targets): refresh tls fingerprint flow across sinks

* fix(tls): align runtime coordinator and http reader updates

* fix(sftp): simplify protocol error mapping

* fix(tls): harmonize material loading behavior

* fix(server): finalize tls material wiring in startup flow

* fix(protos): tighten tls generation cache and deps
 2026-05-24 06:41:15 +00:00
houseme 73bde843d6 refactor(s3): consolidate semantic boundaries and remove s3-common (#3012) 
* refactor(common): introduce rustfs-data-usage core crate

* refactor(concurrency): migrate workers crate into concurrency

* refactor(crypto): migrate appauth token APIs into crypto

* fix docs urls

* remove unused crate

* refactor(data-usage): switch consumers to rustfs-data-usage

* chore(fmt): apply cargo fmt and lockfile sync

* refactor(common): remove data_usage compatibility re-export

* refactor(capacity): move capacity_scope to object-capacity

* refactor(io-metrics): relocate internode metrics from common

* refactor(common): decouple scanner report from madmin

* chore(fmt): normalize import ordering after pre-commit

* refactor(s3): split s3 types and ops crates

* refactor(s3): centralize event version and safe parsing

* refactor(s3): add op-event compatibility guardrails

* refactor(s3): add runtime op-event mismatch observability

* refactor(s3): extract delete event mapping helper

* refactor(s3): extract put event mapping helper

* refactor(s3): consolidate remaining event semantic helpers

* refactor(s3): add op-event coverage checks and observability alerts

* refactor(s3-ops): consolidate op-event semantic mapping

* refactor(scanner): remove last_minute wrapper module

* refactor(scanner): consolidate duplicated data usage models
 2026-05-19 12:50:25 +00:00
Henry Guo bca8b08c2b fix: handle Windows paths in pre-commit tests (#2974) 
Co-authored-by: houseme <housemecn@gmail.com>
 2026-05-15 14:04:51 +00:00
houseme 81754d80b3 refactor(targets): unify queue/connectivity handling and coverage (#2953) 
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: marshawcoco <marshawcoco@gmail.com>
 2026-05-14 04:31:23 +00:00
安正超 404fc816e9 test(targets): cover MySQL probe validation (#2907) 2026-05-11 01:55:48 +00:00
安正超 0375dd39bb fix(targets): handle MySQL DSN scheme case (#2903) 
Co-authored-by: houseme <housemecn@gmail.com>
 2026-05-10 14:25:22 +00:00
JaySon bafff6d207 feat(targets): add check_mysql_server_available probe function (#2884) 
Signed-off-by: JaySon-Huang <tshent@qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-05-10 11:53:45 +00:00
JaySon cb54cecf8c docs(targets): sync AGENTS.md and test doc comments with code (#2881) 
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
 2026-05-09 03:06:11 +00:00
安正超 9288f1ed3a fix(targets): handle postgres dsn redaction scheme case (#2886) 2026-05-09 01:58:23 +00:00
houseme 81ad48dac2 feat(targets): add AMQP support for notify and audit (#2879) 
Co-authored-by: Hyesook Yun <74169420+suk13574@users.noreply.github.com>
 2026-05-09 01:56:26 +00:00
安正超 9df57b8cd8 test(targets): cover Redis env config loading (#2857) 2026-05-07 13:08:11 +00:00
houseme fd37a7d01e fix(targets): probe webhook health by host port (#2854) 2026-05-07 11:52:45 +00:00
houseme 5431b9273d feat(targets): complete redis mysql postgres target wiring (#2842) 
Signed-off-by: jaehanbyun <awbrg789@naver.com>
Signed-off-by: houseme <housemecn@gmail.com>
Signed-off-by: Gunther Xing <jiengup@gmail.com>
Signed-off-by: JaySon-Huang <tshent@qq.com>
Co-authored-by: jaehanbyun <awbrg789@naver.com>
Co-authored-by: Gunther Xing <jiengup@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: JaySon <tshent@qq.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
 2026-05-07 10:00:59 +00:00
houseme 368ef0f16c refactor(targets): unify endpoint source/merge logic and bump rustfs-kafka-async to v1.2.0 (#2654) 
Co-authored-by: Filipe Monteiro <a22407332@alunos.ulht.pt>
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: weisd <im@weisd.in>
Co-authored-by: loverustfs <hello@rustfs.com>
 2026-04-23 09:14:36 +00:00
安正超 8add0126f5 test(targets): cover target config redaction (#2638) 2026-04-22 02:51:41 +00:00
houseme 3ac1d2ab0b fix(security): redact target debug logs and remove eval-based bench hook (#2637) 2026-04-21 21:21:01 +00:00
安正超 1525143a04 test: cover NATS and Pulsar config validation (#2623) 2026-04-21 01:59:05 +00:00
houseme b7a945e453 fix(madmin): restore server_info msgpack compatibility across mixed nodes (#2621) 2026-04-21 01:09:25 +00:00
houseme 3796b684f0 feat(targets): add NATS and Pulsar target support (#2618) 2026-04-21 00:28:19 +00:00
houseme 1cbf156559 refactor(obs): migrate metrics runtime/schema and tighten migration guards (#2584) 2026-04-18 07:51:15 +00:00
Tunglies 49366ee200 chore(lint): clippy rules redundant_clone (#2554) 2026-04-15 13:54:07 +00:00
Tunglies f57dd5a3c7 chore(lint): clippy rules needless_collect (#2522) 2026-04-15 10:00:03 +00:00
houseme 979626c370 refactor(utils): decouple config deps and move sys helpers (#2520) 
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
 2026-04-13 21:05:03 +08:00
houseme d6158c0481 feat(mqtt): migrate client and harden TLS config (#2413) 
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-04-07 15:28:29 +08:00
houseme d2901fd78c feat(admin): add audit target APIs and harden target source handling (#2350) 
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
 2026-04-04 09:07:22 +08:00
houseme 9ce3c7742c fix(targets): pass credentials to MQTT broker availability check (#2192) 
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
 2026-03-17 23:53:28 +08:00
houseme 60aa47bf61 feat(storage): integrate S3Operation into OperationHelper for unified metrics and audit (#2103) 2026-03-08 17:57:33 +08:00
houseme d17d2083d4 feat(targets): enhance webhook TLS support with custom CA and skip-verify (#1994) 
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: heihutu <heihutu@gmail.com>
 2026-02-27 21:24:49 +08:00
Audric 548a39ffe7 fix: return error instead of silently ignoring invalid ARNs in notification config (#1528) 
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2026-01-16 16:12:55 +08:00
houseme a95e549430 Fix/fix improve for audit (#1418) 2026-01-07 18:05:52 +08:00
houseme 8d7cd4cb1b chore: upgrade dependencies and migrate to aws-lc-rs (#1333) 2026-01-02 00:02:34 +08:00