a80699b6dd
feat: add an opt-in NATS JetStream publish path for the notify and audit targets ( #4634 )
...
feat(targets): add an opt-in NATS JetStream publish path for the notify and audit targets
The NATS notify and audit targets publish through NATS Core, which returns
before the server has durably accepted the message. A broker restart or a
connection drop between the publish and the flush loses the event, even though
the send queue has already cleared it, and no acknowledgement gates that clear.
An opt-in JetStream publish path clears a queued event only after the server
returns a durable PublishAck, so delivery is at-least-once across a broker
restart or a reconnect. It applies to both the notify and audit NATS targets, is
off by default, and is byte-identical to the NATS Core path when disabled.
The path includes durable store-and-forward, a stable dedup id sent as the
Nats-Msg-Id header so a replayed event is collapsed by the stream duplicate
window, pre-flight stream validation, and a bounded failed-events store for
terminally-failed and retry-exhausted events. Three configuration keys per
target select it: JETSTREAM_ENABLE, JETSTREAM_STREAM_NAME, and
JETSTREAM_ACK_TIMEOUT_SECS, under the RUSTFS_NOTIFY_NATS_ and RUSTFS_AUDIT_NATS_
prefixes.
The on-disk batch filename separator changes from colon to underscore so
batch names are valid on Windows filesystems, with transparent read-back
of files written under the previous separator. The migration affects the
shared queue store for every target type and lands with this feature
because the store gains its first Windows-exercised paths here.
Co-authored-by: houseme <housemecn@gmail.com >
2026-07-14 15:36:14 +08:00
e12d63234a
test(targets): cover Kafka check SASL validation ( #3136 )
...
test(targets): cover kafka check sasl validation
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-31 04:47:21 +00:00
5bdbd66d09
feat(targets): support Kafka SASL auth ( #3128 )
...
* feat(targets): support Kafka SASL auth
* fix(targets): infer Kafka SASL when enable is blank
---------
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com >
2026-05-30 11:01:17 +00:00
d74e6eb042
refactor(tls): centralize runtime foundation ( #3065 )
...
* refactor(targets): move notify net helpers from utils
* refactor(tls): centralize runtime foundation
* refactor(targets): move notify net helpers from utils
* refactor(tls): centralize runtime foundation
* feat(tls-runtime): add TLS debug state and admin handler
* refactor(tls-runtime): unify TLS debug consumer status view
* fix(tls): address PR3065 review feedback
* refactor(tls): align debug status payload types
* refactor(targets): harden TLS hot reload paths
* fix(targets): resolve review-4348251652 findings
* fix(targets): finalize tls runtime review follow-ups
* fix(targets): harden tls reload and review follow-ups
* fix(targets): align tls reload handling across targets
* fix(targets): finalize tls reload state and metrics updates
* chore(deps): trim unused TLS deps
* style(targets): normalize TLS reload formatting
* refactor(targets): introduce tls runtime adapter path
* chore: update workspace manifests for tls refactor
* fix(tls): stabilize material reload and audit workflow
* fix(targets): refresh tls fingerprint flow across sinks
* fix(tls): align runtime coordinator and http reader updates
* fix(sftp): simplify protocol error mapping
* fix(tls): harmonize material loading behavior
* fix(server): finalize tls material wiring in startup flow
* fix(protos): tighten tls generation cache and deps
2026-05-24 06:41:15 +00:00
404fc816e9
test(targets): cover MySQL probe validation ( #2907 )
2026-05-11 01:55:48 +00:00
bafff6d207
feat(targets): add check_mysql_server_available probe function ( #2884 )
...
Signed-off-by: JaySon-Huang <tshent@qq.com >
Co-authored-by: houseme <housemecn@gmail.com >
2026-05-10 11:53:45 +00:00
81ad48dac2
feat(targets): add AMQP support for notify and audit ( #2879 )
...
Co-authored-by: Hyesook Yun <74169420+suk13574@users.noreply.github.com >
2026-05-09 01:56:26 +00:00
5431b9273d
feat(targets): complete redis mysql postgres target wiring ( #2842 )
...
Signed-off-by: jaehanbyun <awbrg789@naver.com >
Signed-off-by: houseme <housemecn@gmail.com >
Signed-off-by: Gunther Xing <jiengup@gmail.com >
Signed-off-by: JaySon-Huang <tshent@qq.com >
Co-authored-by: jaehanbyun <awbrg789@naver.com >
Co-authored-by: Gunther Xing <jiengup@gmail.com >
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com >
Co-authored-by: cxymds <Cxymds@qq.com >
Co-authored-by: JaySon <tshent@qq.com >
Co-authored-by: 安正超 <anzhengchao@gmail.com >
2026-05-07 10:00:59 +00:00
368ef0f16c
refactor(targets): unify endpoint source/merge logic and bump rustfs-kafka-async to v1.2.0 ( #2654 )
...
Co-authored-by: Filipe Monteiro <a22407332@alunos.ulht.pt >
Co-authored-by: cxymds <Cxymds@qq.com >
Co-authored-by: weisd <im@weisd.in >
Co-authored-by: loverustfs <hello@rustfs.com >
2026-04-23 09:14:36 +00:00
3796b684f0
feat(targets): add NATS and Pulsar target support ( #2618 )
2026-04-21 00:28:19 +00:00
d6158c0481
feat(mqtt): migrate client and harden TLS config ( #2413 )
...
Signed-off-by: houseme <housemecn@gmail.com >
Co-authored-by: heihutu <heihutu@gmail.com >
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
Co-authored-by: 安正超 <anzhengchao@gmail.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-04-07 15:28:29 +08:00
9ce3c7742c
fix(targets): pass credentials to MQTT broker availability check ( #2192 )
...
Signed-off-by: houseme <housemecn@gmail.com >
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com >
2026-03-17 23:53:28 +08:00
b7964081ce
Fix KMS configuration synchronization across cluster nodes ( #855 )
...
* Initial plan
* Add KMS configuration persistence to cluster storage
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* Apply code formatting to KMS configuration changes
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* add comment
* fix fmt
* fix
* Fix overlapping dependabot cargo configurations
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
* improve code for comment and replace `Once_Cell` to `std::sync::OnceLock`
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com >
Co-authored-by: houseme <housemecn@gmail.com >
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com >
2025-11-16 00:05:03 +08:00
aac9b1edb7
chore: improve event and docker-compose ,Improve the permissions of the endpoint health interface, upgrade otel from 0.30.0 to 0.31.0 ( #620 )
...
* feat: improve code for notify
* upgrade starshard version
* upgrade version
* Fix ETag format to comply with HTTP standards by wrapping with quotes (#592 )
* Initial plan
* Fix ETag format to comply with HTTP standards by wrapping with quotes
Co-authored-by: overtrue <1472352+overtrue@users.noreply.github.com >
* bufigx
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com >
Co-authored-by: overtrue <1472352+overtrue@users.noreply.github.com >
Co-authored-by: overtrue <anzhengchao@gmail.com >
* Improve lock (#596 )
* improve lock
Signed-off-by: Mu junxiang <1948535941@qq.com >
* feat(tests): add wait_for_object_absence helper and improve lifecycle test reliability
Signed-off-by: Mu junxiang <1948535941@qq.com >
* chore: remove dirty docs
Signed-off-by: Mu junxiang <1948535941@qq.com >
---------
Signed-off-by: Mu junxiang <1948535941@qq.com >
* feat(append): implement object append operations with state tracking (#599 )
* feat(append): implement object append operations with state tracking
Signed-off-by: junxiang Mu <1948535941@qq.com >
* chore: rebase
Signed-off-by: junxiang Mu <1948535941@qq.com >
---------
Signed-off-by: junxiang Mu <1948535941@qq.com >
* build(deps): upgrade s3s (#595 )
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com >
* fix: validate mqtt broker
* improve code for `import`
* upgrade otel relation crates version
* fix：dep("jsonwebtoken") feature = 'rust_crypto'
* fix
* fix
* fix
* upgrade version
* improve code for ecfs
* chore: improve event and docker-compose ,Improve the permissions of the `endpoint` health interface
* fix
* fix
* fix
* fix
* improve code
* fix
---------
Signed-off-by: Mu junxiang <1948535941@qq.com >
Signed-off-by: junxiang Mu <1948535941@qq.com >
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com >
Co-authored-by: overtrue <1472352+overtrue@users.noreply.github.com >
Co-authored-by: overtrue <anzhengchao@gmail.com >
Co-authored-by: guojidan <63799833+guojidan@users.noreply.github.com >
Co-authored-by: Nugine <nugine@foxmail.com >
Co-authored-by: loverustfs <155562731+loverustfs@users.noreply.github.com >
2025-10-11 09:08:25 +08:00
escapecode