Compare commits

...

864 Commits

Author SHA1 Message Date
houseme 658b8dea66 fix: unify runtime readiness publication and graceful shutdown flow (#3087)
* fix(server): unify runtime readiness and shutdown flow

* refactor(server): decouple readiness shared types

* refactor(server): keep readiness types crate-private

* refactor(server): await protocol shutdown handles

* fix(server): bypass cached startup readiness
2026-05-26 18:49:42 +00:00
Henry Guo 0478505839 fix(heal): restore single disk data during deep heal (#3085)
* fix(heal): restore single disk data during deep heal

* fix(heal): clarify erasure heal step logging

* fix(heal): avoid duplicate deep object heals

* fix(ecstore): preserve volume-not-found walk errors

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-26 17:36:17 +00:00
安正超 0875f09a39 fix: rebuild wiped disks during admin heal (#3084)
* fix: rebuild wiped disks during admin heal

* Preserve forceStart heal admission semantics

* Address heal regression test review comments

* fix(heal): address admin heal review comments

* fix(heal): fix force-start dedup and test polling

* fix(heal): address unresolved review comments

* fix(heal): simplify dedup key for clippy

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-05-26 21:54:29 +08:00
weisd ea74fa7a95 fix(heal): rebuild parity shards during repair (#3086)
Object repair rebuilt missing data shards but left missing parity shards unrecreated. A parity-only repair could therefore complete without restoring the shard contents.

Add a repair-specific reconstruction path that regenerates parity after data shards are available, while keeping normal read decoding data-only. Also shut down repair writers after all blocks are written.

Constraint: Preserve read-path decode behavior and limit parity regeneration to repair.

Confidence: high

Scope-risk: moderate

Directive: Do not route read decoding through parity regeneration without measuring the cost.

Tested: cargo test -p rustfs-ecstore decode_data_and_parity -- --nocapture

Tested: cargo test -p rustfs-ecstore heal_reconstructs_missing_parity_shard -- --nocapture

Tested: cargo test -p rustfs-ecstore erasure_coding -- --nocapture

Tested: cargo test -p rustfs heal_object_marks_missing_shard_disk_dirty_for_capacity_manager -- --nocapture

Tested: cargo test -p rustfs-heal -- --nocapture

Tested: cargo clippy -p rustfs-ecstore --all-targets -- -D warnings

Tested: cargo fmt --all --check

Tested: make pre-commit

Not-tested: Live multi-node disk replacement outside local test harness
2026-05-26 09:49:25 +00:00
Henry Guo 62d1f80dbf fix(data-usage): refresh admin usage after object changes (#3081)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-05-25 14:24:00 +00:00
安正超 0d0a17bb36 fix: include deployment ID in admin info (#3083)
* fix: include deployment ID in admin info

* test(ecstore): avoid mutating global deployment id
2026-05-25 14:23:35 +00:00
houseme 3d2449872a refactor(credentials): derive RPC secret fallback and remove IAM keygen duplication (#3079)
* refactor(credentials): derive rpc secret and remove iam keygen

* fix(credentials): reject default access key RPC secret

* test(credentials): align RPC fallback and add keygen coverage
2026-05-25 11:05:58 +00:00
Demo Macro 5f5b1207e0 fix(ecstore): correct is_truncated logic in ListObjectsV2 pagination (#2997) 2026-05-25 14:59:19 +08:00
Sergei Z. 64feca3550 fix(user): service account expiration handling with RFC3339 (#3078)
fix(user): enhance service account expiration handling with RFC3339 support

Co-authored-by: houseme <housemecn@gmail.com>
2026-05-25 04:27:25 +00:00
houseme f1207a9e28 docs(skills): quote release bump description (#3077)
* docs(skills): quote release bump description

* chore(deps): bump lapin and rumqttc-next
2026-05-24 19:59:29 +00:00
weisd f9475e10dc fix(replication): preserve multipart pending state (#3058)
* Preserve multipart replication recovery state

Multipart uploads previously only scheduled replication after completion, leaving no persisted pending state for scanner recovery if the initial async work was lost. Persist the same pending replication metadata during multipart initialization and let completion evaluate the object metadata that was actually stored.

The scanner heal path also treated ordinary pending objects as delete-replication candidates. Restrict that path to delete markers and version purge state so pending objects remain eligible for object replication heal.

Constraint: Bucket replication recovery depends on persisted object metadata after the async queue is unavailable.
Rejected: Rely only on immediate completion-time scheduling | it cannot recover after process restart or worker loss.
Confidence: high
Scope-risk: moderate
Directive: Keep multipart upload initialization aligned with single PUT replication metadata semantics.
Tested: cargo fmt --all --check
Tested: cargo clippy --workspace --all-features --all-targets -- -D warnings
Tested: LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8 make pre-commit
Tested: Runtime replication outage check confirmed multipart xl.meta stores PENDING status and timestamp.

* fix(replication): preserve version purge scanner state

Role-derived replication configs need target-scoped status strings before scanner heal can build per-target purge status. The duplicated replication-status assignment left version purge status unset, so scanner recovery could lose the target-level purge state.

Constraint: Scanner heal derives per-target purge decisions from version_purge_status_internal.
Rejected: Leave the duplicate as a harmless cleanup | it changes recovery behavior for role-only configs with version purge state.
Confidence: high
Scope-risk: narrow
Directive: Keep role-derived replication and version purge internal status mapping symmetric.
Tested: cargo fmt --all --check
Tested: cargo test -p rustfs-ecstore heal -- --nocapture

---------

Co-authored-by: wly <wlywly0735@126.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-24 14:24:00 +00:00
GatewayJ b0646be756 feat(s3select): improve SelectObjectContent streaming (#3072)
* feat(s3select): improve SelectObjectContent streaming

* fix(s3select): reject empty select expressions

* fix(s3select): address streaming review feedback

---------

Co-authored-by: loverustfs <hello@rustfs.com>
2026-05-24 14:23:47 +00:00
houseme d74e6eb042 refactor(tls): centralize runtime foundation (#3065)
* refactor(targets): move notify net helpers from utils

* refactor(tls): centralize runtime foundation

* refactor(targets): move notify net helpers from utils

* refactor(tls): centralize runtime foundation

* feat(tls-runtime): add TLS debug state and admin handler

* refactor(tls-runtime): unify TLS debug consumer status view

* fix(tls): address PR3065 review feedback

* refactor(tls): align debug status payload types

* refactor(targets): harden TLS hot reload paths

* fix(targets): resolve review-4348251652 findings

* fix(targets): finalize tls runtime review follow-ups

* fix(targets): harden tls reload and review follow-ups

* fix(targets): align tls reload handling across targets

* fix(targets): finalize tls reload state and metrics updates

* chore(deps): trim unused TLS deps

* style(targets): normalize TLS reload formatting

* refactor(targets): introduce tls runtime adapter path

* chore: update workspace manifests for tls refactor

* fix(tls): stabilize material reload and audit workflow

* fix(targets): refresh tls fingerprint flow across sinks

* fix(tls): align runtime coordinator and http reader updates

* fix(sftp): simplify protocol error mapping

* fix(tls): harmonize material loading behavior

* fix(server): finalize tls material wiring in startup flow

* fix(protos): tighten tls generation cache and deps
2026-05-24 06:41:15 +00:00
安正超 8be787387c test(utils): cover bracketed IPv6 zone host parsing (#3073) 2026-05-24 03:18:56 +00:00
houseme c9377161e4 chore(deps): update flake.lock (#3074)
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/d233902' (2026-05-15)
  → 'github:NixOS/nixpkgs/3d8f0f3' (2026-05-23)
• Updated input 'rust-overlay':
    'github:oxalica/rust-overlay/61ec6a4' (2026-05-16)
  → 'github:oxalica/rust-overlay/d9973e2' (2026-05-23)
2026-05-24 03:18:41 +00:00
Henry Guo 522605a055 test(internode): cover adapter metrics validation (#3071)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-05-23 15:23:05 +00:00
安正超 22bcfc474e perf(ecstore): use direct std writes for local disk (#3069)
* perf(ecstore): use direct std writes for local disk

* fix(ecstore): avoid blocking disk writes in local disk path

* fix(ecstore): use tokio async write in local disk path

* chore(ecstore): remove unused AsyncWriteExt import

* fix(ecstore): preserve write semantics and avoid bytes copy

* fix(ecstore): optimize write_all_internal buffer paths

* fix(ecstore): sync write path in local disk

* fix(ecstore): align sync flag docs and avoid ref copy

* chore(rust): format local write path

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-05-23 15:22:49 +00:00
Henry Guo a28cb34381 test(internode): cover RemoteDisk adapter routing (#3070)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-05-23 09:41:25 +00:00
LeonWang0735 cbcfe625ef fix(replication): avoid skipping existing-object backfill for new targets (#2992)
* fix(replication): avoid skipping existing-object backfill for new targets

* optimize(replication): delete redundant line

---------

Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: cxymds <Cxymds@qq.com>
2026-05-23 04:53:08 +00:00
安正超 02a7d3c228 fix: derive run script CORS console port (#3068) 2026-05-23 04:52:46 +00:00
Henry Guo 53b608c089 docs(internode): keep transport adapter OSS scoped (#3067)
docs(internode): keep transport adapter oss scoped

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-05-23 13:00:36 +08:00
Henry Guo 2786a4734a docs(internode): align transport adapter scope (#3064)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-05-22 16:27:27 +00:00
GatewayJ c9f0f25f55 fix: bind run script to localhost (#3063) 2026-05-22 14:28:17 +00:00
houseme 20a4db7c9a fix(tls): resolve RUSTFS_TLS_PATH startup regression (#3059)
* fix(tls): resolve RUSTFS_TLS_PATH startup regression

* fix(tls): adopt PR review follow-ups
2026-05-22 09:09:31 +00:00
houseme 6264be437c fix(storage): add scoped timeout policy and startup fs guardrail (#3056)
* fix(storage): add RUSTFS_NETWORK_MOUNT_MODE for CIFS/NFS backends

* style: fix cargo fmt formatting in disk_store.rs

* fix(storage): add RUSTFS_NETWORK_MOUNT_MODE for CIFS/NFS backends

Extend the TimeoutHealthAction introduced in #2996 to read_metadata,
list_dir, and disk_info operations when RUSTFS_NETWORK_MOUNT_MODE=true.
Also raises all drive operation timeouts to 60s (explicit per-operation
overrides still take precedence).

Closes #2790

* feat(startup): add unsupported filesystem policy guardrail

* chore(deps): refresh lockfile and dependency pins

* feat(ecstore): add scoped timeout health-action policy

* docs(config): document drive timeout health-action policy

* refactor(ecstore): cache timeout health policy per disk wrapper

* fix(storage): add RUSTFS_NETWORK_MOUNT_MODE for CIFS/NFS backends (#2838)

* fix(storage): add RUSTFS_NETWORK_MOUNT_MODE for CIFS/NFS backends

* style: fix cargo fmt formatting in disk_store.rs

* fix(storage): add RUSTFS_NETWORK_MOUNT_MODE for CIFS/NFS backends

Extend the TimeoutHealthAction introduced in #2996 to read_metadata,
list_dir, and disk_info operations when RUSTFS_NETWORK_MOUNT_MODE=true.
Also raises all drive operation timeouts to 60s (explicit per-operation
overrides still take precedence).

Closes #2790

* fix(utils): map verified Linux filesystem magic values (#3051)

* fix(utils): cover sha256 checksum validation (#3052)

* fix(utils): cover sha256 checksum validation

* docs: clarify sha256 checksum validation

---------

Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>

* refactor(config): replace network mount mode with timeout profile preset

* fix(review): align fallback defaults and extend fs-type detection

* fix(review): cache timeout profile and restore probe timeout semantics

* refactor(ecstore): cache timeout health policy lookup

* perf(ecstore): cache active probe timeout per monitor task

---------

Co-authored-by: mistik <mistiklord4@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-05-22 06:37:30 +00:00
weisd 69345fe059 fix(scanner): preserve background heal compatibility (#3041) 2026-05-22 14:38:35 +08:00
Henry Guo b42766f1d3 feat(internode): define transport capabilities (#3047)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-22 02:56:37 +00:00
安正超 3e3fcbf44d fix(utils): cover sha256 checksum validation (#3052)
* fix(utils): cover sha256 checksum validation

* docs: clarify sha256 checksum validation
2026-05-22 02:45:53 +00:00
安正超 0d94437788 fix(utils): map verified Linux filesystem magic values (#3051) 2026-05-22 02:37:50 +00:00
houseme ed2078e025 fix(ecstore): allow expired delete markers on locked buckets (#3048)
* fix: allow expired delete markers on locked buckets

* fix: reject zero-day del marker expiration
2026-05-22 01:39:53 +00:00
Henry Guo 2404bc1657 docs(internode): inventory transport data paths (#3040)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-05-21 15:40:39 +00:00
Henry Guo 0985f0b37b feat(internode): label transport operation metrics (#3045)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-21 15:20:39 +00:00
houseme e69e32285b fix(ecstore): harden multipart part metadata visibility (#3042)
* fix(ecstore): harden multipart part metadata visibility

* fix(ecstore): address PR3042 review follow-ups
2026-05-21 14:47:52 +00:00
Henry Guo 97858baf8e docs(internode): analyze buffer lifecycle (#3046) 2026-05-21 22:52:39 +08:00
Henry Guo 69dcf9e6cb fix(tooling): harden internode transport benchmark setup (#3037)
* refactor(config): centralize internode transport constants

* fix(bench): guard all ripgrep calls behind dry-run check

Move require_cmd rg and metrics collection inside the non-dry-run
path so that --dry-run works on hosts without rg installed.

* feat(tooling): cross-platform protoc setup for Linux and macOS

Make install-protoc.sh support Linux (x86_64, aarch64) alongside
macOS, and bump CI protoc from 29.3 to 33.1 to match the version
required by the gproto build script.

* fix(bench): record internode baseline error counts

* fix(skill): correct YAML frontmatter formatting for release-version-bump

* chore(ci): bump protoc version to 34.1

* fix(tooling): bump protoc 33.1 to 34.1 in install script, restore SKILL.md description

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-05-21 05:47:32 +00:00
houseme 503f89bf0e chore(deps): bump aws sdk and config (#3035) 2026-05-21 01:47:34 +00:00
houseme dcbffb084f chore(deps): refresh workspace deps and linux fs_type gating (#3030)
* chore(deps): refresh workspace deps and linux fs_type gating

- refresh workspace dependency pins and lockfile updates

- remove now-unused crate dependency entries in multiple Cargo.toml files

- enable profiling export defaults in config and scripts/run.sh

- gate os::fs_type module/function/tests to Linux to avoid non-Linux dead_code warnings

* fix(utils): simplify fs_type linux gating

- keep fs_type module-level linux cfg in os::mod

- remove redundant linux cfg on get_fs_type and test module

* chore(deps): bump s3s git revision

- update workspace s3s dependency to rev 507e1312b211c3ddc214b03875d6fabd15d22ed5

- refresh Cargo.lock source entry for s3s

* chore(dev): allow mysql_async git source and env overrides

- allow mysql_async git source in deny.toml allow-git list

- make scripts/run.sh core env vars overrideable via existing shell env

* fix(utils): import get_fs_type in fs_type tests

- add explicit super::get_fs_type import in fs_type test module

- fix Linux E0425 unresolved function errors in unit tests

* chore(dev): tune run script observability defaults

- make profiling export env overrideable in scripts/run.sh

- set RUSTFS_OBS_SAMPLE_RATIO default from 2.0 to 1.0

- update allow-git review window comments in deny.toml

* test(obs): stabilize profiling env alias tests
2026-05-20 15:03:07 +00:00
houseme b2dfdf85de chore(release): prepare 1.0.0-beta.4 (#3032)
* chore(release): prepare 1.0.0-beta.4

* docs(skill): refine rustfs spec changelog rule

* docs(skill): optimize rustfs release bump workflow
2026-05-20 13:56:43 +00:00
houseme 375482a4b6 refactor: converge storage io hot paths (#3029)
* refactor(issue-633): clarify layered io control policies

* refactor(issue-633): consolidate timeout and deadlock layers

* refactor(issue-633): align storage backpressure metadata

* refactor(issue-633): unify storage backpressure transitions

* refactor(issue-633): simplify watermark transition API

* test(issue-633): add storage backpressure transition test

* refactor(issue-633): align storage pipe meta shape

* refactor(issue-633): enrich storage monitor metadata

* refactor(issue-633): finalize storage backpressure convergence

* refactor(issue-633): complete scheduler layer convergence

* refactor(issue-633): reduce concurrency facade config duplication

* refactor(issue-633): migrate storage callsites to final policy names

* chore(issue-633): apply final pre-commit normalization

* refactor(issue-633): unify timeout wrapper dynamic size path

* refactor(issue-633): make concurrency policies copyable

* refactor(issue-633): converge storage io hot paths

* fix(issue-633): honor storage timeout min bound

* fix(storage): avoid timeout calc panic on huge sizes

* refactor(storage): consolidate timeout checks and test attrs

* fix(storage): harden io scheduler core config mapping

* refactor(storage): eliminate patch-on-patch patterns and dead code

- Remove trivial accessor methods on ConcurrencyConfig that just return pub fields
- Remove dead BackpressureEvent/BackpressureEventType types from concurrency crate
- Fix io_schedule test using wrong constructor (from_core_config -> from_scheduler_config)
- Update manager.rs to use config fields directly instead of removed accessors

* fix: adopt review feedback for config guards

* test: remove needless struct update defaults

* fix: harden timeout policy and preserve api alias
2026-05-20 12:00:23 +00:00
Henry Guo 45b04316b9 refactor(ecstore): decouple transport construction from RemoteDisk (#3027)
Move build_internode_data_transport_from_env() out of RemoteDisk::new()
into new_disk(), so RemoteDisk depends only on the
Arc<dyn InternodeDataTransport> trait object. TCP/HTTP remains the
default backend; all data paths unchanged.

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-05-20 11:55:08 +00:00
Henry Guo 19b69abe5c feat(internode): harden p0 transport boundary and baseline tooling (#3017)
* feat(internode): p0 transport baseline and ci hardening

* fix(internode): avoid double wrapping transport errors

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-20 11:06:26 +00:00
安正超 c684438625 fix(obs): add proxied PUT replication metrics (#3020)
fix(obs): add proxied put replication metrics
2026-05-20 05:30:57 +00:00
安正超 11054263c1 fix(utils): map common Linux filesystem magic values (#3023) 2026-05-20 04:00:32 +00:00
安正超 b530a9f0b2 fix(ecstore): remove stale bucket metadata parse TODO (#3021)
* fix(ecstore): remove stale bucket metadata parse TODO

* test: cover stored bucket target config parsing
2026-05-20 03:59:55 +00:00
安正超 dd35538e90 fix(ecstore): remove stale disk TODOs (#3022) 2026-05-20 03:59:34 +00:00
安正超 c727589161 fix(obs): remove stale replication metric TODOs (#3024) 2026-05-20 03:59:16 +00:00
安正超 cde53ca6ad fix(utils): handle IPv6 zones and hex ranges (#3019)
* fix(utils): handle low-risk TODO range parsing

* fix(utils): address scoped IPv6 and range review
2026-05-20 03:56:14 +00:00
Henry Guo e929814edc feat(ecstore): add internode transport boundary and TCP baseline runner (#3010)
* docs: add internode data transport RFC

* feat: add internode operation metrics

* fix feedback

* fix(ci): fallback protoc token to github.token

* feat(ecstore): add internode transport boundary and baseline runner

* feat(internode): harden data transport baseline

* Revert "feat(internode): harden data transport baseline"

This reverts commit 5b8d6b8aa4.

* fix(internode): address baseline review comments

* fix(ci): pin setup-protoc to stable release

* fix(ci): install protoc via apt on linux

* fix(ci): restore protoc install for macos and windows

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-19 22:41:34 +00:00
安正超 a8a5f25af3 perf(ecstore): remove owned write sync regression (#3018) 2026-05-19 22:14:35 +00:00
安正超 54be3cab23 fix(heal): ignore missing response subscribers (#3015)
* fix(heal): ignore missing response subscribers

* fix(heal): ignore missing response subscribers
2026-05-19 15:34:22 +00:00
安正超 1c8fdfddf4 chore(ci): pin RustFS setup-protoc release (#3016)
* chore(ci): use RustFS setup-protoc fork

* chore(ci): pin RustFS setup-protoc release
2026-05-19 15:28:51 +00:00
houseme 73bde843d6 refactor(s3): consolidate semantic boundaries and remove s3-common (#3012)
* refactor(common): introduce rustfs-data-usage core crate

* refactor(concurrency): migrate workers crate into concurrency

* refactor(crypto): migrate appauth token APIs into crypto

* fix docs urls

* remove unused crate

* refactor(data-usage): switch consumers to rustfs-data-usage

* chore(fmt): apply cargo fmt and lockfile sync

* refactor(common): remove data_usage compatibility re-export

* refactor(capacity): move capacity_scope to object-capacity

* refactor(io-metrics): relocate internode metrics from common

* refactor(common): decouple scanner report from madmin

* chore(fmt): normalize import ordering after pre-commit

* refactor(s3): split s3 types and ops crates

* refactor(s3): centralize event version and safe parsing

* refactor(s3): add op-event compatibility guardrails

* refactor(s3): add runtime op-event mismatch observability

* refactor(s3): extract delete event mapping helper

* refactor(s3): extract put event mapping helper

* refactor(s3): consolidate remaining event semantic helpers

* refactor(s3): add op-event coverage checks and observability alerts

* refactor(s3-ops): consolidate op-event semantic mapping

* refactor(scanner): remove last_minute wrapper module

* refactor(scanner): consolidate duplicated data usage models
2026-05-19 12:50:25 +00:00
houseme d6fc70eb12 feat(obs): expose key S3 usecase spans at info (#3013) 2026-05-19 11:00:51 +00:00
houseme 25c6bdf490 perf(filemeta): phase-1~3 rename_data metadata optimization (#3011)
* chore(perf): harden amd64 profiling benchmark flow

* fix(profiling): isolate bench buckets and map protobuf conflict

* perf: avoid blocking owned local writes

* style: format profile admin handler

* docs: clarify observability trace validation

* perf: reduce mkdir overhead on local writes

* perf: add rename_data meta microbenchmark

* perf(filemeta): fast-path data_dir decode in version meta

* perf(filemeta): collapse data-dir lookup into one scan

* perf(filemeta): reduce scan allocs and refresh meta bench

* perf(ecstore): skip mkdir path on read-only open

* perf(filemeta): single-pass unshared data-dir scan

* perf(filemeta): add two-key inline remove fast path

* perf(filemeta): compare remove-two keys by bytes first

* bench(ecstore): add remove_two-only micro benchmark

* bench(ecstore): stabilize rename_data meta benchmark timing

* bench(ecstore): align rename_data path with remove_two

* perf(filemeta): avoid uuid string alloc in remove_two

* perf(filemeta): add fast-path for empty inline data

* perf(filemeta): streamline add_version match branch

* perf(filemeta): fast-return remove_key on miss

* perf(filemeta): speed up add_version insertion lookup

* style(ecstore): normalize formatting in perf-tuning files

* refactor(filemeta): unify inline data removal paths
2026-05-19 10:20:24 +00:00
Henry Guo f695870626 feat(internode): add transport observability (#3007)
* docs: add internode data transport RFC

* feat: add internode operation metrics

* fix feedback

* fix(ci): fallback protoc token to github.token

---------

Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-05-19 07:16:05 +00:00
安正超 aa3f13c0d3 fix(ecstore): stop listing after reaching result limit (#3001)
* fix(ecstore): stop listing after reaching result limit

* fix(ecstore): ignore intentional list cancellation

* fix(ecstore): satisfy list cancellation clippy lint

* fix(ecstore): include entry before limit early return

* fix(ecstore): cancel on limit in gather_results

* fix(ecstore): sync timeout branch with reviewed patch

* fix(ecstore): satisfy cancellation clippy lint

---------

Co-authored-by: loverustfs <hello@rustfs.com>
2026-05-19 05:20:21 +00:00
houseme c4248078d6 chore: update dependencies (#2890)
* chore: update dependencies

* build(deps): bump the dependencies group with 5 updates

* build(deps): bump the dependencies group with 6 updates (#2908)

Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>

* fix(ecstore): narrow Windows URL drive path rewrite

* chore(deps): bump starshard to 1.2.0

* revert(ecstore): restore endpoint Windows path behavior

* up

* up

* up

* perf(notify): use cached snapshot mode for scans

* fix

* chore(deps): bump workspace dependency versions

* chore(deps): refresh pinned dependency references

* chore(ci): align profiling and decommission tooling updates

- enable tokio_unstable cfg in performance profiling build flags\n- bump Rust base image from 1.93 to 1.95 in source and decommission Dockerfiles\n- remove obsolete compose version key from docker-compose-simple.yml\n- add standard Apache-2.0 license header to docker-compose.decommission.yml

* chore(deps): bump the dependencies group across 1 directory with 7 updates (#2994)

Bumps the dependencies group with 7 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [sysinfo](https://github.com/GuillaumeGomez/sysinfo) | `0.39.1` | `0.39.2` |
| [dial9-tokio-telemetry](https://github.com/dial9-rs/dial9-tokio-telemetry) | `0.3.9` | `0.3.10` |
| [opentelemetry](https://github.com/open-telemetry/opentelemetry-rust) | `0.31.0` | `0.32.0` |
| [opentelemetry-otlp](https://github.com/open-telemetry/opentelemetry-rust) | `0.31.1` | `0.32.0` |
| [opentelemetry_sdk](https://github.com/open-telemetry/opentelemetry-rust) | `0.31.0` | `0.32.0` |
| [opentelemetry-semantic-conventions](https://github.com/open-telemetry/opentelemetry-rust) | `0.31.0` | `0.32.0` |
| [opentelemetry-stdout](https://github.com/open-telemetry/opentelemetry-rust) | `0.31.0` | `0.32.0` |



Updates `sysinfo` from 0.39.1 to 0.39.2
- [Changelog](https://github.com/GuillaumeGomez/sysinfo/blob/main/CHANGELOG.md)
- [Commits](https://github.com/GuillaumeGomez/sysinfo/compare/v0.39.1...v0.39.2)

Updates `dial9-tokio-telemetry` from 0.3.9 to 0.3.10
- [Release notes](https://github.com/dial9-rs/dial9-tokio-telemetry/releases)
- [Changelog](https://github.com/dial9-rs/dial9/blob/main/CHANGELOG.md)
- [Commits](https://github.com/dial9-rs/dial9-tokio-telemetry/compare/dial9-tokio-telemetry-v0.3.9...dial9-tokio-telemetry-v0.3.10)

Updates `opentelemetry` from 0.31.0 to 0.32.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-rust/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-rust/blob/main/docs/release_0.32.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-rust/compare/opentelemetry-prometheus-0.31.0...opentelemetry-0.32.0)

Updates `opentelemetry-otlp` from 0.31.1 to 0.32.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-rust/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-rust/blob/main/docs/release_0.32.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-rust/compare/opentelemetry-otlp-0.31.1...opentelemetry-otlp-0.32.0)

Updates `opentelemetry_sdk` from 0.31.0 to 0.32.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-rust/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-rust/blob/main/docs/release_0.32.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-rust/compare/v0.31.0...opentelemetry_sdk-0.32.0)

Updates `opentelemetry-semantic-conventions` from 0.31.0 to 0.32.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-rust/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-rust/blob/main/docs/release_0.32.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-rust/compare/v0.31.0...opentelemetry-semantic-conventions-0.32.0)

Updates `opentelemetry-stdout` from 0.31.0 to 0.32.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-rust/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-rust/blob/main/docs/release_0.32.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-rust/compare/v0.31.0...opentelemetry-stdout-0.32.0)

---
updated-dependencies:
- dependency-name: sysinfo
  dependency-version: 0.39.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: dial9-tokio-telemetry
  dependency-version: 0.3.10
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: dependencies
- dependency-name: opentelemetry
  dependency-version: 0.32.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: opentelemetry-otlp
  dependency-version: 0.32.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: opentelemetry_sdk
  dependency-version: 0.32.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: opentelemetry-semantic-conventions
  dependency-version: 0.32.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: opentelemetry-stdout
  dependency-version: 0.32.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>

* chore(deps): bump workspace dependency versions

* chore(deps): refresh lockfile windows crate graph

* chore(bench): align snapshot mode labels with coverage

* chore(bench): clarify tested snapshot modes

* fix(review): address PR2890 Copilot comments

---------

Signed-off-by: houseme <housemecn@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-19 04:58:42 +00:00
Henry Guo 94da6e34cb fix(notify): parse IPv6 hosts without ports (#3000)
Co-authored-by: Henry Guo <marshawcoco@users.noreply.github.com>
2026-05-19 02:40:21 +00:00
安正超 a9e62dc2c2 perf: avoid blocking hop for owned disk writes (#3004)
* perf: avoid blocking hop for owned disk writes

* fix(ecstore): sync owned write path
2026-05-19 02:27:14 +00:00
yihong ecb6704679 fix: make help color not right (#3005)
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
2026-05-19 02:26:23 +00:00
安正超 e65aebba53 fix(ecstore): guard transition worker max (#3003)
* fix(ecstore): guard transition worker max

* test: clarify env mutation safety

* test(ecstore): document env mutation concurrency contract
2026-05-19 02:25:51 +00:00
weisd a66337bd28 fix: keep scanner walk timeouts from offlining drives (#2996)
* fix: keep scanner walk timeouts from offlining drives

Scanner walk operations can time out on large or slow directory listings without proving the backing drive is faulty. Keep the timeout error local to the scan while preserving failure marking for ordinary disk operations.

Constraint: Scanner walk_dir can include listing work that exceeds the drive timeout under slow storage.

Rejected: Disable timeout failure marking globally | real stuck disk operations must still affect drive health

Confidence: high

Scope-risk: narrow

Directive: Do not route scanner/listing timeout back into drive offline state without reproducing issue #2651

Tested: cargo test -p rustfs-ecstore timeout

Tested: cargo fmt --all --check

Tested: cargo clippy --workspace --all-features --all-targets -- -D warnings

Tested: make pre-commit with en_US.UTF-8 locale

Related: https://github.com/rustfs/rustfs/issues/2651

* fix: keep remote scanner walks from offlining drives

Remote walk_dir uses a streaming request that can hit total or stall timeouts during large scans without proving the remote drive is faulty. Route that scanner path through an explicit ignore action while preserving failure marking for ordinary remote operations.

Also treat Duration::ZERO as no operation timeout for remote health tracking, matching the existing local disk wrapper contract while still marking network-like errors on default paths.

Constraint: Scanner walk_dir streams can be slow because of directory size or consumer backpressure.

Rejected: Disable remote timeout failure marking globally | normal remote disk operations still need to evict failed connections and update drive health

Confidence: high

Scope-risk: narrow

Directive: Do not reintroduce remote scanner timeout failure marking without reproducing issue #2651 against remote disks.

Tested: cargo test -p rustfs-ecstore execute_with_timeout

Tested: cargo test -p rustfs-ecstore timeout

Tested: cargo fmt --all --check

Tested: cargo clippy --workspace --all-features --all-targets -- -D warnings

Tested: make pre-commit with en_US.UTF-8 locale

Related: https://github.com/rustfs/rustfs/issues/2651

* test: prove scanner walk backpressure keeps drives online

Add a local scanner walk regression test where the output writer never makes progress. The test confirms the walk timeout returns without marking the drive faulty, covering a stall source that is not caused by object count or network transfer speed.

Constraint: Scanner walk can block on downstream writer backpressure as well as disk or network IO.\nConfidence: high\nScope-risk: narrow\nTested: cargo test -p rustfs-ecstore walk_dir_writer_backpressure_timeout_does_not_mark_drive_failure\nTested: cargo test -p rustfs-ecstore timeout\nTested: cargo fmt --all --check\nTested: cargo clippy --workspace --all-features --all-targets -- -D warnings\nTested: make pre-commit

---------

Co-authored-by: houseme <housemecn@gmail.com>
2026-05-18 15:04:05 +00:00
houseme bd1e57293f fix: harden lifecycle transition compensation and regression coverage (#2995)
* fix(ecstore): honor transition worker configuration

* fix(ecstore): add transition queue backpressure metrics

* fix(ecstore): schedule transition compensation on enqueue pressure

* fix(ecstore): log transition compensation scheduling

* test(rustfs): add transition compensation fault-injection coverage

* test(rustfs): cover delete after transition compensation

* test(scanner): cover cleanup after transition compensation

* test(rustfs): extend compensation transition coverage

* test(scanner): cover backfill idempotency after compensation

* test(scanner): cover noncurrent expiry after compensation

* test(rustfs): cover versioned delete after compensation

* test(rustfs): cover delete marker lifecycle after compensation

* test(scanner): extend versioned lifecycle compensation coverage

* test(scanner): model versioned delete after compensation

* test(scanner): clarify modeled versioned delete helper

* refactor(ecstore): optimize transition enqueue hot path

* refactor(ecstore): centralize transition runtime constants

* style(ecstore): apply rustfmt for transition timeout helper

* fix(ilm): align queue-full metric semantics

* refactor(ecstore): unify immediate enqueue failure handling

* refactor(ecstore): reuse transition worker env constant

* ci(actions): update setup action inputs
2026-05-18 12:50:43 +00:00
wood 0e888cfef2 fix(ecstore): list_object_v2 error when scanning multipart folder (#2946)
* fix(ecstore): list_object_v2 error when scanning multipart directory with prefix

Signed-off-by: w0od <dingboning02@163.com>

* test(ecstore): cover prefix dir scan with multipart folder support

Signed-off-by: w0od <dingboning02@163.com>

* test(ecstore): harden test shape to improve regression detection

Signed-off-by: w0od <dingboning02@163.com>

* refactor(ecstore): move multipart dir filter into recursion to reduce I/O

Signed-off-by: w0od <dingboning02@163.com>

* test(ecstore): replace scan_dir test with walk_dir integration coverage

Signed-off-by: w0od <dingboning02@163.com>

* refactor(ecstore): remove unnecessary .clone() calls

Signed-off-by: w0od <dingboning02@163.com>

---------

Signed-off-by: w0od <dingboning02@163.com>
2026-05-18 11:01:20 +00:00
安正超 4c9fd789ea docs: update security advisory skill lessons (#2991) 2026-05-18 12:34:27 +08:00
Henry Guo 17ae9f34c7 fix(notify): accept case-insensitive filter rule names (#2990) 2026-05-18 03:41:48 +00:00
houseme cdfe83877b chore(deps): update flake.lock (#2986)
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/b3da656' (2026-05-08)
  → 'github:NixOS/nixpkgs/d233902' (2026-05-15)
• Updated input 'rust-overlay':
    'github:oxalica/rust-overlay/1d634a9' (2026-05-09)
  → 'github:oxalica/rust-overlay/61ec6a4' (2026-05-16)
2026-05-17 03:09:08 +00:00
安正超 be37fd285e ci(nix): avoid requesting review from PR author (#2987) 2026-05-17 11:13:45 +08:00
安正超 095337100e test(policy): validate default policies (#2985) 2026-05-17 01:41:03 +00:00
GatewayJ e0729f5f4d fix(policy): align action-family validation and defaults (#2984)
* fix(policy): align action-family validation and defaults

* test(e2e): add accountinfo service-account roundtrip

* test(policy): add mixed action family cases
2026-05-16 11:19:04 +00:00
houseme 6e12289339 fix(runtime): finalize issue 2941 profiling cleanup (#2983)
* perf(runtime): narrow profiling support and upgrade starshard

* style(notify): normalize starshard imports

* perf(ecstore): reduce list_path_raw coordination overhead

* docs(scripts): add issue 2941 perf capture workflow

* fix(runtime): finalize issue 2941 profiling cleanup

* build(deps): bump quick-xml to 0.40.0

* chore(scripts): untrack local perf capture guide

* fix(scripts): honor label in perf capture output
2026-05-16 11:09:04 +00:00
weisd 9dcf8cb7ea fix: stabilize rebalance start and listing (#2961)
* fix: unblock empty chunked admin rebalance start

Some admin clients send empty POST requests with chunked framing and no explicit content length. The admin rebalance route carries no request body, so normalize that known route before downstream validation sees a missing length.

Constraint: Keep normalization scoped to known empty-body admin routes

Rejected: Relax transfer-encoding handling for S3 routes | broader protocol risk

Confidence: high

Scope-risk: narrow

Tested: cargo test -p rustfs --lib server::layer::tests::

Tested: cargo fmt --all --check

Tested: git diff --check

* fix: keep rebalance listing from stalling

Large object migration was awaited inside the listing callback, so metacache readers could stop being drained while drive walk operations were still writing listing data. Move entry migration into bounded background tasks and wait for them after each set listing completes.

Constraint: Preserve existing rebalance cancellation and first-error propagation

Rejected: Only increase drive walk timeouts | masks callback backpressure and still fails for slower migrations

Confidence: medium

Scope-risk: moderate

Tested: cargo test -p rustfs-ecstore rebalance_unit_tests

Tested: cargo test -p rustfs-ecstore cache_value::metacache_set::tests

Tested: cargo test -p rustfs --lib server::layer::tests::

Tested: cargo clippy -p rustfs-ecstore --lib --all-features -- -D warnings

Tested: cargo clippy -p rustfs-ecstore --tests --all-features -- -D warnings

Tested: cargo fmt --all --check

Tested: git diff --check

---------

Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: cxymds <Cxymds@qq.com>
2026-05-16 05:17:01 +00:00
安正超 6898e720dd fix(security): harden proxy auth and default credentials (#2981)
* fix(security): harden proxy auth and default credentials

* fix(security): address proxy and credential feedback
2026-05-16 04:01:50 +00:00
yihong 824c4f7673 docs: fix some dead links (#2975)
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-16 10:06:53 +08:00
安正超 c0c92cb048 ci(build): honor console asset download fallback (#2980) 2026-05-16 10:06:23 +08:00
Henry Guo bca8b08c2b fix: handle Windows paths in pre-commit tests (#2974)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-15 14:04:51 +00:00
Henry Guo 738fb86611 fix(admin): surface access key policy errors (#2970)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-05-15 08:26:38 +00:00
安正超 092c6bc135 ci(build): pin macOS x86 release runner (#2971) 2026-05-15 11:56:13 +08:00
escapecode 5cda460451 fix(sftp): preserve OPEN-time client attrs as object metadata (#2913)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-15 00:32:06 +00:00
GatewayJ fc8322ed64 bucket policy notify & pba (#2968) 2026-05-14 14:31:40 +00:00
houseme cd2cd74314 ci: force Node24 in Nix workflows with pinned actions (#2966) 2026-05-14 11:13:42 +00:00
Henry Guo ced390b02b test(ecstore): cover managed sse range reads (#2962)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-14 06:56:21 +00:00
Henry Guo 11be278c40 test(notify): cover prefix suffix event dispatch (#2960) 2026-05-14 05:38:38 +00:00
Henry Guo 0893c05540 fix(ecstore): use hex sha256 for delete objects (#2958)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-14 05:34:26 +00:00
houseme 81754d80b3 refactor(targets): unify queue/connectivity handling and coverage (#2953)
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: marshawcoco <marshawcoco@gmail.com>
2026-05-14 04:31:23 +00:00
houseme bdb98598d2 chore(release): prepare 1.0.0-beta.3 (#2957) 2026-05-14 04:27:26 +00:00
安正超 5fccfceb91 test(ecstore): cover walk listing error success paths (#2954) 2026-05-14 01:03:31 +00:00
Henry Guo c1bcee327c fix(notify): keep live listen events active when disabled (#2952)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-13 17:10:14 +00:00
Henry Guo d4d07095f8 fix(ecstore): propagate walk listing errors (#2949)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-13 16:59:49 +00:00
Sergei Z. 8c49671161 Fix #2775 recursive list handling in LocalDisk::scan_dir() (#2923)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-05-13 14:36:23 +00:00
cxymds c65d4071a3 fix(ecstore): map missing metadata to not found (#2944)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-13 13:03:15 +00:00
weisd 4ea805cbc0 fix: preserve pagination when max keys exceed limit (#2943)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-13 12:48:38 +00:00
Henry Guo 26341742e0 fix(ecstore): fail listing on walk_dir producer errors (#2937) 2026-05-13 11:08:05 +00:00
安正超 fd3fb77ed5 fix(sftp): avoid metadata on multipart copy (#2935)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-13 07:44:09 +00:00
weisd c5125b20bd fix(storage): keep storage info RPC map encoded (#2942)
Co-authored-by: cxymds <Cxymds@qq.com>
2026-05-13 06:19:43 +00:00
weisd 9b83de53e1 fix(ecstore): surface prefix listing storage errors (#2940) 2026-05-13 04:24:54 +00:00
houseme d72b8cb88e fix(server): fail fast when configured TLS parsing fails (#2939) 2026-05-13 04:16:39 +00:00
Henry Guo 17eff5a97b fix(tls): ignore Kubernetes secret projection dirs (#2938) 2026-05-13 03:45:42 +00:00
houseme 129cb0f920 fix: make HeadObject consistent after write completion (#2936) 2026-05-13 03:27:38 +00:00
安正超 28c9358482 test(notify): cover encoded key target union (#2934)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-12 15:20:06 +00:00
Henry Guo fe5690ca70 fix(sftp): preserve open attrs metadata (#2929)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-12 13:34:52 +00:00
安正超 25cf164461 test(protocols): cover SFTP host key reload failure (#2928)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-12 10:40:23 +00:00
安正超 8a501846f4 test(iam): cover mixed STS claim policy names (#2932) 2026-05-12 10:39:53 +00:00
GatewayJ b2ba2e5bb3 iam: handle sts claim policy names (#2902)
Co-authored-by: cxymds <Cxymds@qq.com>
2026-05-12 07:10:42 +00:00
安正超 8169bd63e9 test(protocols): cover TLS reload fingerprint ordering (#2927) 2026-05-12 02:50:49 +00:00
houseme 6185d38911 fix(protocols): add hot reload for WebDAV FTPS and SFTP (#2922)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-11 16:25:20 +00:00
cxymds 8ef2c0a5e1 fix(storage): sync transition tier config across peers (#2918)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-11 15:23:29 +00:00
Henry Guo d6813b53a2 fix(ecstore): preserve list marker set index (#2919)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-11 15:20:43 +00:00
Henry Guo 5ca99a8e32 fix(ecstore): fail listing on stalled reader (#2920)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-11 15:19:46 +00:00
Henry Guo 07a01a1123 fix(notify): match filters against decoded event keys (#2921)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-11 15:01:37 +00:00
Henry Guo 941986b331 test(e2e): gate protocol runner by requested features (#2912) 2026-05-11 13:59:24 +00:00
Henry Guo aba65a448c fix(protocols): encode storage client request URIs (#2911) 2026-05-11 10:47:30 +00:00
Henry Guo 1c94f7a066 fix(sftp): classify backend errors by type (#2909) 2026-05-11 02:56:08 +00:00
安正超 77c21dc5fd test(obs): cover replication bandwidth tombstones (#2906) 2026-05-11 01:55:54 +00:00
安正超 404fc816e9 test(targets): cover MySQL probe validation (#2907) 2026-05-11 01:55:48 +00:00
LeonWang0735 61a7820651 optimize(obs):zero and expire removed replication bandwidth series (#2901)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-10 14:26:18 +00:00
安正超 0375dd39bb fix(targets): handle MySQL DSN scheme case (#2903)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-10 14:25:22 +00:00
安正超 f84d8983ed test(s3): promote passing SSE multipart cases (#2900) 2026-05-10 13:16:23 +00:00
JaySon bafff6d207 feat(targets): add check_mysql_server_available probe function (#2884)
Signed-off-by: JaySon-Huang <tshent@qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-10 11:53:45 +00:00
houseme 53ec1b95d8 keep sftp e2e tests buildable (#2897) 2026-05-10 11:52:15 +00:00
安正超 65c62ab2ea test(sftp): cover init session activity stamp (#2898) 2026-05-10 09:57:04 +00:00
安正超 c419cff348 test(sftp): cover init negotiation and platform gating (#2896) 2026-05-10 05:59:10 +00:00
houseme 5a5ffcfac0 chore(deps): update flake.lock (#2894) 2026-05-10 03:48:57 +00:00
escapecode 96b293bf8a feat(sftp): add SFTPv3 protocol support (#2875)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-10 03:48:42 +00:00
houseme 8892cbbdd7 feat: enhance WebDAV support with features and directory operations (#2856) (#2892)
Signed-off-by: giter <giter@users.noreply.github.com>
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: giter <giter@users.noreply.github.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Lijiajie <lijiajie@ffcode.net>
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 唐小鸭 <tangtang1251@qq.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2026-05-09 16:45:09 +00:00
唐小鸭 09c2d15057 fix(sse): Temporarily refactored the SSE design for ECStore (#2813)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: cxymds <Cxymds@qq.com>
2026-05-09 14:06:35 +00:00
Sergei Z. 6275918d92 fix: empty-body requests without content length (#2888)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: OmX <omx@oh-my-codex.dev>
2026-05-09 14:05:53 +00:00
Henry Guo 5879d0b59d fix(server): handle public health before s3 host parsing (#2866)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-09 13:15:37 +00:00
JaySon cb54cecf8c docs(targets): sync AGENTS.md and test doc comments with code (#2881)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-09 03:06:11 +00:00
安正超 9288f1ed3a fix(targets): handle postgres dsn redaction scheme case (#2886) 2026-05-09 01:58:23 +00:00
安正超 8182e2dd38 test(ecstore): cover empty runtime listing candidates (#2889) 2026-05-09 01:57:54 +00:00
houseme 81ad48dac2 feat(targets): add AMQP support for notify and audit (#2879)
Co-authored-by: Hyesook Yun <74169420+suk13574@users.noreply.github.com>
2026-05-09 01:56:26 +00:00
安正超 1582f216fe test(ecstore): cover offline capacity snapshots (#2880) 2026-05-08 16:33:34 +00:00
安正超 5081b8396d test(ecstore): cover system path failure classifier (#2874) 2026-05-08 14:51:11 +00:00
houseme c90bfe2b23 fix(ecstore): harden runtime read-path quorum handling (#2872) 2026-05-08 09:56:39 +00:00
Henry Guo 03045ff2e6 fix(iam): keep error state on initial load failure (#2846)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-05-08 08:26:01 +00:00
安正超 61bd5698bb test(admin): cover pools list response serialization (#2862)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-08 07:32:49 +00:00
安正超 44d2b6a284 [codex] docs: ban rust-refactor-helper skill (#2869) 2026-05-08 15:33:45 +08:00
安正超 fe978488a0 test(admin): cover pool used-size saturation (#2863) 2026-05-08 06:06:54 +00:00
安正超 e209fc6eef test(ecstore): cover store init health reset delegation (#2865) 2026-05-08 05:42:05 +00:00
weisd 97a2775434 fix(ecstore): reset drive health between store init format retries (#2848)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: cxymds <Cxymds@qq.com>
2026-05-08 03:16:58 +00:00
cxymds 9d85b7d23a feat: enrich admin pools list response (#2853)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-07 13:53:25 +00:00
安正超 ba7ee5fbdd docs(security): make advisory skill lesson first (#2860)
Co-authored-by: cxymds <Cxymds@qq.com>
2026-05-07 21:53:26 +08:00
安正超 6896b38dc2 test(s3): promote lifecycle expiration header tests (#2858) 2026-05-07 13:08:42 +00:00
安正超 9df57b8cd8 test(targets): cover Redis env config loading (#2857) 2026-05-07 13:08:11 +00:00
安正超 6ec3a4c4d5 docs(security): refresh advisory lesson states (#2859) 2026-05-07 21:21:15 +08:00
houseme fd37a7d01e fix(targets): probe webhook health by host port (#2854) 2026-05-07 11:52:45 +00:00
Henry Guo 9c0141fbdf docs(io-metrics): fix misleading metrics links (#2849) 2026-05-07 20:00:19 +08:00
cxymds 2a0fbb8d77 fix(ecstore): repair decommission pool quorum (#2847)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-07 11:14:11 +00:00
houseme 5431b9273d feat(targets): complete redis mysql postgres target wiring (#2842)
Signed-off-by: jaehanbyun <awbrg789@naver.com>
Signed-off-by: houseme <housemecn@gmail.com>
Signed-off-by: Gunther Xing <jiengup@gmail.com>
Signed-off-by: JaySon-Huang <tshent@qq.com>
Co-authored-by: jaehanbyun <awbrg789@naver.com>
Co-authored-by: Gunther Xing <jiengup@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: JaySon <tshent@qq.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-05-07 10:00:59 +00:00
安正超 b159d656cc test(admin): cover POST content length compat layer (#2844)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-07 09:51:09 +00:00
cxymds b7815b0694 fix(ecstore): remove startup order sensitivity (#2850) 2026-05-07 09:07:25 +00:00
houseme eaa5ff3053 bump workspace versions and replace cfg-if in crypto (#2851) 2026-05-07 09:04:27 +00:00
houseme f1cd7c1345 feat(rustfs): add ftps/webdav defaults to info output (#2845) 2026-05-07 04:39:56 +00:00
cxymds 06097a3c33 fix(admin): normalize empty admin POST content length (#2843) 2026-05-07 02:52:21 +00:00
安正超 5e0ca006f0 test(replication): cover ETag comparison edge cases (#2840) 2026-05-07 02:48:12 +00:00
安正超 abc07a9dc4 fix(build): quote build script features argument (#2841) 2026-05-07 02:47:53 +00:00
Nikita Bakun 4d6171e996 fix(replication): handle version ID format mismatch with AWS S3 (#2829)
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-05-06 21:30:07 +00:00
GatewayJ 3130670157 test(object-lock): cover default retention delete marker (#2836) 2026-05-06 21:14:00 +00:00
安正超 96d41b6349 test(helm): cover standalone scale-to-zero rendering (#2831) 2026-05-06 21:13:40 +00:00
安正超 9b9e0db9f6 test(lifecycle): cover ILM process time aliases (#2839) 2026-05-06 21:12:22 +00:00
安正超 68fcbffcb6 test(build): cover build script feature flags (#2837) 2026-05-07 05:18:51 +08:00
giter 04712fb6c6 feat: add features option to build script (#2834)
Signed-off-by: giter <giter@users.noreply.github.com>
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-06 15:09:44 +00:00
安正超 4728abcff1 fix(security): document unsafe and TLS overrides (#2835) 2026-05-06 15:09:02 +00:00
cxymds 70be0804ee fix: 2827 lifecycle days next midnight (#2833)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-06 15:00:53 +00:00
安正超 b10db403b6 test(s3): promote passing copy metadata case (#2832) 2026-05-06 13:58:32 +00:00
GatewayJ 9e93d3f47a fix(object-lock): materialize default retention metadata (#2824)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-06 13:48:12 +00:00
cxymds 9f07029373 fix: reload bucket metadata after lifecycle updates (#2822)
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-06 13:01:44 +00:00
Henry Guo 4b36667ba1 fix(policy): avoid logging generated access keys (#2826)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-05-06 12:56:27 +00:00
houseme 41ba34a145 fix(rpc): add issue 2815 regression and docker validation (#2828) 2026-05-06 12:22:13 +00:00
Michael Graff 3898d524fe security: same-origin console CORS, fail-closed helm creds, deny.toml, sample-config hardening (#2769)
Signed-off-by: Michael Graff <explorer@flame.org>
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-05-06 07:34:44 +00:00
Duru Can Celasun 718bec7722 feat(helm-chart): support scale to 0 in standalone mode (#2797)
Signed-off-by: Duru Can Celasun <can@dcc.im>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: cxymds <Cxymds@qq.com>
2026-05-06 06:42:56 +00:00
安正超 50ddec3ffc fix: preserve data on self metadata copy (#2819)
Co-authored-by: cxymds <Cxymds@qq.com>
2026-05-06 06:28:36 +00:00
Henry Guo 7692c0c3bd test(credentials): avoid printing default secret (#2820)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-05-06 05:19:42 +00:00
houseme 3dd0692917 refactor: unify credential env constants and deploy env usage (#2821)
Co-authored-by: Henry Guo <marshawcoco@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-05-06 04:30:29 +00:00
GatewayJ 090d60e00a fix(auth): authorize DeleteObjects per key (#2814) 2026-05-06 00:47:36 +00:00
安正超 c228fabfdb test(admin): cover orphan resync status cleanup (#2816) 2026-05-06 00:47:11 +00:00
安正超 bde569d055 test(getobject): cover buffer threshold edge cases (#2817) 2026-05-06 00:47:02 +00:00
houseme e19b0dd997 fix(notify): improve webhook target diagnostics for env setup (#2810) 2026-05-06 00:28:46 +00:00
houseme 26b3aad069 build(deps): bump the dependencies group with 3 updates (#2812) 2026-05-05 14:05:13 +00:00
安正超 bf893bcb55 fix(admin): clear removed site resync status (#2803) 2026-05-05 13:42:12 +00:00
安正超 60d4598562 test(lock): cover shared waiter abort cleanup (#2811) 2026-05-05 13:41:15 +00:00
houseme 565cbdffed fix(getobject): prevent large-download memory buffering (#2809) 2026-05-05 12:58:26 +00:00
houseme 49b2782d51 fix(lock): make slow-path waiter accounting cancellation-safe (#2805) 2026-05-05 11:19:23 +00:00
houseme 743b87014b fix(notify): validate bucket notification filter rules (#2806) 2026-05-05 10:35:36 +00:00
安正超 3208f930c5 fix(s3): advertise byte ranges on head object (#2802) 2026-05-05 04:29:25 +00:00
安正超 36b3d21c44 test: cover RPC secret trimming fallback (#2796)
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-05 04:26:36 +00:00
cxymds 935bb8a8e2 fix(admin): allow site replication removal with offline peers (#2739)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-05-05 03:54:32 +00:00
安正超 9364ecba67 docs: add security advisory lessons skill (#2801)
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-05 11:59:13 +08:00
Henry Guo 0153710791 fix(server): avoid logging default credential values (#2800)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-05-05 03:44:07 +00:00
Alexander Kharkevich 995e26f5ee fix(rpc): use map-encoded msgpack for all internode RPC responses (#2771)
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-05 03:42:29 +00:00
JaySon e6fdcd1ad6 fix: Fix formatting issue in ARCHITECTURE.md (#2787)
Signed-off-by: JaySon-Huang <tshent@qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-05-05 11:33:55 +08:00
安正超 f016673416 test: cover license verifier success path (#2798) 2026-05-05 02:55:56 +00:00
安正超 42ff6b4d80 test(admin): cover legacy profile auth guards (#2799) 2026-05-05 02:55:45 +00:00
安正超 3ced40f221 test(license): cover public key validation (#2793) 2026-05-04 15:12:59 +00:00
dependabot[bot] a830ab2a5d build(deps): bump the dependencies group with 4 updates (#2785)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-04 10:26:51 +00:00
Joey Shi 52f121ab2a Ensure CreationDate is always present in ListBuckets response (#2783) 2026-05-04 08:20:30 +00:00
安正超 eb8868397e fix: address security review follow-ups (#2781) 2026-05-03 13:53:36 +00:00
安正超 66c38b629d Harden admin and RPC security checks (#2773)
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-03 11:55:09 +00:00
安正超 eb23710d2e fix(security): harden CORS and license handling (#2774) 2026-05-03 11:39:27 +00:00
houseme 4b66155f26 chore(deps): update flake.lock (#2777) 2026-05-03 02:44:53 +00:00
Ramakrishna Chilaka 4978f60254 test(ecstore): cover ranged decode and harden offset bounds (#2758)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-05-02 09:02:50 +00:00
安正超 fe59eb4952 test(admin): cover site identity merge gaps (#2767) 2026-05-02 08:43:47 +00:00
安正超 a455b4377c fix: handle empty multipart list-parts (#2765)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-05-02 02:47:52 +00:00
安正超 08fab34804 test(admin): cover peer identity scheme dedupe (#2764) 2026-05-01 04:11:46 +00:00
安正超 f07fed0c49 test(filemeta): cover no-wait refresh coalescing (#2755) 2026-05-01 02:48:30 +00:00
安正超 dcfbc2612a test(admin): cover site replication scheme normalization (#2757) 2026-05-01 02:48:20 +00:00
安正超 87e1c7aeb6 test(filemeta): cover future cache timestamp refresh (#2762) 2026-05-01 02:48:08 +00:00
安正超 0d7e0a814f test(ci): cover Helm Recreate strategy rendering (#2752) 2026-04-30 12:14:38 +00:00
安正超 ef5ccc232a test(filemeta): cover shared cache reuse (#2754) 2026-04-30 12:14:15 +00:00
GatewayJ c29c8a5a1e fix(filemeta): harden and optimize metacache path (#2724)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-30 07:47:31 +00:00
安正超 403997f2f8 test(ci): cover Helm chart version mapping (#2751) 2026-04-30 06:11:14 +00:00
安正超 e34d75dfdf test(admin): cover site replication absolute URI endpoint (#2745)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-30 06:11:00 +00:00
安正超 132e72ef39 fix(ci): keep Helm chart versions valid for non-beta releases (#2749) 2026-04-30 01:51:12 +00:00
安正超 4f4c759b67 docs: simplify pull request template (#2744) 2026-04-29 20:29:23 +08:00
majinghe e331a26262 feat: helm chart version update (#2738) 2026-04-29 11:14:35 +00:00
安正超 61d2e9fbc3 test(webdav): cover decoded path parsing (#2729)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-29 07:30:40 +00:00
majinghe 946755aa89 feat: helm publish manual trigger support (#2732) 2026-04-29 06:01:22 +00:00
houseme c9622611ed fix(admin): harden site-replication identity handling and local verification for issue 2723 (#2730) 2026-04-29 03:27:55 +00:00
majinghe 7041e628b7 fix: docker image build and helm chart publish error caused by versio… (#2731) 2026-04-29 03:23:47 +00:00
Ramakrishna Chilaka 8e1bd560d8 test(get): reject range with part number (#2725) 2026-04-28 23:15:59 +00:00
majinghe d447da75c1 chore: update version from alpha to beta (#2720)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-28 23:08:10 +00:00
Rafael Peroco 2c9524e2c9 fix(helm): only render rollingUpdate when strategy type is RollingUpdate (#2728) 2026-04-28 23:06:31 +00:00
唐小鸭 e16f1ae639 fix(window): Compatible with Windows Path (#2691)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-28 14:36:59 +00:00
安正超 d79720da1d test(signer): cover header fallback helpers (#2711) 2026-04-28 14:09:56 +00:00
安正超 b747e9817e fix(policy): preserve gateway ListBucket resources (#2710) 2026-04-28 14:09:42 +00:00
giter 90ce72122b fix(webdav): decode URL-encoded filenames in path parsing (#2722)
Signed-off-by: giter <giter@users.noreply.github.com>
2026-04-28 14:06:05 +00:00
houseme c4d5c5c5ec fix(obs): disable profiling export by default and fix Helm env name (#2719) 2026-04-28 11:57:15 +00:00
cxymds c698a0f2b6 fix(policy): allow AssumeRole in system policies (#2718) 2026-04-28 11:01:46 +00:00
houseme 2953558f41 fix(lifecycle): prevent eager date-expiry deletion on config update (#2708) 2026-04-28 10:26:14 +00:00
weisd e0b8c4fd42 fix(storage): avoid faulting local drives on transient timeouts (#2714) 2026-04-28 06:38:31 +00:00
weisd a995ec0315 fix(iam): preserve portable IAM storage and derived auth (#2713) 2026-04-28 05:57:10 +00:00
houseme 946b502527 build(deps): bump the dependencies group with 2 updates (#2709) 2026-04-27 23:53:29 +00:00
GatewayJ 09be06a4d2 fix(ecstore): log walk failures in IAM listing path (#2705)
Co-authored-by: GatewayJ <8352692332qq.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-27 14:13:40 +00:00
安正超 159ddd5bac fix: honor bucket-scoped ListBucket policies with s3:prefix (#2707)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-27 14:13:22 +00:00
houseme a68fe1601f fix(ilm): harden signer failures and guard remote tier delete storms (#2706) 2026-04-27 12:39:02 +00:00
weisd 334184b005 fix(replication): prevent target state loss across buckets (#2704) 2026-04-27 09:27:33 +00:00
GatewayJ cfbd094bc4 fix(iam): propagate cache miss load failures (#2692)
Co-authored-by: GatewayJ <8352692332qq.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-27 09:21:22 +00:00
weisd 468dc3aebd fix(replication): fan out single-bucket rules to all targets (#2701) 2026-04-27 06:51:19 +00:00
dependabot[bot] fd258e877c build(deps): bump the dependencies group with 2 updates (#2694)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-27 02:42:49 +00:00
安正超 4dafb64d58 test(server): cover default module switch source (#2697) 2026-04-27 02:41:38 +00:00
houseme 50d03ef021 perf(memory): add reclaim signals and cache controls (#2689) 2026-04-26 16:42:35 +00:00
houseme 438913b84a feat: add console-managed audit and notify module switches (#2690) 2026-04-26 16:10:41 +00:00
GatewayJ 37a3cbc497 fix(admin): map IAM not found errors to 404 (#2685)
Co-authored-by: GatewayJ <8352692332qq.com>
2026-04-26 14:13:14 +00:00
安正超 a05687b900 test(utils): cover disk-check env alias precedence (#2684) 2026-04-26 07:41:50 +00:00
houseme 561d695237 chore(deps): update flake.lock (#2683) 2026-04-26 00:58:02 +00:00
houseme 59f41eb86a feat(obs): improve metrics coverage and dashboard performance (#2682) 2026-04-25 18:51:29 +00:00
houseme 81854762d4 fix(ecstore): log missing local paths during physical disk independence checks (#2680) 2026-04-25 08:01:01 +00:00
houseme 1e9c75a201 feat(ecstore): enforce local disk topology guardrails and expose device ids (#2679) 2026-04-25 06:02:02 +00:00
安正超 80413e0f8e test(ecstore): cover non-zero offset shard span reads (#2677)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-25 02:31:28 +00:00
houseme b96ccfd110 build(deps): bump the dependencies group with 7 updates (#2676) 2026-04-25 02:15:14 +00:00
Henry Guo c717195de2 fix(admin): harden STS and KMS authorization checks (#2653)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-04-25 01:34:18 +00:00
cxymds 94f64acc87 fix(site-replication): sync IAM and bucket replication (#2671) 2026-04-25 01:33:43 +00:00
唐小鸭 d949d4e794 fix: avoid sending HEAD bodies over TLS HTTP/2 (#2648)
Signed-off-by: 唐小鸭 <tangtang1251@qq.com>
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-04-25 01:33:08 +00:00
houseme 7215761784 feat(trusted-proxies): add switchable simple and legacy modes (#2674)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2026-04-25 01:25:32 +00:00
安正超 f833cd9cbe fix(metrics): keep S3 op counts when chains are disabled (#2675)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-25 01:24:53 +00:00
houseme 13b4500212 feat(obs): improve telemetry stack, replication metrics, and Grafana alignment (#2672)
Co-authored-by: Filipe Monteiro <a22407332@alunos.ulht.pt>
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: weisd <im@weisd.in>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-04-24 13:50:17 +00:00
Henry Guo 2705e3f53b fix(audit): remove unwrap from target shutdown path (#2668)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 09:25:11 +00:00
houseme 92f812fc80 feat(rustfs): gate audit/notify by global env switches (#2669) 2026-04-24 09:14:19 +00:00
Ramakrishna Chilaka fefb308b35 feat(policy): implement BinaryEquals condition evaluation (#2626)
Signed-off-by: Ramakrishna Chilaka <49393831+RamakrishnaChilaka@users.noreply.github.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-24 07:08:11 +00:00
cxymds 8d4caeacad fix(oidc): add federated logout flow (#2667)
Co-authored-by: GatewayJ <835269233@qq.com>
2026-04-24 06:51:31 +00:00
安正超 2860c82e3c fix(console): block disabled health probes from SPA fallback (#2665) 2026-04-24 03:34:57 +00:00
houseme 572dd1264e fix(admin): harden health readiness and add health response controls (#2662) 2026-04-23 17:37:42 +00:00
houseme 47247789ad feat(obs): support OTLP headers and timeout overrides for HTTP exporters (#2661) 2026-04-23 17:02:26 +00:00
安正超 39f7de4450 test(admin): cover list service account authorization (#2650)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-23 13:40:59 +00:00
houseme de6fe816c2 build(deps): bump the dependencies group with 6 updates (#2659) 2026-04-23 12:41:37 +00:00
houseme 368ef0f16c refactor(targets): unify endpoint source/merge logic and bump rustfs-kafka-async to v1.2.0 (#2654)
Co-authored-by: Filipe Monteiro <a22407332@alunos.ulht.pt>
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: weisd <im@weisd.in>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-04-23 09:14:36 +00:00
houseme bc37cc4001 refactor(logging): unify request-id propagation and fallback metrics (#2652) 2026-04-23 07:11:27 +00:00
GatewayJ ecf0db9bb7 fix(admin): enforce owner check for service account update (#2646)
Co-authored-by: GatewayJ <8352692332qq.com>
2026-04-23 01:54:43 +00:00
loverustfs fa1554be7f fix(admin): authorize cross-user ListServiceAccount with ListServiceAccount (#2640) 2026-04-22 12:16:09 +00:00
weisd f08b592c6f fix(storage): list prefix children behind marker objects (#2643) 2026-04-22 11:23:27 +00:00
安正超 8add0126f5 test(targets): cover target config redaction (#2638) 2026-04-22 02:51:41 +00:00
weisd 09a83a8f56 fix: prevent object lock retention race (#2634) 2026-04-22 02:49:38 +00:00
weisd a0f1bb4ff0 fix(lock): prevent stale distributed object locks (#2633) 2026-04-22 02:12:33 +00:00
houseme 3ac1d2ab0b fix(security): redact target debug logs and remove eval-based bench hook (#2637) 2026-04-21 21:21:01 +00:00
唐小鸭 4aafb07173 refactor: update binary field types and conversions in RPC and protofiles (#2619)
Signed-off-by: 唐小鸭 <tangtang1251@qq.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-04-21 14:49:09 +00:00
Samuel Cormier-Iijima 8c76e9838b fix(s3): return 304 Not Modified instead of dropping the connection (#2627)
Signed-off-by: Samuel Cormier-Iijima <samuel@cormier-iijima.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-21 14:28:03 +00:00
weisd d4dcb2ac9d fix(scanner): avoid stalls after abandoned child listings (#2632) 2026-04-21 09:06:26 +00:00
majinghe 41d2812861 feat: add support for external/existing certificate issuer (#2631) 2026-04-21 07:21:43 +00:00
houseme 960c13a34b feat(storage): wire capacity/object perf tuning and add batch benchmark runners (#2628) 2026-04-21 07:20:57 +00:00
Andy Teijelo Pérez 989827e3b5 feat: add OTHER_AUDIENCES config (#2605)
Co-authored-by: GatewayJ <835269233@qq.com>
2026-04-21 03:48:13 +00:00
likewu a77be8f89b fix: some expect error (#2622)
Signed-off-by: likewu <likewu@126.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-21 03:27:50 +00:00
安正超 1525143a04 test: cover NATS and Pulsar config validation (#2623) 2026-04-21 01:59:05 +00:00
houseme b7a945e453 fix(madmin): restore server_info msgpack compatibility across mixed nodes (#2621) 2026-04-21 01:09:25 +00:00
houseme 3796b684f0 feat(targets): add NATS and Pulsar target support (#2618) 2026-04-21 00:28:19 +00:00
cxymds 1511f9eacb fix(lifecycle): correct delete replication fanout (#2609)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-04-20 10:02:59 +00:00
安正超 12f355a3bc test: cover S3 error XML body compatibility (#2606) 2026-04-20 04:27:22 +00:00
安正超 83bac39417 test(s3): promote passing compatibility cases (#2600) 2026-04-19 14:42:56 +00:00
安正超 177fe2ab44 fix(replication): clean targets when deleting config (#2599) 2026-04-19 11:33:01 +00:00
安正超 1d1f00470d test(s3): cover signed equals object reads (#2598) 2026-04-19 02:11:16 +00:00
安正超 457f4e0170 fix(s3): improve GitLab registry compatibility (#2596)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-19 02:10:42 +00:00
安正超 93d0606cbd fix(admin): accept trailing slash listen notification paths (#2595)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-19 02:10:30 +00:00
houseme ae7444ebb4 chore(deps): update flake.lock (#2597) 2026-04-19 00:52:49 +00:00
安正超 b8c788ffca fix: handle raw URI paths in SigV4 (#2593)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-18 17:09:00 +00:00
houseme 9677320f23 fix(scanner): stabilize data usage cache persistence under slow metadata I/O (#2594) 2026-04-18 16:36:28 +00:00
houseme 116db4f5d9 refactor(metrics): unify process sampling and split network IO (#2590) 2026-04-18 15:30:44 +00:00
安正超 f9b5ad17a9 fix(s3): ignore empty conditional ETag headers (#2592)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-18 15:27:46 +00:00
唐小鸭 fb0d096d5d fix(sse). Resolving Nonce Overwriting Issues in Multi-Package Scenarios (#2582)
Signed-off-by: 唐小鸭 <tangtang1251@qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-04-18 14:00:14 +00:00
GatewayJ a5de275875 fix(oidc): prefer username for session identity (#2588)
Co-authored-by: GatewayJ <8352692332qq.com>
2026-04-18 11:13:45 +00:00
GatewayJ dc5ce7d0af test(scanner): avoid flaky noncurrent version counting (#2589)
Co-authored-by: GatewayJ <8352692332qq.com>
2026-04-18 11:13:35 +00:00
安正超 ac443a90ce test(s3): reclassify passing compatibility cases (#2586)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-18 09:15:29 +00:00
安正超 2d0b3227c5 docs: update agent contribution guidelines (#2585)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-18 16:28:07 +08:00
houseme 1cbf156559 refactor(obs): migrate metrics runtime/schema and tighten migration guards (#2584) 2026-04-18 07:51:15 +00:00
GatewayJ 03f8270a60 fix(admin): restore access key listing and guard boot-time uptime (#2580)
Co-authored-by: GatewayJ <8352692332qq.com>
2026-04-17 18:55:38 +00:00
houseme 96fb06f48e refactor(metrics): separate console stream from exports (#2583) 2026-04-17 18:21:08 +00:00
houseme ffcf18f5f3 fix(cache): wire trusted-proxy cache and remove stale cache traces (#2581) 2026-04-17 16:57:21 +00:00
LeonWang0735 38eb0781cf fix(iam): decouple IAM config encryption from root secret rotation (#2558)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-17 14:13:53 +00:00
安正超 ce291ab610 test(utils): cover scanner env alias mapping (#2574)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-17 13:24:16 +00:00
houseme f77ccd5b23 chore: move mcp crate to standalone repository (#2578) 2026-04-17 12:42:26 +00:00
weisd 8dc5ef6ef5 fix(storage): avoid startup UUID disk lookup misses (#2576) 2026-04-17 07:55:47 +00:00
GatewayJ f255b8a9f1 fix(admin): align accountinfo policy with IAM prepare_auth for OIDC console (#2568)
Co-authored-by: GatewayJ <8352692332qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-17 05:38:18 +00:00
houseme 478720d2ee Centralize lifecycle state updates and fix systemd running status (#2567) 2026-04-17 03:20:11 +00:00
majinghe 6b4172998b fix(helm): disable kms default (#2566)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-16 11:51:44 +00:00
houseme 6ce24f3b63 feat(lifecycle): improve ILM compatibility and scanner runtime config (#2534)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: cxymds <Cxymds@qq.com>
2026-04-16 10:28:03 +00:00
majinghe af93d2daba fix: update mtls configuration for standalone and distributed mode (#2565) 2026-04-16 09:26:36 +00:00
houseme 28edfd6190 fix(storage): harden offline drive fail-fast paths (#2564)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2026-04-16 09:21:45 +00:00
Tunglies 579b124726 lint: clippy rules or_fun_call (#2561)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-16 02:48:39 +00:00
majinghe 1ffe23e10f fix: update base image to fix security issue caused by alpine (#2563)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-16 02:44:51 +00:00
majinghe 4615791193 add kms environment variables support in helm chart (#2552)
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-16 02:43:54 +00:00
安正超 b2b92de26c test(ecstore): cover fresh tmp cleanup path (#2562) 2026-04-16 02:12:16 +00:00
Tunglies e05d07494e chore(lint): clippy rules large_futures (#2555)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-16 02:10:13 +00:00
weisd c0d3f53f7a test(filemeta): flush xl meta test fixture before reopen (#2557) 2026-04-16 00:40:46 +00:00
Tunglies 49366ee200 chore(lint): clippy rules redundant_clone (#2554) 2026-04-15 13:54:07 +00:00
Strangerxxx 73e6542eea feat(helm): add generic service and ingress annotation support (#2541)
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-15 10:16:10 +00:00
Tunglies f57dd5a3c7 chore(lint): clippy rules needless_collect (#2522) 2026-04-15 10:00:03 +00:00
安正超 1bda5ed636 test: cover stored-size range handling (#2544)
Co-authored-by: cxymds <Cxymds@qq.com>
2026-04-15 09:37:01 +00:00
weisd 27971f5c03 fix(scanner): preserve totals for compacted folder scans (#2530) 2026-04-15 07:32:13 +00:00
安正超 6506ae8c7b fix(ci): report CLA check for GitHub Merge Queue (#2551) 2026-04-15 17:20:16 +08:00
安正超 8223cda5ff fix(ci): enable merge_group trigger for GitHub Merge Queue (#2549) 2026-04-15 16:27:53 +08:00
majinghe 8152c8e084 fix: add different annotations for different pvc (#2547) 2026-04-15 15:21:38 +08:00
weisd da9dfb51e7 fix(ecstore): cleanup stale put object temp data (#2529) 2026-04-15 14:40:58 +08:00
cxymds c11efce1e4 fix(admin): replicate empty site groups (#2528)
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: GatewayJ <835269233@qq.com>
2026-04-15 14:21:40 +08:00
houseme fa793237fb build(deps): bump the dependencies group with 7 updates (#2546)
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2026-04-15 13:23:57 +08:00
安正超 df109680ae cleanup: remove orphaned chunk fast-path submodule files (#2545)
Co-authored-by: cxymds <Cxymds@qq.com>
2026-04-15 11:50:32 +08:00
cxymds 1eef75b0ca fix(admin): allow clearing group policy (#2527)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: GatewayJ <835269233@qq.com>
2026-04-15 11:35:19 +08:00
安正超 642d83f0e4 revert: remove #2351 chunk I/O and object-io crate (phase 7) (#2543)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-15 10:54:41 +08:00
GatewayJ 16b9189e9b feat(oidc): add roles_claim and jwt:roles policy support (#2509)
Co-authored-by: GatewayJ <8352692332qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-04-15 09:30:24 +08:00
安正超 1979fc7fb1 fix: restore bitrot stream and checksum metadata (#2542) 2026-04-15 08:36:55 +08:00
安正超 68d3dba9fc fix: revert standalone #2351 artifacts (phase 5) (#2535) 2026-04-14 22:35:05 +08:00
安正超 db8ef63674 fix: revert #2351 readers.rs behavioral changes (phase 6) (#2536) 2026-04-14 22:08:38 +08:00
安正超 6d476ae9a1 revert: remove BlockReadable trait and chunk-based I/O from rio, ecstore, disk layers (#2533) 2026-04-14 19:09:02 +08:00
安正超 2a3d127a86 revert: restore PutObjReader and remove PUT zero-copy chunk path (#2351 phase 2) (#2532) 2026-04-14 16:35:03 +08:00
安正超 8abf683178 fix(io-metrics): remove dead GET chunk fast path metrics (#2531) 2026-04-14 15:47:57 +08:00
安正超 5048ff8c69 test(credentials): cover URL-safe secret keys (#2524) 2026-04-14 13:57:23 +08:00
Liam Beckman 711aab73c3 docs: Minor formatting clean up in README.md (#2523)
Signed-off-by: Liam Beckman <lbeckman314@gmail.com>
2026-04-14 08:29:33 +08:00
安正超 315e8134eb test(get): cover reader-backed output context (#2516)
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-13 21:59:13 +08:00
houseme 979626c370 refactor(utils): decouple config deps and move sys helpers (#2520)
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2026-04-13 21:05:03 +08:00
Cocoon-Break 505a566c7c fix: remove dead replace() call in gen_secret_key (credentials.rs) (#2515)
Signed-off-by: cocoon <54054995+kuishou68@users.noreply.github.com>
Signed-off-by: Cocoon-Break <54054995+kuishou68@users.noreply.github.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-13 18:42:50 +08:00
安正超 8efa359a1a docs: add ARCHITECTURE.md for project-wide code navigation (#2519) 2026-04-13 18:26:58 +08:00
dependabot[bot] 522c1675d8 build(deps): bump the dependencies group with 2 updates (#2513)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-13 10:02:27 +08:00
houseme e160633ac4 fix(admin): enforce event and audit target auth (#2508) 2026-04-12 23:59:42 +08:00
安正超 458086dc80 fix(get): remove GET chunk fast path (#2507) 2026-04-12 23:12:23 +08:00
安正超 0e697fa7e1 test(get): cover fast path pre-commit errors (#2502)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-12 20:13:15 +08:00
安正超 30cc481731 refactor(storage): inline acl entrypoints (#2501)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-12 20:12:44 +08:00
houseme b9f2369095 build(deps): bump the dependencies group with 3 updates (#2503) 2026-04-12 12:11:40 +08:00
houseme 4ce374a181 chore(deps): update flake.lock (#2498) 2026-04-12 08:50:40 +08:00
安正超 ff7dc77b5e fix(deploy): auto-create rustfs log directory (#2500) 2026-04-12 08:50:24 +08:00
安正超 b387f6c58a refactor(storage): inline object lock config entrypoints (#2494) 2026-04-12 08:49:51 +08:00
houseme 64508ae770 fix(storage): align archive content-encoding with S3 semantics (#2496) 2026-04-12 02:14:52 +08:00
John 6a2f98b468 fix(obs): respect error log level for http warnings (#2492)
Signed-off-by: 80347547 <jianglong@oppo.com>
Co-authored-by: 80347547 <jianglong@oppo.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-12 01:32:54 +08:00
houseme eb0dc24921 fix(get): validate fast path body before response commit (#2495)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2026-04-12 01:04:22 +08:00
Mohamed Zenadi c8c71e34b6 fix(storage): fix critical bug in range calculation (#2493)
Signed-off-by: Mohamed Zenadi <zeapo@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-12 00:20:15 +08:00
John 11552eb722 fix(deploy): use standard rustfs log directory (#2491)
Signed-off-by: 80347547 <jianglong@oppo.com>
Co-authored-by: 80347547 <jianglong@oppo.com>
2026-04-11 21:48:05 +08:00
安正超 b0b7f56281 refactor(storage): inline object lock metadata writes (#2489) 2026-04-11 20:05:03 +08:00
安正超 120f1021b5 refactor(storage): inline object tagging entrypoints (#2486) 2026-04-11 19:01:24 +08:00
安正超 20e07519a1 refactor(storage): inline object metadata read entrypoints (#2485)
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-11 14:59:38 +08:00
安正超 5c4dadc0b7 test(bucket): cover inline list response params (#2483) 2026-04-11 14:59:17 +08:00
Henry Guo 0f7e7f35e9 test(utils): make retry timer assertions deterministic (#2468) 2026-04-11 12:02:38 +08:00
安正超 5e97377aa0 refactor(storage): drop ecfs test-only helper (#2484) 2026-04-11 11:49:10 +08:00
安正超 b49570d87a refactor(storage): inline extract put-object branch (#2482) 2026-04-11 11:00:11 +08:00
安正超 d70ca1990e refactor(storage): inline list response params (#2476) 2026-04-11 09:25:40 +08:00
Henry Guo 70fdf79297 fix(server): warn on default credentials with console enabled (#2448)
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-11 00:03:06 +08:00
安正超 4472324125 refactor(storage): trim thin s3 forwarding layers (#2474) 2026-04-10 23:17:18 +08:00
houseme b8c45fc9e3 fix(get-object): harden GET fast path against mid-stream regressions (#2472) 2026-04-10 21:38:29 +08:00
weisd a8a2aaa460 fix(ecstore): avoid duplicate keys in ListObjectsV2 (#2467) 2026-04-10 11:29:23 +08:00
Henry Guo 2bf4b14394 docs(readme): fix inconsistent startup instructions (#2465)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-10 10:28:01 +08:00
安正超 59b5e4e722 refactor(app): inline remaining tagging outputs (#2461) 2026-04-10 10:27:27 +08:00
安正超 9ec6465b26 refactor(app): inline object request context types (#2462) 2026-04-10 10:27:08 +08:00
安正超 80cb6b3939 test(filemeta): cover legacy nil UUID metadata (#2464) 2026-04-10 10:26:23 +08:00
weisd 30c8bead63 fix(filemeta): accept nil legacy pool metadata (#2459) 2026-04-10 08:27:41 +08:00
安正超 2125cffd0b refactor(app): inline trivial s3 api outputs (#2460) 2026-04-10 08:07:44 +08:00
安正超 aeef2e67f9 refactor(app): inline bucket tagging outputs (#2455) 2026-04-10 07:47:44 +08:00
安正超 8acf2a51b9 refactor(app): reuse put object request context (#2456) 2026-04-10 07:08:27 +08:00
安正超 33b50a5366 refactor(app): inline bucket replication output (#2454) 2026-04-10 07:07:29 +08:00
安正超 acdd2de21f refactor(app): inline bucket encryption output (#2452) 2026-04-10 07:07:14 +08:00
安正超 b8bfaccbc5 refactor(app): inline multipart transition helper (#2451) 2026-04-10 07:06:50 +08:00
安正超 28d9bbbdcb refactor(storage): remove unused s3 api facades (#2450) 2026-04-10 07:06:30 +08:00
安正超 a9507f8ec7 refactor(app): reuse bucket listing helpers (#2449) 2026-04-10 07:06:11 +08:00
安正超 722cab500c refactor(app): reuse multipart uploads helpers (#2447) 2026-04-10 07:05:40 +08:00
安正超 71e9178182 refactor(app): reuse object tagging validation (#2445)
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-09 23:31:02 +08:00
安正超 aee8a4df51 refactor(app): reuse list parts parsing (#2446) 2026-04-09 22:59:56 +08:00
无心戈 57138fa660 docs: Update RustFS console access URL to port 9001 (#2442)
Signed-off-by: 无心戈 <wanxger@egeeke.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-04-09 21:46:31 +08:00
GatewayJ 8db55de72c fix(iam): return policy JSON object from info_policy (#2395) (#2436)
Co-authored-by: GatewayJ <8352692332qq.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-04-09 21:45:22 +08:00
安正超 90e584af74 test(capacity): cover poisoned scope registry recovery (#2441) 2026-04-09 13:29:00 +08:00
安正超 ff77cb5540 refactor(app): simplify object usecase plumbing (#2438) 2026-04-09 12:54:01 +08:00
安正超 61c0da936b fix(capacity): ignore future write buckets (#2440) 2026-04-09 09:24:04 +08:00
houseme d62114f8d5 refactor(nix): modify git author info (#2437) 2026-04-08 22:09:20 +08:00
houseme 9899985d15 build(deps): bump the dependencies group with 4 updates (#2435)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-04-08 21:34:16 +08:00
houseme 064e21062d fix(capacity): harden scope registry, scan symlink guard, and test temp dir cleanup (#2432)
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2026-04-08 20:58:17 +08:00
安正超 d4ea14c2ba ci: normalize tagged release package filenames (#2425) 2026-04-08 07:25:50 +08:00
安正超 1977d19c29 ci: force Node.js 24 for JS setup actions (#2424) 2026-04-07 23:49:43 +08:00
安正超 d72e7f6f28 Fix remaining review follow-ups for #2361 (#2421)
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: jeadie <jack@spice.ai>
Co-authored-by: Jack Eadie <jack.eadie0@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2026-04-07 23:24:23 +08:00
houseme 79ffecbf14 refactor(storage): remove object cache plumbing (#2422)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-04-07 22:00:53 +08:00
houseme e3000f16e0 fixed: Update service status to Ready (#2423) 2026-04-07 21:14:24 +08:00
houseme ec8f059506 feat: add timing and metrics for query execution phases (#2419)
Signed-off-by: Shreyan Jain <shreyan11@duck.com>
Co-authored-by: Shreyan Jain <shreyan11@duck.com>
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2026-04-07 20:32:32 +08:00
overtrue 9cd1d6b4d5 chore: merge upstream PR 2372 fixes 2026-04-07 19:38:59 +08:00
overtrue bf64bcabf8 fix(kms): restore configure and decrypt compatibility 2026-04-07 19:34:09 +08:00
overtrue 1f598105dd Merge branch 'pr-2372' into codex/merge-pr-2372-upstream 2026-04-07 19:24:18 +08:00
安正超 c4b5b00d9c Merge branch 'main' into feat/kms-vault-transit2 2026-04-07 19:16:28 +08:00
weisd 0b99e02891 feat(ecstore): add stale multipart upload cleanup (#2416) 2026-04-07 18:57:07 +08:00
weisd 15c8c7cecf chore: update s3s to 010ae03 (#2418) 2026-04-07 17:21:22 +08:00
houseme c9c2428373 fix(download): preserve archive download integrity (#2415)
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2026-04-07 16:25:34 +08:00
houseme d6158c0481 feat(mqtt): migrate client and harden TLS config (#2413)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-07 15:28:29 +08:00
weisd 9ed6487e2f fix(scanner): continue scans after traversal errors (#2414) 2026-04-07 14:19:26 +08:00
唐小鸭 72928a43fa Update rustfs/src/admin/handlers/kms_dynamic.rs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: 唐小鸭 <tangtang1251@qq.com>
2026-04-07 13:53:19 +08:00
唐小鸭 55f175b671 Update rustfs/src/admin/handlers/kms_dynamic.rs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: 唐小鸭 <tangtang1251@qq.com>
2026-04-07 13:53:05 +08:00
唐小鸭 3a16c25cd9 Update crates/kms/src/backends/vault_transit.rs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: 唐小鸭 <tangtang1251@qq.com>
2026-04-07 13:52:48 +08:00
majinghe 6963b898ee feat: add support for pvc customized annotations (#2412) 2026-04-07 13:34:49 +08:00
cxymds 7b8d7bdf18 Merge branch 'main' into feat/kms-vault-transit2 2026-04-07 11:11:43 +08:00
majinghe 751bc3d737 fix: move ec configuration from configmap to extraEnv (#2408) 2026-04-07 11:00:21 +08:00
weisd 898857d1c9 fix(iam): keep service account JWT expiry consistent (#2410) 2026-04-07 11:00:07 +08:00
安正超 97b06afd6c fix: enforce multipart authorization checks (#2411) 2026-04-07 10:50:20 +08:00
Alexander Kharkevich 740e4399af fix: skip missing groups in policy_db_get instead of aborting (#2393)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-04-07 09:17:39 +08:00
安正超 832909306e Merge branch 'main' into feat/kms-vault-transit2 2026-04-07 08:36:09 +08:00
houseme 32bf8f5bf3 feat(storage): add direct chunk GET fast path (#2351)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: cxymds <Cxymds@qq.com>
2026-04-07 08:33:46 +08:00
ankohuu 8d27170ce4 fix: limit pyroscope profiling to supported Unix targets (#2399)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-06 23:05:30 +08:00
安正超 dd68a419e3 test(server): cover request context layer propagation (#2398)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-04-06 20:35:37 +08:00
houseme e69d1dc3e6 build(deps): bump the dependencies group with 14 updates (#2407)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-04-06 17:55:38 +08:00
安正超 97d3fb15fb test(metrics): cover Prometheus descriptor names (#2405) 2026-04-06 10:16:53 +08:00
ankohuu 0e5fc4bec1 fix(metrics): use Prometheus-compatible metric names (#2312) (#2317)
Signed-off-by: Shunchao Hu <ankohuu@gmail.com>
2026-04-05 21:39:10 +08:00
安正超 77229fe426 test(admin): cover audit target validation gaps (#2390) 2026-04-04 14:41:35 +08:00
houseme eabbea46d3 refactor(tracing): unify request-context propagation and fix tracing chain breaks (#2394)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-04-04 12:20:59 +08:00
houseme d2901fd78c feat(admin): add audit target APIs and harden target source handling (#2350)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2026-04-04 09:07:22 +08:00
安正超 67863630b2 fix(auth): reject ambiguous case-insensitive claim matches (#2386) 2026-04-04 08:36:14 +08:00
安正超 a9be9af094 ci: bump cla-bot to v0.0.9 (#2389) 2026-04-04 08:35:57 +08:00
houseme 372f004a8d refactor(server): unify TLS loading, optimize HTTP transport, add hot reload (#2388) 2026-04-04 06:37:24 +08:00
安正超 0ac3b5b992 docs: remind agents to clean build artifacts (#2387) 2026-04-03 22:51:29 +08:00
weisd 25512e2635 perf(ecstore): batch delete object lock acquisition (#2374)
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-04-03 21:46:51 +08:00
Logan Ye 2d91e2f580 fix(oidc): support case-insensitive claim name matching (#2362)
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-04-03 21:45:56 +08:00
GatewayJ c244943313 feat(iam): retry OIDC discovery with issuer URL slash variants (#2360)
Co-authored-by: GatewayJ <8352692332qq.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-04-03 21:10:27 +08:00
xxkeming b3f31ad694 fix(build): enable tokio_unstable in build script (#2376)
Co-authored-by: houseme <housemecn@gmail.com>
2026-04-03 21:09:30 +08:00
Andy Brown c4efb46827 fix(notify): emit delete webhooks for prefix deletes and align replication headers (#2383)
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-04-03 21:09:05 +08:00
安正超 1fe036cb70 ci: update CLA workflow for corrected comments (#2384) 2026-04-03 20:49:09 +08:00
weisd 5d302febb7 fix(rio): preserve reader capabilities and crypto safety (#2363) 2026-04-03 13:57:42 +08:00
weisd 6a114cd2e0 fix: bump s3s for presigned checksum handling (#2379) 2026-04-03 13:27:56 +08:00
安正超 6696703343 test: cover delete-group percent decoding (#2373) 2026-04-03 11:30:56 +08:00
安正超 aefb99f1da Merge branch 'main' into feat/kms-vault-transit2 2026-04-03 11:26:51 +08:00
安正超 c44309c16a ci: bump cla-bot to v0.0.6 (#2377) 2026-04-03 11:21:14 +08:00
安正超 c3361e38d6 ci: bump cla-bot to v0.0.5 (#2375) 2026-04-03 10:55:40 +08:00
reatang 30a757a9da feat(kms): add vault transit engine 2026-04-03 00:46:00 +08:00
GatewayJ 84f58af628 fix(admin): percent-decode group name in DELETE /v3/group/{group} (#2358)
Co-authored-by: GatewayJ <8352692332qq.com>
2026-04-02 23:52:04 +08:00
安正超 890837aee8 docs: update AGENTS pre-commit policy (#2370)
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-02 22:26:05 +08:00
安正超 c513275741 Update CLA Bot token usage in workflow
Signed-off-by: 安正超 <anzhengchao@gmail.com>
2026-04-02 22:07:40 +08:00
安正超 9d3191e55b Modify CLA workflow permissions and cleanup (#2369)
Signed-off-by: 安正超 <anzhengchao@gmail.com>
2026-04-02 22:02:35 +08:00
安正超 6fba01fb65 ci: use GitHub App tokens for CLA bot (#2368) 2026-04-02 20:42:48 +08:00
安正超 a8af7c9617 ci: integrate CLA bot checks (#2367) 2026-04-02 20:21:02 +08:00
安正超 bd36cf3588 test(filemeta): cover legacy delete marker decoding (#2333)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-31 22:08:45 +08:00
安正超 8893de1cad test(admin): cover empty kms key aliases (#2331)
Co-authored-by: cxymds <Cxymds@qq.com>
2026-03-31 22:08:21 +08:00
安正超 d3dee898ee test(object): cover zero-copy selection heuristics (#2338)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: cxymds <Cxymds@qq.com>
2026-03-31 22:07:58 +08:00
cxymds d960acb170 fix(admin): reconcile site replication peer identity (#2356) 2026-03-31 18:12:56 +08:00
weisd e86c6b726f fix(lock): split distributed read and write quorum (#2355) 2026-03-31 16:16:13 +08:00
weisd 15995aae14 feat(admin): complete site replication support (#2346) 2026-03-31 13:01:38 +08:00
lunrenyi 6bf0c542a1 docs: add x-cmd and nix installation options (#2306)
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-30 22:05:46 +08:00
majinghe d56e839f20 feat: add extra env support for helm chart (#2340) 2026-03-30 22:03:36 +08:00
weisd 0fb070e912 feat(s3): support metadata extensions for bucket listings (#2344) 2026-03-30 22:03:13 +08:00
majinghe 172086ff42 fix: change the condition for httproute (#2345)
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-30 22:03:01 +08:00
安正超 0b0c10b323 docs(agents): enforce constant reuse rules (#2348) 2026-03-30 21:50:14 +08:00
安正超 b256154f25 test(admin): cover tier, bucket metadata, and kms aliases (#2334) 2026-03-30 21:48:42 +08:00
安正超 d5f05993a3 test(ecstore): cover read offset overflow (#2341) 2026-03-30 20:48:03 +08:00
安正超 d0ea41e190 test(ecstore): cover inline bitrot offset reads (#2337) 2026-03-30 20:40:26 +08:00
安正超 74b2c70602 test(admin): cover heal alias routes (#2329) 2026-03-30 20:11:54 +08:00
houseme 16db18216d fix: populate tagging notification principalId and object metadata (#2342) 2026-03-30 14:32:39 +08:00
dependabot[bot] 387c385dfa build(deps): bump rustc-hash from 2.1.1 to 2.1.2 in the dependencies group (#2339)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-30 09:09:27 +08:00
houseme dd9e093dcc perf(capacity): tune default capacity settings, sync docs, and fix refresh/metrics correctness (#2336)
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-30 05:33:56 +08:00
houseme 7172e151de fix: address correctness, safety, and concurrency issues (#2327)
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-30 00:30:57 +08:00
安正超 860a37d3a8 test(admin): cover alias parsing edge cases (#2326) 2026-03-29 20:10:19 +08:00
安正超 3515615e79 test(s3): cover anonymous write-offset rejection (#2320) 2026-03-29 20:06:16 +08:00
安正超 924c4b17a6 test(storage): cover write-offset pre-auth rejection (#2319) 2026-03-29 20:06:04 +08:00
安正超 7fb405526b test(filemeta): cover legacy delete marker fallback (#2322) 2026-03-29 20:05:48 +08:00
安正超 4764b849cb fix(admin): route root heal start through heal_format (#2323) 2026-03-29 20:05:37 +08:00
安正超 0c6bb6add5 test(admin): cover compatible alias routes (#2328) 2026-03-29 20:05:20 +08:00
安正超 3578baf501 test(object-lock): cover validation gaps (#2318) 2026-03-29 19:23:30 +08:00
安正超 939a69f9c2 test(s3): complete snowball auto-extract compatibility (#2324) 2026-03-29 19:19:45 +08:00
GatewayJ 3366bd2464 feat(iam,admin): prepared IAM auth, ExistingObjectTag, admin permission checks (#2315)
Signed-off-by: GatewayJ <835269233@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: GatewayJ <8352692332qq.com>
2026-03-29 19:18:16 +08:00
houseme 263e504c0c refactor(capacity): optimize capacity management module (#2325)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2026-03-29 17:49:30 +08:00
安正超 f98664ea4a test(s3): complete snowball auto-extract coverage (#2313) 2026-03-28 22:56:42 +08:00
安正超 9536ed8d38 test(storage): cover write-offset rejection (#2316)
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-28 22:56:03 +08:00
安正超 ef33e43032 feat(admin): align heal compatibility routes (#2311) 2026-03-28 19:57:22 +08:00
安正超 c20f2555eb feat(admin): add MinIO-compatible admin aliases (#2307) 2026-03-28 08:19:56 +08:00
weisd 5e21c398f5 fix(filemeta): support legacy xl.meta compatibility (#2304) 2026-03-27 13:42:06 +08:00
houseme af46a61fde build(deps): bump the dependencies group with 6 updates (#2303)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-27 12:47:48 +08:00
majinghe 14e4d94666 add ec environment variables in helm chart (#2290)
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-27 09:40:30 +08:00
weisd 0779036535 feat(s3): reject write-offset-bytes requests compatibly (#2295) 2026-03-26 15:48:12 +08:00
weisd d637c4d342 fix(object-lock): recover remaining s3 tests (#2294) 2026-03-26 12:11:34 +08:00
安正超 a236b0d01d feat(ecstore): implement decommission and rebalance (#2281)
Co-authored-by: weisd <im@weisd.in>
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-26 11:44:02 +08:00
weisd 59c437d901 feat(object-lock): complete legal hold enforcement (#2293) 2026-03-26 10:58:10 +08:00
houseme 0c42916fa9 ci(build): enable tokio_unstable flag in Build RustFS job (#2289) 2026-03-25 17:03:21 +08:00
weisd 41dcebda44 fix(tier): sweep transitioned copies from delete handlers (#2287) 2026-03-25 16:06:36 +08:00
houseme fb2ced4d27 feat(obs): integrate dial9-tokio-telemetry for runtime tracing (#2285)
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2026-03-25 14:23:58 +08:00
weisd 2681731443 fix(checksum): align multipart CRC64NVME with full object (#2286) 2026-03-25 12:44:46 +08:00
houseme 19b8389dc4 fix(disk): Fix Usage Report Capacity Calculation (#2274)
Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: heihutu <heihutu@gmail.com>
2026-03-24 23:47:30 +08:00
安正超 8c8d157418 fix(object): always unregister deadlock-tracked get requests (#2275)
Signed-off-by: heihutu <heihutu@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: cxymds <Cxymds@qq.com>
2026-03-24 19:09:45 +08:00
weisd 28f57b228c feat(s3): advance parity coverage (#2278) 2026-03-24 17:29:33 +08:00
heihutu 8aa59b12cb refactor(auth): Improve UI access token login issue (#2277) 2026-03-24 14:48:37 +08:00
cxymds dad9a7d708 fix(ecstore): honor lifecycle tag filters (#2264)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-03-24 14:25:43 +08:00
cxymds 5ea6d8a7e6 fix(ecstore): preserve transition object metadata (#2263)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
2026-03-24 14:06:29 +08:00
cxymds 75e6902f46 feat(admin): add persisted OIDC config APIs (#2267)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-03-24 12:13:41 +08:00
majinghe 24d359a867 fix: CVE-2026-22184 fix in docker image (#2276)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-03-24 11:36:40 +08:00
Peter Olds ca62b0c163 fix(Helm): Remove duplicate imagePullSecrets block (#2260)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: cxymds <Cxymds@qq.com>
2026-03-24 09:54:33 +08:00
heihutu b7789c8e08 refactor(docker): image grafana/tempo:2.10.3 (#2273) 2026-03-23 23:25:58 +08:00
dependabot[bot] e95eb92612 build(deps): bump zip from 8.3.1 to 8.4.0 in the dependencies group (#2270)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-23 20:50:17 +08:00
dependabot[bot] 2583d1e49b build(deps): bump the dependencies group with 3 updates (#2259)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: wly <wlywly0735@126.com>
2026-03-23 19:56:09 +08:00
LeonWang0735 990acbcd4b fix(admin): decode compat payload in set-remote-target (#2216)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: 马登山 <Cxymds@qq.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-03-23 18:22:07 +08:00
weisd 05dc131a49 perf(storage): optimize internode RPC transfer path (#2262)
Co-authored-by: momoda693 <momoda693@gmail.com>
2026-03-23 17:09:49 +08:00
cxymds 236142a682 fix(ecstore): repair lifecycle transition and restore flows (#2240)
Co-authored-by: heihutu <heihutu@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: weisd <im@weisd.in>
2026-03-23 12:29:13 +08:00
weisd 2e7abfbd63 fix(ecstore): preserve raw metadata read semantics (#2258)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
2026-03-23 10:42:16 +08:00
houseme 6cb094e30a fix(object): Fix concurrent request hang issue in S3 range read workloads (#2251)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-03-23 09:55:08 +08:00
heihutu ff40e2bc79 build(deps): bump the dependencies group with 3 updates (#2257) 2026-03-22 17:24:42 +08:00
安正超 99dbe70a89 fix(madmin): handle blank service-account expiration (#2254)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-03-22 13:44:05 +08:00
weisd a42320848c fix(ecstore): invalidate xl.meta cache after writes and rename (#2255) 2026-03-22 11:30:29 +08:00
heihutu 8b4e5b2540 chore(deps): update flake.lock (#2253) 2026-03-22 09:00:17 +08:00
GatewayJ 19d3a23a13 fix(admin): console self password for STS sessions (#1923) (#2250)
Co-authored-by: GatewayJ <8352692332qq.com>
2026-03-21 22:10:38 +08:00
weisd 95850c1bcd fix(admin): accept IAM service account exports (#2249)
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-21 13:44:46 +08:00
houseme 628481be7c fix(tracing): Fix distributed tracing context linking (#2247) 2026-03-21 11:07:35 +08:00
houseme 0047bcd3ac build(deps): bump the dependencies group with 4 updates (#2243) 2026-03-20 23:22:57 +08:00
houseme fafbc4fe1d feat(info): add --json flag for JSON output and markdown table format by default (#2245) 2026-03-20 22:53:03 +08:00
马登山 f11c307aec fix(admin): avoid tier stats panic on missing tier (#2238)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-03-20 20:57:33 +08:00
heihutu 3c28f0a0ba feat(metrics): migrate system monitoring from rustfs-obs to rustfs-metrics (#2242)
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-20 18:52:33 +08:00
houseme 28f86a505e feat: Add --info command and refactor config module (#2234)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-20 01:16:45 +08:00
Trent Houliston e1a278aaf8 fix(iam): preserve trailing slash in OIDC issuer URL (#2228) 2026-03-19 13:52:59 +08:00
weisd 35e1f28f23 fix: support legacy bucket metadata decoding (#2227) 2026-03-19 12:43:53 +08:00
weisd c6715259b1 fix(ecstore): keep bucket created time when metadata is epoch (#2220)
Co-authored-by: momoda693 <momoda693@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-19 01:02:34 +08:00
weisd df8f0edaea fix(ecstore): support legacy pool meta decoding (#2218)
Co-authored-by: momoda693 <momoda693@gmail.com>
2026-03-19 00:20:24 +08:00
houseme a4411a24d6 refactor(metrics): modularize collectors and add comprehensive metrics support (#2208)
Co-authored-by: overtrue <anzhengchao@gmail.com>
Co-authored-by: heihutu <heihutu@gmail.com>
2026-03-18 23:52:42 +08:00
houseme 4a807d80e3 build(deps): bump the dependencies group with 2 updates (#2211) 2026-03-18 21:39:01 +08:00
安正超 f095f56e20 fix(ci): revert docker alpha latest tag (#2209) 2026-03-18 21:05:52 +08:00
weisd b9b7d86ae4 feat: improve legacy metadata and admin compatibility (#2202) 2026-03-18 21:05:09 +08:00
马登山 84077adf17 fix(admin): avoid unbounded metrics sampling by default (#2203) 2026-03-18 14:43:17 +08:00
马登山 237c933f38 test(lifecycle): add prefix regression and zero-day checks (#2201) 2026-03-18 13:20:00 +08:00
安正超 c20b3c7f19 fix(ecstore): handle EODM rules without due date (#2198) 2026-03-18 12:54:41 +08:00
安正超 c9a2fd756c feat(rustfs): add optional license gating feature (#2197) 2026-03-18 08:45:05 +08:00
dependabot[bot] bddb0d0a05 build(deps): bump astral-tokio-tar from 0.5.6 to 0.6.0 (#2196)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-18 08:22:12 +08:00
houseme 9ce3c7742c fix(targets): pass credentials to MQTT broker availability check (#2192)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-17 23:53:28 +08:00
houseme b5d881f399 fix(notify): Fix XML Filter parsing and add comprehensive tests (#2191) 2026-03-17 23:08:07 +08:00
安正超 ce1f7cfdcb chore(skills): add repository-local workflow skills (#2190) 2026-03-17 22:13:46 +08:00
马登山 c66c6d97ec fix(lifecycle): respect Filter.Prefix and safe delete marker expiry (#2185)
Signed-off-by: likewu <likewu@126.com>
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: likewu <likewu@126.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-17 18:45:38 +08:00
dependabot[bot] be89b5fc6a build(deps): bump lz4_flex from 0.12.0 to 0.12.1 (#2181)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-17 10:10:41 +08:00
houseme 94cdb89e29 feat(obs): add init_obs_with_config API and signature guard test (#2175)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
2026-03-16 18:17:55 +08:00
heihutu 06dff96c09 chore(deps): update flake.lock (#2173) 2026-03-16 16:01:36 +08:00
安正超 c1d5106acc feat(ci): allow selecting build platforms in build workflow (#2171) 2026-03-15 22:01:44 +08:00
heihutu 0a2411f59c chore(deps): update flake.lock (#2169)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-15 16:10:12 +08:00
houseme 1ede71b881 chore: update nix-flake-update.yml to use FLAKE_UPDATE_TOKEN for user… (#2168) 2026-03-15 14:49:38 +08:00
github-actions[bot] 4fb7059e6f chore(deps): update flake.lock (#2165) 2026-03-15 10:26:05 +08:00
安正超 2ad275ecc3 fix(helm): quote obs stdout configmap value (#2166)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: 马登山 <Cxymds@qq.com>
2026-03-15 10:11:25 +08:00
Philip Schmid 9179fd5608 fix(helm): merge customAnnotations with class-specific ingress annotations (#2161)
Signed-off-by: Philip Schmid <philip.schmid@protonmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-15 09:22:12 +08:00
LeonWang0735 7f1cdaedad feat(replication): add bandwidth-aware reporting for bucket replication metrics (#2141) 2026-03-15 09:03:10 +08:00
houseme 7f3459f5a8 fix(obs): fixed unresolved import super::local::ensure_dir_permissions (#2164) 2026-03-15 00:33:06 +08:00
yxrxy d3cff7d033 feat(webdav): add WebDAV protocol gateway (#2158)
Signed-off-by: yxrxy <1532529704@qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: 马登山 <Cxymds@qq.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-03-14 23:06:53 +08:00
majinghe f66a90c1b2 fix: fix github action error caused by oltp modification (#2163)
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-14 22:15:35 +08:00
majinghe afcaaf66fc feat: add support for obs enpoint support in helm chart (#2160) 2026-03-14 21:44:44 +08:00
安正超 a1104b45f6 fix(obs): honor target-only rust_log directives (#2159)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-14 11:14:46 +08:00
bcdax110 82d9452736 docs: fix incorrect UID in Docker Quick Start of README_ZH (#2149)
Signed-off-by: bcdax110 <1711382287@qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-14 09:26:07 +08:00
houseme 6e0f034ad1 refactor(obs): enhance log rotation robustness and refine filter logic (#2155)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-14 09:20:35 +08:00
houseme 593a58c161 refactor(obs): optimize logging with custom RollingAppender and improved cleanup (#2151)
Signed-off-by: houseme <housemecn@gmail.com>
Signed-off-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-13 13:20:27 +08:00
houseme f83bf95b04 feat(ecstore): Skip rustls provider install if already present (#2145)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2026-03-12 18:02:19 +08:00
安正超 aa88b1976a fix(ecstore): avoid warm tier init panics (#2144) 2026-03-12 13:52:49 +08:00
安正超 e2f741d41f fix(helm): use canonical scanner start delay env (#2142) 2026-03-12 10:06:42 +08:00
安正超 ad54293d7e fix(admin): propagate heal handler background errors (#2124) 2026-03-12 10:06:12 +08:00
安正超 83fb530609 refactor(config): normalize scanner env naming (#2129) 2026-03-11 22:41:41 +08:00
安正超 aa84d34bf8 fix(auth): preserve IAMAuth clone and correct missing-key error (#2123) 2026-03-11 21:59:12 +08:00
安正超 df57f0c033 fix(workers): clamp worker release count (#2122) 2026-03-11 21:59:00 +08:00
安正超 c47dec8549 fix(signer): avoid panics in v2 signing for missing data (#2121) 2026-03-11 21:58:40 +08:00
安正超 fdbe12ec95 fix(scanner): respect configured scan start delay (#2119) 2026-03-11 21:56:48 +08:00
安正超 b2e8078971 fix(policy): avoid unicode panic in variable resolver (#2115) 2026-03-11 21:56:32 +08:00
安正超 ac43a44a00 [codex] fix scanner first cycle startup delay (#2137) 2026-03-11 20:02:01 +08:00
安正超 5625f04697 fix(common): remove panic paths in runtime helpers (#2116)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
2026-03-11 18:12:37 +08:00
安正超 e1f24f764d fix(credentials): harden masked debug output (#2114)
Signed-off-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
2026-03-11 15:40:37 +08:00
安正超 7d7e0b2654 fix(utils): harden panic-prone paths (#2113)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
2026-03-11 15:16:03 +08:00
安正超 9908a44c38 fix(protocols): return errors instead of panics for sync signatures (#2120)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
2026-03-11 11:22:20 +08:00
evan slack 4b480727d6 feat(perf): Add configurable bitrot skip for reads (#2110)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-03-11 10:59:00 +08:00
simon-escapecode f00d01ec2d fix: resolve silent failure in MQTT bucket event notifications (#2112)
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-11 10:08:30 +08:00
dependabot[bot] 7e8c7fa2b2 build(deps): bump quinn-proto from 0.11.13 to 0.11.14 (#2127)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-11 09:30:53 +08:00
安正超 845ad1fa16 fix(obs): avoid panic in telemetry init and clamp sampler boundaries (#2118)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-11 01:32:46 +08:00
安正超 bb4fbf5ae2 fix(notify): ignore disabled targets when sending events (#2117)
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-11 00:37:30 +08:00
安正超 3df7105dae fix(server): init event notifier when partial notify configured (#2125) 2026-03-10 23:52:40 +08:00
evan slack b3da8ae269 feat(scanner): Add dynamic throttling presets (#2095)
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: GatewayJ <835269233@qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: weisd <im@weisd.in>
2026-03-10 16:12:56 +08:00
majinghe 67e5f5e3c3 feat: add metrics support in helm chart (#2109)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-10 12:08:36 +08:00
majinghe 296efea42f change ghcr username and password name due to github restrict (#2108) 2026-03-09 21:44:35 +08:00
GatewayJ 16946c5a54 fix: allow root to bypass bucket policy deny for policy management APIs (#2102)
Co-authored-by: GatewayJ <8352692332qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-03-09 20:36:29 +08:00
majinghe 73d29e95dd feat:add docker image support for quay.io and ghcr.io (#2107)
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-09 16:22:28 +08:00
dependabot[bot] e930c5c281 build(deps): bump libc from 0.2.182 to 0.2.183 in the dependencies group (#2106)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-09 16:03:17 +08:00
安正超 9d03029959 fix(iam): sync user cache on load-user notifications (#2104) 2026-03-09 09:36:02 +08:00
loverustfs a02c354ef5 Fix image url error
Fix image url error

Signed-off-by: loverustfs <hello@rustfs.com>
2026-03-08 23:39:50 +08:00
houseme 60aa47bf61 feat(storage): integrate S3Operation into OperationHelper for unified metrics and audit (#2103) 2026-03-08 17:57:33 +08:00
houseme 8e4a1ef917 refactor(protocols): replace tar with astral-tokio-tar for async processing (#2099)
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2026-03-08 15:18:15 +08:00
Peter Hamilton b035d10abb fix(metrics): Remove high cardinality labels causing memory leak (#2098)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-03-08 13:01:11 +08:00
github-actions[bot] 2180e9e7a1 chore(deps): update flake.lock (#2097)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-08 10:51:32 +08:00
evan slack 57e49e6737 feat(obs): Add metric to count all s3 operations (#2088)
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-08 10:19:20 +08:00
Senol Colak b07383760f Add OpenStack Swift API Support (#2066)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <noreply@github.com>
2026-03-08 01:11:35 +08:00
evan slack 7c94be4e8c fix(obs): Remove high cardinality label on rustfs_api_requests_total (#2087)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-03-07 20:46:33 +08:00
evan slack d52a10c5fb chore(obs): Improve tracing instrumentation (#2086)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-03-07 20:03:20 +08:00
安正超 8c4735ff88 docs: scope AGENTS instructions by directory (#2083) 2026-03-05 17:25:37 +08:00
LeonWang0735 a0503168d4 fix(heal):heal failed replication via must_replicate instead of check replicate_delete (#2072) 2026-03-05 15:47:36 +08:00
安正超 b73059dcf2 fix(admin): allow non-consoleAdmin self password update (#2082) 2026-03-05 15:47:21 +08:00
weisd ed18b3da75 Fix data usage cache and scanner (#2074) 2026-03-04 19:55:01 +08:00
houseme 05032cf887 chore: update dependencies and workspace resolver (#2073) 2026-03-04 19:22:54 +08:00
唐小鸭 f89cdfe5b3 update s3s 0.14.0-dev (#2070)
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-04 01:07:24 +08:00
houseme f4b523c236 build(deps): bump the dependencies group with 7 updates (#2069) 2026-03-04 00:42:03 +08:00
安正超 c6209ba59d ci: optimize workflow runtime and remove redundant pipeline work (#2065) 2026-03-03 20:56:37 +08:00
houseme 5e7495a042 build(obs): restrict pyroscope dependency to unix targets (#2064) 2026-03-03 20:41:37 +08:00
evan slack ac4b13def1 feat(obs): Optional continuous CPU profiling with grafana pyroscope (#2035)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
2026-03-03 15:28:58 +08:00
安正超 08e1f4670b fix: restore default CORS fallback and STS object ACL ownership (#2053)
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-03 01:08:50 +08:00
唐小鸭 fff96a0921 fix sse-options (#2056) 2026-03-03 01:08:37 +08:00
唐小鸭 f17725a2ea fix(sse): allow PUT/GET without KMS when no SSE or bucket default (#2054)
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-03 00:44:23 +08:00
houseme bf957e3523 remove rustflags target cpu (#2052) 2026-03-02 23:59:34 +08:00
Rafael Herrero a6090b98dc fix(iam): remove incorrect trailing slash from OIDC issuer URL (#2050) 2026-03-02 19:48:12 +08:00
houseme 2ac07c95a8 refactor(obs): enhance log cleanup and rotation (#2040) 2026-03-02 16:28:32 +08:00
安正超 e157a88f09 fix: support query-only presigned URL access (#2046) 2026-03-02 15:46:50 +08:00
安正超 01a75b5f58 Add env variable alias compatibility warnings (#2044) 2026-03-02 15:34:19 +08:00
GatewayJ 2cb8db36a5 fix(iam): user group policy and delete group (fixes #2028) (#2043)
Co-authored-by: GatewayJ <8352692332qq.com>
2026-03-02 14:19:01 +08:00
weisd e3815aa101 fix(ecstore): add etag fallback when mod_time unavailable in metadata (#2042) 2026-03-02 13:50:17 +08:00
dependabot[bot] fd32507ce5 build(deps): bump datafusion from 52.1.0 to 52.2.0 in the dependencies group (#2037)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-02 11:37:24 +08:00
GatewayJ ba32fd9d96 fix(s3): allow anonymous access when PublicAccessBlock config is miss… (#2039)
Co-authored-by: GatewayJ <8352692332qq.com>
2026-03-02 11:37:00 +08:00
安正超 273dbc9c38 feat(s3): return 409 BucketAlreadyExists when non-owner creates existing bucket (#2034) 2026-03-01 22:53:41 +08:00
安正超 f0c5d762f3 feat(s3): enforce RestrictPublicBuckets for anonymous access (#2033)
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-01 20:50:19 +08:00
houseme c452f24487 Optimize log cleanup and rotation, update dependencies (#2032)
Co-authored-by: heihutu <heihutu@gmail.com>
2026-03-01 20:09:52 +08:00
安正超 798e620088 fix(s3): add x-amz-grant-* headers to policy condition values (#2031) 2026-03-01 19:00:04 +08:00
安正超 e5e1010c31 fix(s3): return InvalidRange when CopySourceRange exceeds source object size (#2029) 2026-03-01 17:46:31 +08:00
安正超 8aecc7267b fix(s3): implement S3-compliant CORS and bucket existence checks (#2026) 2026-03-01 16:02:02 +08:00
安正超 f42b155f59 fix(s3): allow Object Lock on versioned buckets and reject invalid checksums (#2024) 2026-03-01 14:19:02 +08:00
Smig d13c423d50 Bump Helm Chart version to 0.0.83 and appVersion to 1.0.0-alpha.83 (#2019)
Signed-off-by: Smig <89040888+smiggiddy@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-01 13:15:50 +08:00
安正超 fbb162d8bb test(s3): promote 145 passing tests to implemented list (#2023) 2026-03-01 12:55:49 +08:00
github-actions[bot] 7c52af22e5 chore(deps): update flake.lock (#2020)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-03-01 12:43:12 +08:00
安正超 7a83b818b8 fix(policy): address review feedback from #2018 (#2021)
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
2026-03-01 11:05:20 +08:00
heihutu 595f916ba3 build(deps): bump the dependencies group with 2 updates (#2017)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: 唐小鸭 <tangtang1251@qq.com>
2026-03-01 10:04:05 +08:00
安正超 7eb136faf0 feat(policy): add Service principal, ArnLike/IfExists conditions, and logging error ordering (#2018) 2026-03-01 08:44:42 +08:00
heihutu 2c01b8c49d feat(obs): add advanced log management configuration (#2016)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: 唐小鸭 <tangtang1251@qq.com>
2026-03-01 03:23:48 +08:00
安正超 e7466eb1cc fix: policy StringNotEquals double negation and delete_objects version mapping (#2015) 2026-03-01 03:13:52 +08:00
evan slack fd1b903531 fix(obs): Update observability docker compose stack (#2010) 2026-03-01 03:03:50 +08:00
安正超 fe884eabfc fix(s3): improve S3 API compatibility for versioning, SSE, and policy (#2013) 2026-03-01 02:21:13 +08:00
安正超 0701e1c35f chore(s3-tests): promote 42 passing tests to implemented list (#2011) 2026-03-01 01:39:08 +08:00
唐小鸭 568c07ced9 fix: implement handling for "aws-chunked" Content-Encoding (#2009)
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-03-01 01:22:12 +08:00
安正超 f93db578df fix(s3): reject invalid SSE algorithm (aes:kms) in PutObject (#2008) 2026-03-01 00:56:26 +08:00
安正超 1872bdcedd fix(s3): reject SSE-C with partial headers per S3 spec (#2007) 2026-02-28 22:56:35 +08:00
安正超 27ff35e574 fix: avoid region fallback panic paths (#2006) 2026-02-28 22:35:56 +08:00
安正超 aa3f960b3d Fix: validate SSE headers in object read/write paths (#2005) 2026-02-28 18:07:56 +08:00
安正超 212b7ae8e1 Update logo image link in README.md (#2004)
Signed-off-by: 安正超 <anzhengchao@gmail.com>
2026-02-28 16:29:11 +08:00
安正超 b4a633ebc6 fix(ecstore): set expiration header for put object via lifecycle prediction (#2003) 2026-02-28 16:21:58 +08:00
安正超 3f5ccb20fc fix(s3): normalize GetObjectAttributes ETag XML response (#2002) 2026-02-28 14:53:53 +08:00
houseme 274b6f8bc7 build(deps): bump the dependencies group with 4 updates (#2001) 2026-02-28 12:01:17 +08:00
安正超 a24cbbb7a6 fix(s3): return proper HTTP 400 for SSE-C validation errors (#1998) 2026-02-28 10:24:46 +08:00
安正超 af6c32efac refactor: improve code quality with safer error handling, trait decomposition, and dead code cleanup (#1997) 2026-02-28 01:19:47 +08:00
Brayan Jules 7ce23c6b54 fix(ecstore): allow trailing slash in object names to match S3 behavior (#1996)
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-02-27 22:46:42 +08:00
evan slack dcbc67eb91 perf(lock): Use global lock manager, instead of one per request (#1848)
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-27 22:26:32 +08:00
LoganZ2 e73b17aff6 fix(scanner): skip recent IO-error objects (#1860)
Signed-off-by: LoganZ2 <103290230+LoganZ2@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-02-27 22:25:52 +08:00
GatewayJ 55396f13d4 feat: policy add object tag (#1908)
Co-authored-by: GatewayJ <8352692332qq.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-02-27 22:24:57 +08:00
Senol Colak b69183aadf Openstack Keystone integration - v1 keeps the same mechanism as (#1961)
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-02-27 22:23:35 +08:00
houseme d17d2083d4 feat(targets): enhance webhook TLS support with custom CA and skip-verify (#1994)
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: heihutu <heihutu@gmail.com>
2026-02-27 21:24:49 +08:00
evan slack bdb2a9e9b7 fix(dashboard): Rename grafana dashboard rustfs.yaml -> rustfs.json (#1987)
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-02-27 15:49:34 +08:00
安正超 9d2b8822cf refactor: stabilize heal format recovery integration tests (#1984)
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-27 15:26:19 +08:00
houseme 3433dfa88e feat(config): refine defaults and improve region handling (#1990)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-27 15:02:29 +08:00
weisd 68cc0a5df7 chore(heal): remove unused global response broadcast (#1991) 2026-02-27 14:12:03 +08:00
weisd 368bba3345 fix(ecstore): adjust unformatted disk error mapping (#1988) 2026-02-27 13:45:33 +08:00
安正超 b23a1a4ff9 refactor(app): remove dead objects/ code and migrate put_object_extract to usecase layer (#1980)
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-27 10:24:48 +08:00
LeonWang0735 10140be6d8 fix(replication): handle TLS CA trust and force-delete replication edge cases (#1983) 2026-02-27 08:40:39 +08:00
heihutu c32b6f2f37 refactor region parsing (#1981) 2026-02-27 02:34:09 +08:00
heihutu d983638391 build: update docker config and refine s3s region handling (#1976)
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-27 01:21:12 +08:00
安正超 eb07f084cb refactor(app): complete phase 5 gate and equivalence guards (#1979) 2026-02-26 23:05:24 +08:00
安正超 09aa6d9f6f refactor(app): remove remaining global access in main init (#1978) 2026-02-26 22:14:51 +08:00
安正超 a3c76618f3 refactor(ci): add layered dependency guard baseline (#1977) 2026-02-26 21:55:45 +08:00
安正超 2c85721654 refactor(app): centralize context resolvers for admin/server paths (#1975) 2026-02-26 20:41:11 +08:00
安正超 dafb31d208 refactor(rpc): use node name accessor in health handlers (#1972) 2026-02-26 19:37:16 +08:00
安正超 40903ec2af refactor(admin): move KMS management handlers (#1971) 2026-02-26 15:29:51 +08:00
安正超 49579129c1 refactor(app): decouple AppContext adapters from GLOBAL statics (#1970) 2026-02-26 14:54:45 +08:00
安正超 40692f18ed fix(iam): address PR 1875 review issues for OIDC STS flows (#1969) 2026-02-26 14:38:57 +08:00
Jeff Poegel c35ef84a8c feat(iam): add OpenID Connect SSO with claim-based policy resolution (#1875)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-02-26 14:03:17 +08:00
安正超 0f8bc461d6 refactor(admin): route kms handlers through app context (#1967)
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-26 13:31:59 +08:00
evan slack ae6eacd7e3 fix(typo): change dang_ling to dangling (#1968) 2026-02-26 13:03:36 +08:00
安正超 4b82cc20bb refactor(admin): route kms handlers via app context (#1965) 2026-02-26 10:32:16 +08:00
安正超 1c01c3d73a refactor(app): route buffer config through AppContext (#1964) 2026-02-26 09:19:59 +08:00
安正超 1a549d78ca refactor(storage): converge put-object quota metadata context (#1963) 2026-02-26 09:18:07 +08:00
安正超 7909a57634 refactor(server): route config access through AppContext (#1960)
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-26 00:04:03 +08:00
安正超 fd86d0bd0f refactor(admin): route tier config manager through AppContext (#1959) 2026-02-25 23:11:00 +08:00
安正超 4c08e18812 refactor(app): converge lower-priority global reads via AppContext (#1958) 2026-02-25 22:30:53 +08:00
安正超 dc795a494a refactor(app): remove multipart metadata global fallback (#1957)
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
2026-02-25 21:47:35 +08:00
houseme 024a3107d8 build(deps): bump the dependencies group with 4 updates (#1955) 2026-02-25 19:21:20 +08:00
安正超 aea7f41149 refactor(app): route admin/object globals through AppContext (#1954) 2026-02-25 18:01:52 +08:00
majinghe 52090d72d6 fix: add liveness and readiness probe (#1953) 2026-02-25 15:43:46 +08:00
安正超 d774d6821b refactor(app): route metadata/endpoints access through AppContext (#1949) 2026-02-25 15:07:09 +08:00
安正超 672c255567 fix: restore SSE baseline on latest main (#1951) 2026-02-25 14:19:04 +08:00
GatewayJ 62b51b5649 feat: admin permission check (#1783)
Signed-off-by: GatewayJ <835269233@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-25 11:58:30 +08:00
安正超 0d9e5f1e93 refactor(app): add iam and notify interface boundaries (#1948) 2026-02-25 09:49:02 +08:00
安正超 b48f273c7d refactor(filemeta): split filemeta into focused submodules (#1946) 2026-02-25 08:38:32 +08:00
安正超 7f132a290c refactor(ecstore): split set_disk.rs into submodules (#1945) 2026-02-25 07:41:08 +08:00
安正超 095b77795d refactor(ecstore): split store.rs into store submodules (#1942) 2026-02-25 06:35:27 +08:00
安正超 aac4a6c25f refactor(storage): split tonic_service into rpc modules (#1939) 2026-02-24 23:04:38 +08:00
安正超 5ed4772ed8 refactor(ecstore): split store_api into focused modules (#1938) 2026-02-24 22:31:46 +08:00
安正超 f4874ec89d refactor(storage): extract remaining s3_api response builders (#1937) 2026-02-24 21:57:43 +08:00
安正超 1b1fd6295d test(admin): cover kms list-keys route registration (#1936) 2026-02-24 21:31:19 +08:00
安正超 c864d14c9e [codex] Refactor P1-07: slim KMS handler ownership (#1935) 2026-02-24 21:12:03 +08:00
yxrxy deb1dbedbb fix(ftps): Fix basename extraction and implement recursive delete (#1920)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
2026-02-24 20:49:57 +08:00
mkrueger92 3b024a9dc5 rustfs#1916 Allow existing secrets to be used for tls certs in ingress (#1918)
Signed-off-by: mkrueger92 <7305571+mkrueger92@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-24 20:34:08 +08:00
安正超 c692777ead refactor(app): migrate delete-objects and listing orchestration (#1933) 2026-02-24 20:09:01 +08:00
安正超 c10084867a refactor(app): migrate multipart list and copy-part orchestration (#1932) 2026-02-24 19:27:41 +08:00
0xdx2 17b3054a77 feat(s3select): add JSON handling and flattening for EcObjectStore (#1930)
Signed-off-by: 0xdx2 <xuedamon2@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-24 18:05:34 +08:00
LeonWang0735 06d12a8ec8 feat(replication):add replication bandwidth throttle monitor and reader (#1885)
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-24 15:21:45 +08:00
Niraj Yadav 8f00d1fbb0 feat(admin): implement handler for delete group (#1901)
Signed-off-by: Niraj Yadav <niryadav@redhat.com>
Signed-off-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: yxrxy <yxrxytrigger@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-02-24 13:00:46 +08:00
安正超 f9da807bdf test: add regression coverage for access action mapping (#1928) 2026-02-24 12:27:58 +08:00
cxymds 49eda934d9 fix: policy-action-1903 (#1927)
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-02-24 11:21:35 +08:00
安正超 e556b64996 refactor(app): migrate bucket ACL/location/list-buckets orchestration (#1924) 2026-02-24 11:10:21 +08:00
安正超 bc026b746e refactor(app): migrate object lock and attributes flows to usecase (#1922) 2026-02-23 22:15:24 +08:00
安正超 588631b02a refactor(app): migrate object ACL/tagging flows to usecase (#1921) 2026-02-23 21:19:57 +08:00
安正超 0f631e6dd2 refactor(app): migrate bucket config flows to usecase (#1919) 2026-02-23 20:11:18 +08:00
dependabot[bot] 045988e062 build(deps): bump the dependencies group with 15 updates (#1912)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-23 17:57:53 +08:00
安正超 1614cc1b2c refactor(app): migrate restore/select and admin info orchestration (#1917) 2026-02-23 14:20:07 +08:00
安正超 5b8cbaf7c7 refactor: migrate multipart orchestration to usecase (#1915) 2026-02-23 13:10:58 +08:00
安正超 3cdd2b313b refactor(app): migrate bucket sub-operation flows (#1914) 2026-02-23 12:15:30 +08:00
安正超 d1768aa1c3 refactor(app): migrate create/delete/head bucket flows (#1913) 2026-02-23 11:05:56 +08:00
安正超 d9c97c5c52 refactor(app): migrate copy/delete/head object flows (#1911) 2026-02-23 10:32:53 +08:00
安正超 cf1d109bb9 refactor(app): route put/get/listv2 through usecases (#1910) 2026-02-22 23:36:48 +08:00
安正超 84053484e6 refactor(app): add AppContext skeleton wiring (#1909) 2026-02-22 22:41:42 +08:00
安正超 4211652991 refactor(app): add application layer module entry (#1907) 2026-02-22 22:15:37 +08:00
安正超 4a6e81d427 refactor(storage): extract object-lock response builders from ecfs (#1906) 2026-02-22 12:27:07 +08:00
安正超 094e6a7319 refactor(storage): extract tagging helpers from ecfs (#1881) 2026-02-22 11:20:41 +08:00
github-actions[bot] 6972a7b4b2 chore(deps): update flake.lock (#1905)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-02-22 10:08:24 +08:00
evan slack 23f7ffe36b fix(startup): Only monitor disk health after format loading (#1854)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-02-21 20:46:14 +08:00
LeonWang0735 f31cd4b716 fix(replication): replicate delete all versions to targets (#1898)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-02-21 20:12:05 +08:00
loverustfs da63b5e562 Fix/x86 64 compat drop target cpu native (#1895) 2026-02-21 10:24:14 +08:00
loverustfs 5d737eaeb7 fix(ecstore): invalidate GlobalFileCache after write_all_private to fix DeleteMarker visibility (#1890) 2026-02-20 22:47:45 +08:00
cui cde66e0a46 fix: uncompress -> compress (#1855)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-02-20 17:20:32 +08:00
etak64n 1a4a84bebe fix: remove deprecated darwin.apple_sdk references from flake.nix (#1884)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-02-20 16:22:06 +08:00
Burak Bozacı db70a2bed0 Feature/deployment probe override (#1876)
Co-authored-by: capitansec <capitansec@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-20 16:19:51 +08:00
安正超 583377d2a5 refactor(storage): extract ACL response builders into s3_api (#1880) 2026-02-19 22:57:35 +08:00
安正超 a4e8e1fd5e refactor(storage): extract ListBuckets response assembly (#1879) 2026-02-19 22:35:47 +08:00
LeonWang0735 c7211c9df7 fix:correctly handle replicate delete (#1850)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-02-19 13:23:29 +08:00
Miguel Caballer Fernandez 7ae0415715 Increase ACCESS_KEY_MAX_LEN to 128 (#1870)
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-18 22:00:16 +08:00
Rohmilchkaese 3f4cb6883e fix(helm): apply traefikAnnotations and gate TLS secret on certManager.enabled (#1864)
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-18 18:59:41 +08:00
houseme d345ace326 perf(obs): optimize metrics recorder and telemetry initialization (#1859) 2026-02-18 07:05:43 +08:00
evan slack 9da332c47d perf(metrics): Cache metric handles instead of creating each call (#1852)
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
2026-02-18 01:03:35 +08:00
evan slack 8010284aa3 obs(export): Add env vars to selectivly disable exporting traces/metrics/logs (#1853)
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-18 00:38:30 +08:00
loverustfs cf633569a2 fix: remove duplicate common prefixes for slash delimiter (#1797) (#1841)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2026-02-17 21:36:31 +08:00
evan slack 229f0f89c8 perf(read): Remove unecessary allocation in read_xl_meta_no_data (#1846)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-02-16 19:08:12 +08:00
安正超 d19edd9a2c refactor(storage): use named params for multipart list APIs (#1833) 2026-02-16 11:50:07 +08:00
dependabot[bot] e6c032cc92 build(deps): bump the dependencies group with 5 updates (#1845) 2026-02-16 10:18:23 +08:00
唐小鸭 4413878739 fix(compress): downgrade non-compressible log level and expand excluded file types (#1780) 2026-02-16 03:36:22 +08:00
evan slack 9786d9b004 metrics(scanner): Add metrics to scanner (#1823)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
2026-02-15 18:36:40 +08:00
Jasmine Lowen 🦁 bffeacf1d2 chore(nix): update flake lock & fix devshell+package (#1805) 2026-02-15 18:01:58 +08:00
houseme c7f1a18cc5 chore(deps): bump zip from 7.4.0 to 8.0.0 (#1837) 2026-02-15 17:40:22 +08:00
heihutu f4028a4641 chore(deps): update flake.lock (#1835)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-15 16:58:01 +08:00
Jasmine Lowen 🦁 21ef6d505e feat(config): allow specifying keys via files (key files) (#1814)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
2026-02-15 16:28:52 +08:00
majinghe da15d622a0 fix: gateway api listener name hardcode issue (#1834)
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-15 16:01:40 +08:00
majinghe 2d4d240508 feat: add existing pvc claim for standalone (#1829)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-02-15 15:49:37 +08:00
evan slack 2093a13308 logging(disks): Propogate storage disk init error, improve logging (#1825)
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-15 15:28:07 +08:00
loverustfs e41ddad003 docs: reformat CLA.md for better readability 2026-02-15 14:40:22 +08:00
loverustfs 8bc2db750f docs: update contributor license agreement 2026-02-15 14:36:01 +08:00
安正超 21ade0aaa7 refactor(storage): use named params for ListObjectVersions (#1832) 2026-02-15 13:12:00 +08:00
安正超 2debc14e4d refactor(storage): extract ListObjectsV2 parameter parsing (#1831) 2026-02-15 12:55:07 +08:00
安正超 2fadb16365 refactor(storage): extract list object versions helpers (#1830) 2026-02-15 12:32:26 +08:00
安正超 339a5db668 refactor(storage): extract multipart list param parser (#1817)
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-15 11:50:19 +08:00
realyashnegi 715cf33b89 fix(admin): return 503 when health deps are not ready (#1824) 2026-02-15 10:26:47 +08:00
houseme 4895c180e1 ci(Flake): optimize nix-flake-update workflow (#1827)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
2026-02-15 09:21:43 +08:00
houseme 1554e7e76a ci: optimize and translate nix workflow (#1821) 2026-02-15 00:37:57 +08:00
houseme 8512a38f68 build(deps): bump the dependencies group with 16 updates (#1820) 2026-02-14 23:43:09 +08:00
evan slack 9fe3d5621e feat(observability): Add additional metric panels to grafana dashboard (#1778)
Signed-off-by: evan slack <51209817+evanofslack@users.noreply.github.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-14 21:41:07 +08:00
安正超 d3ff6ff36a refactor(storage): centralize S3 response/error helpers (#1818) 2026-02-14 21:20:57 +08:00
安正超 546485a8ee refactor(storage): extract list_parts parameter parsing (#1816) 2026-02-14 20:55:14 +08:00
安正超 c701f30bd3 refactor(storage): extract ListMultipartUploads response builder (#1815) 2026-02-14 20:44:23 +08:00
安正超 6d6a2b7ed6 refactor(storage): extract ListParts response assembly helper (#1812) 2026-02-14 13:58:18 +08:00
安正超 257e31a4b4 refactor(storage): extract ListObjectsV2 response assembly helper (#1811) 2026-02-14 12:31:10 +08:00
安正超 22ae004205 refactor(storage): extract list_objects v1 response builder (#1810) 2026-02-14 11:50:15 +08:00
GatewayJ fb0267981d fix(iam): STS parent groups fallback and session policy debug for #1423 (#1804)
Co-authored-by: 安正超 <anzhengchao@gmail.com>
2026-02-14 11:40:36 +08:00
安正超 eaeb83aa1c refactor(storage): add s3_api facade and extract read helper (#1803)
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-14 11:23:17 +08:00
shadow1runner 8e1fcd4792 fix(helm): add {{ .Release.Namespace }} for kustomize v5.8 compat, closes #1808 (#1809)
Co-authored-by: Helmut Wolf <3902045+shadow1runner@users.noreply.github.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-02-14 10:25:13 +08:00
yxrxy 23f79ae88f fix: improve IAM and quota authorization (#1781)
Signed-off-by: yxrxy <yxrxytrigger@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-02-14 10:09:47 +08:00
安正超 b3daa80e72 refactor(admin): extract kms management route registration (#1801) 2026-02-13 23:21:04 +08:00
安正超 986a259a9b refactor(admin): move kms key route registration (#1799) 2026-02-13 19:18:15 +08:00
安正超 53d601e6ec refactor(admin): move kms dynamic route registration (#1798) 2026-02-13 18:34:02 +08:00
安正超 6bf4fd1273 refactor(admin): extract user policy binding route registration (#1796) 2026-02-13 17:11:48 +08:00
安正超 c4a68d3efe refactor(admin): extract user lifecycle route registration (#1795) 2026-02-13 13:42:25 +08:00
安正超 cbb4329428 refactor(admin): extract user IAM route registration (#1794) 2026-02-13 12:35:59 +08:00
安正超 2fc36bb52e fix: restore s3 compatibility regressions and CI coverage (#1793) 2026-02-13 12:26:52 +08:00
安正超 921cfb849c refactor(admin): move accountinfo route registration (#1790) 2026-02-13 10:36:54 +08:00
安正超 bfc924a70b refactor(admin): move route registration into handler modules (#1789) 2026-02-12 23:27:35 +08:00
安正超 7d8f7a12ba refactor(admin): modularize handlers and route registration (#1787) 2026-02-12 21:28:48 +08:00
安正超 4203adaac1 refactor(admin): split remaining handlers into modules (#1782) 2026-02-12 20:56:52 +08:00
evan slack c60be70d4d tool(agents): AGENTS.md instructs to respond in english, not chinese (#1775) 2026-02-12 10:48:32 +08:00
安正超 2edf0ed747 refactor(admin): extract health check handler module (#1777) 2026-02-12 10:46:07 +08:00
evan slack 9824171995 feat(observability): Add grafana dashboard, observability changes (#1770)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-02-11 15:55:08 +08:00
heihutu ecceb8fd1c build(deps): bump the dependencies group with 2+ updates (#1769)
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-11 09:39:47 +08:00
Tyooughtul 1184806c3f Fix/resolve pr 1710 (#1743) 2026-02-11 08:24:55 +08:00
houseme 4411c625e2 feat(metrics): async collection with configurable intervals & graceful shutdown (#1768) 2026-02-10 21:37:24 +08:00
Md. Amdadul Bari Imad c07ed61989 fix(entrypoint): remove dead HTTP URL check in volume filtering (#1761)
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-10 12:07:08 +08:00
yxrxy da58f8e291 fix: Allow non-admin users to read bucket quota configuration. (#1759)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
2026-02-10 11:23:21 +08:00
mengyu-sxyz aa011ade19 fix: improve part size calculation in optimal_part_info function (#1532)
Signed-off-by: mengyu-sxyz <mengyu@sentio.xyz>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
2026-02-10 10:56:47 +08:00
LoganZ2 ccf3b29df5 fix: stabilize head metadata responses and heal tests (#1732)
Signed-off-by: LoganZ2 <103290230+LoganZ2@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-10 09:44:14 +08:00
evan slack 682b5bbb2f perf(scanner): Change DataUseageEntry from clone to borrow (#1757)
Signed-off-by: evan slack <51209817+evanofslack@users.noreply.github.com>
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-10 08:53:26 +08:00
LeonWang0735 f4e9ef2edc fix(replication): avoid re-replication loop in Active-Active replication (#1751)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-02-09 14:11:30 +08:00
dependabot[bot] ff8c1c782a build(deps): bump libunftp from 0.21.0 to 0.22.0 in the dependencies group across 1 directory (#1756)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-09 13:43:47 +08:00
houseme ca6076fe18 build(deps): bump the dependencies group with 5 updates (#1755) 2026-02-09 11:31:43 +08:00
evan slack 58ee140324 perf(regex): Compile bucket validation regex once (#1753) 2026-02-09 10:50:17 +08:00
thorntonmc 927f3a57d7 perf(quota): Skip expensive usage checks when no quota configured (#1749)
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-08 22:37:53 +08:00
majinghe a574285ab2 feat: add support for mtls with kubernetes installation (#1741) 2026-02-08 09:31:58 +08:00
LeonWang0735 60793c17d7 fix: persist replication status and timestamp after replicate_object (#1747)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-07 12:50:37 +08:00
houseme 0b870d6301 build(deps): bump the dependencies group with 19 updates (#1745) 2026-02-07 12:22:14 +08:00
Isaac Mills d635ee8d2e Propogate tracing context from HTTP requests into spans (#1739)
Signed-off-by: Isaac Mills <57533634+StratusFearMe21@users.noreply.github.com>
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-07 11:52:31 +08:00
yxrxy 5c2eda356e feat: migrate FTP/SFTP to protocols crate and update dependencies (#1580)
Signed-off-by: yxrxy <yxrxytrigger@gmail.com>
Signed-off-by: houseme <housemecn@gmail.com>
Signed-off-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-02-06 20:58:42 +08:00
LeonWang0735 a2b88a79ec test(e2e_test): add automated cluster environment for conditional PUT race test (#1673)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-02-06 15:01:59 +08:00
LeonWang0735 6eb90e7df9 fix: fetch_owner set to true when calling list_objects_v2 in the list_objects function (#1730)
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-05 22:48:32 +08:00
majinghe 466429b958 feat: add contour as ingress controller with http proxy (#1729)
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-05 21:04:18 +08:00
GatewayJ c8411fd62c fix: bucket policy id field serde (#1726) 2026-02-05 18:38:57 +08:00
houseme 6bba41f11f Fix/fix issues #1564 (#1708)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-05 13:45:14 +08:00
majinghe e30781654d enhancement: add support for http to https redirect for traefik gatew… (#1712) 2026-02-04 20:21:42 +08:00
唐小鸭 7a42af922d Refactor: refactor SSE layer and KMS subsystem (#1703)
Co-authored-by: houseme <housemecn@gmail.com>
2026-02-04 16:10:33 +08:00
weisd 4d19b069c3 fix: replication delete (#1714) 2026-02-04 13:39:35 +08:00
loverustfs a4563f7b41 fix: return null versionId when suspended (#1066) (#1709) 2026-02-04 09:22:52 +08:00
dependabot[bot] 174e12bf66 build(deps): bump bytes from 1.11.0 to 1.11.1 (#1711)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-04 08:41:30 +08:00
majinghe f03034b99e feat: add glibc based docker image support (#1705)
Signed-off-by: majinghe <42570491+majinghe@users.noreply.github.com>
2026-02-03 21:27:20 +08:00
LeonWang0735 36f14acbe9 fix: object lock compliance mode allows deletion (#1687)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-02-03 17:06:24 +08:00
majinghe 2f66f15524 feat: add obs log rotations environment variables (#1702) 2026-02-03 10:35:30 +08:00
houseme cb468fb32f Refactor trusted-proxies: modernize utils, improve safety, and fix clippy lints (#1693)
Co-authored-by: majinghe <42570491+majinghe@users.noreply.github.com>
Co-authored-by: GatewayJ <835269233@qq.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: houseme <4829346+houseme@users.noreply.github.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
2026-02-03 01:06:22 +08:00
houseme d1a70176a2 fix: Fixed that account_id returns normal value (#1695) 2026-02-02 20:15:54 +08:00
LeonWang0735 ec4458f846 Fix/correctly handle terraform s3 backend with versioned bucket (#1686)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-02-02 12:43:07 +08:00
yxrxy 00ccc19e27 Revert "fix: resolve Issue #1465 - IAM credential change crash (#1535)" (#1685) 2026-02-02 07:14:53 +08:00
houseme 07cf2feaad fix(pprof): Fixed the problem that pprof crate does not support the window platform (#1681)
Signed-off-by: houseme <housemecn@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-01 00:12:55 +08:00
likewu 087f58b7c8 fix(lifecycle): lifecycle fixes (#1625)
Signed-off-by: likewu <likewu@126.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-01-31 19:14:47 +08:00
moechs 6fc35e442c fix: add gatewayApi.enabled check to TraefikService template (#1679)
Signed-off-by: moechs <68768084+moechs@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-01-31 17:42:27 +08:00
majinghe a798b20308 fix: init container security hardened for operation permission error (#1680) 2026-01-31 17:00:42 +08:00
houseme 38b779b924 feat(profiling): support cross-platform memory profiling with mimalloc and pprof (#1674) 2026-01-30 22:23:49 +08:00
weisd 1aba8c10b9 refactor: Remove unused data usage collection code (#1672)
Co-authored-by: houseme <housemecn@gmail.com>
2026-01-30 19:08:39 +08:00
houseme 90ed75a3dc refactor(utils/os): Optimize Windows OS utilities and add safety comments (#1671)
Co-authored-by: weisd <weishidavip@163.com>
2026-01-30 16:13:42 +08:00
weisd dce117840c refactor: NamespaceLock (nslock), AHM→Heal Crate, and Lock/Clippy Fixes (#1664)
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: weisd <2057561+weisd@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-01-30 13:13:41 +08:00
majinghe 1c085590ca fix: traefik gateway api support with sticky session (#1660)
Co-authored-by: houseme <housemecn@gmail.com>
2026-01-30 12:34:31 +08:00
houseme 2ee81496b0 fix: deduplicate disks in capacity calculation to prevent inflation (#1656) 2026-01-30 00:03:21 +08:00
LeonWang0735 022e3dfc21 fix:s3 tests fix (#1652)
Co-authored-by: houseme <housemecn@gmail.com>
2026-01-29 19:35:03 +08:00
majinghe 6b15e727f5 feat: add virtual host mode support for kubernetes installation (#1655) 2026-01-29 17:55:24 +08:00
zhangwenlong ab84da24ef fix: build error on loongarch64 (#904)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: 0xdx2 <xuedamon2@gmail.com>
2026-01-29 17:41:56 +08:00
houseme e377c7e7f9 fix(head): clearer NoSuchKey for prefix keys and validate part numbers (#1638) 2026-01-29 15:40:51 +08:00
houseme 7c497a30b2 build(deps): bump the dependencies group with 12 updates (#1650) 2026-01-29 14:21:35 +08:00
evan slack 51e8a4820f fix: map unversioned destination replication error correctly (#1645) 2026-01-29 10:13:36 +08:00
loverustfs a81bbed551 Fix align 2026-01-28 22:03:42 +08:00
loverustfs 072de6b025 Update README to streamline badge links (#1643)
Signed-off-by: loverustfs <hello@rustfs.com>
2026-01-28 21:58:48 +08:00
loverustfs 9269cb779b Add badge to readme
Add badge to readme

Signed-off-by: loverustfs <hello@rustfs.com>
2026-01-28 21:57:22 +08:00
安正超 fff175dcdd chore: remove skills and docs dir. (#1631) 2026-01-27 22:43:22 +08:00
LeonWang0735 26c0230e8f fix:use RFC1123 format for last-modified header in 304 responses (#1627)
Co-authored-by: houseme <housemecn@gmail.com>
2026-01-27 19:06:31 +08:00
LeonWang0735 8edb1affc0 Fix:s3 compatibility (#1617)
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
2026-01-27 16:56:31 +08:00
heihutu 74759b6e99 fix: missing object.key in S3 event notifications for multipart uploads #1609 (#1624)
Co-authored-by: houseme <housemecn@gmail.com>
2026-01-27 13:32:56 +08:00
yxrxy db29c0cae2 fix: missing object.key in S3 event notifications for multipart uploads (#1621)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-01-27 11:48:45 +08:00
weisd 6ab7b75fd4 fix bug (#1615)
Co-authored-by: houseme <housemecn@gmail.com>
2026-01-27 09:54:13 +08:00
houseme 2108c4ad28 fix: remove plaintext credential logging (#1619) 2026-01-27 01:47:39 +08:00
houseme d251b9fb35 fix: unify path handling to use S3-standard forward slashes on all platforms (#1555)
Signed-off-by: houseme <housemecn@gmail.com>
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: LeonWang0735 <wlywly0735@126.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-01-26 18:49:21 +08:00
dependabot[bot] 172bed0ff2 build(deps): bump the dependencies group with 3 updates (#1612)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-01-26 11:00:56 +08:00
安正超 1db7bac2dc fix: readme list. (#1608) 2026-01-25 09:35:12 +08:00
安正超 4890fb25c1 fix: listobjects v2 pagination (#1607) 2026-01-25 09:30:08 +08:00
Dat Truong 3838a13606 Update README with docker-buildx.sh features (#1585)
Signed-off-by: Dat Truong <truongminhdat07@gmail.com>
2026-01-25 09:25:26 +08:00
houseme c28134c957 Add support for success_action_status and success_action_redirect in AWS S3 POST object uploads (#1606) 2026-01-25 02:50:29 +08:00
安正超 173dad27d1 fix: preserve exact JSON format in bucket policy GET response (#1598)
Co-authored-by: loverustfs <hello@rustfs.com>
2026-01-24 23:02:01 +08:00
GatewayJ 9285acba06 feat: object retention (#1589) 2026-01-24 22:12:45 +08:00
LeonWang0735 db5e72e475 Fix/correctly handle compression (#1594) 2026-01-24 10:55:10 +08:00
安正超 16160b7b84 fix: use main user for s3tests tenant to prevent teardown failures (#1597) 2026-01-24 10:05:10 +08:00
安正超 461ba3aeba fix: handle duplicate part numbers in CompleteMultipartUpload (#1584)
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: houseme <housemecn@gmail.com>
2026-01-23 22:41:46 +08:00
yxrxy 65de487eba fix: resolve Issue #1465 - IAM credential change crash (#1535)
Co-authored-by: heihutu <30542132+heihutu@users.noreply.github.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
2026-01-23 15:11:01 +08:00
houseme e5284a85ed fix: Fixed detection warnings in rust v1.93.0 (#1591) 2026-01-23 13:09:41 +08:00
安正超 fd08be7be2 chore: update README. (#1586) 2026-01-22 22:44:28 +08:00
安正超 43bf846633 fix: correct max_keys field in list_object_versions response (#1576)
Signed-off-by: 安正超 <anzhengchao@gmail.com>
Co-authored-by: loverustfs <hello@rustfs.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-01-22 20:58:03 +08:00
1412 changed files with 317353 additions and 109991 deletions
@@ -0,0 +1,78 @@
---
name: code-change-verification
description: Verify code changes by identifying correctness, regression, security, and performance risks from diffs or patches, then produce prioritized findings with file/line evidence and concrete fixes. Use when reviewing commits, PRs, and merged patches before/after release.
---
# Code Change Verification
Use this skill to review code changes consistently before merge, before release, and during incident follow-up.
## Quick Start
1. Read the scope: commit, PR, patch, or file list.
2. Map each changed area by risk and user impact.
3. Inspect each risky change in context.
4. Report findings first, ordered by severity.
5. Close with residual risks and verification recommendations.
## Core Workflow
### 1) Scope and assumptions
- Confirm change source (diff, commit, PR, files), target branch, language/runtime, and version.
- If context is missing, state assumptions before deeper analysis.
- Focus only on requested scope; avoid reviewing unrelated files.
### 2) Risk map
- Prioritize in this order:
- Data correctness and user-visible behavior
- API/contract compatibility
- Security and authz/authn boundaries
- Concurrency and lifecycle correctness
- Performance and resource usage
- Give higher priority to stateful paths, migration logic, defaults, and error handling.
### 3) Evidence-based inspection
- Read each modified hunk with neighboring context.
- Trace call paths and call-site expectations.
- Check for:
- invariant breaks and missing guards
- unchecked assumptions and null/empty/error-path handling
- stale tests, fixtures, and configs
- hidden coupling to shared helpers/constants/features
- If a point is uncertain, mark it as an open question instead of guessing.
### 4) Findings-first output
- Order findings by severity:
- P0: critical failure, security breach, or data loss risk
- P1: high-impact regression
- P2: medium risk correctness gap
- P3: low risk/quality debt
- For each finding include:
- Severity
- `path:line` reference
- concise issue statement
- impact and likely failure mode
- specific fix or mitigation
- validation step to confirm
- If no issues exist, explicitly state `No findings` and why.
### 5) Close
- Report assumptions and unknowns.
- Suggest targeted checks (tests, canary checks, logs/metrics, migration validation).
## Output Template
1. Findings
2. No findings (if applicable)
3. Assumptions / Unknowns
4. Recommended verification steps
## Finding Template
- `[P1] Missing timeout for downstream call`
- Location: `path/to/file.rs:123`
- Issue: ...
- Impact: ...
- Fix suggestion: ...
- Validation: ...
@@ -0,0 +1,4 @@
interface:
display_name: "Code Change Verification"
short_description: "Prioritize risks and verify code changes before merge."
default_prompt: "Inspect a patch or diff, identify correctness/security/regression risks, and return prioritized findings with file/line evidence and fixes."
@@ -0,0 +1,91 @@
---
name: pr-creation-checker
description: Prepare PR-ready diffs by validating scope, checking required verification steps, drafting a compliant English PR title/body, and surfacing blockers before opening or updating a pull request in RustFS.
---
# PR Creation Checker
Use this skill before `gh pr create`, before `gh pr edit`, or when reviewing whether a branch is ready for PR.
## Read sources of truth first
- Read `AGENTS.md`.
- Read `.github/pull_request_template.md`.
- Use `Makefile` and `.config/make/` for local quality commands.
- Use `.github/workflows/ci.yml` for CI expectations.
- Do not restate long command matrices or template sections from memory when the files exist.
## Workflow
1. Collect PR context
- Confirm base branch, current branch, change goal, and scope.
- Confirm whether the task is: draft a new PR, update an existing PR, or preflight-check readiness.
- Confirm whether the branch includes only intended changes.
2. Inspect change scope
- Review the diff and summarize what changed.
- Call out unrelated edits, generated artifacts, logs, or secrets as blockers.
- Mark risky areas explicitly: auth, storage, config, network, migrations, breaking changes.
- Scan the diff for newly added string literals and confirm whether they duplicate values already defined as constants/enums/typed wrappers in the same module or shared modules.
- Treat introducing a new hardcoded literal where a project constant already exists as a likely regression risk; require either a refactor to reuse the constant or an explicit exception explanation in the PR body.
3. Verify readiness requirements
- Require `make pre-commit` before marking the PR ready.
- If `make` is unavailable, use the equivalent commands from `.config/make/`.
- Add scope-specific verification commands when the changed area needs more than the baseline.
- If required checks fail, stop and return `BLOCKED`.
4. Draft PR metadata
- Write the PR title in English using Conventional Commits and keep it within 72 characters.
- If a generic PR workflow suggests a different title format, ignore it and follow the repository rule instead.
- In RustFS, do not use tool-specific prefixes such as `[codex]` when the repository requires Conventional Commits.
- Keep the PR body in English.
- Use the exact section headings from `.github/pull_request_template.md`.
- Fill non-applicable sections with `N/A`.
- Include verification commands in the PR description.
- Do not include local filesystem paths in the PR body unless the user explicitly asks for them.
- Prefer repo-relative paths, command names, and concise summaries over machine-specific paths such as `/Users/...`.
5. Prepare reviewer context
- Summarize why the change exists.
- Summarize what was verified.
- Call out risks, rollout notes, config impact, and rollback notes when applicable.
- Mention assumptions or missing context instead of guessing.
6. Prepare CLI-safe output
- When proposing `gh pr create` or `gh pr edit`, use `--body-file`, never inline `--body` for multiline markdown.
- Return a ready-to-save PR body plus a short title.
- If not ready, return blockers first and list the minimum steps needed to unblock.
## Output format
### Status
- `READY` or `BLOCKED`
### Title
- `<type>(<scope>): <summary>`
### PR Body
- Reproduce the repository template headings exactly.
- Fill every section.
- Omit local absolute paths unless explicitly required.
### Verification
- List each command run.
- State pass/fail.
### Risks
- List breaking changes, config changes, migration impact, or `N/A`.
## Blocker rules
- Return `BLOCKED` if `make pre-commit` has not passed.
- Return `BLOCKED` if the diff contains unrelated changes that are not acknowledged.
- Return `BLOCKED` if required template sections are missing.
- Return `BLOCKED` if the title/body is not in English.
- Return `BLOCKED` if the title does not follow the repository's Conventional Commit rule.
- Return `BLOCKED` if the diff introduces string literals that should use existing constants but did not.
## Reference
- Use [pr-readiness-checklist.md](references/pr-readiness-checklist.md) for a short final pass before opening or editing the PR.
@@ -0,0 +1,4 @@
interface:
display_name: "PR Creation Checker"
short_description: "Draft RustFS-ready PRs with checks, template, and blockers."
default_prompt: "Inspect a branch or diff, verify required PR checks, and produce a compliant English PR title/body plus blockers or readiness status."
@@ -0,0 +1,15 @@
# PR Readiness Checklist
- Confirm the branch is based on current `main`.
- Confirm the diff matches the stated scope.
- Confirm no secrets, logs, temp files, or unrelated refactors are included.
- Confirm `make pre-commit` passed, or document why it could not run.
- Confirm extra verification commands are listed for risky changes.
- Confirm the PR title uses Conventional Commits and stays within 72 characters.
- Confirm the PR title does not use tool-specific prefixes such as `[codex]`.
- Confirm the PR body is in English.
- Confirm the PR body keeps the exact headings from `.github/pull_request_template.md`.
- Confirm non-applicable sections are filled with `N/A`.
- Confirm the PR body does not include local absolute paths unless explicitly required.
- Confirm multiline GitHub CLI commands use `--body-file`.
- Confirm new hardcoded string literals were not introduced for values already represented by existing constants/enums (including protocol labels, error identifiers, headers, and metric names), or record a justified exception.
@@ -0,0 +1,124 @@
---
name: rustfs-release-version-bump
description: "Publish a RustFS alpha/beta/stable release with an auditable flow: confirm target version and scope, update workspace and release assets (including strict rustfs.spec changelog identity/date/version format), run required verification, and finish with commit, push, and GitHub PR creation."
---
# RustFS Release Version Bump
Use this skill to publish a RustFS release (alpha, beta, or stable) with a minimal, auditable diff and a complete ship flow (`edit -> verify -> commit -> push -> PR`).
Validated baseline: release pattern used in PR `#2957`.
## Required inputs
- Exact target version, for example `1.0.0-beta.4`.
- Delivery scope:
- Local only (`edit/verify`).
- Local + git (`commit/push`).
- Full GitHub flow (`commit/push/PR`).
If target version is missing or ambiguous, stop and ask before editing.
## Read before editing
- `AGENTS.md` (root and nearest path-specific files).
- `.github/pull_request_template.md`.
- Current branch status and diff against `origin/main`.
## Default release file scope
Treat the following file list as the default checklist for each release bump:
- `Cargo.toml`
- `Cargo.lock`
- `README.md`
- `README_ZH.md`
- `flake.nix`
- `helm/rustfs/Chart.yaml`
- `rustfs.spec`
Only drop a file when the current repository release process clearly no longer requires it.
## Hard release policy
- Docker doc tags use `<version>` (for example `rustfs/rustfs:1.0.0-beta.4`), not `v<version>`.
- Helm chart version mapping follows `beta.N -> 0.N.0`.
- `rustfs.spec` `Release` uses prerelease suffix only (for example `beta.4`).
- Do not change these rules without explicit confirmation.
## Step-by-step workflow
1. Confirm intent and isolate scope
- Confirm target version string exactly.
- Confirm whether user requested local-only or full GitHub flow.
- Inspect current branch and ensure only release-related files are touched for this task.
2. Update workspace versions
- Bump `[workspace.package].version` in `Cargo.toml`.
- Bump internal workspace crate dependency versions in `Cargo.toml`.
- Update `Cargo.lock` so workspace package versions match target version.
- Re-scan for partial leftovers.
3. Update release assets
- `README.md` and `README_ZH.md`: update versioned Docker examples to target version.
- `flake.nix`: update package version to target version.
- `helm/rustfs/Chart.yaml`:
- `appVersion` = target version.
- `version` follows chart mapping rule, for example:
- `1.0.0-beta.3` -> `0.3.0`
- `1.0.0-beta.4` -> `0.4.0`
- `rustfs.spec`:
- Set `Release` to prerelease suffix (example `beta.4`).
- Add/update top changelog entry with exact format:
- `* Thu May 20 2026 houseme <housemecn@gmail.com>`
- `- Update RPM package to RustFS 1.0.0-beta.4`
- Changelog identity and time must come from current environment:
- `git config --get user.name`
- `git config --get user.email`
- `date '+%a %b %d %Y'`
- Changelog version text must match target release version exactly.
4. Verify before shipping
- Run:
- `cargo fmt --all`
- `cargo fmt --all --check`
- `make pre-commit`
- If verification passes, run `cargo clean`.
- If `make pre-commit` fails, return `BLOCKED` with root cause and do not silently widen scope to fix unrelated issues unless user asks.
5. Commit strategy
- Preferred split when both parts changed:
- `chore(release): prepare <version>` for `Cargo.toml` and `Cargo.lock`.
- `chore(release): align release assets for <version>` for docs and packaging files.
- If user asks for one commit, use one commit.
- Stage only intended release files; do not include unrelated working tree changes.
6. Push and PR
- Push branch:
- `git push -u origin <branch>` (first push), or `git push` (tracking already exists).
- Create PR with template headings unchanged:
- `gh pr create --base main --head <branch> --title ... --body-file ...`
- PR title/body must be English.
- Use `N/A` for non-applicable template sections.
- Include verification commands and any `BLOCKED` reason clearly.
## Recommended check commands
- `git status --short --branch`
- `git diff --name-only origin/main...HEAD`
- `git diff --stat origin/main...HEAD`
- `rg -n "<old_version>|<new_version>" Cargo.toml Cargo.lock README.md README_ZH.md flake.nix helm/rustfs/Chart.yaml rustfs.spec`
- `cargo fmt --all`
- `cargo fmt --all --check`
- `make pre-commit`
- `cargo clean`
## Output contract
When using this skill, always report:
- Target version.
- Files changed.
- Any assumptions or uncertainties requiring confirmation.
- Verification result (`PASSED` or `BLOCKED`) with key evidence.
- Commit message(s) used.
- Push status and PR URL when GitHub flow is requested.
@@ -0,0 +1,4 @@
interface:
display_name: "RustFS Release Bump"
short_description: "Prepare RustFS release branches like PR #2957."
default_prompt: "Use $rustfs-release-version-bump to prepare a RustFS release version, ask about any unclear version policy, and finish the commit/push/PR flow."
@@ -0,0 +1,119 @@
---
name: security-advisory-lessons
description: Apply RustFS security lessons distilled from repository GitHub Security Advisories. Use when making or reviewing RustFS code changes, doing security checks, handling PR review for auth/authz, IAM, storage, RPC, logging, CORS, console/browser, encryption, policy, or endpoint changes, and when deciding which security regression tests are required.
---
# RustFS Security Advisory Lessons
Use this skill as a RustFS-specific security lens before changing or approving code. For the distilled advisory lessons and review patterns, read [advisory-patterns.md](references/advisory-patterns.md).
When currentness matters, fetch the live advisory inventory instead of relying on this skill as a status mirror:
```bash
gh api repos/rustfs/rustfs/security-advisories --paginate \
--jq '.[] | {ghsa_id,state,severity,summary,updated_at}'
```
Fetch full advisory details only when the live summary suggests a new or changed lesson:
```bash
gh api repos/rustfs/rustfs/security-advisories/<GHSA_ID>
```
For the full pattern map, read [advisory-patterns.md](references/advisory-patterns.md).
## Workflow
### 1. Scope the change
- Identify touched routes, handlers, storage paths, credentials, logs, browser surfaces, CI/release code, and policy checks.
- Treat these paths as security-sensitive by default: `rustfs/src/admin/`, `rustfs/src/storage/`, `rustfs/src/auth.rs`, `rustfs/src/server/layer.rs`, `crates/iam/`, `crates/policy/`, `crates/credentials/`, `crates/ecstore/src/rpc/`, `crates/rio/`, and console preview/auth code.
### 2. Map to advisory classes
- Read [advisory-patterns.md](references/advisory-patterns.md) for matching GHSA lessons.
- Do not rely on advisory titles alone. Confirm whether the issue is authentication, authorization, input validation, storage invariant, browser isolation, logging, or operational hardening.
### 3. Verify fail-closed behavior
- Check that unauthenticated, wrong-permission, cross-user, cross-bucket, malformed-input, and default-config cases fail explicitly.
- Prefer exact action/permission checks over broad helper calls or inferred ownership.
- Confirm lower storage/RPC layers do not bypass checks done in upper layers.
### 4. Require regression evidence
- For behavior changes, add focused negative tests that reproduce the advisory class.
- For sensitive fixes, include tests for the bypass form, not only the happy path.
- If a test is impractical, explain the residual risk and provide a manual verification command.
### 5. Report clearly
- Lead with concrete findings and file/line evidence.
- Separate proven vulnerabilities from hardening risks.
- Avoid exaggerating unauthenticated impact when the code actually rejects unauthenticated requests but allows a low-privileged authenticated bypass.
## Advisory-Derived Guardrails
### Auth and admin authorization
- Every admin or diagnostic route needs an explicit authn and authz story. Route registration, router whitelist, and handler-level authorization must agree.
- Match the admin action to the operation exactly. Copy-paste action constants are a known RustFS vulnerability class.
- Avoid authentication-only helpers for state-changing admin APIs; use `validate_admin_request` or the established equivalent with the right `AdminAction`.
- Do not assume admin-action `Resource` scoping constrains blast radius unless the policy engine actually enforces resources for that action.
### IAM and service accounts
- Treat imported IAM payload fields as attacker-controlled: `parent`, `claims`, `accessKey`, `secretKey`, status, policy names, and groups.
- For service account create/update/import, prove parent ownership or root/admin authority before writing credentials or claims.
- Do not let `deny_only` or "no explicit deny" become an allow decision that skips required allow checks.
- Test cross-user list/update/import flows with wrong, correct, self, parent, and root identities.
### S3 copy, multipart, and presigned POST
- Multipart copy must enforce source `GetObject` and destination `PutObject` semantics equivalent to `CopyObject`, including copy-source and policy conditions.
- Do not let `CreateMultipartUpload`, `UploadPartCopy`, `CompleteMultipartUpload`, or `AbortMultipartUpload` return success without authorization.
- Presigned POST policies are server-side contracts. Enforce `content-length-range`, key prefix, exact metadata/content-type, and all signed policy conditions.
### Paths, object keys, and filesystem access
- Never join untrusted bucket/object/RPC path strings onto filesystem roots without normalization and boundary checks.
- Reject or safely handle `..`, absolute paths, URL-encoded traversal, platform separators, empty components, and paths that canonicalize outside the intended root.
- Validate both S3 object-key paths and internode/RPC disk paths; storage helpers can bypass S3 authorization if they trust already-parsed paths.
### Secrets, default credentials, and crypto
- Do not ship hard-coded shared tokens, HMAC secrets, private keys, or production test keys.
- Defaults for internode/RPC auth must fail closed for network-reachable deployments or require explicit opt-in with loud warnings.
- License or token validation must use signatures with embedded public/verifying keys only; do not use private-key decryption as authenticity.
- Plan key rotation and key IDs when removing exposed keys.
### Logging and debug output
- Logs must never include access keys beyond safe identifiers, secret keys, session tokens, JWT claims, HMAC secrets, expected signatures, license secrets, or raw response bodies containing credentials.
- Treat `Debug` implementations, `?value` tracing, merged config dumps, and dependency-level HTTP body logging as leak surfaces.
- Add log-capture tests or targeted unit tests for redaction wrappers when changing credential structs or response bodies.
### RPC, parsing, and panic safety
- Treat all RPC payload bytes as attacker-controlled. Replace `unwrap`, `expect`, and panic-prone deserialization with typed errors.
- Malformed request tests should cover empty bytes, truncated MessagePack/protobuf, invalid enum values, stale timestamps, and invalid signatures.
- RPC authentication must be independently strong; do not depend on S3 admin credentials unless the fallback is explicit and safe.
### Browser, CORS, and console surfaces
- Do not reflect arbitrary `Origin` while also allowing credentials. Default CORS should be no CORS unless explicitly configured.
- Do not render user-controlled object content in a same-origin iframe with console credentials available to JavaScript.
- Prefer origin separation for object preview/download, `nosniff`, CSP, strict content-type handling, and avoiding durable credentials in `localStorage`.
- Console license/version-like metadata endpoints should expose only coarse public data unless authenticated, especially subject names and expiration timestamps.
### Profiling, debug, and health endpoints
- Profiling and debug endpoints are not health checks. They require admin auth, opt-in enablement, rate limiting, and safe responses.
- Do not return absolute filesystem paths or other deployment layout in unauthenticated or low-privilege responses.
- Ensure health endpoint allowlists cannot accidentally include expensive diagnostics.
### Trusted proxy and network identity
- Only honor `X-Forwarded-For` or `X-Real-IP` when the request came from a configured trusted proxy.
- Direct clients must use the socket peer address for `aws:SourceIp` and policy condition evaluation.
- Add tests for direct spoofed headers and trusted-proxy headers.
### SSE and storage invariants
- Encryption metadata is not proof that bytes were encrypted on disk.
- When touching reader/writer wrappers such as hashing, encryption, compression, or warp readers, verify wrapper order and inspect stored bytes in regression tests.
- Avoid helper shortcuts that unwrap nested readers and accidentally bypass encryption or integrity layers.
## Review Prompts
Use these prompts while reviewing a diff:
- Could a low-privileged authenticated user reach this path with the wrong action, parent, bucket, or source object?
- Does a public/default/empty config change security behavior from fail-closed to fail-open?
- Is any attacker-controlled value later used as a path, policy condition, credential identity, log field, URL, Origin, or response body?
- Is the same operation implemented in multiple paths, such as `CopyObject` vs `UploadPartCopy`, and do all paths enforce the same security contract?
- Does the test prove the exploit form is denied, or only that the intended form still works?
@@ -0,0 +1,4 @@
interface:
display_name: "Security Advisory Lessons"
short_description: "Apply advisory lessons in reviews."
default_prompt: "Review code changes against past RustFS security advisory lessons and report concrete risks, missing tests, and recommended fixes."
@@ -0,0 +1,93 @@
# RustFS Advisory Pattern Map
This file is a lesson map, not an advisory inventory mirror. It keeps durable security patterns distilled from RustFS GitHub Security Advisories.
When current advisory state, severity, URLs, or full text matters, fetch it live:
```bash
gh api repos/rustfs/rustfs/security-advisories --paginate \
--jq '.[] | {ghsa_id,state,severity,summary,updated_at}'
gh api repos/rustfs/rustfs/security-advisories/<GHSA_ID>
```
Update this file only when an advisory adds or changes a reusable lesson, affected surface, validation pattern, or regression-test expectation. Do not update it for state-only, URL-only, count-only, or timestamp-only changes.
## Pattern Index
### Admin authorization and route exposure
- `GHSA-pfcq-4gjr-6gjm`: notification target endpoints accepted authenticated users but skipped admin authorization. Lesson: distinguish authn from authz; admin target CRUD must call the operation-specific admin authorization path.
- `GHSA-mm2q-qcmx-gw4w`: `ListServiceAccount` used `UpdateServiceAccountAdminAction`, while update lacked target ownership checks. Lesson: exact action constants and ownership checks are both required; information disclosure can chain into secret rotation and takeover.
- `GHSA-vcwh-pff9-64cc`: `ImportIam` checked `ExportIAMAction` for an import/write operation. Lesson: every admin handler must authorize the action it actually performs.
- `GHSA-jqmc-mg33-v45g` and `GHSA-8784-9m7f-c6p6`: `/profile/cpu` and `/profile/memory` were whitelisted from auth and allowed expensive diagnostics plus path disclosure. Lesson: profiling/debug endpoints need admin auth, opt-in, rate limits, and non-sensitive responses.
- `GHSA-xp32-gxq2-3v52`: console license metadata endpoint was public and exposed subject and expiration fields. Lesson: management metadata endpoints should require admin auth or return only coarse public status.
### IAM import, service accounts, and privilege boundaries
- `GHSA-566f-q62r-wcr8`: `ImportIam` accepted attacker-controlled service account `parent`, `claims`, `accessKey`, and `secretKey`, enabling persistent backdoor accounts under root. Lesson: imported IAM payloads are untrusted data and must be validated against privilege boundaries.
- `GHSA-xgr5-qc6w-vcg9`: `deny_only=true` skipped allow checks and let restricted service accounts mint unrestricted children. Lesson: deny-only logic must never become implicit allow for privilege creation.
- `GHSA-mm2q-qcmx-gw4w`: leaked service account access keys plus update-without-ownership formed an escalation chain. Lesson: service-account identifiers are security-sensitive because update APIs consume them.
### S3 copy, multipart, and upload policy validation
- `GHSA-mx42-j6wv-px98`: `UploadPartCopy` missed source authorization and allowed cross-bucket object exfiltration. Lesson: multipart copy must enforce the same source and destination contract as `CopyObject`.
- `GHSA-wfxj-ph3v-7mjf`: `UploadPartCopy` checked source and destination independently but missed destination copy-source policy constraints. Lesson: source read and destination write checks are not sufficient when policy constrains allowed copy sources.
- `GHSA-w5fh-f8xh-5x3p`: presigned POST accepted uploads without enforcing signed policy conditions. Lesson: parse and enforce all POST policy constraints server-side, including size, key prefix, and content type.
### Filesystem paths and object key traversal
- `GHSA-pq29-69jg-9mxc`: RPC `read_file_stream` joined untrusted paths under a volume directory without canonical boundary checks. Lesson: `PathBuf::join` plus length checks are not path security.
- `GHSA-8r6f-hmq2-28rg`: object keys containing traversal sequences bypassed bucket/object authorization when mapped to filesystem paths. Lesson: reject traversal at object-key parsing and verify final storage paths remain under the expected bucket/key root.
### Secrets, defaults, and cryptographic misuse
- `GHSA-h956-rh7x-ppgj`: gRPC used the hard-coded token `rustfs rpc` on both client and server. Lesson: source-visible shared tokens are authentication bypasses.
- `GHSA-r5qv-rc46-hv8q`: internode RPC HMAC secret fell back to the public default `rustfsadmin`. Lesson: RPC/internode auth must fail closed instead of silently using public defaults.
- `GHSA-923g-jp7v-f97f`: license verification embedded a production RSA private key and used private-key decryption as authenticity. Lesson: ship verifying/public keys only and use real signature verification.
### Sensitive logging and debug output
- `GHSA-r54g-49rx-98cr`: STS credentials were logged at info level. Lesson: generated credentials must never be logged in plaintext.
- `GHSA-8cm2-h255-v749`: debug logs leaked session tokens, secret keys, JWT claims, and raw STS response bodies. Lesson: redaction must cover custom `Debug` implementations and dependency response-body logging.
- `GHSA-333v-68xh-8mmq`: invalid RPC signature logging included the shared HMAC secret and expected signature. Lesson: error paths often leak secrets; never log raw secrets or derived authenticators.
### RPC input validation and panic safety
- `GHSA-gw2x-q739-qhcr`: malformed gRPC `GetMetrics` payloads reached `unwrap()` on deserialization and caused remote DoS. Lesson: every network/RPC deserialization failure returns an error, not a panic.
- `GHSA-h956-rh7x-ppgj` and `GHSA-r5qv-rc46-hv8q`: weak RPC auth increased reachability of otherwise internal handlers. Lesson: panic bugs become more severe when internode auth is weak or defaulted.
### Browser, CORS, and console isolation
- `GHSA-v9fg-3cr2-277j`: object preview rendered attacker-controlled HTML in a same-origin iframe, exposing console credentials stored in `localStorage`. Lesson: user content must be origin-isolated from the console and protected with `nosniff`, CSP, and strict content-type handling.
- `GHSA-x5xv-223c-8vm7`: default CORS reflected arbitrary origins with credentials. Lesson: never combine reflected origins with `Access-Control-Allow-Credentials: true`; default should be fail-closed.
### Trusted proxy and source IP conditions
- `GHSA-fc6g-2gcp-2qrq`: `aws:SourceIp` trusted client-supplied `X-Forwarded-For` or `X-Real-IP`. Lesson: forwarded IP headers are valid only behind configured trusted proxies; direct clients use socket peer IP.
### SSE and on-disk storage invariants
- `GHSA-xrrf-67jm-3c2r`: SSE metadata reported encryption while reader composition bypassed `EncryptReader` and stored plaintext. Lesson: test actual bytes on disk and wrapper order, not only API metadata.
## Useful Search Seeds
Use these targeted searches when a diff touches security-sensitive code:
```bash
rg -n "validate_admin_request|check_permissions|AdminAction::|deny_only|is_allowed" rustfs crates
rg -n "UploadPartCopy|upload_part_copy|CompleteMultipart|PostObject|content-length-range|starts-with" rustfs crates
rg -n "PathBuf::join|canonicalize|\\.\\.|x-forwarded-for|x-real-ip|SourceIp" rustfs crates
rg -n "DEFAULT_SECRET|DEFAULT_ACCESS|TEST_PRIVATE_KEY|rustfs rpc|RUSTFS_RPC_SECRET" rustfs crates
rg -n "debug!|trace!|info!|error!|\\?resp|\\?merged_config|session_token|secret_key" rustfs crates
rg -n "HashReader|EncryptReader|SSE|server-side encryption|Access-Control-Allow-Credentials|Origin" rustfs crates
```
## Minimum Regression Test Expectations
- Authz fixes: include unauthenticated, valid low-privilege, wrong-action, correct-action, owner, non-owner, and root/admin cases as applicable.
- IAM fixes: include import/update/list service-account cases with attacker-controlled parent, claims, access key, secret key, and policy.
- Copy/upload fixes: include cross-bucket, cross-user, source-denied, destination-denied, copy-source-condition, and multipart completion cases.
- Path fixes: include encoded traversal, absolute path, nested traversal, valid object keys that resemble traversal text but should be rejected, and canonical boundary checks.
- Logging fixes: assert redacted output for structs and response bodies that may contain credentials.
- Browser/CORS fixes: assert no credentials on reflected/default origins, correct behavior for explicit allowlists, and no same-origin script execution for previewed object content.
- SSE fixes: inspect stored bytes and verify API metadata, read-back behavior, and on-disk ciphertext together.
@@ -0,0 +1,66 @@
---
name: test-coverage-improver
description: Run project coverage checks, rank high-risk gaps, and propose high-impact tests to improve regression confidence for changed and critical code paths before release.
---
# Test Coverage Improver
Use this skill when you need a prioritized, risk-aware plan to improve tests from coverage results.
## Usage assumptions
- Focus scope is either changed lines/files, a module, or the whole repository.
- Coverage artifact must be generated or provided in a supported format.
- If required context is missing, call out assumptions explicitly before proposing work.
## Workflow
1. Define scope and baseline
- Confirm target language, framework, and branch.
- Confirm whether the scope is changed files only or full-repo.
2. Produce coverage snapshot
- Rust: `cargo llvm-cov` (or `cargo tarpaulin`) with existing repo config.
- JavaScript/TypeScript: `npm test -- --coverage` and read `coverage/coverage-final.json`.
- Python: `pytest --cov=<pkg> --cov-report=json` and read `coverage.json`.
- Collect total, per-file, and changed-line coverage.
3. Rank highest-risk gaps
- Prioritize changed code, branch coverage gaps, and low-confidence boundaries.
- Apply the risk rubric in [coverage-prioritization.md](references/coverage-prioritization.md).
- Keep shortlist to 58 gaps.
- For each gap, capture: file, lines, uncovered branches, and estimated risk score.
4. Propose high-impact tests
- For each shortlisted gap, output:
- Intent and expected behavior.
- Normal, edge, and failure scenarios.
- Assertions and side effects to verify.
- Setup needs (fixtures, mocks, integration dependencies).
- Estimated effort (`S/M/L`).
5. Close with validation plan
- State which gaps remain after proposals.
- Provide concrete verification command and acceptance threshold.
- List assumptions or blockers (environment, fixtures, flaky dependencies).
## Output template
### Coverage Snapshot
- total / branch coverage
- changed-file coverage
- top missing regions by size
### Top Gaps (ranked)
- `path:line-range` | risk score | why critical
### Test Proposals
- `path:line-range`
- Test name
- scenarios
- assertions
- effort
### Validation Plan
- command
- pass criteria
- remaining risk
@@ -0,0 +1,4 @@
interface:
display_name: "Test Coverage Improver"
short_description: "Find top uncovered risk areas and propose high-impact tests."
default_prompt: "Run coverage checks, identify largest gaps, and recommend highest-impact test cases to improve risk coverage."
@@ -0,0 +1,25 @@
# Coverage Gap Prioritization Guide
Use this rubric for each uncovered area.
Score = (Criticality × 2) + CoverageDebt + (Volatility × 0.5)
- Criticality:
- 5: authz/authn, data-loss, payment/consistency path
- 4: state mutation, cache invalidation, scheduling
- 3: error handling + fallbacks in user-visible flows
- 2: parsing/format conversion paths
- 1: logging-only or low-impact utilities
- CoverageDebt:
- 0: 05 uncovered lines
- 1: 620 uncovered lines
- 2: 2140 uncovered lines
- 3: 41+ uncovered lines
- Volatility:
- 1: stable legacy code with few recent edits
- 2: changed in last 2 releases
- 3: touched in last 30 days or currently in active PR
Sort by score descending, then by business impact.
+26
View File
@@ -0,0 +1,26 @@
# Copyright 2024 RustFS Team
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# RustFS Cargo configuration
# Enable tokio_unstable cfg for dial9-tokio-telemetry support
# This allows dial9 to hook into Tokio's internal runtime events
[build]
# Enable Tokio unstable features required by dial9-tokio-telemetry for runtime tracing.
# See: https://docs.rs/tokio/latest/tokio/#unstable-features
rustflags = ["--cfg", "tokio_unstable"]
# Enable frame pointers for CPU profiling (Linux only, optional but recommended)
# Uncomment the following line for better CPU profiling data
# rustflags = ["--cfg", "tokio_unstable", "-C", "force-frame-pointers=yes"]
-94
View File
@@ -1,94 +0,0 @@
---
name: coding-guidelines
description: "Use when asking about Rust code style or best practices. Keywords: naming, formatting, comment, clippy, rustfmt, lint, code style, best practice, P.NAM, G.FMT, code review, naming convention, variable naming, function naming, type naming, 命名规范, 代码风格, 格式化, 最佳实践, 代码审查, 怎么命名"
source: https://rust-coding-guidelines.github.io/rust-coding-guidelines-zh/
---
# Rust Coding Guidelines (50 Core Rules)
## Naming (Rust-Specific)
| Rule | Guideline |
|------|-----------|
| No `get_` prefix | `fn name()` not `fn get_name()` |
| Iterator convention | `iter()` / `iter_mut()` / `into_iter()` |
| Conversion naming | `as_` (cheap &), `to_` (expensive), `into_` (ownership) |
| Static var prefix | `G_CONFIG` for `static`, no prefix for `const` |
## Data Types
| Rule | Guideline |
|------|-----------|
| Use newtypes | `struct Email(String)` for domain semantics |
| Prefer slice patterns | `if let [first, .., last] = slice` |
| Pre-allocate | `Vec::with_capacity()`, `String::with_capacity()` |
| Avoid Vec abuse | Use arrays for fixed sizes |
## Strings
| Rule | Guideline |
|------|-----------|
| Prefer bytes | `s.bytes()` over `s.chars()` when ASCII |
| Use `Cow<str>` | When might modify borrowed data |
| Use `format!` | Over string concatenation with `+` |
| Avoid nested iteration | `contains()` on string is O(n*m) |
## Error Handling
| Rule | Guideline |
|------|-----------|
| Use `?` propagation | Not `try!()` macro |
| `expect()` over `unwrap()` | When value guaranteed |
| Assertions for invariants | `assert!` at function entry |
## Memory
| Rule | Guideline |
|------|-----------|
| Meaningful lifetimes | `'src`, `'ctx` not just `'a` |
| `try_borrow()` for RefCell | Avoid panic |
| Shadowing for transformation | `let x = x.parse()?` |
## Concurrency
| Rule | Guideline |
|------|-----------|
| Identify lock ordering | Prevent deadlocks |
| Atomics for primitives | Not Mutex for bool/usize |
| Choose memory order carefully | Relaxed/Acquire/Release/SeqCst |
## Async
| Rule | Guideline |
|------|-----------|
| Sync for CPU-bound | Async is for I/O |
| Don't hold locks across await | Use scoped guards |
## Macros
| Rule | Guideline |
|------|-----------|
| Avoid unless necessary | Prefer functions/generics |
| Follow Rust syntax | Macro input should look like Rust |
## Deprecated → Better
| Deprecated | Better | Since |
|------------|--------|-------|
| `lazy_static!` | `std::sync::OnceLock` | 1.70 |
| `once_cell::Lazy` | `std::sync::LazyLock` | 1.80 |
| `std::sync::mpsc` | `crossbeam::channel` | - |
| `std::sync::Mutex` | `parking_lot::Mutex` | - |
| `failure`/`error-chain` | `thiserror`/`anyhow` | - |
| `try!()` | `?` operator | 2018 |
## Quick Reference
```
Naming: snake_case (fn/var), CamelCase (type), SCREAMING_CASE (const)
Format: rustfmt (just use it)
Docs: /// for public items, //! for module docs
Lint: #![warn(clippy::all)]
```
Claude knows Rust conventions well. These are the non-obvious Rust-specific rules.
@@ -1,16 +0,0 @@
# Clippy Lint → Rule Mapping
| Clippy Lint | Category | Fix |
|-------------|----------|-----|
| `unwrap_used` | Error | Use `?` or `expect()` |
| `needless_clone` | Perf | Use reference |
| `await_holding_lock` | Async | Scope guard before await |
| `linkedlist` | Perf | Use Vec/VecDeque |
| `wildcard_imports` | Style | Explicit imports |
| `missing_safety_doc` | Safety | Add `# Safety` doc |
| `undocumented_unsafe_blocks` | Safety | Add `// SAFETY:` |
| `transmute_ptr_to_ptr` | Safety | Use `pointer::cast()` |
| `large_stack_arrays` | Mem | Use Vec or Box |
| `too_many_arguments` | Design | Use struct params |
For unsafe-related lints → see `unsafe-checker` skill.
@@ -1,6 +0,0 @@
# Complete Rules Reference
For the full 500+ rules, see:
- Source: https://rust-coding-guidelines.github.io/rust-coding-guidelines-zh/
Core rules are in `../SKILL.md`.
-49
View File
@@ -1,49 +0,0 @@
---
name: core-actionbook
# Internal tool - no description to prevent auto-triggering
# Used by: rust-learner agents for pre-computed selectors
---
# Actionbook
Pre-computed action manuals for browser automation. Agents receive structured page information instead of parsing entire HTML.
## Workflow
1. **search_actions** - Search by keyword, returns URL-based action IDs with content previews
2. **get_action_by_id** - Get full action manual with page details, DOM structure, and element selectors
3. **Execute** - Use returned selectors with your browser automation tool
## MCP Tools
- `search_actions` - Search by keyword. Returns: URL-based action IDs, content previews, relevance scores
- `get_action_by_id` - Get full action details. Returns: action content, page element selectors (CSS/XPath), element types, allowed methods (click, type, extract), document metadata
### Parameters
**search_actions**:
- `query` (required): Search keyword (e.g., "airbnb search", "google login")
- `type`: `vector` | `fulltext` | `hybrid` (default)
- `limit`: Max results (default: 5)
- `sourceIds`: Filter by source IDs (comma-separated)
- `minScore`: Minimum relevance score (0-1)
**get_action_by_id**:
- `id` (required): URL-based action ID (e.g., `example.com/page`)
## Example Response
```json
{
"title": "Airbnb Search",
"url": "www.airbnb.com/search",
"elements": [
{
"name": "location_input",
"selector": "input[data-testid='structured-search-input-field-query']",
"type": "textbox",
"methods": ["type", "fill"]
}
]
}
```
-115
View File
@@ -1,115 +0,0 @@
---
name: core-agent-browser
# Internal tool - no description to prevent auto-triggering
# Used by: rust-learner, docs-researcher, crate-researcher agents
---
# Browser Automation with agent-browser
## Priority Note
For fetching Rust/crate information, use this priority order:
1. **rust-learner skill** - Orchestrates actionbook + browser-fetcher
2. **actionbook MCP** - Pre-computed selectors for known sites
3. **agent-browser CLI** - Direct browser automation (last resort)
Use agent-browser directly only when:
- actionbook has no pre-computed selectors for the target site
- You need interactive browser testing/automation
- You need screenshots or form filling
## Quick start
```bash
agent-browser open <url> # Navigate to page
agent-browser snapshot -i # Get interactive elements with refs
agent-browser click @e1 # Click element by ref
agent-browser fill @e2 "text" # Fill input by ref
agent-browser close # Close browser
```
## Core workflow
1. Navigate: `agent-browser open <url>`
2. Snapshot: `agent-browser snapshot -i` (returns elements with refs like `@e1`, `@e2`)
3. Interact using refs from the snapshot
4. Re-snapshot after navigation or significant DOM changes
## Commands
### Navigation
```bash
agent-browser open <url> # Navigate to URL
agent-browser back # Go back
agent-browser forward # Go forward
agent-browser reload # Reload page
agent-browser close # Close browser
```
### Snapshot (page analysis)
```bash
agent-browser snapshot # Full accessibility tree
agent-browser snapshot -i # Interactive elements only (recommended)
agent-browser snapshot -c # Compact output
agent-browser snapshot -d 3 # Limit depth to 3
```
### Interactions (use @refs from snapshot)
```bash
agent-browser click @e1 # Click
agent-browser dblclick @e1 # Double-click
agent-browser fill @e2 "text" # Clear and type
agent-browser type @e2 "text" # Type without clearing
agent-browser press Enter # Press key
agent-browser press Control+a # Key combination
agent-browser hover @e1 # Hover
agent-browser check @e1 # Check checkbox
agent-browser uncheck @e1 # Uncheck checkbox
agent-browser select @e1 "value" # Select dropdown
agent-browser scroll down 500 # Scroll page
agent-browser scrollintoview @e1 # Scroll element into view
```
### Get information
```bash
agent-browser get text @e1 # Get element text
agent-browser get value @e1 # Get input value
agent-browser get title # Get page title
agent-browser get url # Get current URL
```
### Screenshots
```bash
agent-browser screenshot # Screenshot to stdout
agent-browser screenshot path.png # Save to file
agent-browser screenshot --full # Full page
```
### Wait
```bash
agent-browser wait @e1 # Wait for element
agent-browser wait 2000 # Wait milliseconds
agent-browser wait --text "Success" # Wait for text
agent-browser wait --load networkidle # Wait for network idle
```
### Semantic locators (alternative to refs)
```bash
agent-browser find role button click --name "Submit"
agent-browser find text "Sign In" click
agent-browser find label "Email" fill "user@test.com"
```
## Example: Form submission
```bash
agent-browser open https://example.com/form
agent-browser snapshot -i
# Output shows: textbox "Email" [ref=e1], textbox "Password" [ref=e2], button "Submit" [ref=e3]
agent-browser fill @e1 "user@example.com"
agent-browser fill @e2 "password123"
agent-browser click @e3
agent-browser wait --load networkidle
agent-browser snapshot -i # Check result
```
@@ -1,87 +0,0 @@
---
name: core-dynamic-skills
# Command-based tool - no description to prevent auto-triggering
# Triggered by: /sync-crate-skills, /clean-crate-skills, /update-crate-skill
---
# Dynamic Skills Manager
Orchestrates on-demand generation of crate-specific skills based on project dependencies.
## Concept
Dynamic skills are:
- Generated locally at `~/.claude/skills/`
- Based on Cargo.toml dependencies
- Created using llms.txt from docs.rs
- Versioned and updatable
- Not committed to the rust-skills repository
## Trigger Scenarios
### Prompt-on-Open
When entering a directory with Cargo.toml:
1. Detect Cargo.toml (single or workspace)
2. Parse dependencies list
3. Check which crates are missing skills
4. If missing: "Found X dependencies without skills. Sync now?"
5. If confirmed: run `/sync-crate-skills`
### Manual Commands
- `/sync-crate-skills` - Sync all dependencies
- `/clean-crate-skills [crate]` - Remove skills
- `/update-crate-skill <crate>` - Update specific skill
## Architecture
```
Cargo.toml
Parse dependencies
For each crate:
├─ Check ~/.claude/skills/{crate}/
├─ If missing: Check actionbook for llms.txt
│ ├─ Found: /create-skills-via-llms
│ └─ Not found: /create-llms-for-skills first
└─ Load skill
```
## Local Skills Directory
```
~/.claude/skills/
├── tokio/
│ ├── SKILL.md
│ └── references/
├── serde/
│ ├── SKILL.md
│ └── references/
└── axum/
├── SKILL.md
└── references/
```
## Workflow Priority
1. **actionbook MCP** - Check for pre-generated llms.txt
2. **/create-llms-for-skills** - Generate llms.txt from docs.rs
3. **/create-skills-via-llms** - Create skills from llms.txt
## Workspace Support
For Cargo workspace projects:
1. Parse root Cargo.toml for `[workspace] members`
2. Collect all member Cargo.toml paths
3. Aggregate all dependencies
4. Deduplicate before skill generation
## Related Commands
- `/sync-crate-skills` - Main sync command
- `/clean-crate-skills` - Cleanup command
- `/update-crate-skill` - Update command
- `/create-llms-for-skills` - Generate llms.txt
- `/create-skills-via-llms` - Create skills from llms.txt
@@ -1,99 +0,0 @@
---
name: core-fix-skill-docs
# Internal maintenance tool - no description to prevent auto-triggering
# Triggered by: /fix-skill-docs command
---
# Fix Skill Documentation
Check and fix missing reference files in dynamic skills.
## Usage
```
/fix-skill-docs [crate_name] [--check-only] [--remove-invalid]
```
**Arguments:**
- `crate_name`: Specific crate to check (optional, defaults to all)
- `--check-only`: Only report issues, don't fix
- `--remove-invalid`: Remove invalid references instead of creating files
## Instructions
### 1. Scan Skills Directory
```bash
# If crate_name provided
skill_dir=~/.claude/skills/{crate_name}
# Otherwise scan all
for dir in ~/.claude/skills/*/; do
# Process each skill
done
```
### 2. Parse SKILL.md for References
Extract referenced files from Documentation section:
```markdown
## Documentation
- `./references/file1.md` - Description
```
### 3. Check File Existence
```bash
if [ ! -f "{skill_dir}/references/{filename}" ]; then
echo "MISSING: {filename}"
fi
```
### 4. Report Status
```
=== {crate_name} ===
SKILL.md: ✅
references/:
- sync.md: ✅
- runtime.md: ❌ MISSING
Action needed: 1 file missing
```
### 5. Fix Missing Files
**--check-only**: Only report, don't fix.
**--remove-invalid**: Update SKILL.md to remove invalid references.
**Default**: Generate missing files using agent-browser:
```bash
agent-browser "Navigate to docs.rs/{crate_name}/latest/{crate_name}/{module}/
Extract documentation for {topic}. Save as markdown."
```
### 6. Update SKILL.md
Ensure Documentation section matches actual files.
## Tool Priority
1. **agent-browser CLI** - Generate missing documentation
2. **WebFetch** - Fallback if agent-browser unavailable
3. **Edit SKILL.md** - Remove invalid references (--remove-invalid)
## Example
```bash
# Check all skills
/fix-skill-docs --check-only
# Fix specific crate
/fix-skill-docs tokio
# Remove invalid references
/fix-skill-docs tokio --remove-invalid
```
-160
View File
@@ -1,160 +0,0 @@
---
name: domain-cli
description: "Use when building CLI tools. Keywords: CLI, command line, terminal, clap, structopt, argument parsing, subcommand, interactive, TUI, ratatui, crossterm, indicatif, progress bar, colored output, shell completion, config file, environment variable, 命令行, 终端应用, 参数解析"
globs: ["**/Cargo.toml"]
---
# CLI Domain
> **Layer 3: Domain Constraints**
## Domain Constraints → Design Implications
| Domain Rule | Design Constraint | Rust Implication |
|-------------|-------------------|------------------|
| User ergonomics | Clear help, errors | clap derive macros |
| Config precedence | CLI > env > file | Layered config loading |
| Exit codes | Non-zero on error | Proper Result handling |
| Stdout/stderr | Data vs errors | eprintln! for errors |
| Interruptible | Handle Ctrl+C | Signal handling |
---
## Critical Constraints
### User Communication
```
RULE: Errors to stderr, data to stdout
WHY: Pipeable output, scriptability
RUST: eprintln! for errors, println! for data
```
### Configuration Priority
```
RULE: CLI args > env vars > config file > defaults
WHY: User expectation, override capability
RUST: Layered config with clap + figment/config
```
### Exit Codes
```
RULE: Return non-zero on any error
WHY: Script integration, automation
RUST: main() -> Result<(), Error> or explicit exit()
```
---
## Trace Down ↓
From constraints to design (Layer 2):
```
"Need argument parsing"
↓ m05-type-driven: Derive structs for args
↓ clap: #[derive(Parser)]
"Need config layering"
↓ m09-domain: Config as domain object
↓ figment/config: Layer sources
"Need progress display"
↓ m12-lifecycle: Progress bar as RAII
↓ indicatif: ProgressBar
```
---
## Key Crates
| Purpose | Crate |
|---------|-------|
| Argument parsing | clap |
| Interactive prompts | dialoguer |
| Progress bars | indicatif |
| Colored output | colored |
| Terminal UI | ratatui |
| Terminal control | crossterm |
| Console utilities | console |
## Design Patterns
| Pattern | Purpose | Implementation |
|---------|---------|----------------|
| Args struct | Type-safe args | `#[derive(Parser)]` |
| Subcommands | Command hierarchy | `#[derive(Subcommand)]` |
| Config layers | Override precedence | CLI > env > file |
| Progress | User feedback | `ProgressBar::new(len)` |
## Code Pattern: CLI Structure
```rust
use clap::{Parser, Subcommand};
#[derive(Parser)]
#[command(name = "myapp", about = "My CLI tool")]
struct Cli {
/// Enable verbose output
#[arg(short, long)]
verbose: bool,
#[command(subcommand)]
command: Commands,
}
#[derive(Subcommand)]
enum Commands {
/// Initialize a new project
Init { name: String },
/// Run the application
Run {
#[arg(short, long)]
port: Option<u16>,
},
}
fn main() -> anyhow::Result<()> {
let cli = Cli::parse();
match cli.command {
Commands::Init { name } => init_project(&name)?,
Commands::Run { port } => run_server(port.unwrap_or(8080))?,
}
Ok(())
}
```
---
## Common Mistakes
| Mistake | Domain Violation | Fix |
|---------|-----------------|-----|
| Errors to stdout | Breaks piping | eprintln! |
| No help text | Poor UX | #[arg(help = "...")] |
| Panic on error | Bad exit code | Result + proper handling |
| No progress for long ops | User uncertainty | indicatif |
---
## Trace to Layer 1
| Constraint | Layer 2 Pattern | Layer 1 Implementation |
|------------|-----------------|------------------------|
| Type-safe args | Derive macros | clap Parser |
| Error handling | Result propagation | anyhow + exit codes |
| User feedback | Progress RAII | indicatif ProgressBar |
| Config precedence | Builder pattern | Layered sources |
---
## Related Skills
| When | See |
|------|-----|
| Error handling | m06-error-handling |
| Type-driven args | m05-type-driven |
| Progress lifecycle | m12-lifecycle |
| Async CLI | m07-concurrency |
-165
View File
@@ -1,165 +0,0 @@
---
name: domain-cloud-native
description: "Use when building cloud-native apps. Keywords: kubernetes, k8s, docker, container, grpc, tonic, microservice, service mesh, observability, tracing, metrics, health check, cloud, deployment, 云原生, 微服务, 容器"
---
# Cloud-Native Domain
> **Layer 3: Domain Constraints**
## Domain Constraints → Design Implications
| Domain Rule | Design Constraint | Rust Implication |
|-------------|-------------------|------------------|
| 12-Factor | Config from env | Environment-based config |
| Observability | Metrics + traces | tracing + opentelemetry |
| Health checks | Liveness/readiness | Dedicated endpoints |
| Graceful shutdown | Clean termination | Signal handling |
| Horizontal scale | Stateless design | No local state |
| Container-friendly | Small binaries | Release optimization |
---
## Critical Constraints
### Stateless Design
```
RULE: No local persistent state
WHY: Pods can be killed/rescheduled anytime
RUST: External state (Redis, DB), no static mut
```
### Graceful Shutdown
```
RULE: Handle SIGTERM, drain connections
WHY: Zero-downtime deployments
RUST: tokio::signal + graceful shutdown
```
### Observability
```
RULE: Every request must be traceable
WHY: Debugging distributed systems
RUST: tracing spans, opentelemetry export
```
---
## Trace Down ↓
From constraints to design (Layer 2):
```
"Need distributed tracing"
↓ m12-lifecycle: Span lifecycle
↓ tracing + opentelemetry
"Need graceful shutdown"
↓ m07-concurrency: Signal handling
↓ m12-lifecycle: Connection draining
"Need health checks"
↓ domain-web: HTTP endpoints
↓ m06-error-handling: Health status
```
---
## Key Crates
| Purpose | Crate |
|---------|-------|
| gRPC | tonic |
| Kubernetes | kube, kube-runtime |
| Docker | bollard |
| Tracing | tracing, opentelemetry |
| Metrics | prometheus, metrics |
| Config | config, figment |
| Health | HTTP endpoints |
## Design Patterns
| Pattern | Purpose | Implementation |
|---------|---------|----------------|
| gRPC services | Service mesh | tonic + tower |
| K8s operators | Custom resources | kube-runtime Controller |
| Observability | Debugging | tracing + OTEL |
| Health checks | Orchestration | `/health`, `/ready` |
| Config | 12-factor | Env vars + secrets |
## Code Pattern: Graceful Shutdown
```rust
use tokio::signal;
async fn run_server() -> anyhow::Result<()> {
let app = Router::new()
.route("/health", get(health))
.route("/ready", get(ready));
let addr = SocketAddr::from(([0, 0, 0, 0], 8080));
axum::Server::bind(&addr)
.serve(app.into_make_service())
.with_graceful_shutdown(shutdown_signal())
.await?;
Ok(())
}
async fn shutdown_signal() {
signal::ctrl_c().await.expect("failed to listen for ctrl+c");
tracing::info!("shutdown signal received");
}
```
## Health Check Pattern
```rust
async fn health() -> StatusCode {
StatusCode::OK
}
async fn ready(State(db): State<Arc<DbPool>>) -> StatusCode {
match db.ping().await {
Ok(_) => StatusCode::OK,
Err(_) => StatusCode::SERVICE_UNAVAILABLE,
}
}
```
---
## Common Mistakes
| Mistake | Domain Violation | Fix |
|---------|-----------------|-----|
| Local file state | Not stateless | External storage |
| No SIGTERM handling | Hard kills | Graceful shutdown |
| No tracing | Can't debug | tracing spans |
| Static config | Not 12-factor | Env vars |
---
## Trace to Layer 1
| Constraint | Layer 2 Pattern | Layer 1 Implementation |
|------------|-----------------|------------------------|
| Stateless | External state | Arc<Client> for external |
| Graceful shutdown | Signal handling | tokio::signal |
| Tracing | Span lifecycle | tracing + OTEL |
| Health checks | HTTP endpoints | Dedicated routes |
---
## Related Skills
| When | See |
|------|-----|
| Async patterns | m07-concurrency |
| HTTP endpoints | domain-web |
| Error handling | m13-domain-error |
| Resource lifecycle | m12-lifecycle |
-170
View File
@@ -1,170 +0,0 @@
---
name: domain-embedded
description: "Use when developing embedded/no_std Rust. Keywords: embedded, no_std, microcontroller, MCU, ARM, RISC-V, bare metal, firmware, HAL, PAC, RTIC, embassy, interrupt, DMA, peripheral, GPIO, SPI, I2C, UART, embedded-hal, cortex-m, esp32, stm32, nrf, 嵌入式, 单片机, 固件, 裸机"
globs: ["**/Cargo.toml", "**/.cargo/config.toml"]
---
# Embedded Domain
> **Layer 3: Domain Constraints**
## Domain Constraints → Design Implications
| Domain Rule | Design Constraint | Rust Implication |
|-------------|-------------------|------------------|
| No heap | Stack allocation | heapless, no Box/Vec |
| No std | Core only | #![no_std] |
| Real-time | Predictable timing | No dynamic alloc |
| Resource limited | Minimal memory | Static buffers |
| Hardware safety | Safe peripheral access | HAL + ownership |
| Interrupt safe | No blocking in ISR | Atomic, critical sections |
---
## Critical Constraints
### No Dynamic Allocation
```
RULE: Cannot use heap (no allocator)
WHY: Deterministic memory, no OOM
RUST: heapless::Vec<T, N>, arrays
```
### Interrupt Safety
```
RULE: Shared state must be interrupt-safe
WHY: ISR can preempt at any time
RUST: Mutex<RefCell<T>> + critical section
```
### Hardware Ownership
```
RULE: Peripherals must have clear ownership
WHY: Prevent conflicting access
RUST: HAL takes ownership, singletons
```
---
## Trace Down ↓
From constraints to design (Layer 2):
```
"Need no_std compatible data structures"
↓ m02-resource: heapless collections
↓ Static sizing: heapless::Vec<T, N>
"Need interrupt-safe state"
↓ m03-mutability: Mutex<RefCell<Option<T>>>
↓ m07-concurrency: Critical sections
"Need peripheral ownership"
↓ m01-ownership: Singleton pattern
↓ m12-lifecycle: RAII for hardware
```
---
## Layer Stack
| Layer | Examples | Purpose |
|-------|----------|---------|
| PAC | stm32f4, esp32c3 | Register access |
| HAL | stm32f4xx-hal | Hardware abstraction |
| Framework | RTIC, Embassy | Concurrency |
| Traits | embedded-hal | Portable drivers |
## Framework Comparison
| Framework | Style | Best For |
|-----------|-------|----------|
| RTIC | Priority-based | Interrupt-driven apps |
| Embassy | Async | Complex state machines |
| Bare metal | Manual | Simple apps |
## Key Crates
| Purpose | Crate |
|---------|-------|
| Runtime (ARM) | cortex-m-rt |
| Panic handler | panic-halt, panic-probe |
| Collections | heapless |
| HAL traits | embedded-hal |
| Logging | defmt |
| Flash/debug | probe-run |
## Design Patterns
| Pattern | Purpose | Implementation |
|---------|---------|----------------|
| no_std setup | Bare metal | `#![no_std]` + `#![no_main]` |
| Entry point | Startup | `#[entry]` or embassy |
| Static state | ISR access | `Mutex<RefCell<Option<T>>>` |
| Fixed buffers | No heap | `heapless::Vec<T, N>` |
## Code Pattern: Static Peripheral
```rust
#![no_std]
#![no_main]
use cortex_m::interrupt::{self, Mutex};
use core::cell::RefCell;
static LED: Mutex<RefCell<Option<Led>>> = Mutex::new(RefCell::new(None));
#[entry]
fn main() -> ! {
let dp = pac::Peripherals::take().unwrap();
let led = Led::new(dp.GPIOA);
interrupt::free(|cs| {
LED.borrow(cs).replace(Some(led));
});
loop {
interrupt::free(|cs| {
if let Some(led) = LED.borrow(cs).borrow_mut().as_mut() {
led.toggle();
}
});
}
}
```
---
## Common Mistakes
| Mistake | Domain Violation | Fix |
|---------|-----------------|-----|
| Using Vec | Heap allocation | heapless::Vec |
| No critical section | Race with ISR | Mutex + interrupt::free |
| Blocking in ISR | Missed interrupts | Defer to main loop |
| Unsafe peripheral | Hardware conflict | HAL ownership |
---
## Trace to Layer 1
| Constraint | Layer 2 Pattern | Layer 1 Implementation |
|------------|-----------------|------------------------|
| No heap | Static collections | heapless::Vec<T, N> |
| ISR safety | Critical sections | Mutex<RefCell<T>> |
| Hardware ownership | Singleton | take().unwrap() |
| no_std | Core-only | #![no_std], #![no_main] |
---
## Related Skills
| When | See |
|------|-----|
| Static memory | m02-resource |
| Interior mutability | m03-mutability |
| Interrupt patterns | m07-concurrency |
| Unsafe for hardware | unsafe-checker |
-145
View File
@@ -1,145 +0,0 @@
---
name: domain-fintech
description: "Use when building fintech apps. Keywords: fintech, trading, decimal, currency, financial, money, transaction, ledger, payment, exchange rate, precision, rounding, accounting, 金融, 交易系统, 货币, 支付"
---
# FinTech Domain
> **Layer 3: Domain Constraints**
## Domain Constraints → Design Implications
| Domain Rule | Design Constraint | Rust Implication |
|-------------|-------------------|------------------|
| Audit trail | Immutable records | Arc<T>, no mutation |
| Precision | No floating point | rust_decimal |
| Consistency | Transaction boundaries | Clear ownership |
| Compliance | Complete logging | Structured tracing |
| Reproducibility | Deterministic execution | No race conditions |
---
## Critical Constraints
### Financial Precision
```
RULE: Never use f64 for money
WHY: Floating point loses precision
RUST: Use rust_decimal::Decimal
```
### Audit Requirements
```
RULE: All transactions must be immutable and traceable
WHY: Regulatory compliance, dispute resolution
RUST: Arc<T> for sharing, event sourcing pattern
```
### Consistency
```
RULE: Money can't disappear or appear
WHY: Double-entry accounting principles
RUST: Transaction types with validated totals
```
---
## Trace Down ↓
From constraints to design (Layer 2):
```
"Need immutable transaction records"
↓ m09-domain: Model as Value Objects
↓ m01-ownership: Use Arc for shared immutable data
"Need precise decimal math"
↓ m05-type-driven: Newtype for Currency/Amount
↓ rust_decimal: Use Decimal type
"Need transaction boundaries"
↓ m12-lifecycle: RAII for transaction scope
↓ m09-domain: Aggregate boundaries
```
---
## Key Crates
| Purpose | Crate |
|---------|-------|
| Decimal math | rust_decimal |
| Date/time | chrono, time |
| UUID | uuid |
| Serialization | serde |
| Validation | validator |
## Design Patterns
| Pattern | Purpose | Implementation |
|---------|---------|----------------|
| Currency newtype | Type safety | `struct Amount(Decimal);` |
| Transaction | Atomic operations | Event sourcing |
| Audit log | Traceability | Structured logging with trace IDs |
| Ledger | Double-entry | Debit/credit balance |
## Code Pattern: Currency Type
```rust
use rust_decimal::Decimal;
#[derive(Clone, Debug, PartialEq)]
pub struct Amount {
value: Decimal,
currency: Currency,
}
impl Amount {
pub fn new(value: Decimal, currency: Currency) -> Self {
Self { value, currency }
}
pub fn add(&self, other: &Amount) -> Result<Amount, CurrencyMismatch> {
if self.currency != other.currency {
return Err(CurrencyMismatch);
}
Ok(Amount::new(self.value + other.value, self.currency))
}
}
```
---
## Common Mistakes
| Mistake | Domain Violation | Fix |
|---------|-----------------|-----|
| Using f64 | Precision loss | rust_decimal |
| Mutable transaction | Audit trail broken | Immutable + events |
| String for amount | No validation | Validated newtype |
| Silent overflow | Money disappears | Checked arithmetic |
---
## Trace to Layer 1
| Constraint | Layer 2 Pattern | Layer 1 Implementation |
|------------|-----------------|------------------------|
| Immutable records | Event sourcing | Arc<T>, Clone |
| Transaction scope | Aggregate | Owned children |
| Precision | Value Object | rust_decimal newtype |
| Thread-safe sharing | Shared immutable | Arc (not Rc) |
---
## Related Skills
| When | See |
|------|-----|
| Value Object design | m09-domain |
| Ownership for immutable | m01-ownership |
| Arc for sharing | m02-resource |
| Error handling | m13-domain-error |
-167
View File
@@ -1,167 +0,0 @@
---
name: domain-iot
description: "Use when building IoT apps. Keywords: IoT, Internet of Things, sensor, MQTT, device, edge computing, telemetry, actuator, smart home, gateway, protocol, 物联网, 传感器, 边缘计算, 智能家居"
---
# IoT Domain
> **Layer 3: Domain Constraints**
## Domain Constraints → Design Implications
| Domain Rule | Design Constraint | Rust Implication |
|-------------|-------------------|------------------|
| Unreliable network | Offline-first | Local buffering |
| Power constraints | Efficient code | Sleep modes, minimal alloc |
| Resource limits | Small footprint | no_std where needed |
| Security | Encrypted comms | TLS, signed firmware |
| Reliability | Self-recovery | Watchdog, error handling |
| OTA updates | Safe upgrades | Rollback capability |
---
## Critical Constraints
### Network Unreliability
```
RULE: Network can fail at any time
WHY: Wireless, remote locations
RUST: Local queue, retry with backoff
```
### Power Management
```
RULE: Minimize power consumption
WHY: Battery life, energy costs
RUST: Sleep modes, efficient algorithms
```
### Device Security
```
RULE: All communication encrypted
WHY: Physical access possible
RUST: TLS, signed messages
```
---
## Trace Down ↓
From constraints to design (Layer 2):
```
"Need offline-first design"
↓ m12-lifecycle: Local buffer with persistence
↓ m13-domain-error: Retry with backoff
"Need power efficiency"
↓ domain-embedded: no_std patterns
↓ m10-performance: Minimal allocations
"Need reliable messaging"
↓ m07-concurrency: Async with timeout
↓ MQTT: QoS levels
```
---
## Environment Comparison
| Environment | Stack | Crates |
|-------------|-------|--------|
| Linux gateway | tokio + std | rumqttc, reqwest |
| MCU device | embassy + no_std | embedded-hal |
| Hybrid | Split workloads | Both |
## Key Crates
| Purpose | Crate |
|---------|-------|
| MQTT (std) | rumqttc, paho-mqtt |
| Embedded | embedded-hal, embassy |
| Async (std) | tokio |
| Async (no_std) | embassy |
| Logging (no_std) | defmt |
| Logging (std) | tracing |
## Design Patterns
| Pattern | Purpose | Implementation |
|---------|---------|----------------|
| Pub/Sub | Device comms | MQTT topics |
| Edge compute | Local processing | Filter before upload |
| OTA updates | Firmware upgrade | Signed + rollback |
| Power mgmt | Battery life | Sleep + wake events |
| Store & forward | Network reliability | Local queue |
## Code Pattern: MQTT Client
```rust
use rumqttc::{AsyncClient, MqttOptions, QoS};
async fn run_mqtt() -> anyhow::Result<()> {
let mut options = MqttOptions::new("device-1", "broker.example.com", 1883);
options.set_keep_alive(Duration::from_secs(30));
let (client, mut eventloop) = AsyncClient::new(options, 10);
// Subscribe to commands
client.subscribe("devices/device-1/commands", QoS::AtLeastOnce).await?;
// Publish telemetry
tokio::spawn(async move {
loop {
let data = read_sensor().await;
client.publish("devices/device-1/telemetry", QoS::AtLeastOnce, false, data).await.ok();
tokio::time::sleep(Duration::from_secs(60)).await;
}
});
// Process events
loop {
match eventloop.poll().await {
Ok(event) => handle_event(event).await,
Err(e) => {
tracing::error!("MQTT error: {}", e);
tokio::time::sleep(Duration::from_secs(5)).await;
}
}
}
}
```
---
## Common Mistakes
| Mistake | Domain Violation | Fix |
|---------|-----------------|-----|
| No retry logic | Lost data | Exponential backoff |
| Always-on radio | Battery drain | Sleep between sends |
| Unencrypted MQTT | Security risk | TLS |
| No local buffer | Network outage = data loss | Persist locally |
---
## Trace to Layer 1
| Constraint | Layer 2 Pattern | Layer 1 Implementation |
|------------|-----------------|------------------------|
| Offline-first | Store & forward | Local queue + flush |
| Power efficiency | Sleep patterns | Timer-based wake |
| Network reliability | Retry | tokio-retry, backoff |
| Security | TLS | rustls, native-tls |
---
## Related Skills
| When | See |
|------|-----|
| Embedded patterns | domain-embedded |
| Async patterns | m07-concurrency |
| Error recovery | m13-domain-error |
| Performance | m10-performance |
-180
View File
@@ -1,180 +0,0 @@
---
name: domain-ml
description: "Use when building ML/AI apps in Rust. Keywords: machine learning, ML, AI, tensor, model, inference, neural network, deep learning, training, prediction, ndarray, tch-rs, burn, candle, 机器学习, 人工智能, 模型推理"
---
# Machine Learning Domain
> **Layer 3: Domain Constraints**
## Domain Constraints → Design Implications
| Domain Rule | Design Constraint | Rust Implication |
|-------------|-------------------|------------------|
| Large data | Efficient memory | Zero-copy, streaming |
| GPU acceleration | CUDA/Metal support | candle, tch-rs |
| Model portability | Standard formats | ONNX |
| Batch processing | Throughput over latency | Batched inference |
| Numerical precision | Float handling | ndarray, careful f32/f64 |
| Reproducibility | Deterministic | Seeded random, versioning |
---
## Critical Constraints
### Memory Efficiency
```
RULE: Avoid copying large tensors
WHY: Memory bandwidth is bottleneck
RUST: References, views, in-place ops
```
### GPU Utilization
```
RULE: Batch operations for GPU efficiency
WHY: GPU overhead per kernel launch
RUST: Batch sizes, async data loading
```
### Model Portability
```
RULE: Use standard model formats
WHY: Train in Python, deploy in Rust
RUST: ONNX via tract or candle
```
---
## Trace Down ↓
From constraints to design (Layer 2):
```
"Need efficient data pipelines"
↓ m10-performance: Streaming, batching
↓ polars: Lazy evaluation
"Need GPU inference"
↓ m07-concurrency: Async data loading
↓ candle/tch-rs: CUDA backend
"Need model loading"
↓ m12-lifecycle: Lazy init, caching
↓ tract: ONNX runtime
```
---
## Use Case → Framework
| Use Case | Recommended | Why |
|----------|-------------|-----|
| Inference only | tract (ONNX) | Lightweight, portable |
| Training + inference | candle, burn | Pure Rust, GPU |
| PyTorch models | tch-rs | Direct bindings |
| Data pipelines | polars | Fast, lazy eval |
## Key Crates
| Purpose | Crate |
|---------|-------|
| Tensors | ndarray |
| ONNX inference | tract |
| ML framework | candle, burn |
| PyTorch bindings | tch-rs |
| Data processing | polars |
| Embeddings | fastembed |
## Design Patterns
| Pattern | Purpose | Implementation |
|---------|---------|----------------|
| Model loading | Once, reuse | `OnceLock<Model>` |
| Batching | Throughput | Collect then process |
| Streaming | Large data | Iterator-based |
| GPU async | Parallelism | Data loading parallel to compute |
## Code Pattern: Inference Server
```rust
use std::sync::OnceLock;
use tract_onnx::prelude::*;
static MODEL: OnceLock<SimplePlan<TypedFact, Box<dyn TypedOp>, Graph<TypedFact, Box<dyn TypedOp>>>> = OnceLock::new();
fn get_model() -> &'static SimplePlan<...> {
MODEL.get_or_init(|| {
tract_onnx::onnx()
.model_for_path("model.onnx")
.unwrap()
.into_optimized()
.unwrap()
.into_runnable()
.unwrap()
})
}
async fn predict(input: Vec<f32>) -> anyhow::Result<Vec<f32>> {
let model = get_model();
let input = tract_ndarray::arr1(&input).into_shape((1, input.len()))?;
let result = model.run(tvec!(input.into()))?;
Ok(result[0].to_array_view::<f32>()?.iter().copied().collect())
}
```
## Code Pattern: Batched Inference
```rust
async fn batch_predict(inputs: Vec<Vec<f32>>, batch_size: usize) -> Vec<Vec<f32>> {
let mut results = Vec::with_capacity(inputs.len());
for batch in inputs.chunks(batch_size) {
// Stack inputs into batch tensor
let batch_tensor = stack_inputs(batch);
// Run inference on batch
let batch_output = model.run(batch_tensor).await;
// Unstack results
results.extend(unstack_outputs(batch_output));
}
results
}
```
---
## Common Mistakes
| Mistake | Domain Violation | Fix |
|---------|-----------------|-----|
| Clone tensors | Memory waste | Use views |
| Single inference | GPU underutilized | Batch processing |
| Load model per request | Slow | Singleton pattern |
| Sync data loading | GPU idle | Async pipeline |
---
## Trace to Layer 1
| Constraint | Layer 2 Pattern | Layer 1 Implementation |
|------------|-----------------|------------------------|
| Memory efficiency | Zero-copy | ndarray views |
| Model singleton | Lazy init | OnceLock<Model> |
| Batch processing | Chunked iteration | chunks() + parallel |
| GPU async | Concurrent loading | tokio::spawn + GPU |
---
## Related Skills
| When | See |
|------|-----|
| Performance | m10-performance |
| Lazy initialization | m12-lifecycle |
| Async patterns | m07-concurrency |
| Memory efficiency | m01-ownership |
-155
View File
@@ -1,155 +0,0 @@
---
name: domain-web
description: "Use when building web services. Keywords: web server, HTTP, REST API, GraphQL, WebSocket, axum, actix, warp, rocket, tower, hyper, reqwest, middleware, router, handler, extractor, state management, authentication, authorization, JWT, session, cookie, CORS, rate limiting, web 开发, HTTP 服务, API 设计, 中间件, 路由"
globs: ["**/Cargo.toml"]
---
# Web Domain
> **Layer 3: Domain Constraints**
## Domain Constraints → Design Implications
| Domain Rule | Design Constraint | Rust Implication |
|-------------|-------------------|------------------|
| Stateless HTTP | No request-local globals | State in extractors |
| Concurrency | Handle many connections | Async, Send + Sync |
| Latency SLA | Fast response | Efficient ownership |
| Security | Input validation | Type-safe extractors |
| Observability | Request tracing | tracing + tower layers |
---
## Critical Constraints
### Async by Default
```
RULE: Web handlers must not block
WHY: Block one task = block many requests
RUST: async/await, spawn_blocking for CPU work
```
### State Management
```
RULE: Shared state must be thread-safe
WHY: Handlers run on any thread
RUST: Arc<T>, Arc<RwLock<T>> for mutable
```
### Request Lifecycle
```
RULE: Resources live only for request duration
WHY: Memory management, no leaks
RUST: Extractors, proper ownership
```
---
## Trace Down ↓
From constraints to design (Layer 2):
```
"Need shared application state"
↓ m07-concurrency: Use Arc for thread-safe sharing
↓ m02-resource: Arc<RwLock<T>> for mutable state
"Need request validation"
↓ m05-type-driven: Validated extractors
↓ m06-error-handling: IntoResponse for errors
"Need middleware stack"
↓ m12-lifecycle: Tower layers
↓ m04-zero-cost: Trait-based composition
```
---
## Framework Comparison
| Framework | Style | Best For |
|-----------|-------|----------|
| axum | Functional, tower | Modern APIs |
| actix-web | Actor-based | High performance |
| warp | Filter composition | Composable APIs |
| rocket | Macro-driven | Rapid development |
## Key Crates
| Purpose | Crate |
|---------|-------|
| HTTP server | axum, actix-web |
| HTTP client | reqwest |
| JSON | serde_json |
| Auth/JWT | jsonwebtoken |
| Session | tower-sessions |
| Database | sqlx, diesel |
| Middleware | tower |
## Design Patterns
| Pattern | Purpose | Implementation |
|---------|---------|----------------|
| Extractors | Request parsing | `State(db)`, `Json(payload)` |
| Error response | Unified errors | `impl IntoResponse` |
| Middleware | Cross-cutting | Tower layers |
| Shared state | App config | `Arc<AppState>` |
## Code Pattern: Axum Handler
```rust
async fn handler(
State(db): State<Arc<DbPool>>,
Json(payload): Json<CreateUser>,
) -> Result<Json<User>, AppError> {
let user = db.create_user(&payload).await?;
Ok(Json(user))
}
// Error handling
impl IntoResponse for AppError {
fn into_response(self) -> Response {
let (status, message) = match self {
Self::NotFound => (StatusCode::NOT_FOUND, "Not found"),
Self::Internal(_) => (StatusCode::INTERNAL_SERVER_ERROR, "Internal error"),
};
(status, Json(json!({"error": message}))).into_response()
}
}
```
---
## Common Mistakes
| Mistake | Domain Violation | Fix |
|---------|-----------------|-----|
| Blocking in handler | Latency spike | spawn_blocking |
| Rc in state | Not Send + Sync | Use Arc |
| No validation | Security risk | Type-safe extractors |
| No error response | Bad UX | IntoResponse impl |
---
## Trace to Layer 1
| Constraint | Layer 2 Pattern | Layer 1 Implementation |
|------------|-----------------|------------------------|
| Async handlers | Async/await | tokio runtime |
| Thread-safe state | Shared state | Arc<T>, Arc<RwLock<T>> |
| Request lifecycle | Extractors | Ownership via From<Request> |
| Middleware | Tower layers | Trait-based composition |
---
## Related Skills
| When | See |
|------|-----|
| Async patterns | m07-concurrency |
| State management | m02-resource |
| Error handling | m06-error-handling |
| Middleware design | m12-lifecycle |
-133
View File
@@ -1,133 +0,0 @@
---
name: m01-ownership
description: "CRITICAL: Use for ownership/borrow/lifetime issues. Triggers: E0382, E0597, E0506, E0507, E0515, E0716, E0106, value moved, borrowed value does not live long enough, cannot move out of, use of moved value, ownership, borrow, lifetime, 'a, 'static, move, clone, Copy, 所有权, 借用, 生命周期"
---
# Ownership & Lifetimes
> **Layer 1: Language Mechanics**
## Core Question
**Who should own this data, and for how long?**
Before fixing ownership errors, understand the data's role:
- Is it shared or exclusive?
- Is it short-lived or long-lived?
- Is it transformed or just read?
---
## Error → Design Question
| Error | Don't Just Say | Ask Instead |
|-------|----------------|-------------|
| E0382 | "Clone it" | Who should own this data? |
| E0597 | "Extend lifetime" | Is the scope boundary correct? |
| E0506 | "End borrow first" | Should mutation happen elsewhere? |
| E0507 | "Clone before move" | Why are we moving from a reference? |
| E0515 | "Return owned" | Should caller own the data? |
| E0716 | "Bind to variable" | Why is this temporary? |
| E0106 | "Add 'a" | What is the actual lifetime relationship? |
---
## Thinking Prompt
Before fixing an ownership error, ask:
1. **What is this data's domain role?**
- Entity (unique identity) → owned
- Value Object (interchangeable) → clone/copy OK
- Temporary (computation result) → maybe restructure
2. **Is the ownership design intentional?**
- By design → work within constraints
- Accidental → consider redesign
3. **Fix symptom or redesign?**
- If Strike 3 (3rd attempt) → escalate to Layer 2
---
## Trace Up ↑
When errors persist, trace to design layer:
```
E0382 (moved value)
↑ Ask: What design choice led to this ownership pattern?
↑ Check: m09-domain (is this Entity or Value Object?)
↑ Check: domain-* (what constraints apply?)
```
| Persistent Error | Trace To | Question |
|-----------------|----------|----------|
| E0382 repeated | m02-resource | Should use Arc/Rc for sharing? |
| E0597 repeated | m09-domain | Is scope boundary at right place? |
| E0506/E0507 | m03-mutability | Should use interior mutability? |
---
## Trace Down ↓
From design decisions to implementation:
```
"Data needs to be shared immutably"
↓ Use: Arc<T> (multi-thread) or Rc<T> (single-thread)
"Data needs exclusive ownership"
↓ Use: move semantics, take ownership
"Data is read-only view"
↓ Use: &T (immutable borrow)
```
---
## Quick Reference
| Pattern | Ownership | Cost | Use When |
|---------|-----------|------|----------|
| Move | Transfer | Zero | Caller doesn't need data |
| `&T` | Borrow | Zero | Read-only access |
| `&mut T` | Exclusive borrow | Zero | Need to modify |
| `clone()` | Duplicate | Alloc + copy | Actually need a copy |
| `Rc<T>` | Shared (single) | Ref count | Single-thread sharing |
| `Arc<T>` | Shared (multi) | Atomic ref count | Multi-thread sharing |
| `Cow<T>` | Clone-on-write | Alloc if mutated | Might modify |
## Error Code Reference
| Error | Cause | Quick Fix |
|-------|-------|-----------|
| E0382 | Value moved | Clone, reference, or redesign ownership |
| E0597 | Reference outlives owner | Extend owner scope or restructure |
| E0506 | Assign while borrowed | End borrow before mutation |
| E0507 | Move out of borrowed | Clone or use reference |
| E0515 | Return local reference | Return owned value |
| E0716 | Temporary dropped | Bind to variable |
| E0106 | Missing lifetime | Add `'a` annotation |
---
## Anti-Patterns
| Anti-Pattern | Why Bad | Better |
|--------------|---------|--------|
| `.clone()` everywhere | Hides design issues | Design ownership properly |
| Fight borrow checker | Increases complexity | Work with the compiler |
| `'static` for everything | Restricts flexibility | Use appropriate lifetimes |
| Leak with `Box::leak` | Memory leak | Proper lifetime design |
---
## Related Skills
| When | See |
|------|-----|
| Need smart pointers | m02-resource |
| Need interior mutability | m03-mutability |
| Data is domain entity | m09-domain |
| Learning ownership concepts | m14-mental-model |
-222
View File
@@ -1,222 +0,0 @@
# Ownership: Comparison with Other Languages
## Rust vs C++
### Memory Management
| Aspect | Rust | C++ |
|--------|------|-----|
| Default | Move semantics | Copy semantics (pre-C++11) |
| Move | `let b = a;` (a invalidated) | `auto b = std::move(a);` (a valid but unspecified) |
| Copy | `let b = a.clone();` | `auto b = a;` |
| Safety | Compile-time enforcement | Runtime responsibility |
### Rust Move vs C++ Move
```rust
// Rust: after move, 'a' is INVALID
let a = String::from("hello");
let b = a; // a moved
// println!("{}", a); // COMPILE ERROR
// Equivalent in C++:
// std::string a = "hello";
// std::string b = std::move(a);
// std::cout << a; // UNDEFINED (compiles but buggy)
```
### Smart Pointers
| Rust | C++ | Purpose |
|------|-----|---------|
| `Box<T>` | `std::unique_ptr<T>` | Unique ownership |
| `Rc<T>` | `std::shared_ptr<T>` | Shared ownership |
| `Arc<T>` | `std::shared_ptr<T>` + atomic | Thread-safe shared |
| `RefCell<T>` | (manual runtime checks) | Interior mutability |
---
## Rust vs Go
### Memory Model
| Aspect | Rust | Go |
|--------|------|-----|
| Memory | Stack + heap, explicit | GC manages all |
| Ownership | Enforced at compile-time | None (GC handles) |
| Null | `Option<T>` | `nil` for pointers |
| Concurrency | `Send`/`Sync` traits | Channels (less strict) |
### Sharing Data
```rust
// Rust: explicit about sharing
use std::sync::Arc;
let data = Arc::new(vec![1, 2, 3]);
let data_clone = Arc::clone(&data);
std::thread::spawn(move || {
println!("{:?}", data_clone);
});
// Go: implicit sharing
// data := []int{1, 2, 3}
// go func() {
// fmt.Println(data) // potential race condition
// }()
```
### Why No GC in Rust
1. **Deterministic destruction**: Resources freed exactly when scope ends
2. **Zero-cost**: No GC pauses or overhead
3. **Embeddable**: Works in OS kernels, embedded systems
4. **Predictable latency**: Critical for real-time systems
---
## Rust vs Java/C#
### Reference Semantics
| Aspect | Rust | Java/C# |
|--------|------|---------|
| Objects | Owned by default | Reference by default |
| Null | `Option<T>` | `null` (nullable) |
| Immutability | Default | Must use `final`/`readonly` |
| Copy | Explicit `.clone()` | Reference copy (shallow) |
### Comparison
```rust
// Rust: clear ownership
fn process(data: Vec<i32>) { // takes ownership
// data is ours, will be freed at end
}
let numbers = vec![1, 2, 3];
process(numbers);
// numbers is invalid here
// Java: ambiguous ownership
// void process(List<Integer> data) {
// // Who owns data? Caller? Callee? Both?
// // Can caller still use it?
// }
```
---
## Rust vs Python
### Memory Model
| Aspect | Rust | Python |
|--------|------|--------|
| Typing | Static, compile-time | Dynamic, runtime |
| Memory | Ownership-based | Reference counting + GC |
| Mutability | Default immutable | Default mutable |
| Performance | Native, zero-cost | Interpreted, higher overhead |
### Common Pattern Translation
```rust
// Rust: borrowing iteration
let items = vec!["a", "b", "c"];
for item in &items {
println!("{}", item);
}
// items still usable
// Python: iteration doesn't consume
// items = ["a", "b", "c"]
// for item in items:
// print(item)
// items still usable (different reason - ref counting)
```
---
## Unique Rust Concepts
### Concepts Other Languages Lack
1. **Borrow Checker**: No other mainstream language has compile-time borrow checking
2. **Lifetimes**: Explicit annotation of reference validity
3. **Move by Default**: Values move, not copy
4. **No Null**: `Option<T>` instead of null pointers
5. **Affine Types**: Values can be used at most once
### Learning Curve Areas
| Concept | Coming From | Key Insight |
|---------|-------------|-------------|
| Ownership | GC languages | Think about who "owns" data |
| Borrowing | C/C++ | Like references but checked |
| Lifetimes | Any | Explicit scope of validity |
| Move | C++ | Move is default, not copy |
---
## Mental Model Shifts
### From GC Languages (Java, Go, Python)
```
Before: "Memory just works, GC handles it"
After: "I explicitly decide who owns data and when it's freed"
```
Key shifts:
- Think about ownership at design time
- Returning references requires lifetime thinking
- No more `null` - use `Option<T>`
### From C/C++
```
Before: "I manually manage memory and hope I get it right"
After: "Compiler enforces correctness, I fight the borrow checker"
```
Key shifts:
- Trust the compiler's errors
- Move is the default (unlike C++ copy)
- Smart pointers are idiomatic, not overhead
### From Functional Languages (Haskell, ML)
```
Before: "Everything is immutable, copying is fine"
After: "Mutability is explicit, ownership prevents aliasing"
```
Key shifts:
- Mutability is safe because of ownership rules
- No persistent data structures needed (usually)
- Performance characteristics are explicit
---
## Performance Trade-offs
| Language | Memory Overhead | Latency | Throughput |
|----------|-----------------|---------|------------|
| Rust | Minimal (no GC) | Predictable | Excellent |
| C++ | Minimal | Predictable | Excellent |
| Go | GC overhead | GC pauses | Good |
| Java | GC overhead | GC pauses | Good |
| Python | High (ref counting + GC) | Variable | Lower |
### When Rust Ownership Wins
1. **Real-time systems**: No GC pauses
2. **Embedded**: No runtime overhead
3. **High-performance**: Zero-cost abstractions
4. **Concurrent**: Data races prevented at compile time
### When GC Might Be Preferable
1. **Rapid prototyping**: Less mental overhead
2. **Complex object graphs**: Cycles are tricky in Rust
3. **GUI applications**: Object lifetimes are dynamic
4. **Small programs**: Overhead doesn't matter
@@ -1,339 +0,0 @@
# Ownership Best Practices
## API Design Patterns
### 1. Prefer Borrowing Over Ownership
```rust
// BAD: takes ownership unnecessarily
fn print_name(name: String) {
println!("Name: {}", name);
}
// GOOD: borrows instead
fn print_name(name: &str) {
println!("Name: {}", name);
}
// Caller benefits:
let name = String::from("Alice");
print_name(&name); // can reuse name
print_name(&name); // still valid
```
### 2. Return Owned Values from Constructors
```rust
// GOOD: return owned value
impl User {
fn new(name: &str) -> Self {
User {
name: name.to_string(),
}
}
}
// GOOD: accept Into<String> for flexibility
impl User {
fn new(name: impl Into<String>) -> Self {
User {
name: name.into(),
}
}
}
// Usage:
let u1 = User::new("Alice"); // &str
let u2 = User::new(String::from("Bob")); // String
```
### 3. Use AsRef for Generic Borrowing
```rust
// GOOD: accepts both &str and String
fn process<S: AsRef<str>>(input: S) {
let s = input.as_ref();
println!("{}", s);
}
process("literal"); // &str
process(String::from("owned")); // String
process(&String::from("ref")); // &String
```
### 4. Cow for Clone-on-Write
```rust
use std::borrow::Cow;
// Return borrowed when possible, owned when needed
fn maybe_modify(s: &str, uppercase: bool) -> Cow<'_, str> {
if uppercase {
Cow::Owned(s.to_uppercase()) // allocates
} else {
Cow::Borrowed(s) // zero-cost
}
}
let input = "hello";
let result = maybe_modify(input, false);
// result is borrowed, no allocation
```
---
## Struct Design Patterns
### 1. Owned Fields vs References
```rust
// Use owned fields for most cases
struct User {
name: String,
email: String,
}
// Use references only when lifetime is clear
struct UserView<'a> {
name: &'a str,
email: &'a str,
}
// Pattern: owned data + view for efficiency
impl User {
fn view(&self) -> UserView<'_> {
UserView {
name: &self.name,
email: &self.email,
}
}
}
```
### 2. Builder Pattern with Ownership
```rust
#[derive(Default)]
struct RequestBuilder {
url: Option<String>,
method: Option<String>,
body: Option<Vec<u8>>,
}
impl RequestBuilder {
fn new() -> Self {
Self::default()
}
// Take self by value for chaining
fn url(mut self, url: impl Into<String>) -> Self {
self.url = Some(url.into());
self
}
fn method(mut self, method: impl Into<String>) -> Self {
self.method = Some(method.into());
self
}
fn build(self) -> Result<Request, Error> {
Ok(Request {
url: self.url.ok_or(Error::MissingUrl)?,
method: self.method.unwrap_or_else(|| "GET".to_string()),
body: self.body.unwrap_or_default(),
})
}
}
// Usage:
let req = RequestBuilder::new()
.url("https://example.com")
.method("POST")
.build()?;
```
### 3. Interior Mutability When Needed
```rust
use std::cell::RefCell;
use std::rc::Rc;
// Shared mutable state in single-threaded context
struct Counter {
value: Rc<RefCell<u32>>,
}
impl Counter {
fn new() -> Self {
Counter {
value: Rc::new(RefCell::new(0)),
}
}
fn increment(&self) {
*self.value.borrow_mut() += 1;
}
fn get(&self) -> u32 {
*self.value.borrow()
}
fn clone_handle(&self) -> Self {
Counter {
value: Rc::clone(&self.value),
}
}
}
```
---
## Collection Patterns
### 1. Efficient Iteration
```rust
let items = vec![1, 2, 3, 4, 5];
// Iterate by reference (no move)
for item in &items {
println!("{}", item);
}
// Iterate by mutable reference
for item in &mut items.clone() {
*item *= 2;
}
// Consume with into_iter when done
let sum: i32 = items.into_iter().sum();
```
### 2. Collecting Results
```rust
// Collect into owned collection
let strings: Vec<String> = (0..5)
.map(|i| format!("item_{}", i))
.collect();
// Collect references
let refs: Vec<&str> = strings.iter().map(|s| s.as_str()).collect();
// Collect with transformation
let result: Result<Vec<i32>, _> = ["1", "2", "3"]
.iter()
.map(|s| s.parse::<i32>())
.collect();
```
### 3. Entry API for Maps
```rust
use std::collections::HashMap;
let mut map: HashMap<String, Vec<i32>> = HashMap::new();
// Efficient: don't search twice
map.entry("key".to_string())
.or_insert_with(Vec::new)
.push(42);
// With entry modification
map.entry("key".to_string())
.and_modify(|v| v.push(43))
.or_insert_with(|| vec![43]);
```
---
## Error Handling with Ownership
### 1. Preserve Context in Errors
```rust
use std::error::Error;
use std::fmt;
#[derive(Debug)]
struct ParseError {
input: String, // owns the problematic input
message: String,
}
impl fmt::Display for ParseError {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "Failed to parse '{}': {}", self.input, self.message)
}
}
fn parse(input: &str) -> Result<i32, ParseError> {
input.parse().map_err(|_| ParseError {
input: input.to_string(), // clone for error context
message: "not a valid integer".to_string(),
})
}
```
### 2. Ownership in Result Chains
```rust
fn process_data(path: &str) -> Result<ProcessedData, Error> {
let content = std::fs::read_to_string(path)?; // owned String
let parsed = parse_content(&content)?; // borrow
let processed = transform(parsed)?; // ownership moves
Ok(processed) // return owned
}
```
---
## Performance Considerations
### 1. Avoid Unnecessary Clones
```rust
// BAD: cloning just to compare
fn contains_item(items: &[String], target: &str) -> bool {
items.iter().any(|s| s.clone() == target) // unnecessary clone
}
// GOOD: compare references
fn contains_item(items: &[String], target: &str) -> bool {
items.iter().any(|s| s == target) // String implements PartialEq<str>
}
```
### 2. Use Slices for Flexibility
```rust
// BAD: requires Vec
fn sum(numbers: &Vec<i32>) -> i32 {
numbers.iter().sum()
}
// GOOD: accepts any slice
fn sum(numbers: &[i32]) -> i32 {
numbers.iter().sum()
}
// Now works with:
sum(&vec![1, 2, 3]); // Vec
sum(&[1, 2, 3]); // array
sum(&array[1..3]); // slice
```
### 3. In-Place Mutation
```rust
// BAD: allocates new String
fn make_uppercase(s: &str) -> String {
s.to_uppercase()
}
// GOOD when you own the data: mutate in place
fn make_uppercase(mut s: String) -> String {
s.make_ascii_uppercase(); // in-place for ASCII
s
}
```
@@ -1,265 +0,0 @@
# Common Ownership Errors & Fixes
## E0382: Use of Moved Value
### Error Pattern
```rust
let s = String::from("hello");
let s2 = s; // s moved here
println!("{}", s); // ERROR: value borrowed after move
```
### Fix Options
**Option 1: Clone (if ownership not needed)**
```rust
let s = String::from("hello");
let s2 = s.clone(); // s is cloned
println!("{}", s); // OK: s still valid
```
**Option 2: Borrow (if modification not needed)**
```rust
let s = String::from("hello");
let s2 = &s; // borrow, not move
println!("{}", s); // OK
println!("{}", s2); // OK
```
**Option 3: Use Rc/Arc (for shared ownership)**
```rust
use std::rc::Rc;
let s = Rc::new(String::from("hello"));
let s2 = Rc::clone(&s); // shared ownership
println!("{}", s); // OK
println!("{}", s2); // OK
```
---
## E0597: Borrowed Value Does Not Live Long Enough
### Error Pattern
```rust
fn get_str() -> &str {
let s = String::from("hello");
&s // ERROR: s dropped here, but reference returned
}
```
### Fix Options
**Option 1: Return owned value**
```rust
fn get_str() -> String {
String::from("hello") // return owned value
}
```
**Option 2: Use 'static lifetime**
```rust
fn get_str() -> &'static str {
"hello" // string literal has 'static lifetime
}
```
**Option 3: Accept reference parameter**
```rust
fn get_str<'a>(s: &'a str) -> &'a str {
s // return reference with same lifetime as input
}
```
---
## E0499: Cannot Borrow as Mutable More Than Once
### Error Pattern
```rust
let mut s = String::from("hello");
let r1 = &mut s;
let r2 = &mut s; // ERROR: second mutable borrow
println!("{}, {}", r1, r2);
```
### Fix Options
**Option 1: Sequential borrows**
```rust
let mut s = String::from("hello");
{
let r1 = &mut s;
r1.push_str(" world");
} // r1 goes out of scope
let r2 = &mut s; // OK: r1 no longer exists
```
**Option 2: Use RefCell for interior mutability**
```rust
use std::cell::RefCell;
let s = RefCell::new(String::from("hello"));
let mut r1 = s.borrow_mut();
// drop r1 before borrowing again
drop(r1);
let mut r2 = s.borrow_mut();
```
---
## E0502: Cannot Borrow as Mutable While Immutable Borrow Exists
### Error Pattern
```rust
let mut v = vec![1, 2, 3];
let first = &v[0]; // immutable borrow
v.push(4); // ERROR: mutable borrow while immutable exists
println!("{}", first);
```
### Fix Options
**Option 1: Finish using immutable borrow first**
```rust
let mut v = vec![1, 2, 3];
let first = v[0]; // copy value, not borrow
v.push(4); // OK
println!("{}", first); // OK: using copied value
```
**Option 2: Clone before mutating**
```rust
let mut v = vec![1, 2, 3];
let first = v[0].clone(); // if T: Clone
v.push(4);
println!("{}", first);
```
---
## E0507: Cannot Move Out of Borrowed Content
### Error Pattern
```rust
fn take_string(s: &String) {
let moved = *s; // ERROR: cannot move out of borrowed content
}
```
### Fix Options
**Option 1: Clone**
```rust
fn take_string(s: &String) {
let cloned = s.clone();
}
```
**Option 2: Take ownership in function signature**
```rust
fn take_string(s: String) { // take ownership
let moved = s;
}
```
**Option 3: Use mem::take for Option/Default types**
```rust
fn take_from_option(opt: &mut Option<String>) -> Option<String> {
std::mem::take(opt) // replaces with None, returns owned value
}
```
---
## E0515: Return Local Reference
### Error Pattern
```rust
fn create_string() -> &String {
let s = String::from("hello");
&s // ERROR: cannot return reference to local variable
}
```
### Fix Options
**Option 1: Return owned value**
```rust
fn create_string() -> String {
String::from("hello")
}
```
**Option 2: Use static/const**
```rust
fn get_static_str() -> &'static str {
"hello"
}
```
---
## E0716: Temporary Value Dropped While Borrowed
### Error Pattern
```rust
let r: &str = &String::from("hello"); // ERROR: temporary dropped
println!("{}", r);
```
### Fix Options
**Option 1: Bind to variable first**
```rust
let s = String::from("hello");
let r: &str = &s;
println!("{}", r);
```
**Option 2: Use let binding with reference**
```rust
let r: &str = {
let s = String::from("hello");
// s.as_str() // ERROR: still temporary
Box::leak(s.into_boxed_str()) // extreme: leak for 'static
};
```
---
## Pattern: Loop Ownership Issues
### Error Pattern
```rust
let strings = vec![String::from("a"), String::from("b")];
for s in strings {
println!("{}", s);
}
// ERROR: strings moved into loop
println!("{:?}", strings);
```
### Fix Options
**Option 1: Iterate by reference**
```rust
let strings = vec![String::from("a"), String::from("b")];
for s in &strings {
println!("{}", s);
}
println!("{:?}", strings); // OK
```
**Option 2: Use iter()**
```rust
for s in strings.iter() {
println!("{}", s);
}
```
**Option 3: Clone if needed**
```rust
for s in strings.clone() {
// consumes cloned vec
}
println!("{:?}", strings); // original still available
```
@@ -1,229 +0,0 @@
# Lifetime Patterns
## Basic Lifetime Annotation
### When Required
```rust
// ERROR: missing lifetime specifier
fn longest(x: &str, y: &str) -> &str {
if x.len() > y.len() { x } else { y }
}
// FIX: explicit lifetime
fn longest<'a>(x: &'a str, y: &'a str) -> &'a str {
if x.len() > y.len() { x } else { y }
}
```
### Lifetime Elision Rules
1. Each input reference gets its own lifetime
2. If one input lifetime, output uses same
3. If `&self` or `&mut self`, output uses self's lifetime
```rust
// These are equivalent (elision applies):
fn first_word(s: &str) -> &str { ... }
fn first_word<'a>(s: &'a str) -> &'a str { ... }
// Method with self (elision applies):
impl MyStruct {
fn get_ref(&self) -> &str { ... }
// Equivalent to:
fn get_ref<'a>(&'a self) -> &'a str { ... }
}
```
---
## Struct Lifetimes
### Struct Holding References
```rust
// Struct must declare lifetime for references
struct Excerpt<'a> {
part: &'a str,
}
impl<'a> Excerpt<'a> {
fn level(&self) -> i32 { 3 }
// Return reference tied to self's lifetime
fn get_part(&self) -> &str {
self.part
}
}
```
### Multiple Lifetimes in Struct
```rust
struct Multi<'a, 'b> {
x: &'a str,
y: &'b str,
}
// Use when references may have different lifetimes
fn make_multi<'a, 'b>(x: &'a str, y: &'b str) -> Multi<'a, 'b> {
Multi { x, y }
}
```
---
## 'static Lifetime
### When to Use
```rust
// String literals are 'static
let s: &'static str = "hello";
// Owned data can be leaked to 'static
let leaked: &'static str = Box::leak(String::from("hello").into_boxed_str());
// Thread spawn requires 'static or move
std::thread::spawn(move || {
// closure owns data, satisfies 'static
});
```
### Avoid Overusing 'static
```rust
// BAD: requires 'static unnecessarily
fn process(s: &'static str) { ... }
// GOOD: use generic lifetime
fn process<'a>(s: &'a str) { ... }
// or
fn process(s: &str) { ... } // lifetime elision
```
---
## Higher-Ranked Trait Bounds (HRTB)
### for<'a> Syntax
```rust
// Function that works with any lifetime
fn apply_to_ref<F>(f: F)
where
F: for<'a> Fn(&'a str) -> &'a str,
{
let s = String::from("hello");
let result = f(&s);
println!("{}", result);
}
```
### Common Use: Closure Bounds
```rust
// Closure that borrows any lifetime
fn filter_refs<F>(items: &[&str], pred: F) -> Vec<&str>
where
F: for<'a> Fn(&'a str) -> bool,
{
items.iter().copied().filter(|s| pred(s)).collect()
}
```
---
## Lifetime Bounds
### 'a: 'b (Outlives)
```rust
// 'a must live at least as long as 'b
fn coerce<'a, 'b>(x: &'a str) -> &'b str
where
'a: 'b,
{
x
}
```
### T: 'a (Type Outlives Lifetime)
```rust
// T must live at least as long as 'a
struct Wrapper<'a, T: 'a> {
value: &'a T,
}
// Common pattern with trait objects
fn use_trait<'a, T: MyTrait + 'a>(t: &'a T) { ... }
```
---
## Common Lifetime Mistakes
### Mistake 1: Returning Reference to Local
```rust
// WRONG
fn dangle() -> &String {
let s = String::from("hello");
&s // s dropped, reference invalid
}
// RIGHT
fn no_dangle() -> String {
String::from("hello")
}
```
### Mistake 2: Conflicting Lifetimes
```rust
// WRONG: might return reference to y which has shorter lifetime
fn wrong<'a, 'b>(x: &'a str, y: &'b str) -> &'a str {
y // ERROR: 'b might not live as long as 'a
}
// RIGHT: use same lifetime or add bound
fn right<'a>(x: &'a str, y: &'a str) -> &'a str {
y // OK: both have lifetime 'a
}
```
### Mistake 3: Struct Outlives Reference
```rust
// WRONG: s might outlive the string it references
let r;
{
let s = String::from("hello");
r = Excerpt { part: &s }; // ERROR
}
println!("{}", r.part); // s already dropped
// RIGHT: ensure source outlives struct
let s = String::from("hello");
let r = Excerpt { part: &s };
println!("{}", r.part); // OK: s still in scope
```
---
## Subtyping and Variance
### Covariance
```rust
// &'a T is covariant in 'a
// Can use &'long where &'short expected
fn example<'short, 'long: 'short>(long_ref: &'long str) {
let short_ref: &'short str = long_ref; // OK: covariance
}
```
### Invariance
```rust
// &'a mut T is invariant in 'a
fn example<'a, 'b>(x: &'a mut &'b str, y: &'b str) {
*x = y; // ERROR if 'a and 'b are different
}
```
### Practical Impact
```rust
// This works due to covariance
fn accept_any<'a>(s: &'a str) { ... }
let s = String::from("hello");
let long_lived: &str = &s;
accept_any(long_lived); // 'long coerces to 'short
```
-158
View File
@@ -1,158 +0,0 @@
---
name: m02-resource
description: "CRITICAL: Use for smart pointers and resource management. Triggers: Box, Rc, Arc, Weak, RefCell, Cell, smart pointer, heap allocation, reference counting, RAII, Drop, should I use Box or Rc, when to use Arc vs Rc, 智能指针, 引用计数, 堆分配"
---
# Resource Management
> **Layer 1: Language Mechanics**
## Core Question
**What ownership pattern does this resource need?**
Before choosing a smart pointer, understand:
- Is ownership single or shared?
- Is access single-threaded or multi-threaded?
- Are there potential cycles?
---
## Error → Design Question
| Error | Don't Just Say | Ask Instead |
|-------|----------------|-------------|
| "Need heap allocation" | "Use Box" | Why can't this be on stack? |
| Rc memory leak | "Use Weak" | Is the cycle necessary in design? |
| RefCell panic | "Use try_borrow" | Is runtime check the right approach? |
| Arc overhead complaint | "Accept it" | Is multi-thread access actually needed? |
---
## Thinking Prompt
Before choosing a smart pointer:
1. **What's the ownership model?**
- Single owner → Box or owned value
- Shared ownership → Rc/Arc
- Weak reference → Weak
2. **What's the thread context?**
- Single-thread → Rc, Cell, RefCell
- Multi-thread → Arc, Mutex, RwLock
3. **Are there cycles?**
- Yes → One direction must be Weak
- No → Regular Rc/Arc is fine
---
## Trace Up ↑
When pointer choice is unclear, trace to design:
```
"Should I use Arc or Rc?"
↑ Ask: Is this data shared across threads?
↑ Check: m07-concurrency (thread model)
↑ Check: domain-* (performance constraints)
```
| Situation | Trace To | Question |
|-----------|----------|----------|
| Rc vs Arc confusion | m07-concurrency | What's the concurrency model? |
| RefCell panics | m03-mutability | Is interior mutability right here? |
| Memory leaks | m12-lifecycle | Where should cleanup happen? |
---
## Trace Down ↓
From design to implementation:
```
"Need single-owner heap data"
↓ Use: Box<T>
"Need shared immutable data (single-thread)"
↓ Use: Rc<T>
"Need shared immutable data (multi-thread)"
↓ Use: Arc<T>
"Need to break reference cycle"
↓ Use: Weak<T>
"Need shared mutable data"
↓ Single-thread: Rc<RefCell<T>>
↓ Multi-thread: Arc<Mutex<T>> or Arc<RwLock<T>>
```
---
## Quick Reference
| Type | Ownership | Thread-Safe | Use When |
|------|-----------|-------------|----------|
| `Box<T>` | Single | Yes | Heap allocation, recursive types |
| `Rc<T>` | Shared | No | Single-thread shared ownership |
| `Arc<T>` | Shared | Yes | Multi-thread shared ownership |
| `Weak<T>` | Weak ref | Same as Rc/Arc | Break reference cycles |
| `Cell<T>` | Single | No | Interior mutability (Copy types) |
| `RefCell<T>` | Single | No | Interior mutability (runtime check) |
## Decision Flowchart
```
Need heap allocation?
├─ Yes → Single owner?
│ ├─ Yes → Box<T>
│ └─ No → Multi-thread?
│ ├─ Yes → Arc<T>
│ └─ No → Rc<T>
└─ No → Stack allocation (default)
Have reference cycles?
├─ Yes → Use Weak for one direction
└─ No → Regular Rc/Arc
Need interior mutability?
├─ Yes → Thread-safe needed?
│ ├─ Yes → Mutex<T> or RwLock<T>
│ └─ No → T: Copy? → Cell<T> : RefCell<T>
└─ No → Use &mut T
```
---
## Common Errors
| Problem | Cause | Fix |
|---------|-------|-----|
| Rc cycle leak | Mutual strong refs | Use Weak for one direction |
| RefCell panic | Borrow conflict at runtime | Use try_borrow or restructure |
| Arc overhead | Atomic ops in hot path | Consider Rc if single-threaded |
| Box unnecessary | Data fits on stack | Remove Box |
---
## Anti-Patterns
| Anti-Pattern | Why Bad | Better |
|--------------|---------|--------|
| Arc everywhere | Unnecessary atomic overhead | Use Rc for single-thread |
| RefCell everywhere | Runtime panics | Design clear ownership |
| Box for small types | Unnecessary allocation | Stack allocation |
| Ignore Weak for cycles | Memory leaks | Design parent-child with Weak |
---
## Related Skills
| When | See |
|------|-----|
| Ownership errors | m01-ownership |
| Interior mutability details | m03-mutability |
| Multi-thread context | m07-concurrency |
| Resource lifecycle | m12-lifecycle |
-152
View File
@@ -1,152 +0,0 @@
---
name: m03-mutability
description: "CRITICAL: Use for mutability issues. Triggers: E0596, E0499, E0502, cannot borrow as mutable, already borrowed as immutable, mut, &mut, interior mutability, Cell, RefCell, Mutex, RwLock, 可变性, 内部可变性, 借用冲突"
---
# Mutability
> **Layer 1: Language Mechanics**
## Core Question
**Why does this data need to change, and who can change it?**
Before adding interior mutability, understand:
- Is mutation essential or accidental complexity?
- Who should control mutation?
- Is the mutation pattern safe?
---
## Error → Design Question
| Error | Don't Just Say | Ask Instead |
|-------|----------------|-------------|
| E0596 | "Add mut" | Should this really be mutable? |
| E0499 | "Split borrows" | Is the data structure right? |
| E0502 | "Separate scopes" | Why do we need both borrows? |
| RefCell panic | "Use try_borrow" | Is runtime check appropriate? |
---
## Thinking Prompt
Before adding mutability:
1. **Is mutation necessary?**
- Maybe transform → return new value
- Maybe builder → construct immutably
2. **Who controls mutation?**
- External caller → `&mut T`
- Internal logic → interior mutability
- Concurrent access → synchronized mutability
3. **What's the thread context?**
- Single-thread → Cell/RefCell
- Multi-thread → Mutex/RwLock/Atomic
---
## Trace Up ↑
When mutability conflicts persist:
```
E0499/E0502 (borrow conflicts)
↑ Ask: Is the data structure designed correctly?
↑ Check: m09-domain (should data be split?)
↑ Check: m07-concurrency (is async involved?)
```
| Persistent Error | Trace To | Question |
|-----------------|----------|----------|
| Repeated borrow conflicts | m09-domain | Should data be restructured? |
| RefCell in async | m07-concurrency | Is Send/Sync needed? |
| Mutex deadlocks | m07-concurrency | Is the lock design right? |
---
## Trace Down ↓
From design to implementation:
```
"Need mutable access from &self"
↓ T: Copy → Cell<T>
↓ T: !Copy → RefCell<T>
"Need thread-safe mutation"
↓ Simple counters → AtomicXxx
↓ Complex data → Mutex<T> or RwLock<T>
"Need shared mutable state"
↓ Single-thread: Rc<RefCell<T>>
↓ Multi-thread: Arc<Mutex<T>>
```
---
## Borrow Rules
```
At any time, you can have EITHER:
├─ Multiple &T (immutable borrows)
└─ OR one &mut T (mutable borrow)
Never both simultaneously.
```
## Quick Reference
| Pattern | Thread-Safe | Runtime Cost | Use When |
|---------|-------------|--------------|----------|
| `&mut T` | N/A | Zero | Exclusive mutable access |
| `Cell<T>` | No | Zero | Copy types, no refs needed |
| `RefCell<T>` | No | Runtime check | Non-Copy, need runtime borrow |
| `Mutex<T>` | Yes | Lock contention | Thread-safe mutation |
| `RwLock<T>` | Yes | Lock contention | Many readers, few writers |
| `Atomic*` | Yes | Minimal | Simple types (bool, usize) |
## Error Code Reference
| Error | Cause | Quick Fix |
|-------|-------|-----------|
| E0596 | Borrowing immutable as mutable | Add `mut` or redesign |
| E0499 | Multiple mutable borrows | Restructure code flow |
| E0502 | &mut while & exists | Separate borrow scopes |
---
## Interior Mutability Decision
| Scenario | Choose |
|----------|--------|
| T: Copy, single-thread | `Cell<T>` |
| T: !Copy, single-thread | `RefCell<T>` |
| T: Copy, multi-thread | `AtomicXxx` |
| T: !Copy, multi-thread | `Mutex<T>` or `RwLock<T>` |
| Read-heavy, multi-thread | `RwLock<T>` |
| Simple flags/counters | `AtomicBool`, `AtomicUsize` |
---
## Anti-Patterns
| Anti-Pattern | Why Bad | Better |
|--------------|---------|--------|
| RefCell everywhere | Runtime panics | Clear ownership design |
| Mutex for single-thread | Unnecessary overhead | RefCell |
| Ignore RefCell panic | Hard to debug | Handle or restructure |
| Lock inside hot loop | Performance killer | Batch operations |
---
## Related Skills
| When | See |
|------|-----|
| Smart pointer choice | m02-resource |
| Thread safety | m07-concurrency |
| Data structure design | m09-domain |
| Anti-patterns | m15-anti-pattern |
-164
View File
@@ -1,164 +0,0 @@
---
name: m04-zero-cost
description: "CRITICAL: Use for generics, traits, zero-cost abstraction. Triggers: E0277, E0308, E0599, generic, trait, impl, dyn, where, monomorphization, static dispatch, dynamic dispatch, impl Trait, trait bound not satisfied, 泛型, 特征, 零成本抽象, 单态化"
---
# Zero-Cost Abstraction
> **Layer 1: Language Mechanics**
## Core Question
**Do we need compile-time or runtime polymorphism?**
Before choosing between generics and trait objects:
- Is the type known at compile time?
- Is a heterogeneous collection needed?
- What's the performance priority?
---
## Error → Design Question
| Error | Don't Just Say | Ask Instead |
|-------|----------------|-------------|
| E0277 | "Add trait bound" | Is this abstraction at the right level? |
| E0308 | "Fix the type" | Should types be unified or distinct? |
| E0599 | "Import the trait" | Is the trait the right abstraction? |
| E0038 | "Make object-safe" | Do we really need dynamic dispatch? |
---
## Thinking Prompt
Before adding trait bounds:
1. **What abstraction is needed?**
- Same behavior, different types → trait
- Different behavior, same type → enum
- No abstraction needed → concrete type
2. **When is type known?**
- Compile time → generics (static dispatch)
- Runtime → trait objects (dynamic dispatch)
3. **What's the trade-off priority?**
- Performance → generics
- Compile time → trait objects
- Flexibility → depends
---
## Trace Up ↑
When type system fights back:
```
E0277 (trait bound not satisfied)
↑ Ask: Is the abstraction level correct?
↑ Check: m09-domain (what behavior is being abstracted?)
↑ Check: m05-type-driven (should use newtype?)
```
| Persistent Error | Trace To | Question |
|-----------------|----------|----------|
| Complex trait bounds | m09-domain | Is the abstraction right? |
| Object safety issues | m05-type-driven | Can typestate help? |
| Type explosion | m10-performance | Accept dyn overhead? |
---
## Trace Down ↓
From design to implementation:
```
"Need to abstract over types with same behavior"
↓ Types known at compile time → impl Trait or generics
↓ Types determined at runtime → dyn Trait
"Need collection of different types"
↓ Closed set → enum
↓ Open set → Vec<Box<dyn Trait>>
"Need to return different types"
↓ Same type → impl Trait
↓ Different types → Box<dyn Trait>
```
---
## Quick Reference
| Pattern | Dispatch | Code Size | Runtime Cost |
|---------|----------|-----------|--------------|
| `fn foo<T: Trait>()` | Static | +bloat | Zero |
| `fn foo(x: &dyn Trait)` | Dynamic | Minimal | vtable lookup |
| `impl Trait` return | Static | +bloat | Zero |
| `Box<dyn Trait>` | Dynamic | Minimal | Allocation + vtable |
## Syntax Comparison
```rust
// Static dispatch - type known at compile time
fn process(x: impl Display) { } // argument position
fn process<T: Display>(x: T) { } // explicit generic
fn get() -> impl Display { } // return position
// Dynamic dispatch - type determined at runtime
fn process(x: &dyn Display) { } // reference
fn process(x: Box<dyn Display>) { } // owned
```
## Error Code Reference
| Error | Cause | Quick Fix |
|-------|-------|-----------|
| E0277 | Type doesn't impl trait | Add impl or change bound |
| E0308 | Type mismatch | Check generic params |
| E0599 | No method found | Import trait with `use` |
| E0038 | Trait not object-safe | Use generics or redesign |
---
## Decision Guide
| Scenario | Choose | Why |
|----------|--------|-----|
| Performance critical | Generics | Zero runtime cost |
| Heterogeneous collection | `dyn Trait` | Different types at runtime |
| Plugin architecture | `dyn Trait` | Unknown types at compile |
| Reduce compile time | `dyn Trait` | Less monomorphization |
| Small, known type set | `enum` | No indirection |
---
## Object Safety
A trait is object-safe if it:
- Doesn't have `Self: Sized` bound
- Doesn't return `Self`
- Doesn't have generic methods
- Uses `where Self: Sized` for non-object-safe methods
---
## Anti-Patterns
| Anti-Pattern | Why Bad | Better |
|--------------|---------|--------|
| Over-generic everything | Compile time, complexity | Concrete types when possible |
| `dyn` for known types | Unnecessary indirection | Generics |
| Complex trait hierarchies | Hard to understand | Simpler design |
| Ignore object safety | Limits flexibility | Plan for dyn if needed |
---
## Related Skills
| When | See |
|------|-----|
| Type-driven design | m05-type-driven |
| Domain abstraction | m09-domain |
| Performance concerns | m10-performance |
| Send/Sync bounds | m07-concurrency |
-174
View File
@@ -1,174 +0,0 @@
---
name: m05-type-driven
description: "CRITICAL: Use for type-driven design. Triggers: type state, PhantomData, newtype, marker trait, builder pattern, make invalid states unrepresentable, compile-time validation, sealed trait, ZST, 类型状态, 新类型模式, 类型驱动设计"
---
# Type-Driven Design
> **Layer 1: Language Mechanics**
## Core Question
**How can the type system prevent invalid states?**
Before reaching for runtime checks:
- Can the compiler catch this error?
- Can invalid states be unrepresentable?
- Can the type encode the invariant?
---
## Error → Design Question
| Pattern | Don't Just Say | Ask Instead |
|---------|----------------|-------------|
| Primitive obsession | "It's just a string" | What does this value represent? |
| Boolean flags | "Add an is_valid flag" | Can states be types? |
| Optional everywhere | "Check for None" | Is absence really possible? |
| Validation at runtime | "Return Err if invalid" | Can we validate at construction? |
---
## Thinking Prompt
Before adding runtime validation:
1. **Can the type encode the constraint?**
- Numeric range → bounded types or newtypes
- Valid states → type state pattern
- Semantic meaning → newtype
2. **When is validation possible?**
- At construction → validated newtype
- At state transition → type state
- Only at runtime → Result with clear error
3. **Who needs to know the invariant?**
- Compiler → type-level encoding
- API users → clear type signatures
- Runtime only → documentation
---
## Trace Up ↑
When type design is unclear:
```
"Need to validate email format"
↑ Ask: Is this a domain value object?
↑ Check: m09-domain (Email as Value Object)
↑ Check: domain-* (validation requirements)
```
| Situation | Trace To | Question |
|-----------|----------|----------|
| What types to create | m09-domain | What's the domain model? |
| State machine design | m09-domain | What are valid transitions? |
| Marker trait usage | m04-zero-cost | Static or dynamic dispatch? |
---
## Trace Down ↓
From design to implementation:
```
"Need type-safe wrapper for primitives"
↓ Newtype: struct UserId(u64);
"Need compile-time state validation"
↓ Type State: Connection<Connected>
"Need to track phantom type parameters"
↓ PhantomData: PhantomData<T>
"Need capability markers"
↓ Marker Trait: trait Validated {}
"Need gradual construction"
↓ Builder: Builder::new().field(x).build()
```
---
## Quick Reference
| Pattern | Purpose | Example |
|---------|---------|---------|
| Newtype | Type safety | `struct UserId(u64);` |
| Type State | State machine | `Connection<Connected>` |
| PhantomData | Variance/lifetime | `PhantomData<&'a T>` |
| Marker Trait | Capability flag | `trait Validated {}` |
| Builder | Gradual construction | `Builder::new().name("x").build()` |
| Sealed Trait | Prevent external impl | `mod private { pub trait Sealed {} }` |
## Pattern Examples
### Newtype
```rust
struct Email(String); // Not just any string
impl Email {
pub fn new(s: &str) -> Result<Self, ValidationError> {
// Validate once, trust forever
validate_email(s)?;
Ok(Self(s.to_string()))
}
}
```
### Type State
```rust
struct Connection<State>(TcpStream, PhantomData<State>);
struct Disconnected;
struct Connected;
struct Authenticated;
impl Connection<Disconnected> {
fn connect(self) -> Connection<Connected> { ... }
}
impl Connection<Connected> {
fn authenticate(self) -> Connection<Authenticated> { ... }
}
```
---
## Decision Guide
| Need | Pattern |
|------|---------|
| Type safety for primitives | Newtype |
| Compile-time state validation | Type State |
| Lifetime/variance markers | PhantomData |
| Capability flags | Marker Trait |
| Gradual construction | Builder |
| Closed set of impls | Sealed Trait |
| Zero-sized type marker | ZST struct |
---
## Anti-Patterns
| Anti-Pattern | Why Bad | Better |
|--------------|---------|--------|
| Boolean flags for states | Runtime errors | Type state |
| String for semantic types | No type safety | Newtype |
| Option for uninitialized | Unclear invariant | Builder |
| Public fields with invariants | Invariant violation | Private + validated new() |
---
## Related Skills
| When | See |
|------|-----|
| Domain modeling | m09-domain |
| Trait design | m04-zero-cost |
| Error handling in constructors | m06-error-handling |
| Anti-patterns | m15-anti-pattern |
-165
View File
@@ -1,165 +0,0 @@
---
name: m06-error-handling
description: "CRITICAL: Use for error handling. Triggers: Result, Option, Error, ?, unwrap, expect, panic, anyhow, thiserror, when to panic vs return Result, custom error, error propagation, 错误处理, Result 用法, 什么时候用 panic"
---
# Error Handling
> **Layer 1: Language Mechanics**
## Core Question
**Is this failure expected or a bug?**
Before choosing error handling strategy:
- Can this fail in normal operation?
- Who should handle this failure?
- What context does the caller need?
---
## Error → Design Question
| Pattern | Don't Just Say | Ask Instead |
|---------|----------------|-------------|
| unwrap panics | "Use ?" | Is None/Err actually possible here? |
| Type mismatch on ? | "Use anyhow" | Are error types designed correctly? |
| Lost error context | "Add .context()" | What does the caller need to know? |
| Too many error variants | "Use Box<dyn Error>" | Is error granularity right? |
---
## Thinking Prompt
Before handling an error:
1. **What kind of failure is this?**
- Expected → Result<T, E>
- Absence normal → Option<T>
- Bug/invariant → panic!
- Unrecoverable → panic!
2. **Who handles this?**
- Caller → propagate with ?
- Current function → match/if-let
- User → friendly error message
- Programmer → panic with message
3. **What context is needed?**
- Type of error → thiserror variants
- Call chain → anyhow::Context
- Debug info → anyhow or tracing
---
## Trace Up ↑
When error strategy is unclear:
```
"Should I return Result or Option?"
↑ Ask: Is absence/failure normal or exceptional?
↑ Check: m09-domain (what does domain say?)
↑ Check: domain-* (error handling requirements)
```
| Situation | Trace To | Question |
|-----------|----------|----------|
| Too many unwraps | m09-domain | Is the data model right? |
| Error context design | m13-domain-error | What recovery is needed? |
| Library vs app errors | m11-ecosystem | Who are the consumers? |
---
## Trace Down ↓
From design to implementation:
```
"Expected failure, library code"
↓ Use: thiserror for typed errors
"Expected failure, application code"
↓ Use: anyhow for ergonomic errors
"Absence is normal (find, get, lookup)"
↓ Use: Option<T>
"Bug or invariant violation"
↓ Use: panic!, assert!, unreachable!
"Need to propagate with context"
↓ Use: .context("what was happening")
```
---
## Quick Reference
| Pattern | When | Example |
|---------|------|---------|
| `Result<T, E>` | Recoverable error | `fn read() -> Result<String, io::Error>` |
| `Option<T>` | Absence is normal | `fn find() -> Option<&Item>` |
| `?` | Propagate error | `let data = file.read()?;` |
| `unwrap()` | Dev/test only | `config.get("key").unwrap()` |
| `expect()` | Invariant holds | `env.get("HOME").expect("HOME set")` |
| `panic!` | Unrecoverable | `panic!("critical failure")` |
## Library vs Application
| Context | Error Crate | Why |
|---------|-------------|-----|
| Library | `thiserror` | Typed errors for consumers |
| Application | `anyhow` | Ergonomic error handling |
| Mixed | Both | thiserror at boundaries, anyhow internally |
## Decision Flowchart
```
Is failure expected?
├─ Yes → Is absence the only "failure"?
│ ├─ Yes → Option<T>
│ └─ No → Result<T, E>
│ ├─ Library → thiserror
│ └─ Application → anyhow
└─ No → Is it a bug?
├─ Yes → panic!, assert!
└─ No → Consider if really unrecoverable
Use ? → Need context?
├─ Yes → .context("message")
└─ No → Plain ?
```
---
## Common Errors
| Error | Cause | Fix |
|-------|-------|-----|
| `unwrap()` panic | Unhandled None/Err | Use `?` or match |
| Type mismatch | Different error types | Use `anyhow` or `From` |
| Lost context | `?` without context | Add `.context()` |
| `cannot use ?` | Missing Result return | Return `Result<(), E>` |
---
## Anti-Patterns
| Anti-Pattern | Why Bad | Better |
|--------------|---------|--------|
| `.unwrap()` everywhere | Panics in production | `.expect("reason")` or `?` |
| Ignore errors silently | Bugs hidden | Handle or propagate |
| `panic!` for expected errors | Bad UX, no recovery | Result |
| Box<dyn Error> everywhere | Lost type info | thiserror |
---
## Related Skills
| When | See |
|------|-----|
| Domain error strategy | m13-domain-error |
| Crate boundaries | m11-ecosystem |
| Type-safe errors | m05-type-driven |
| Mental models | m14-mental-model |
@@ -1,332 +0,0 @@
# Error Handling: Library vs Application
## Library Error Design
### Principles
1. **Define specific error types** - Don't use `anyhow` in libraries
2. **Implement std::error::Error** - For compatibility
3. **Provide error variants** - Let users match on errors
4. **Include source errors** - Enable error chains
5. **Be `Send + Sync`** - For async compatibility
### Example: Library Error Type
```rust
// lib.rs
use thiserror::Error;
#[derive(Error, Debug)]
pub enum DatabaseError {
#[error("connection failed: {host}:{port}")]
ConnectionFailed {
host: String,
port: u16,
#[source]
source: std::io::Error,
},
#[error("query failed: {query}")]
QueryFailed {
query: String,
#[source]
source: SqlError,
},
#[error("record not found: {table}.{id}")]
NotFound { table: String, id: String },
#[error("constraint violation: {0}")]
ConstraintViolation(String),
}
// Public Result alias
pub type Result<T> = std::result::Result<T, DatabaseError>;
// Library functions
pub fn connect(host: &str, port: u16) -> Result<Connection> {
// ...
}
pub fn query(conn: &Connection, sql: &str) -> Result<Rows> {
// ...
}
```
### Library Usage of Errors
```rust
impl Database {
pub fn get_user(&self, id: &str) -> Result<User> {
let rows = self.query(&format!("SELECT * FROM users WHERE id = '{}'", id))?;
rows.first()
.cloned()
.ok_or_else(|| DatabaseError::NotFound {
table: "users".to_string(),
id: id.to_string(),
})
}
}
```
---
## Application Error Design
### Principles
1. **Use anyhow for convenience** - Or custom unified error
2. **Add context liberally** - Help debugging
3. **Log at boundaries** - Don't log in libraries
4. **Convert to user-friendly messages** - For display
### Example: Application Error Handling
```rust
// main.rs
use anyhow::{Context, Result};
use tracing::{error, info};
async fn run_server() -> Result<()> {
let config = load_config()
.context("failed to load configuration")?;
let db = Database::connect(&config.db_url)
.await
.context("failed to connect to database")?;
let server = Server::new(config.port)
.context("failed to create server")?;
info!("Server starting on port {}", config.port);
server.run(db).await
.context("server error")?;
Ok(())
}
#[tokio::main]
async fn main() {
tracing_subscriber::init();
if let Err(e) = run_server().await {
error!("Application error: {:#}", e);
std::process::exit(1);
}
}
```
### Converting Library Errors
```rust
use mylib::DatabaseError;
async fn get_user_handler(id: &str) -> Result<Response> {
match db.get_user(id).await {
Ok(user) => Ok(Response::json(user)),
Err(DatabaseError::NotFound { .. }) => {
Ok(Response::not_found("User not found"))
}
Err(DatabaseError::ConnectionFailed { .. }) => {
error!("Database connection failed");
Ok(Response::internal_error("Service unavailable"))
}
Err(e) => {
error!("Database error: {}", e);
Err(e.into()) // Convert to anyhow::Error
}
}
}
```
---
## Error Handling Layers
```
┌─────────────────────────────────────┐
│ Application Layer │
│ - Use anyhow or unified error │
│ - Add context at boundaries │
│ - Log errors │
│ - Convert to user messages │
└─────────────────────────────────────┘
│ calls
┌─────────────────────────────────────┐
│ Service Layer │
│ - Map between error types │
│ - Add business context │
│ - Handle recoverable errors │
└─────────────────────────────────────┘
│ calls
┌─────────────────────────────────────┐
│ Library Layer │
│ - Define specific error types │
│ - Use thiserror │
│ - Include source errors │
│ - No logging │
└─────────────────────────────────────┘
```
---
## Practical Examples
### HTTP API Error Response
```rust
use axum::{response::IntoResponse, http::StatusCode};
use serde::Serialize;
#[derive(Serialize)]
struct ErrorResponse {
error: String,
code: String,
}
enum AppError {
NotFound(String),
BadRequest(String),
Internal(anyhow::Error),
}
impl IntoResponse for AppError {
fn into_response(self) -> axum::response::Response {
let (status, error, code) = match self {
AppError::NotFound(msg) => {
(StatusCode::NOT_FOUND, msg, "NOT_FOUND")
}
AppError::BadRequest(msg) => {
(StatusCode::BAD_REQUEST, msg, "BAD_REQUEST")
}
AppError::Internal(e) => {
tracing::error!("Internal error: {:#}", e);
(
StatusCode::INTERNAL_SERVER_ERROR,
"Internal server error".to_string(),
"INTERNAL_ERROR",
)
}
};
let body = ErrorResponse {
error,
code: code.to_string(),
};
(status, axum::Json(body)).into_response()
}
}
```
### CLI Error Handling
```rust
use anyhow::{Context, Result};
use clap::Parser;
#[derive(Parser)]
struct Args {
#[arg(short, long)]
config: String,
}
fn main() {
if let Err(e) = run() {
eprintln!("Error: {:#}", e);
std::process::exit(1);
}
}
fn run() -> Result<()> {
let args = Args::parse();
let config = std::fs::read_to_string(&args.config)
.context(format!("Failed to read config file: {}", args.config))?;
let parsed: Config = toml::from_str(&config)
.context("Failed to parse config file")?;
process(parsed)?;
println!("Done!");
Ok(())
}
```
---
## Testing Error Handling
### Testing Error Cases
```rust
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_not_found_error() {
let result = db.get_user("nonexistent");
assert!(matches!(
result,
Err(DatabaseError::NotFound { table, id })
if table == "users" && id == "nonexistent"
));
}
#[test]
fn test_error_message() {
let err = DatabaseError::NotFound {
table: "users".to_string(),
id: "123".to_string(),
};
assert_eq!(err.to_string(), "record not found: users.123");
}
#[test]
fn test_error_chain() {
let io_err = std::io::Error::new(
std::io::ErrorKind::ConnectionRefused,
"connection refused"
);
let err = DatabaseError::ConnectionFailed {
host: "localhost".to_string(),
port: 5432,
source: io_err,
};
// Check source is preserved
assert!(err.source().is_some());
}
}
```
### Testing with anyhow
```rust
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_with_context() -> anyhow::Result<()> {
let result = process("valid input")?;
assert_eq!(result, expected);
Ok(())
}
#[test]
fn test_error_context() {
let err = process("invalid")
.context("processing failed")
.unwrap_err();
// Check error chain contains expected text
let chain = format!("{:#}", err);
assert!(chain.contains("processing failed"));
}
}
```
@@ -1,404 +0,0 @@
# Error Handling Patterns
## The ? Operator
### Basic Usage
```rust
fn read_config() -> Result<Config, io::Error> {
let content = std::fs::read_to_string("config.toml")?;
let config: Config = toml::from_str(&content)?; // needs From impl
Ok(config)
}
```
### With Different Error Types
```rust
use std::error::Error;
// Box<dyn Error> for quick prototyping
fn process() -> Result<(), Box<dyn Error>> {
let file = std::fs::read_to_string("data.txt")?;
let num: i32 = file.trim().parse()?; // different error type
Ok(())
}
```
### Custom Conversion with From
```rust
#[derive(Debug)]
enum MyError {
Io(std::io::Error),
Parse(std::num::ParseIntError),
}
impl From<std::io::Error> for MyError {
fn from(err: std::io::Error) -> Self {
MyError::Io(err)
}
}
impl From<std::num::ParseIntError> for MyError {
fn from(err: std::num::ParseIntError) -> Self {
MyError::Parse(err)
}
}
fn process() -> Result<i32, MyError> {
let content = std::fs::read_to_string("num.txt")?; // auto-converts
let num: i32 = content.trim().parse()?; // auto-converts
Ok(num)
}
```
---
## Error Type Design
### Simple Enum Error
```rust
#[derive(Debug, Clone, PartialEq)]
pub enum ConfigError {
NotFound,
InvalidFormat,
MissingField(String),
}
impl std::fmt::Display for ConfigError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
ConfigError::NotFound => write!(f, "configuration file not found"),
ConfigError::InvalidFormat => write!(f, "invalid configuration format"),
ConfigError::MissingField(field) => write!(f, "missing field: {}", field),
}
}
}
impl std::error::Error for ConfigError {}
```
### Error with Source (Wrapping)
```rust
#[derive(Debug)]
pub struct AppError {
kind: AppErrorKind,
source: Option<Box<dyn std::error::Error + Send + Sync>>,
}
#[derive(Debug, Clone, Copy)]
pub enum AppErrorKind {
Config,
Database,
Network,
}
impl std::fmt::Display for AppError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self.kind {
AppErrorKind::Config => write!(f, "configuration error"),
AppErrorKind::Database => write!(f, "database error"),
AppErrorKind::Network => write!(f, "network error"),
}
}
}
impl std::error::Error for AppError {
fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
self.source.as_ref().map(|e| e.as_ref() as _)
}
}
```
---
## Using thiserror
### Basic Usage
```rust
use thiserror::Error;
#[derive(Error, Debug)]
pub enum DataError {
#[error("file not found: {path}")]
NotFound { path: String },
#[error("invalid data format")]
InvalidFormat,
#[error("IO error")]
Io(#[from] std::io::Error),
#[error("parse error: {0}")]
Parse(#[from] std::num::ParseIntError),
}
// Usage
fn load_data(path: &str) -> Result<Data, DataError> {
let content = std::fs::read_to_string(path)
.map_err(|_| DataError::NotFound { path: path.to_string() })?;
let num: i32 = content.trim().parse()?; // auto-converts with #[from]
Ok(Data { value: num })
}
```
### Transparent Wrapper
```rust
use thiserror::Error;
#[derive(Error, Debug)]
#[error(transparent)]
pub struct MyError(#[from] InnerError);
// Useful for newtype error wrappers
```
---
## Using anyhow
### For Applications
```rust
use anyhow::{Context, Result, bail, ensure};
fn process_file(path: &str) -> Result<Data> {
let content = std::fs::read_to_string(path)
.context("failed to read config file")?;
ensure!(!content.is_empty(), "config file is empty");
let data: Data = serde_json::from_str(&content)
.context("failed to parse JSON")?;
if data.version < 1 {
bail!("unsupported config version: {}", data.version);
}
Ok(data)
}
fn main() -> Result<()> {
let data = process_file("config.json")
.context("failed to load configuration")?;
Ok(())
}
```
### Error Chain
```rust
use anyhow::{Context, Result};
fn deep_function() -> Result<()> {
std::fs::read_to_string("missing.txt")
.context("failed to read file")?;
Ok(())
}
fn middle_function() -> Result<()> {
deep_function()
.context("failed in deep function")?;
Ok(())
}
fn top_function() -> Result<()> {
middle_function()
.context("failed in middle function")?;
Ok(())
}
// Error output shows full chain:
// Error: failed in middle function
// Caused by:
// 0: failed in deep function
// 1: failed to read file
// 2: No such file or directory (os error 2)
```
---
## Option Handling
### Converting Option to Result
```rust
fn find_user(id: u32) -> Option<User> { ... }
// Using ok_or for static error
fn get_user(id: u32) -> Result<User, &'static str> {
find_user(id).ok_or("user not found")
}
// Using ok_or_else for dynamic error
fn get_user(id: u32) -> Result<User, String> {
find_user(id).ok_or_else(|| format!("user {} not found", id))
}
```
### Chaining Options
```rust
fn get_nested_value(data: &Data) -> Option<&str> {
data.config
.as_ref()?
.nested
.as_ref()?
.value
.as_deref()
}
// Equivalent with and_then
fn get_nested_value(data: &Data) -> Option<&str> {
data.config
.as_ref()
.and_then(|c| c.nested.as_ref())
.and_then(|n| n.value.as_deref())
}
```
---
## Pattern: Result Combinators
### map and map_err
```rust
fn parse_port(s: &str) -> Result<u16, ParseError> {
s.parse::<u16>()
.map_err(|e| ParseError::InvalidPort(e))
}
fn get_url(config: &Config) -> Result<String, Error> {
config.url()
.map(|u| format!("https://{}", u))
}
```
### and_then (flatMap)
```rust
fn validate_and_save(input: &str) -> Result<(), Error> {
validate(input)
.and_then(|valid| save(valid))
.and_then(|saved| notify(saved))
}
```
### unwrap_or and unwrap_or_else
```rust
// Default value
let port = config.port().unwrap_or(8080);
// Computed default
let port = config.port().unwrap_or_else(|| find_free_port());
// Default for Result
let data = load_data().unwrap_or_default();
```
---
## Pattern: Early Return vs Combinators
### Early Return Style
```rust
fn process(input: &str) -> Result<Output, Error> {
let step1 = validate(input)?;
if !step1.is_valid {
return Err(Error::Invalid);
}
let step2 = transform(step1)?;
let step3 = save(step2)?;
Ok(step3)
}
```
### Combinator Style
```rust
fn process(input: &str) -> Result<Output, Error> {
validate(input)
.and_then(|s| {
if s.is_valid {
Ok(s)
} else {
Err(Error::Invalid)
}
})
.and_then(transform)
.and_then(save)
}
```
### When to Use Which
| Style | Best For |
|-------|----------|
| Early return (`?`) | Most cases, clearer flow |
| Combinators | Functional pipelines, one-liners |
| Match | Complex branching on errors |
---
## Panic vs Result
### When to Panic
```rust
// 1. Unrecoverable programmer error
fn get_config() -> &'static Config {
CONFIG.get().expect("config must be initialized")
}
// 2. In tests
#[test]
fn test_parsing() {
let result = parse("valid").unwrap(); // OK in tests
assert_eq!(result, expected);
}
// 3. Prototype/examples
fn main() {
let data = load().unwrap(); // OK for quick examples
}
```
### When to Return Result
```rust
// 1. Any I/O operation
fn read_file(path: &str) -> Result<String, io::Error>
// 2. User input validation
fn parse_port(s: &str) -> Result<u16, ParseError>
// 3. Network operations
async fn fetch(url: &str) -> Result<Response, Error>
// 4. Anything that can fail at runtime
fn connect(addr: &str) -> Result<Connection, Error>
```
---
## Error Context Best Practices
### Add Context at Boundaries
```rust
fn load_user_config(user_id: u64) -> Result<Config, Error> {
let path = format!("/home/{}/config.toml", user_id);
std::fs::read_to_string(&path)
.context(format!("failed to read config for user {}", user_id))?
// NOT: .context("failed to read file") // too generic
// ...
}
```
### Include Relevant Data
```rust
// Good: includes the problematic value
fn parse_age(s: &str) -> Result<u8, Error> {
s.parse()
.context(format!("invalid age value: '{}'", s))
}
// Bad: no context about what failed
fn parse_age(s: &str) -> Result<u8, Error> {
s.parse()
.context("parse error")
}
```
-221
View File
@@ -1,221 +0,0 @@
---
name: m07-concurrency
description: "CRITICAL: Use for concurrency/async. Triggers: E0277 Send Sync, cannot be sent between threads, thread, spawn, channel, mpsc, Mutex, RwLock, Atomic, async, await, Future, tokio, deadlock, race condition, 并发, 线程, 异步, 死锁"
---
# Concurrency
> **Layer 1: Language Mechanics**
## Core Question
**Is this CPU-bound or I/O-bound, and what's the sharing model?**
Before choosing concurrency primitives:
- What's the workload type?
- What data needs to be shared?
- What's the thread safety requirement?
---
## Error → Design Question
| Error | Don't Just Say | Ask Instead |
|-------|----------------|-------------|
| E0277 Send | "Add Send bound" | Should this type cross threads? |
| E0277 Sync | "Wrap in Mutex" | Is shared access really needed? |
| Future not Send | "Use spawn_local" | Is async the right choice? |
| Deadlock | "Reorder locks" | Is the locking design correct? |
---
## Thinking Prompt
Before adding concurrency:
1. **What's the workload?**
- CPU-bound → threads (std::thread, rayon)
- I/O-bound → async (tokio, async-std)
- Mixed → hybrid approach
2. **What's the sharing model?**
- No sharing → message passing (channels)
- Immutable sharing → Arc<T>
- Mutable sharing → Arc<Mutex<T>> or Arc<RwLock<T>>
3. **What are the Send/Sync requirements?**
- Cross-thread ownership → Send
- Cross-thread references → Sync
- Single-thread async → spawn_local
---
## Trace Up ↑ (MANDATORY)
**CRITICAL**: Don't just fix the error. Trace UP to find domain constraints.
### Domain Detection Table
| Context Keywords | Load Domain Skill | Key Constraint |
|-----------------|-------------------|----------------|
| Web API, HTTP, axum, actix, handler | **domain-web** | Handlers run on any thread |
| 交易, 支付, trading, payment | **domain-fintech** | Audit + thread safety |
| gRPC, kubernetes, microservice | **domain-cloud-native** | Distributed tracing |
| CLI, terminal, clap | **domain-cli** | Usually single-thread OK |
### Example: Web API + Rc Error
```
"Rc cannot be sent between threads" in Web API context
↑ DETECT: "Web API" → Load domain-web
↑ FIND: domain-web says "Shared state must be thread-safe"
↑ FIND: domain-web says "Rc in state" is Common Mistake
↓ DESIGN: Use Arc<T> with State extractor
↓ IMPL: axum::extract::State<Arc<AppConfig>>
```
### Generic Trace
```
"Send not satisfied for my type"
↑ Ask: What domain is this? Load domain-* skill
↑ Ask: Does this type need to cross thread boundaries?
↑ Check: m09-domain (is the data model correct?)
```
| Situation | Trace To | Question |
|-----------|----------|----------|
| Send/Sync in Web | **domain-web** | What's the state management pattern? |
| Send/Sync in CLI | **domain-cli** | Is multi-thread really needed? |
| Mutex vs channels | m09-domain | Shared state or message passing? |
| Async vs threads | m10-performance | What's the workload profile? |
---
## Trace Down ↓
From design to implementation:
```
"Need parallelism for CPU work"
↓ Use: std::thread or rayon
"Need concurrency for I/O"
↓ Use: async/await with tokio
"Need to share immutable data across threads"
↓ Use: Arc<T>
"Need to share mutable data across threads"
↓ Use: Arc<Mutex<T>> or Arc<RwLock<T>>
↓ Or: channels for message passing
"Need simple atomic operations"
↓ Use: AtomicBool, AtomicUsize, etc.
```
---
## Send/Sync Markers
| Marker | Meaning | Example |
|--------|---------|---------|
| `Send` | Can transfer ownership between threads | Most types |
| `Sync` | Can share references between threads | `Arc<T>` |
| `!Send` | Must stay on one thread | `Rc<T>` |
| `!Sync` | No shared refs across threads | `RefCell<T>` |
## Quick Reference
| Pattern | Thread-Safe | Blocking | Use When |
|---------|-------------|----------|----------|
| `std::thread` | Yes | Yes | CPU-bound parallelism |
| `async/await` | Yes | No | I/O-bound concurrency |
| `Mutex<T>` | Yes | Yes | Shared mutable state |
| `RwLock<T>` | Yes | Yes | Read-heavy shared state |
| `mpsc::channel` | Yes | Optional | Message passing |
| `Arc<Mutex<T>>` | Yes | Yes | Shared mutable across threads |
## Decision Flowchart
```
What type of work?
├─ CPU-bound → std::thread or rayon
├─ I/O-bound → async/await
└─ Mixed → hybrid (spawn_blocking)
Need to share data?
├─ No → message passing (channels)
├─ Immutable → Arc<T>
└─ Mutable →
├─ Read-heavy → Arc<RwLock<T>>
└─ Write-heavy → Arc<Mutex<T>>
└─ Simple counter → AtomicUsize
Async context?
├─ Type is Send → tokio::spawn
├─ Type is !Send → spawn_local
└─ Blocking code → spawn_blocking
```
---
## Common Errors
| Error | Cause | Fix |
|-------|-------|-----|
| E0277 `Send` not satisfied | Non-Send in async | Use Arc or spawn_local |
| E0277 `Sync` not satisfied | Non-Sync shared | Wrap with Mutex |
| Deadlock | Lock ordering | Consistent lock order |
| `future is not Send` | Non-Send across await | Drop before await |
| `MutexGuard` across await | Guard held during suspend | Scope guard properly |
---
## Anti-Patterns
| Anti-Pattern | Why Bad | Better |
|--------------|---------|--------|
| Arc<Mutex<T>> everywhere | Contention, complexity | Message passing |
| thread::sleep in async | Blocks executor | tokio::time::sleep |
| Holding locks across await | Blocks other tasks | Scope locks tightly |
| Ignoring deadlock risk | Hard to debug | Lock ordering, try_lock |
---
## Async-Specific Patterns
### Avoid MutexGuard Across Await
```rust
// Bad: guard held across await
let guard = mutex.lock().await;
do_async().await; // guard still held!
// Good: scope the lock
{
let guard = mutex.lock().await;
// use guard
} // guard dropped
do_async().await;
```
### Non-Send Types in Async
```rust
// Rc is !Send, can't cross await in spawned task
// Option 1: use Arc instead
// Option 2: use spawn_local (single-thread runtime)
// Option 3: ensure Rc is dropped before .await
```
---
## Related Skills
| When | See |
|------|-----|
| Smart pointer choice | m02-resource |
| Interior mutability | m03-mutability |
| Performance tuning | m10-performance |
| Domain concurrency needs | domain-* |
@@ -1,312 +0,0 @@
# Concurrency: Comparison with Other Languages
## Rust vs Go
### Concurrency Model
| Aspect | Rust | Go |
|--------|------|-----|
| Model | Ownership + Send/Sync | CSP (Communicating Sequential Processes) |
| Primitives | Arc, Mutex, channels | goroutines, channels |
| Safety | Compile-time | Runtime (race detector) |
| Async | async/await + runtime | Built-in scheduler |
### Goroutines vs Rust Tasks
```rust
// Rust: explicit about thread safety
use std::sync::Arc;
use tokio::sync::Mutex;
let data = Arc::new(Mutex::new(vec![]));
let data_clone = Arc::clone(&data);
tokio::spawn(async move {
let mut guard = data_clone.lock().await;
guard.push(1); // Safe: Mutex protects access
});
// Go: implicit sharing (potential race)
// data := []int{}
// go func() {
// data = append(data, 1) // RACE CONDITION!
// }()
```
### Channel Comparison
```rust
// Rust: typed channels with ownership
use tokio::sync::mpsc;
let (tx, mut rx) = mpsc::channel::<String>(100);
tokio::spawn(async move {
tx.send("hello".to_string()).await.unwrap();
// tx is moved, can't be used elsewhere
});
// Go: channels are more flexible but less safe
// ch := make(chan string, 100)
// go func() {
// ch <- "hello"
// // ch can still be used anywhere
// }()
```
---
## Rust vs Java
### Thread Safety Model
| Aspect | Rust | Java |
|--------|------|------|
| Safety | Compile-time (Send/Sync) | Runtime (synchronized, volatile) |
| Null | No null (Option) | NullPointerException risk |
| Locks | RAII (drop releases) | try-finally or try-with-resources |
| Memory | No GC | GC with stop-the-world |
### Synchronization Comparison
```rust
// Rust: lock is tied to data
use std::sync::Mutex;
let data = Mutex::new(vec![1, 2, 3]);
{
let mut guard = data.lock().unwrap();
guard.push(4);
} // lock released automatically
// Java: lock and data are separate
// List<Integer> data = new ArrayList<>();
// synchronized(data) {
// data.add(4);
// } // easy to forget synchronization elsewhere
```
### Thread Pool Comparison
```rust
// Rust: rayon for data parallelism
use rayon::prelude::*;
let sum: i32 = (0..1000)
.into_par_iter()
.map(|x| x * x)
.sum();
// Java: Stream API
// int sum = IntStream.range(0, 1000)
// .parallel()
// .map(x -> x * x)
// .sum();
```
---
## Rust vs C++
### Safety Guarantees
| Aspect | Rust | C++ |
|--------|------|-----|
| Data races | Prevented at compile-time | Undefined behavior |
| Deadlocks | Not prevented (same as C++) | Not prevented |
| Thread safety | Send/Sync traits | Convention only |
| Memory ordering | Explicit Ordering enum | memory_order enum |
### Atomic Comparison
```rust
// Rust: clear memory ordering
use std::sync::atomic::{AtomicI32, Ordering};
let counter = AtomicI32::new(0);
counter.fetch_add(1, Ordering::SeqCst);
let value = counter.load(Ordering::Acquire);
// C++: similar but without safety
// std::atomic<int> counter{0};
// counter.fetch_add(1, std::memory_order_seq_cst);
// int value = counter.load(std::memory_order_acquire);
```
### Mutex Comparison
```rust
// Rust: data protected by Mutex
use std::sync::Mutex;
struct SafeCounter {
count: Mutex<i32>, // Mutex contains the data
}
impl SafeCounter {
fn increment(&self) {
*self.count.lock().unwrap() += 1;
}
}
// C++: mutex separate from data (error-prone)
// class Counter {
// std::mutex mtx;
// int count; // NOT protected by type system
// public:
// void increment() {
// std::lock_guard<std::mutex> lock(mtx);
// count++;
// }
// void unsafe_increment() {
// count++; // Compiles! But wrong.
// }
// };
```
---
## Async Models Comparison
| Language | Model | Runtime |
|----------|-------|---------|
| Rust | async/await, zero-cost | tokio, async-std (bring your own) |
| Go | goroutines | Built-in scheduler |
| JavaScript | async/await, Promises | Event loop (single-threaded) |
| Python | async/await | asyncio (single-threaded) |
| Java | CompletableFuture, Virtual Threads | ForkJoinPool, Loom |
### Rust vs JavaScript Async
```rust
// Rust: async requires explicit runtime, can use multiple threads
#[tokio::main]
async fn main() {
let results = tokio::join!(
fetch("url1"), // runs concurrently
fetch("url2"),
);
}
// JavaScript: single-threaded event loop
// async function main() {
// const results = await Promise.all([
// fetch("url1"),
// fetch("url2"),
// ]);
// }
```
### Rust vs Python Async
```rust
// Rust: true parallelism possible
#[tokio::main(flavor = "multi_thread")]
async fn main() {
let handles: Vec<_> = urls
.into_iter()
.map(|url| tokio::spawn(fetch(url))) // spawns on thread pool
.collect();
for handle in handles {
let _ = handle.await;
}
}
// Python: asyncio is single-threaded (use ProcessPoolExecutor for CPU)
# async def main():
# tasks = [asyncio.create_task(fetch(url)) for url in urls]
# await asyncio.gather(*tasks) # all on same thread
```
---
## Send and Sync: Rust's Unique Feature
No other mainstream language has compile-time thread safety markers:
| Trait | Meaning | Auto-impl |
|-------|---------|-----------|
| `Send` | Safe to transfer between threads | Most types |
| `Sync` | Safe to share `&T` between threads | Types with thread-safe `&` |
| `!Send` | Must stay on one thread | Rc, raw pointers |
| `!Sync` | References can't be shared | RefCell, Cell |
### Why This Matters
```rust
// Rust PREVENTS this at compile time:
use std::rc::Rc;
let rc = Rc::new(42);
std::thread::spawn(move || {
println!("{}", rc); // ERROR: Rc is not Send
});
// In other languages, this would be a runtime bug:
// - Go: race detector might catch it
// - Java: undefined behavior
// - Python: GIL usually saves you
// - C++: undefined behavior
```
---
## Performance Characteristics
| Aspect | Rust | Go | Java | C++ |
|--------|------|-----|------|-----|
| Thread overhead | System threads or M:N | M:N (goroutines) | System or virtual | System threads |
| Context switch | OS-level or cooperative | Cheap (goroutines) | OS-level | OS-level |
| Memory | Predictable (no GC) | GC pauses | GC pauses | Predictable |
| Async overhead | Zero-cost futures | Runtime overhead | Boxing overhead | Depends |
### When to Use What
| Scenario | Best Choice |
|----------|-------------|
| CPU-bound parallelism | Rust (rayon), C++ |
| I/O-bound concurrency | Rust (tokio), Go, Node.js |
| Low latency required | Rust, C++ |
| Rapid development | Go, Python |
| Complex concurrent state | Rust (compile-time safety) |
---
## Mental Model Shifts
### From Go
```
Before: "Just use goroutines and channels"
After: "Explicitly declare what can be shared and how"
```
Key shifts:
- `Arc<Mutex<T>>` instead of implicit sharing
- Compiler enforces thread safety
- Async needs explicit runtime
### From Java
```
Before: "synchronized everywhere, hope for the best"
After: "Types encode thread safety, compiler enforces"
```
Key shifts:
- No need for synchronized keyword
- Mutex contains data, not separate
- No GC pauses in critical sections
### From C++
```
Before: "Be careful, read the docs, use sanitizers"
After: "Compiler catches data races, trust the type system"
```
Key shifts:
- Send/Sync replace convention
- RAII locks are mandatory, not optional
- Much harder to write incorrect concurrent code
@@ -1,396 +0,0 @@
# Thread-Based Concurrency Patterns
## Thread Spawning Best Practices
### Basic Thread Spawn
```rust
use std::thread;
fn main() {
let handle = thread::spawn(|| {
println!("Hello from thread!");
42 // return value
});
let result = handle.join().unwrap();
println!("Thread returned: {}", result);
}
```
### Named Threads for Debugging
```rust
use std::thread;
let builder = thread::Builder::new()
.name("worker-1".to_string())
.stack_size(32 * 1024); // 32KB stack
let handle = builder.spawn(|| {
println!("Thread name: {:?}", thread::current().name());
}).unwrap();
```
### Scoped Threads (No 'static Required)
```rust
use std::thread;
fn process_data(data: &[u32]) -> Vec<u32> {
thread::scope(|s| {
let handles: Vec<_> = data
.chunks(2)
.map(|chunk| {
s.spawn(|| {
chunk.iter().map(|x| x * 2).collect::<Vec<_>>()
})
})
.collect();
handles
.into_iter()
.flat_map(|h| h.join().unwrap())
.collect()
})
}
fn main() {
let data = vec![1, 2, 3, 4, 5, 6];
let result = process_data(&data); // No 'static needed!
println!("{:?}", result);
}
```
---
## Shared State Patterns
### Arc + Mutex (Read-Write)
```rust
use std::sync::{Arc, Mutex};
use std::thread;
fn shared_counter() {
let counter = Arc::new(Mutex::new(0));
let mut handles = vec![];
for _ in 0..10 {
let counter = Arc::clone(&counter);
let handle = thread::spawn(move || {
let mut num = counter.lock().unwrap();
*num += 1;
});
handles.push(handle);
}
for handle in handles {
handle.join().unwrap();
}
println!("Result: {}", *counter.lock().unwrap());
}
```
### Arc + RwLock (Read-Heavy)
```rust
use std::sync::{Arc, RwLock};
use std::thread;
fn read_heavy_cache() {
let cache = Arc::new(RwLock::new(vec![1, 2, 3]));
// Many readers
for i in 0..5 {
let cache = Arc::clone(&cache);
thread::spawn(move || {
let data = cache.read().unwrap();
println!("Reader {}: {:?}", i, *data);
});
}
// Occasional writer
{
let cache = Arc::clone(&cache);
thread::spawn(move || {
let mut data = cache.write().unwrap();
data.push(4);
println!("Writer: added element");
});
}
}
```
### Atomic for Simple Types
```rust
use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::Arc;
use std::thread;
fn atomic_counter() {
let counter = Arc::new(AtomicUsize::new(0));
let mut handles = vec![];
for _ in 0..10 {
let counter = Arc::clone(&counter);
handles.push(thread::spawn(move || {
for _ in 0..1000 {
counter.fetch_add(1, Ordering::SeqCst);
}
}));
}
for handle in handles {
handle.join().unwrap();
}
println!("Result: {}", counter.load(Ordering::SeqCst));
}
```
---
## Channel Patterns
### MPSC Channel
```rust
use std::sync::mpsc;
use std::thread;
fn producer_consumer() {
let (tx, rx) = mpsc::channel();
// Multiple producers
for i in 0..3 {
let tx = tx.clone();
thread::spawn(move || {
for j in 0..5 {
tx.send(format!("msg {}-{}", i, j)).unwrap();
}
});
}
drop(tx); // Drop original sender
// Single consumer
for received in rx {
println!("Got: {}", received);
}
}
```
### Sync Channel (Bounded)
```rust
use std::sync::mpsc;
use std::thread;
fn bounded_channel() {
let (tx, rx) = mpsc::sync_channel(2); // buffer size 2
thread::spawn(move || {
for i in 0..5 {
println!("Sending {}", i);
tx.send(i).unwrap(); // blocks if buffer full
println!("Sent {}", i);
}
});
thread::sleep(std::time::Duration::from_millis(500));
for received in rx {
println!("Received: {}", received);
thread::sleep(std::time::Duration::from_millis(100));
}
}
```
---
## Thread Pool Patterns
### Using rayon for Parallel Iteration
```rust
use rayon::prelude::*;
fn parallel_map() {
let numbers: Vec<i32> = (0..1000).collect();
let squares: Vec<i32> = numbers
.par_iter() // parallel iterator
.map(|x| x * x)
.collect();
println!("Processed {} items", squares.len());
}
fn parallel_filter_map() {
let data: Vec<String> = get_data();
let results: Vec<_> = data
.par_iter()
.filter(|s| !s.is_empty())
.map(|s| expensive_process(s))
.collect();
}
```
### Custom Thread Pool with crossbeam
```rust
use crossbeam::channel;
use std::thread;
fn custom_pool(num_workers: usize) {
let (tx, rx) = channel::bounded::<Box<dyn FnOnce() + Send>>(100);
// Spawn workers
let workers: Vec<_> = (0..num_workers)
.map(|_| {
let rx = rx.clone();
thread::spawn(move || {
while let Ok(task) = rx.recv() {
task();
}
})
})
.collect();
// Submit tasks
for i in 0..100 {
tx.send(Box::new(move || {
println!("Processing task {}", i);
})).unwrap();
}
drop(tx); // Close channel
for worker in workers {
worker.join().unwrap();
}
}
```
---
## Synchronization Primitives
### Barrier (Wait for All)
```rust
use std::sync::{Arc, Barrier};
use std::thread;
fn barrier_example() {
let barrier = Arc::new(Barrier::new(3));
let mut handles = vec![];
for i in 0..3 {
let barrier = Arc::clone(&barrier);
handles.push(thread::spawn(move || {
println!("Thread {} starting", i);
thread::sleep(std::time::Duration::from_millis(i as u64 * 100));
barrier.wait(); // All threads wait here
println!("Thread {} after barrier", i);
}));
}
for handle in handles {
handle.join().unwrap();
}
}
```
### Condvar (Condition Variable)
```rust
use std::sync::{Arc, Condvar, Mutex};
use std::thread;
fn condvar_example() {
let pair = Arc::new((Mutex::new(false), Condvar::new()));
let pair_clone = Arc::clone(&pair);
// Waiter thread
let waiter = thread::spawn(move || {
let (lock, cvar) = &*pair_clone;
let mut started = lock.lock().unwrap();
while !*started {
started = cvar.wait(started).unwrap();
}
println!("Waiter: condition met!");
});
// Notifier
thread::sleep(std::time::Duration::from_millis(100));
let (lock, cvar) = &*pair;
{
let mut started = lock.lock().unwrap();
*started = true;
}
cvar.notify_one();
waiter.join().unwrap();
}
```
### Once (One-Time Initialization)
```rust
use std::sync::Once;
static INIT: Once = Once::new();
static mut CONFIG: Option<Config> = None;
fn get_config() -> &'static Config {
INIT.call_once(|| {
unsafe {
CONFIG = Some(load_config());
}
});
unsafe { CONFIG.as_ref().unwrap() }
}
// Better: use once_cell or lazy_static
use once_cell::sync::Lazy;
static CONFIG: Lazy<Config> = Lazy::new(|| {
load_config()
});
```
---
## Error Handling in Threads
### Handling Panics
```rust
use std::thread;
fn handle_panic() {
let handle = thread::spawn(|| {
panic!("Thread panicked!");
});
match handle.join() {
Ok(_) => println!("Thread completed successfully"),
Err(e) => {
if let Some(s) = e.downcast_ref::<&str>() {
println!("Thread panicked with: {}", s);
} else if let Some(s) = e.downcast_ref::<String>() {
println!("Thread panicked with: {}", s);
} else {
println!("Thread panicked with unknown error");
}
}
}
}
```
### Catching Panics
```rust
use std::panic;
fn catch_panic() {
let result = panic::catch_unwind(|| {
risky_operation()
});
match result {
Ok(value) => println!("Success: {:?}", value),
Err(_) => println!("Operation panicked, continuing..."),
}
}
```
@@ -1,409 +0,0 @@
# Async Patterns in Rust
## Task Spawning
### Basic Spawn
```rust
use tokio::task;
#[tokio::main]
async fn main() {
// Spawn a task that runs concurrently
let handle = task::spawn(async {
expensive_computation().await
});
// Do other work while task runs
other_work().await;
// Wait for result
let result = handle.await.unwrap();
}
```
### Spawn with Shared State
```rust
use std::sync::Arc;
use tokio::sync::Mutex;
async fn process_with_state() {
let state = Arc::new(Mutex::new(vec![]));
let handles: Vec<_> = (0..10)
.map(|i| {
let state = Arc::clone(&state);
tokio::spawn(async move {
let mut guard = state.lock().await;
guard.push(i);
})
})
.collect();
// Wait for all tasks
for handle in handles {
handle.await.unwrap();
}
}
```
---
## Select Pattern
### Racing Multiple Futures
```rust
use tokio::select;
use tokio::time::{sleep, Duration};
async fn first_response() {
select! {
result = fetch_from_server_a() => {
println!("A responded first: {:?}", result);
}
result = fetch_from_server_b() => {
println!("B responded first: {:?}", result);
}
}
}
```
### Select with Timeout
```rust
use tokio::time::timeout;
async fn with_timeout() -> Result<Data, Error> {
select! {
result = fetch_data() => result,
_ = sleep(Duration::from_secs(5)) => {
Err(Error::Timeout)
}
}
}
// Or use timeout directly
async fn with_timeout2() -> Result<Data, Error> {
timeout(Duration::from_secs(5), fetch_data())
.await
.map_err(|_| Error::Timeout)?
}
```
### Select with Channel
```rust
use tokio::sync::mpsc;
async fn process_messages(mut rx: mpsc::Receiver<Message>) {
loop {
select! {
Some(msg) = rx.recv() => {
handle_message(msg).await;
}
_ = tokio::signal::ctrl_c() => {
println!("Shutting down...");
break;
}
}
}
}
```
---
## Channel Patterns
### MPSC (Multi-Producer, Single-Consumer)
```rust
use tokio::sync::mpsc;
async fn producer_consumer() {
let (tx, mut rx) = mpsc::channel(100);
// Spawn producers
for i in 0..3 {
let tx = tx.clone();
tokio::spawn(async move {
tx.send(format!("Message from {}", i)).await.unwrap();
});
}
// Drop original sender so channel closes
drop(tx);
// Consume
while let Some(msg) = rx.recv().await {
println!("Received: {}", msg);
}
}
```
### Oneshot (Single-Shot Response)
```rust
use tokio::sync::oneshot;
async fn request_response() {
let (tx, rx) = oneshot::channel();
tokio::spawn(async move {
let result = compute_something().await;
tx.send(result).unwrap();
});
// Wait for response
let response = rx.await.unwrap();
}
```
### Broadcast (Multi-Consumer)
```rust
use tokio::sync::broadcast;
async fn pub_sub() {
let (tx, _) = broadcast::channel(16);
// Subscribe multiple consumers
let mut rx1 = tx.subscribe();
let mut rx2 = tx.subscribe();
tokio::spawn(async move {
while let Ok(msg) = rx1.recv().await {
println!("Consumer 1: {}", msg);
}
});
tokio::spawn(async move {
while let Ok(msg) = rx2.recv().await {
println!("Consumer 2: {}", msg);
}
});
// Publish
tx.send("Hello").unwrap();
}
```
### Watch (Single Latest Value)
```rust
use tokio::sync::watch;
async fn config_updates() {
let (tx, mut rx) = watch::channel(Config::default());
// Consumer watches for changes
tokio::spawn(async move {
while rx.changed().await.is_ok() {
let config = rx.borrow();
apply_config(&config);
}
});
// Update config
tx.send(Config::new()).unwrap();
}
```
---
## Structured Concurrency
### JoinSet for Task Groups
```rust
use tokio::task::JoinSet;
async fn parallel_fetch(urls: Vec<String>) -> Vec<Result<Response, Error>> {
let mut set = JoinSet::new();
for url in urls {
set.spawn(async move {
fetch(&url).await
});
}
let mut results = vec![];
while let Some(res) = set.join_next().await {
results.push(res.unwrap());
}
results
}
```
### Scoped Tasks (no 'static)
```rust
// Using tokio-scoped or async-scoped crate
use async_scoped::TokioScope;
async fn scoped_example(data: &[u32]) {
let results = TokioScope::scope_and_block(|scope| {
for item in data {
scope.spawn(async move {
process(item).await
});
}
});
}
```
---
## Cancellation Patterns
### Using CancellationToken
```rust
use tokio_util::sync::CancellationToken;
async fn cancellable_task(token: CancellationToken) {
loop {
select! {
_ = token.cancelled() => {
println!("Task cancelled");
break;
}
_ = do_work() => {
// Continue working
}
}
}
}
async fn main_with_cancellation() {
let token = CancellationToken::new();
let task_token = token.clone();
let handle = tokio::spawn(cancellable_task(task_token));
// Cancel after some condition
tokio::time::sleep(Duration::from_secs(5)).await;
token.cancel();
handle.await.unwrap();
}
```
### Graceful Shutdown
```rust
async fn serve_with_shutdown(shutdown: impl Future) {
let server = TcpListener::bind("0.0.0.0:8080").await.unwrap();
loop {
select! {
Ok((socket, _)) = server.accept() => {
tokio::spawn(handle_connection(socket));
}
_ = &mut shutdown => {
println!("Shutting down...");
break;
}
}
}
}
#[tokio::main]
async fn main() {
let ctrl_c = async {
tokio::signal::ctrl_c().await.unwrap();
};
serve_with_shutdown(ctrl_c).await;
}
```
---
## Backpressure Patterns
### Bounded Channels
```rust
use tokio::sync::mpsc;
async fn with_backpressure() {
// Buffer of 10 - producers will wait if full
let (tx, mut rx) = mpsc::channel(10);
let producer = tokio::spawn(async move {
for i in 0..1000 {
// This will wait if channel is full
tx.send(i).await.unwrap();
}
});
let consumer = tokio::spawn(async move {
while let Some(item) = rx.recv().await {
// Slow consumer
tokio::time::sleep(Duration::from_millis(10)).await;
process(item);
}
});
let _ = tokio::join!(producer, consumer);
}
```
### Semaphore for Rate Limiting
```rust
use tokio::sync::Semaphore;
use std::sync::Arc;
async fn rate_limited_requests(urls: Vec<String>) {
let semaphore = Arc::new(Semaphore::new(10)); // max 10 concurrent
let handles: Vec<_> = urls
.into_iter()
.map(|url| {
let sem = Arc::clone(&semaphore);
tokio::spawn(async move {
let _permit = sem.acquire().await.unwrap();
fetch(&url).await
})
})
.collect();
for handle in handles {
handle.await.unwrap();
}
}
```
---
## Error Handling in Async
### Propagating Errors
```rust
async fn fetch_and_parse(url: &str) -> Result<Data, Error> {
let response = fetch(url).await?;
let data = parse(response).await?;
Ok(data)
}
```
### Handling Task Panics
```rust
async fn robust_spawn() {
let handle = tokio::spawn(async {
risky_operation().await
});
match handle.await {
Ok(result) => println!("Success: {:?}", result),
Err(e) if e.is_panic() => {
println!("Task panicked: {:?}", e);
}
Err(e) => {
println!("Task cancelled: {:?}", e);
}
}
}
```
### Try-Join for Multiple Results
```rust
use tokio::try_join;
async fn fetch_all() -> Result<(A, B, C), Error> {
// All must succeed, or first error returned
try_join!(
fetch_a(),
fetch_b(),
fetch_c(),
)
}
```
@@ -1,331 +0,0 @@
# Common Concurrency Errors & Fixes
## E0277: Cannot Send Between Threads
### Error Pattern
```rust
use std::rc::Rc;
let data = Rc::new(42);
std::thread::spawn(move || {
println!("{}", data); // ERROR: Rc<i32> cannot be sent between threads
});
```
### Fix Options
**Option 1: Use Arc instead**
```rust
use std::sync::Arc;
let data = Arc::new(42);
let data_clone = Arc::clone(&data);
std::thread::spawn(move || {
println!("{}", data_clone); // OK: Arc is Send
});
```
**Option 2: Move owned data**
```rust
let data = 42; // i32 is Copy and Send
std::thread::spawn(move || {
println!("{}", data); // OK
});
```
---
## E0277: Cannot Share Between Threads (Not Sync)
### Error Pattern
```rust
use std::cell::RefCell;
use std::sync::Arc;
let data = Arc::new(RefCell::new(42));
// ERROR: RefCell is not Sync
```
### Fix Options
**Option 1: Use Mutex for thread-safe interior mutability**
```rust
use std::sync::{Arc, Mutex};
let data = Arc::new(Mutex::new(42));
let data_clone = Arc::clone(&data);
std::thread::spawn(move || {
let mut guard = data_clone.lock().unwrap();
*guard += 1;
});
```
**Option 2: Use RwLock for read-heavy workloads**
```rust
use std::sync::{Arc, RwLock};
let data = Arc::new(RwLock::new(42));
let data_clone = Arc::clone(&data);
std::thread::spawn(move || {
let guard = data_clone.read().unwrap();
println!("{}", *guard);
});
```
---
## Deadlock Patterns
### Pattern 1: Lock Ordering Deadlock
```rust
// DANGER: potential deadlock
use std::sync::{Arc, Mutex};
let a = Arc::new(Mutex::new(1));
let b = Arc::new(Mutex::new(2));
// Thread 1: locks a then b
let a1 = Arc::clone(&a);
let b1 = Arc::clone(&b);
std::thread::spawn(move || {
let _a = a1.lock().unwrap();
let _b = b1.lock().unwrap(); // waits for b
});
// Thread 2: locks b then a (opposite order!)
let a2 = Arc::clone(&a);
let b2 = Arc::clone(&b);
std::thread::spawn(move || {
let _b = b2.lock().unwrap();
let _a = a2.lock().unwrap(); // waits for a - DEADLOCK
});
```
### Fix: Consistent Lock Ordering
```rust
// SAFE: always lock in same order (a before b)
std::thread::spawn(move || {
let _a = a1.lock().unwrap();
let _b = b1.lock().unwrap();
});
std::thread::spawn(move || {
let _a = a2.lock().unwrap(); // same order
let _b = b2.lock().unwrap();
});
```
### Pattern 2: Self-Deadlock
```rust
// DANGER: locking same mutex twice
let m = Mutex::new(42);
let _g1 = m.lock().unwrap();
let _g2 = m.lock().unwrap(); // DEADLOCK on std::Mutex
// FIX: use parking_lot::ReentrantMutex if needed
// or restructure code to avoid double locking
```
---
## Mutex Guard Across Await
### Error Pattern
```rust
use std::sync::Mutex;
use tokio::time::sleep;
async fn bad_async() {
let m = Mutex::new(42);
let guard = m.lock().unwrap();
sleep(Duration::from_secs(1)).await; // WARNING: guard held across await
println!("{}", *guard);
}
```
### Fix Options
**Option 1: Scope the lock**
```rust
async fn good_async() {
let m = Mutex::new(42);
let value = {
let guard = m.lock().unwrap();
*guard // copy value
}; // guard dropped here
sleep(Duration::from_secs(1)).await;
println!("{}", value);
}
```
**Option 2: Use tokio::sync::Mutex**
```rust
use tokio::sync::Mutex;
async fn good_async() {
let m = Mutex::new(42);
let guard = m.lock().await; // async lock
sleep(Duration::from_secs(1)).await; // OK with tokio::Mutex
println!("{}", *guard);
}
```
---
## Data Race Prevention
### Pattern: Missing Synchronization
```rust
// This WON'T compile - Rust prevents data races
use std::sync::Arc;
let data = Arc::new(0);
let d1 = Arc::clone(&data);
let d2 = Arc::clone(&data);
std::thread::spawn(move || {
// *d1 += 1; // ERROR: cannot mutate through Arc
});
std::thread::spawn(move || {
// *d2 += 1; // ERROR: cannot mutate through Arc
});
```
### Fix: Add Synchronization
```rust
use std::sync::{Arc, Mutex};
use std::sync::atomic::{AtomicI32, Ordering};
// Option 1: Mutex
let data = Arc::new(Mutex::new(0));
let d1 = Arc::clone(&data);
std::thread::spawn(move || {
*d1.lock().unwrap() += 1;
});
// Option 2: Atomic (for simple types)
let data = Arc::new(AtomicI32::new(0));
let d1 = Arc::clone(&data);
std::thread::spawn(move || {
d1.fetch_add(1, Ordering::SeqCst);
});
```
---
## Channel Errors
### Disconnected Channel
```rust
use std::sync::mpsc;
let (tx, rx) = mpsc::channel();
drop(tx); // sender dropped
match rx.recv() {
Ok(v) => println!("{}", v),
Err(_) => println!("channel disconnected"), // this happens
}
```
### Fix: Handle Disconnection
```rust
// Use try_recv for non-blocking
loop {
match rx.try_recv() {
Ok(msg) => handle(msg),
Err(TryRecvError::Empty) => continue,
Err(TryRecvError::Disconnected) => break,
}
}
// Or iterate (stops on disconnect)
for msg in rx {
handle(msg);
}
```
---
## Async Common Errors
### Forgetting to Spawn
```rust
// WRONG: future not polled
async fn fetch_data() -> Result<Data, Error> { ... }
fn process() {
fetch_data(); // does nothing! returns Future that's dropped
}
// RIGHT: await or spawn
async fn process() {
let data = fetch_data().await; // awaited
}
fn process_sync() {
tokio::spawn(fetch_data()); // spawned
}
```
### Blocking in Async Context
```rust
// WRONG: blocks the executor
async fn bad() {
std::thread::sleep(Duration::from_secs(1)); // blocks!
std::fs::read_to_string("file.txt").unwrap(); // blocks!
}
// RIGHT: use async versions
async fn good() {
tokio::time::sleep(Duration::from_secs(1)).await;
tokio::fs::read_to_string("file.txt").await.unwrap();
}
// Or spawn_blocking for CPU-bound work
async fn compute() {
let result = tokio::task::spawn_blocking(|| {
heavy_computation() // OK to block here
}).await.unwrap();
}
```
---
## Thread Panic Handling
### Unhandled Panic
```rust
let handle = std::thread::spawn(|| {
panic!("oops");
});
// Main thread continues, might miss the error
handle.join().unwrap(); // panics here
```
### Proper Error Handling
```rust
let handle = std::thread::spawn(|| {
panic!("oops");
});
match handle.join() {
Ok(result) => println!("Success: {:?}", result),
Err(e) => println!("Thread panicked: {:?}", e),
}
// For async: use catch_unwind
use std::panic;
async fn safe_task() {
let result = panic::catch_unwind(|| {
risky_operation()
});
match result {
Ok(v) => use_value(v),
Err(_) => log_error("task panicked"),
}
}
```
-173
View File
@@ -1,173 +0,0 @@
---
name: m09-domain
description: "CRITICAL: Use for domain modeling. Triggers: domain model, DDD, domain-driven design, entity, value object, aggregate, repository pattern, business rules, validation, invariant, 领域模型, 领域驱动设计, 业务规则"
---
# Domain Modeling
> **Layer 2: Design Choices**
## Core Question
**What is this concept's role in the domain?**
Before modeling in code, understand:
- Is it an Entity (identity matters) or Value Object (interchangeable)?
- What invariants must be maintained?
- Where are the aggregate boundaries?
---
## Domain Concept → Rust Pattern
| Domain Concept | Rust Pattern | Ownership Implication |
|----------------|--------------|----------------------|
| Entity | struct + Id | Owned, unique identity |
| Value Object | struct + Clone/Copy | Shareable, immutable |
| Aggregate Root | struct owns children | Clear ownership tree |
| Repository | trait | Abstracts persistence |
| Domain Event | enum | Captures state changes |
| Service | impl block / free fn | Stateless operations |
---
## Thinking Prompt
Before creating a domain type:
1. **What's the concept's identity?**
- Needs unique identity → Entity (Id field)
- Interchangeable by value → Value Object (Clone/Copy)
2. **What invariants must hold?**
- Always valid → private fields + validated constructor
- Transition rules → type state pattern
3. **Who owns this data?**
- Single owner (parent) → owned field
- Shared reference → Arc/Rc
- Weak reference → Weak
---
## Trace Up ↑
To domain constraints (Layer 3):
```
"How should I model a Transaction?"
↑ Ask: What domain rules govern transactions?
↑ Check: domain-fintech (audit, precision requirements)
↑ Check: Business stakeholders (what invariants?)
```
| Design Question | Trace To | Ask |
|-----------------|----------|-----|
| Entity vs Value Object | domain-* | What makes two instances "the same"? |
| Aggregate boundaries | domain-* | What must be consistent together? |
| Validation rules | domain-* | What business rules apply? |
---
## Trace Down ↓
To implementation (Layer 1):
```
"Model as Entity"
↓ m01-ownership: Owned, unique
↓ m05-type-driven: Newtype for Id
"Model as Value Object"
↓ m01-ownership: Clone/Copy OK
↓ m05-type-driven: Validate at construction
"Model as Aggregate"
↓ m01-ownership: Parent owns children
↓ m02-resource: Consider Rc for shared within aggregate
```
---
## Quick Reference
| DDD Concept | Rust Pattern | Example |
|-------------|--------------|---------|
| Value Object | Newtype | `struct Email(String);` |
| Entity | Struct + ID | `struct User { id: UserId, ... }` |
| Aggregate | Module boundary | `mod order { ... }` |
| Repository | Trait | `trait UserRepo { fn find(...) }` |
| Domain Event | Enum | `enum OrderEvent { Created, ... }` |
## Pattern Templates
### Value Object
```rust
struct Email(String);
impl Email {
pub fn new(s: &str) -> Result<Self, ValidationError> {
validate_email(s)?;
Ok(Self(s.to_string()))
}
}
```
### Entity
```rust
struct UserId(Uuid);
struct User {
id: UserId,
email: Email,
// ... other fields
}
impl PartialEq for User {
fn eq(&self, other: &Self) -> bool {
self.id == other.id // Identity equality
}
}
```
### Aggregate
```rust
mod order {
pub struct Order {
id: OrderId,
items: Vec<OrderItem>, // Owned children
// ...
}
impl Order {
pub fn add_item(&mut self, item: OrderItem) {
// Enforce aggregate invariants
}
}
}
```
---
## Common Mistakes
| Mistake | Why Wrong | Better |
|---------|-----------|--------|
| Primitive obsession | No type safety | Newtype wrappers |
| Public fields with invariants | Invariants violated | Private + accessor |
| Leaked aggregate internals | Broken encapsulation | Methods on root |
| String for semantic types | No validation | Validated newtype |
---
## Related Skills
| When | See |
|------|-----|
| Type-driven implementation | m05-type-driven |
| Ownership for aggregates | m01-ownership |
| Domain error handling | m13-domain-error |
| Specific domain rules | domain-* |
-156
View File
@@ -1,156 +0,0 @@
---
name: m10-performance
description: "CRITICAL: Use for performance optimization. Triggers: performance, optimization, benchmark, profiling, flamegraph, criterion, slow, fast, allocation, cache, SIMD, make it faster, 性能优化, 基准测试"
---
# Performance Optimization
> **Layer 2: Design Choices**
## Core Question
**What's the bottleneck, and is optimization worth it?**
Before optimizing:
- Have you measured? (Don't guess)
- What's the acceptable performance?
- Will optimization add complexity?
---
## Performance Decision → Implementation
| Goal | Design Choice | Implementation |
|------|---------------|----------------|
| Reduce allocations | Pre-allocate, reuse | `with_capacity`, object pools |
| Improve cache | Contiguous data | `Vec`, `SmallVec` |
| Parallelize | Data parallelism | `rayon`, threads |
| Avoid copies | Zero-copy | References, `Cow<T>` |
| Reduce indirection | Inline data | `smallvec`, arrays |
---
## Thinking Prompt
Before optimizing:
1. **Have you measured?**
- Profile first → flamegraph, perf
- Benchmark → criterion, cargo bench
- Identify actual hotspots
2. **What's the priority?**
- Algorithm (10x-1000x improvement)
- Data structure (2x-10x)
- Allocation (2x-5x)
- Cache (1.5x-3x)
3. **What's the trade-off?**
- Complexity vs speed
- Memory vs CPU
- Latency vs throughput
---
## Trace Up ↑
To domain constraints (Layer 3):
```
"How fast does this need to be?"
↑ Ask: What's the performance SLA?
↑ Check: domain-* (latency requirements)
↑ Check: Business requirements (acceptable response time)
```
| Question | Trace To | Ask |
|----------|----------|-----|
| Latency requirements | domain-* | What's acceptable response time? |
| Throughput needs | domain-* | How many requests per second? |
| Memory constraints | domain-* | What's the memory budget? |
---
## Trace Down ↓
To implementation (Layer 1):
```
"Need to reduce allocations"
↓ m01-ownership: Use references, avoid clone
↓ m02-resource: Pre-allocate with_capacity
"Need to parallelize"
↓ m07-concurrency: Choose rayon or threads
↓ m07-concurrency: Consider async for I/O-bound
"Need cache efficiency"
↓ Data layout: Prefer Vec over HashMap when possible
↓ Access patterns: Sequential over random access
```
---
## Quick Reference
| Tool | Purpose |
|------|---------|
| `cargo bench` | Micro-benchmarks |
| `criterion` | Statistical benchmarks |
| `perf` / `flamegraph` | CPU profiling |
| `heaptrack` | Allocation tracking |
| `valgrind` / `cachegrind` | Cache analysis |
## Optimization Priority
```
1. Algorithm choice (10x - 1000x)
2. Data structure (2x - 10x)
3. Allocation reduction (2x - 5x)
4. Cache optimization (1.5x - 3x)
5. SIMD/Parallelism (2x - 8x)
```
## Common Techniques
| Technique | When | How |
|-----------|------|-----|
| Pre-allocation | Known size | `Vec::with_capacity(n)` |
| Avoid cloning | Hot paths | Use references or `Cow<T>` |
| Batch operations | Many small ops | Collect then process |
| SmallVec | Usually small | `smallvec::SmallVec<[T; N]>` |
| Inline buffers | Fixed-size data | Arrays over Vec |
---
## Common Mistakes
| Mistake | Why Wrong | Better |
|---------|-----------|--------|
| Optimize without profiling | Wrong target | Profile first |
| Benchmark in debug mode | Meaningless | Always `--release` |
| Use LinkedList | Cache unfriendly | `Vec` or `VecDeque` |
| Hidden `.clone()` | Unnecessary allocs | Use references |
| Premature optimization | Wasted effort | Make it work first |
---
## Anti-Patterns
| Anti-Pattern | Why Bad | Better |
|--------------|---------|--------|
| Clone to avoid lifetimes | Performance cost | Proper ownership |
| Box everything | Indirection cost | Stack when possible |
| HashMap for small sets | Overhead | Vec with linear search |
| String concat in loop | O(n^2) | `String::with_capacity` or `format!` |
---
## Related Skills
| When | See |
|------|-----|
| Reducing clones | m01-ownership |
| Concurrency options | m07-concurrency |
| Smart pointer choice | m02-resource |
| Domain requirements | domain-* |
@@ -1,365 +0,0 @@
# Rust Performance Optimization Guide
## Profiling First
### Tools
```bash
# CPU profiling
cargo install flamegraph
cargo flamegraph --bin myapp
# Memory profiling
cargo install cargo-instruments # macOS
heaptrack ./target/release/myapp # Linux
# Benchmarking
cargo bench # with criterion
# Cache analysis
valgrind --tool=cachegrind ./target/release/myapp
```
### Criterion Benchmarks
```rust
use criterion::{criterion_group, criterion_main, Criterion};
fn benchmark_parse(c: &mut Criterion) {
let input = "test data".repeat(1000);
c.bench_function("parse_v1", |b| {
b.iter(|| parse_v1(&input))
});
c.bench_function("parse_v2", |b| {
b.iter(|| parse_v2(&input))
});
}
criterion_group!(benches, benchmark_parse);
criterion_main!(benches);
```
---
## Common Optimizations
### 1. Avoid Unnecessary Allocations
```rust
// BAD: allocates on every call
fn to_uppercase(s: &str) -> String {
s.to_uppercase()
}
// GOOD: return Cow, allocate only if needed
use std::borrow::Cow;
fn to_uppercase(s: &str) -> Cow<'_, str> {
if s.chars().all(|c| c.is_uppercase()) {
Cow::Borrowed(s)
} else {
Cow::Owned(s.to_uppercase())
}
}
```
### 2. Reuse Allocations
```rust
// BAD: creates new Vec each iteration
for item in items {
let mut buffer = Vec::new();
process(&mut buffer, item);
}
// GOOD: reuse buffer
let mut buffer = Vec::new();
for item in items {
buffer.clear();
process(&mut buffer, item);
}
```
### 3. Use Appropriate Collections
| Need | Collection | Notes |
|------|------------|-------|
| Sequential access | `Vec<T>` | Best cache locality |
| Random access by key | `HashMap<K, V>` | O(1) lookup |
| Ordered keys | `BTreeMap<K, V>` | O(log n) lookup |
| Small sets (<20) | `Vec<T>` + linear search | Lower overhead |
| FIFO queue | `VecDeque<T>` | O(1) push/pop both ends |
### 4. Pre-allocate Capacity
```rust
// BAD: many reallocations
let mut v = Vec::new();
for i in 0..10000 {
v.push(i);
}
// GOOD: single allocation
let mut v = Vec::with_capacity(10000);
for i in 0..10000 {
v.push(i);
}
```
---
## String Optimization
### Avoid String Concatenation in Loops
```rust
// BAD: O(n²) allocations
let mut result = String::new();
for s in strings {
result = result + &s;
}
// GOOD: O(n) with push_str
let mut result = String::new();
for s in strings {
result.push_str(&s);
}
// BETTER: pre-calculate capacity
let total_len: usize = strings.iter().map(|s| s.len()).sum();
let mut result = String::with_capacity(total_len);
for s in strings {
result.push_str(&s);
}
// BEST: use join for simple cases
let result = strings.join("");
```
### Use &str When Possible
```rust
// BAD: requires allocation
fn greet(name: String) {
println!("Hello, {}", name);
}
// GOOD: borrows, no allocation
fn greet(name: &str) {
println!("Hello, {}", name);
}
// Works with both:
greet("world"); // &str
greet(&String::from("world")); // &String coerces to &str
```
---
## Iterator Optimization
### Use Iterators Over Indexing
```rust
// BAD: bounds checking on each access
let mut sum = 0;
for i in 0..vec.len() {
sum += vec[i];
}
// GOOD: no bounds checking
let sum: i32 = vec.iter().sum();
// GOOD: when index needed
for (i, item) in vec.iter().enumerate() {
// ...
}
```
### Lazy Evaluation
```rust
// Iterators are lazy - computation happens at collect
let result: Vec<_> = data
.iter()
.filter(|x| x.is_valid())
.map(|x| x.process())
.take(10) // stop after 10 items
.collect();
```
### Avoid Collecting When Not Needed
```rust
// BAD: unnecessary intermediate allocation
let filtered: Vec<_> = items.iter().filter(|x| x.valid).collect();
let count = filtered.len();
// GOOD: no allocation
let count = items.iter().filter(|x| x.valid).count();
```
---
## Parallelism with Rayon
```rust
use rayon::prelude::*;
// Sequential
let sum: i32 = (0..1_000_000).map(|x| x * x).sum();
// Parallel (automatic work stealing)
let sum: i32 = (0..1_000_000).into_par_iter().map(|x| x * x).sum();
// Parallel with custom chunk size
let results: Vec<_> = data
.par_chunks(1000)
.map(|chunk| process_chunk(chunk))
.collect();
```
---
## Memory Layout
### Use Appropriate Integer Sizes
```rust
// If values are small, use smaller types
struct Item {
count: u8, // 0-255, not u64
flags: u8, // small enum
id: u32, // if 4 billion is enough
}
```
### Pack Structs Efficiently
```rust
// BAD: 24 bytes due to padding
struct Bad {
a: u8, // 1 byte + 7 padding
b: u64, // 8 bytes
c: u8, // 1 byte + 7 padding
}
// GOOD: 16 bytes (or use #[repr(packed)])
struct Good {
b: u64, // 8 bytes
a: u8, // 1 byte
c: u8, // 1 byte + 6 padding
}
```
### Box Large Values
```rust
// Large enum variants waste space
enum Message {
Quit,
Data([u8; 10000]), // all variants are 10000+ bytes
}
// Better: box the large variant
enum Message {
Quit,
Data(Box<[u8; 10000]>), // variants are pointer-sized
}
```
---
## Async Performance
### Avoid Blocking in Async
```rust
// BAD: blocks the executor
async fn bad() {
std::thread::sleep(Duration::from_secs(1)); // blocking!
std::fs::read_to_string("file.txt").unwrap(); // blocking!
}
// GOOD: use async versions
async fn good() {
tokio::time::sleep(Duration::from_secs(1)).await;
tokio::fs::read_to_string("file.txt").await.unwrap();
}
// For CPU work: spawn_blocking
async fn compute() -> i32 {
tokio::task::spawn_blocking(|| {
heavy_computation()
}).await.unwrap()
}
```
### Buffer Async I/O
```rust
use tokio::io::{AsyncBufReadExt, BufReader};
// BAD: many small reads
async fn bad(file: File) {
let mut byte = [0u8];
while file.read(&mut byte).await.unwrap() > 0 {
process(byte[0]);
}
}
// GOOD: buffered reading
async fn good(file: File) {
let reader = BufReader::new(file);
let mut lines = reader.lines();
while let Some(line) = lines.next_line().await.unwrap() {
process(&line);
}
}
```
---
## Release Build Optimization
### Cargo.toml Settings
```toml
[profile.release]
lto = true # Link-time optimization
codegen-units = 1 # Single codegen unit (slower compile, faster code)
panic = "abort" # Smaller binary, no unwinding
strip = true # Strip symbols
[profile.release-fast]
inherits = "release"
opt-level = 3 # Maximum optimization
[profile.release-small]
inherits = "release"
opt-level = "s" # Optimize for size
```
### Compile-Time Assertions
```rust
// Zero runtime cost
const _: () = assert!(std::mem::size_of::<MyStruct>() <= 64);
```
---
## Checklist
Before optimizing:
- [ ] Profile to find actual bottlenecks
- [ ] Have benchmarks to measure improvement
- [ ] Consider if optimization is worth complexity
Common wins:
- [ ] Reduce allocations (Cow, reuse buffers)
- [ ] Use appropriate collections
- [ ] Pre-allocate with_capacity
- [ ] Use iterators instead of indexing
- [ ] Enable LTO for release builds
- [ ] Use rayon for parallel workloads
-155
View File
@@ -1,155 +0,0 @@
---
name: m11-ecosystem
description: "Use when integrating crates or ecosystem questions. Keywords: E0425, E0433, E0603, crate, cargo, dependency, feature flag, workspace, which crate to use, using external C libraries, creating Python extensions, PyO3, wasm, WebAssembly, bindgen, cbindgen, napi-rs, cannot find, private, crate recommendation, best crate for, Cargo.toml, features, crate 推荐, 依赖管理, 特性标志, 工作空间, Python 绑定"
---
# Ecosystem Integration
> **Layer 2: Design Choices**
## Core Question
**What's the right crate for this job, and how should it integrate?**
Before adding dependencies:
- Is there a standard solution?
- What's the maintenance status?
- What's the API stability?
---
## Integration Decision → Implementation
| Need | Choice | Crates |
|------|--------|--------|
| Serialization | Derive-based | serde, serde_json |
| Async runtime | tokio or async-std | tokio (most popular) |
| HTTP client | Ergonomic | reqwest |
| HTTP server | Modern | axum, actix-web |
| Database | SQL or ORM | sqlx, diesel |
| CLI parsing | Derive-based | clap |
| Error handling | App vs lib | anyhow, thiserror |
| Logging | Facade | tracing, log |
---
## Thinking Prompt
Before adding a dependency:
1. **Is it well-maintained?**
- Recent commits?
- Active issue response?
- Breaking changes frequency?
2. **What's the scope?**
- Do you need the full crate or just a feature?
- Can feature flags reduce bloat?
3. **How does it integrate?**
- Trait-based or concrete types?
- Sync or async?
- What bounds does it require?
---
## Trace Up ↑
To domain constraints (Layer 3):
```
"Which HTTP framework should I use?"
↑ Ask: What are the performance requirements?
↑ Check: domain-web (latency, throughput needs)
↑ Check: Team expertise (familiarity with framework)
```
| Question | Trace To | Ask |
|----------|----------|-----|
| Framework choice | domain-* | What constraints matter? |
| Library vs build | domain-* | What's the deployment model? |
| API design | domain-* | Who are the consumers? |
---
## Trace Down ↓
To implementation (Layer 1):
```
"Integrate external crate"
↓ m04-zero-cost: Trait bounds and generics
↓ m06-error-handling: Error type compatibility
"FFI integration"
↓ unsafe-checker: Safety requirements
↓ m12-lifecycle: Resource cleanup
```
---
## Quick Reference
### Language Interop
| Integration | Crate/Tool | Use Case |
|-------------|------------|----------|
| C/C++ → Rust | `bindgen` | Auto-generate bindings |
| Rust → C | `cbindgen` | Export C headers |
| Python ↔ Rust | `pyo3` | Python extensions |
| Node.js ↔ Rust | `napi-rs` | Node addons |
| WebAssembly | `wasm-bindgen` | Browser/WASI |
### Cargo Features
| Feature | Purpose |
|---------|---------|
| `[features]` | Optional functionality |
| `default = [...]` | Default features |
| `feature = "serde"` | Conditional deps |
| `[workspace]` | Multi-crate projects |
## Error Code Reference
| Error | Cause | Fix |
|-------|-------|-----|
| E0433 | Can't find crate | Add to Cargo.toml |
| E0603 | Private item | Check crate docs |
| Feature not enabled | Optional feature | Enable in `features` |
| Version conflict | Incompatible deps | `cargo update` or pin |
| Duplicate types | Different crate versions | Unify in workspace |
---
## Crate Selection Criteria
| Criterion | Good Sign | Warning Sign |
|-----------|-----------|--------------|
| Maintenance | Recent commits | Years inactive |
| Community | Active issues/PRs | No response |
| Documentation | Examples, API docs | Minimal docs |
| Stability | Semantic versioning | Frequent breaking |
| Dependencies | Minimal, well-known | Heavy, obscure |
---
## Anti-Patterns
| Anti-Pattern | Why Bad | Better |
|--------------|---------|--------|
| `extern crate` | Outdated (2018+) | Just `use` |
| `#[macro_use]` | Global pollution | Explicit import |
| Wildcard deps `*` | Unpredictable | Specific versions |
| Too many deps | Supply chain risk | Evaluate necessity |
| Vendoring everything | Maintenance burden | Trust crates.io |
---
## Related Skills
| When | See |
|------|-----|
| Error type design | m06-error-handling |
| Trait integration | m04-zero-cost |
| FFI safety | unsafe-checker |
| Resource management | m12-lifecycle |
-176
View File
@@ -1,176 +0,0 @@
---
name: m12-lifecycle
description: "Use when designing resource lifecycles. Keywords: RAII, Drop, resource lifecycle, connection pool, lazy initialization, connection pool design, resource cleanup patterns, cleanup, scope, OnceCell, Lazy, once_cell, OnceLock, transaction, session management, when is Drop called, cleanup on error, guard pattern, scope guard, 资源生命周期, 连接池, 惰性初始化, 资源清理, RAII 模式"
---
# Resource Lifecycle
> **Layer 2: Design Choices**
## Core Question
**When should this resource be created, used, and cleaned up?**
Before implementing lifecycle:
- What's the resource's scope?
- Who owns the cleanup responsibility?
- What happens on error?
---
## Lifecycle Pattern → Implementation
| Pattern | When | Implementation |
|---------|------|----------------|
| RAII | Auto cleanup | `Drop` trait |
| Lazy init | Deferred creation | `OnceLock`, `LazyLock` |
| Pool | Reuse expensive resources | `r2d2`, `deadpool` |
| Guard | Scoped access | `MutexGuard` pattern |
| Scope | Transaction boundary | Custom struct + Drop |
---
## Thinking Prompt
Before designing lifecycle:
1. **What's the resource cost?**
- Cheap → create per use
- Expensive → pool or cache
- Global → lazy singleton
2. **What's the scope?**
- Function-local → stack allocation
- Request-scoped → passed or extracted
- Application-wide → static or Arc
3. **What about errors?**
- Cleanup must happen → Drop
- Cleanup is optional → explicit close
- Cleanup can fail → Result from close
---
## Trace Up ↑
To domain constraints (Layer 3):
```
"How should I manage database connections?"
↑ Ask: What's the connection cost?
↑ Check: domain-* (latency requirements)
↑ Check: Infrastructure (connection limits)
```
| Question | Trace To | Ask |
|----------|----------|-----|
| Connection pooling | domain-* | What's acceptable latency? |
| Resource limits | domain-* | What are infra constraints? |
| Transaction scope | domain-* | What must be atomic? |
---
## Trace Down ↓
To implementation (Layer 1):
```
"Need automatic cleanup"
↓ m02-resource: Implement Drop
↓ m01-ownership: Clear owner for cleanup
"Need lazy initialization"
↓ m03-mutability: OnceLock for thread-safe
↓ m07-concurrency: LazyLock for sync
"Need connection pool"
↓ m07-concurrency: Thread-safe pool
↓ m02-resource: Arc for sharing
```
---
## Quick Reference
| Pattern | Type | Use Case |
|---------|------|----------|
| RAII | `Drop` trait | Auto cleanup on scope exit |
| Lazy Init | `OnceLock`, `LazyLock` | Deferred initialization |
| Pool | `r2d2`, `deadpool` | Connection reuse |
| Guard | `MutexGuard` | Scoped lock release |
| Scope | Custom struct | Transaction boundaries |
## Lifecycle Events
| Event | Rust Mechanism |
|-------|----------------|
| Creation | `new()`, `Default` |
| Lazy Init | `OnceLock::get_or_init` |
| Usage | `&self`, `&mut self` |
| Cleanup | `Drop::drop()` |
## Pattern Templates
### RAII Guard
```rust
struct FileGuard {
path: PathBuf,
_handle: File,
}
impl Drop for FileGuard {
fn drop(&mut self) {
// Cleanup: remove temp file
let _ = std::fs::remove_file(&self.path);
}
}
```
### Lazy Singleton
```rust
use std::sync::OnceLock;
static CONFIG: OnceLock<Config> = OnceLock::new();
fn get_config() -> &'static Config {
CONFIG.get_or_init(|| {
Config::load().expect("config required")
})
}
```
---
## Common Errors
| Error | Cause | Fix |
|-------|-------|-----|
| Resource leak | Forgot Drop | Implement Drop or RAII wrapper |
| Double free | Manual memory | Let Rust handle |
| Use after drop | Dangling reference | Check lifetimes |
| E0509 move out of Drop | Moving owned field | `Option::take()` |
| Pool exhaustion | Not returned | Ensure Drop returns |
---
## Anti-Patterns
| Anti-Pattern | Why Bad | Better |
|--------------|---------|--------|
| Manual cleanup | Easy to forget | RAII/Drop |
| `lazy_static!` | External dep | `std::sync::OnceLock` |
| Global mutable state | Thread unsafety | `OnceLock` or proper sync |
| Forget to close | Resource leak | Drop impl |
---
## Related Skills
| When | See |
|------|-----|
| Smart pointers | m02-resource |
| Thread-safe init | m07-concurrency |
| Domain scopes | m09-domain |
| Error in cleanup | m06-error-handling |
-179
View File
@@ -1,179 +0,0 @@
---
name: m13-domain-error
description: "Use when designing domain error handling. Keywords: domain error, error categorization, recovery strategy, retry, fallback, domain error hierarchy, user-facing vs internal errors, error code design, circuit breaker, graceful degradation, resilience, error context, backoff, retry with backoff, error recovery, transient vs permanent error, 领域错误, 错误分类, 恢复策略, 重试, 熔断器, 优雅降级"
---
# Domain Error Strategy
> **Layer 2: Design Choices**
## Core Question
**Who needs to handle this error, and how should they recover?**
Before designing error types:
- Is this user-facing or internal?
- Is recovery possible?
- What context is needed for debugging?
---
## Error Categorization
| Error Type | Audience | Recovery | Example |
|------------|----------|----------|---------|
| User-facing | End users | Guide action | `InvalidEmail`, `NotFound` |
| Internal | Developers | Debug info | `DatabaseError`, `ParseError` |
| System | Ops/SRE | Monitor/alert | `ConnectionTimeout`, `RateLimited` |
| Transient | Automation | Retry | `NetworkError`, `ServiceUnavailable` |
| Permanent | Human | Investigate | `ConfigInvalid`, `DataCorrupted` |
---
## Thinking Prompt
Before designing error types:
1. **Who sees this error?**
- End user → friendly message, actionable
- Developer → detailed, debuggable
- Ops → structured, alertable
2. **Can we recover?**
- Transient → retry with backoff
- Degradable → fallback value
- Permanent → fail fast, alert
3. **What context is needed?**
- Call chain → anyhow::Context
- Request ID → structured logging
- Input data → error payload
---
## Trace Up ↑
To domain constraints (Layer 3):
```
"How should I handle payment failures?"
↑ Ask: What are the business rules for retries?
↑ Check: domain-fintech (transaction requirements)
↑ Check: SLA (availability requirements)
```
| Question | Trace To | Ask |
|----------|----------|-----|
| Retry policy | domain-* | What's acceptable latency for retry? |
| User experience | domain-* | What message should users see? |
| Compliance | domain-* | What must be logged for audit? |
---
## Trace Down ↓
To implementation (Layer 1):
```
"Need typed errors"
↓ m06-error-handling: thiserror for library
↓ m04-zero-cost: Error enum design
"Need error context"
↓ m06-error-handling: anyhow::Context
↓ Logging: tracing with fields
"Need retry logic"
↓ m07-concurrency: async retry patterns
↓ Crates: tokio-retry, backoff
```
---
## Quick Reference
| Recovery Pattern | When | Implementation |
|------------------|------|----------------|
| Retry | Transient failures | exponential backoff |
| Fallback | Degraded mode | cached/default value |
| Circuit Breaker | Cascading failures | failsafe-rs |
| Timeout | Slow operations | `tokio::time::timeout` |
| Bulkhead | Isolation | separate thread pools |
## Error Hierarchy
```rust
#[derive(thiserror::Error, Debug)]
pub enum AppError {
// User-facing
#[error("Invalid input: {0}")]
Validation(String),
// Transient (retryable)
#[error("Service temporarily unavailable")]
ServiceUnavailable(#[source] reqwest::Error),
// Internal (log details, show generic)
#[error("Internal error")]
Internal(#[source] anyhow::Error),
}
impl AppError {
pub fn is_retryable(&self) -> bool {
matches!(self, Self::ServiceUnavailable(_))
}
}
```
## Retry Pattern
```rust
use tokio_retry::{Retry, strategy::ExponentialBackoff};
async fn with_retry<F, T, E>(f: F) -> Result<T, E>
where
F: Fn() -> impl Future<Output = Result<T, E>>,
E: std::fmt::Debug,
{
let strategy = ExponentialBackoff::from_millis(100)
.max_delay(Duration::from_secs(10))
.take(5);
Retry::spawn(strategy, || f()).await
}
```
---
## Common Mistakes
| Mistake | Why Wrong | Better |
|---------|-----------|--------|
| Same error for all | No actionability | Categorize by audience |
| Retry everything | Wasted resources | Only transient errors |
| Infinite retry | DoS self | Max attempts + backoff |
| Expose internal errors | Security risk | User-friendly messages |
| No context | Hard to debug | .context() everywhere |
---
## Anti-Patterns
| Anti-Pattern | Why Bad | Better |
|--------------|---------|--------|
| String errors | No structure | thiserror types |
| panic! for recoverable | Bad UX | Result with context |
| Ignore errors | Silent failures | Log or propagate |
| Box<dyn Error> everywhere | Lost type info | thiserror |
| Error in happy path | Performance | Early validation |
---
## Related Skills
| When | See |
|------|-----|
| Error handling basics | m06-error-handling |
| Retry implementation | m07-concurrency |
| Domain modeling | m09-domain |
| User-facing APIs | domain-* |
-176
View File
@@ -1,176 +0,0 @@
---
name: m14-mental-model
description: "Use when learning Rust concepts. Keywords: mental model, how to think about ownership, understanding borrow checker, visualizing memory layout, analogy, misconception, explaining ownership, why does Rust, help me understand, confused about, learning Rust, explain like I'm, ELI5, intuition for, coming from Java, coming from Python, 心智模型, 如何理解所有权, 学习 Rust, Rust 入门, 为什么 Rust"
---
# Mental Models
> **Layer 2: Design Choices**
## Core Question
**What's the right way to think about this Rust concept?**
When learning or explaining Rust:
- What's the correct mental model?
- What misconceptions should be avoided?
- What analogies help understanding?
---
## Key Mental Models
| Concept | Mental Model | Analogy |
|---------|--------------|---------|
| Ownership | Unique key | Only one person has the house key |
| Move | Key handover | Giving away your key |
| `&T` | Lending for reading | Lending a book |
| `&mut T` | Exclusive editing | Only you can edit the doc |
| Lifetime `'a` | Valid scope | "Ticket valid until..." |
| `Box<T>` | Heap pointer | Remote control to TV |
| `Rc<T>` | Shared ownership | Multiple remotes, last turns off |
| `Arc<T>` | Thread-safe Rc | Remotes from any room |
---
## Coming From Other Languages
| From | Key Shift |
|------|-----------|
| Java/C# | Values are owned, not references by default |
| C/C++ | Compiler enforces safety rules |
| Python/Go | No GC, deterministic destruction |
| Functional | Mutability is safe via ownership |
| JavaScript | No null, use Option instead |
---
## Thinking Prompt
When confused about Rust:
1. **What's the ownership model?**
- Who owns this data?
- How long does it live?
- Who can access it?
2. **What guarantee is Rust providing?**
- No data races
- No dangling pointers
- No use-after-free
3. **What's the compiler telling me?**
- Error = violation of safety rule
- Solution = work with the rules
---
## Trace Up ↑
To design understanding (Layer 2):
```
"Why can't I do X in Rust?"
↑ Ask: What safety guarantee would be violated?
↑ Check: m01-m07 for the rule being enforced
↑ Ask: What's the intended design pattern?
```
---
## Trace Down ↓
To implementation (Layer 1):
```
"I understand the concept, now how do I implement?"
↓ m01-ownership: Ownership patterns
↓ m02-resource: Smart pointer choice
↓ m07-concurrency: Thread safety
```
---
## Common Misconceptions
| Error | Wrong Model | Correct Model |
|-------|-------------|---------------|
| E0382 use after move | GC cleans up | Ownership = unique key transfer |
| E0502 borrow conflict | Multiple writers OK | Only one writer at a time |
| E0499 multiple mut borrows | Aliased mutation | Exclusive access for mutation |
| E0106 missing lifetime | Ignoring scope | References have validity scope |
| E0507 cannot move from `&T` | Implicit clone | References don't own data |
## Deprecated Thinking
| Deprecated | Better |
|------------|--------|
| "Rust is like C++" | Different ownership model |
| "Lifetimes are GC" | Compile-time validity scope |
| "Clone solves everything" | Restructure ownership |
| "Fight the borrow checker" | Work with the compiler |
| "`unsafe` to avoid rules" | Understand safe patterns first |
---
## Ownership Visualization
```
Stack Heap
+----------------+ +----------------+
| main() | | |
| s1 ─────────────────────> │ "hello" |
| | | |
| fn takes(s) { | | |
| s2 (moved) ─────────────> │ "hello" |
| } | | (s1 invalid) |
+----------------+ +----------------+
After move: s1 is no longer valid
```
## Reference Visualization
```
+----------------+
| data: String |────────────> "hello"
+----------------+
│ &data (immutable borrow)
+------+------+
| reader1 reader2 (multiple OK)
+------+------+
+----------------+
| data: String |────────────> "hello"
+----------------+
│ &mut data (mutable borrow)
+------+
| writer (only one)
+------+
```
---
## Learning Path
| Stage | Focus | Skills |
|-------|-------|--------|
| Beginner | Ownership basics | m01-ownership, m14-mental-model |
| Intermediate | Smart pointers, error handling | m02, m06 |
| Advanced | Concurrency, unsafe | m07, unsafe-checker |
| Expert | Design patterns | m09-m15, domain-* |
---
## Related Skills
| When | See |
|------|-----|
| Ownership errors | m01-ownership |
| Smart pointers | m02-resource |
| Concurrency | m07-concurrency |
| Anti-patterns | m15-anti-pattern |
@@ -1,286 +0,0 @@
# Thinking in Rust: Mental Models
## Core Mental Models
### 1. Ownership as Resource Management
```
Traditional: "Who has a pointer to this data?"
Rust: "Who OWNS this data and is responsible for freeing it?"
```
Key insight: Every value has exactly one owner. When the owner goes out of scope, the value is dropped.
```rust
{
let s = String::from("hello"); // s owns the String
// use s...
} // s goes out of scope, String is dropped (memory freed)
```
### 2. Borrowing as Temporary Access
```
Traditional: "I'll just read from this pointer"
Rust: "I'm borrowing this value, owner still responsible for it"
```
Key insight: Borrows are like library books - you can read them, but must return them.
```rust
fn print_length(s: &String) { // borrows s
println!("{}", s.len());
} // borrow ends, caller still owns s
let my_string = String::from("hello");
print_length(&my_string); // lend to function
println!("{}", my_string); // still have it
```
### 3. Lifetimes as Validity Scopes
```
Traditional: "Hope this pointer is still valid"
Rust: "Compiler tracks exactly how long references are valid"
```
Key insight: A reference can't outlive the data it points to.
```rust
fn longest<'a>(x: &'a str, y: &'a str) -> &'a str {
// 'a means: the returned reference is valid as long as BOTH inputs are valid
if x.len() > y.len() { x } else { y }
}
```
---
## Shifting Perspectives
### From "Everything is a Reference" (Java/C#)
Java mental model:
```java
// Everything is implicitly a reference
User user = new User("Alice"); // user is a reference
List<User> users = new ArrayList<>();
users.add(user); // shares the reference
user.setName("Bob"); // affects the list too!
```
Rust mental model:
```rust
// Values are owned, sharing is explicit
let user = User::new("Alice"); // user is owned
let mut users = vec![];
users.push(user); // user moved into vec, can't use user anymore
// user.set_name("Bob"); // ERROR: user was moved
// If you need sharing:
use std::rc::Rc;
let user = Rc::new(User::new("Alice"));
let user2 = Rc::clone(&user); // explicit shared ownership
```
### From "Manual Memory Management" (C/C++)
C mental model:
```c
char* s = malloc(100);
// ... must remember to free(s) ...
// ... what if we return early? ...
// ... what if an exception occurs? ...
free(s);
```
Rust mental model:
```rust
let s = String::with_capacity(100);
// ... use s ...
// No need to free - Rust drops s automatically when scope ends
// Even with early returns, panics, or any control flow
```
### From "Garbage Collection" (Go/Python)
GC mental model:
```python
# Create objects, GC will figure it out
users = []
for name in names:
users.append(User(name))
# GC runs sometime later, when it feels like it
```
Rust mental model:
```rust
let users: Vec<User> = names
.iter()
.map(|name| User::new(name))
.collect();
// Memory is freed EXACTLY when users goes out of scope
// Deterministic, no GC pauses, no unpredictable memory usage
```
---
## Key Questions to Ask
### When Designing Functions
1. **Does this function need to own the data, or just read it?**
- Need to keep it: take ownership (`fn process(data: Vec<T>)`)
- Just reading: borrow (`fn process(data: &[T])`)
- Need to modify: mutable borrow (`fn process(data: &mut Vec<T>)`)
2. **Does the return value contain references to inputs?**
- Yes: need lifetime annotations
- No: lifetime elision usually works
### When Designing Structs
1. **Should this struct own its data or reference it?**
- Long-lived, independent: own (`name: String`)
- Short-lived view: reference (`name: &'a str`)
2. **Do multiple parts need to access the same data?**
- Single-threaded: `Rc<T>` or `Rc<RefCell<T>>`
- Multi-threaded: `Arc<T>` or `Arc<Mutex<T>>`
### When Hitting Borrow Checker Errors
1. **Am I trying to use a value after moving it?**
- Clone it, borrow it, or restructure the code
2. **Am I trying to have multiple mutable references?**
- Scope the mutations, use interior mutability, or redesign
3. **Does a reference outlive its source?**
- Return owned data instead, or use `'static`
---
## Common Patterns
### The Clone Escape Hatch
When fighting the borrow checker, `.clone()` often works:
```rust
// Can't do this - double borrow
let mut map = HashMap::new();
for key in map.keys() {
map.insert(key.clone(), process(key)); // ERROR: map borrowed twice
}
// Clone to escape
let keys: Vec<_> = map.keys().cloned().collect();
for key in keys {
map.insert(key.clone(), process(&key)); // OK
}
```
But ask: "Is there a better design?" Often, restructuring is better than cloning.
### The "Make It Own" Pattern
When lifetimes get complex, make the struct own its data:
```rust
// Complex: struct with references
struct Parser<'a> {
input: &'a str,
current: &'a str,
}
// Simpler: struct owns data
struct Parser {
input: String,
position: usize,
}
```
### The "Split the Borrow" Pattern
```rust
struct Data {
field_a: Vec<i32>,
field_b: Vec<i32>,
}
// Can't borrow self mutably twice
fn process(&mut self) {
// for a in &self.field_a {
// self.field_b.push(*a); // ERROR
// }
// Split the borrow
let Data { field_a, field_b } = self;
for a in field_a.iter() {
field_b.push(*a); // OK: separate borrows
}
}
```
---
## The Rust Way
### Embrace the Type System
```rust
// Don't: stringly-typed
fn connect(host: &str, port: &str) { ... }
connect("8080", "localhost"); // oops, wrong order
// Do: strongly-typed
struct Host(String);
struct Port(u16);
fn connect(host: Host, port: Port) { ... }
// connect(Port(8080), Host("localhost".into())); // compile error!
```
### Make Invalid States Unrepresentable
```rust
// Don't: runtime checks
struct Connection {
socket: Option<Socket>,
connected: bool,
}
// Do: types enforce states
enum Connection {
Disconnected,
Connected { socket: Socket },
}
```
### Let the Compiler Guide You
```rust
// Start with what you want
fn process(data: ???) -> ???
// Let compiler errors tell you:
// - What types are needed
// - What lifetimes are needed
// - What bounds are needed
// The error messages are documentation!
```
---
## Summary: The Rust Mental Model
1. **Values have owners** - exactly one at a time
2. **Borrowing is lending** - temporary access, owner retains responsibility
3. **Lifetimes are scopes** - compiler tracks validity
4. **Types encode constraints** - use them to prevent bugs
5. **The compiler is your friend** - work with it, not against it
When stuck:
- Clone to make progress
- Restructure to own instead of borrow
- Ask: "What is the compiler trying to tell me?"
-159
View File
@@ -1,159 +0,0 @@
---
name: m15-anti-pattern
description: "Use when reviewing code for anti-patterns. Keywords: anti-pattern, common mistake, pitfall, code smell, bad practice, code review, is this an anti-pattern, better way to do this, common mistake to avoid, why is this bad, idiomatic way, beginner mistake, fighting borrow checker, clone everywhere, unwrap in production, should I refactor, 反模式, 常见错误, 代码异味, 最佳实践, 地道写法"
---
# Anti-Patterns
> **Layer 2: Design Choices**
## Core Question
**Is this pattern hiding a design problem?**
When reviewing code:
- Is this solving the symptom or the cause?
- Is there a more idiomatic approach?
- Does this fight or flow with Rust?
---
## Anti-Pattern → Better Pattern
| Anti-Pattern | Why Bad | Better |
|--------------|---------|--------|
| `.clone()` everywhere | Hides ownership issues | Proper references or ownership |
| `.unwrap()` in production | Runtime panics | `?`, `expect`, or handling |
| `Rc` when single owner | Unnecessary overhead | Simple ownership |
| `unsafe` for convenience | UB risk | Find safe pattern |
| OOP via `Deref` | Misleading API | Composition, traits |
| Giant match arms | Unmaintainable | Extract to methods |
| `String` everywhere | Allocation waste | `&str`, `Cow<str>` |
| Ignoring `#[must_use]` | Lost errors | Handle or `let _ =` |
---
## Thinking Prompt
When seeing suspicious code:
1. **Is this symptom or cause?**
- Clone to avoid borrow? → Ownership design issue
- Unwrap "because it won't fail"? → Unhandled case
2. **What would idiomatic code look like?**
- References instead of clones
- Iterators instead of index loops
- Pattern matching instead of flags
3. **Does this fight Rust?**
- Fighting borrow checker → restructure
- Excessive unsafe → find safe pattern
---
## Trace Up ↑
To design understanding:
```
"Why does my code have so many clones?"
↑ Ask: Is the ownership model correct?
↑ Check: m09-domain (data flow design)
↑ Check: m01-ownership (reference patterns)
```
| Anti-Pattern | Trace To | Question |
|--------------|----------|----------|
| Clone everywhere | m01-ownership | Who should own this data? |
| Unwrap everywhere | m06-error-handling | What's the error strategy? |
| Rc everywhere | m09-domain | Is ownership clear? |
| Fighting lifetimes | m09-domain | Should data structure change? |
---
## Trace Down ↓
To implementation (Layer 1):
```
"Replace clone with proper ownership"
↓ m01-ownership: Reference patterns
↓ m02-resource: Smart pointer if needed
"Replace unwrap with proper handling"
↓ m06-error-handling: ? operator
↓ m06-error-handling: expect with message
```
---
## Top 5 Beginner Mistakes
| Rank | Mistake | Fix |
|------|---------|-----|
| 1 | Clone to escape borrow checker | Use references |
| 2 | Unwrap in production | Propagate with `?` |
| 3 | String for everything | Use `&str` |
| 4 | Index loops | Use iterators |
| 5 | Fighting lifetimes | Restructure to own data |
## Code Smell → Refactoring
| Smell | Indicates | Refactoring |
|-------|-----------|-------------|
| Many `.clone()` | Ownership unclear | Clarify data flow |
| Many `.unwrap()` | Error handling missing | Add proper handling |
| Many `pub` fields | Encapsulation broken | Private + accessors |
| Deep nesting | Complex logic | Extract methods |
| Long functions | Multiple responsibilities | Split |
| Giant enums | Missing abstraction | Trait + types |
---
## Common Error Patterns
| Error | Anti-Pattern Cause | Fix |
|-------|-------------------|-----|
| E0382 use after move | Cloning vs ownership | Proper references |
| Panic in production | Unwrap everywhere | ?, matching |
| Slow performance | String for all text | &str, Cow |
| Borrow checker fights | Wrong structure | Restructure |
| Memory bloat | Rc/Arc everywhere | Simple ownership |
---
## Deprecated → Better
| Deprecated | Better |
|------------|--------|
| Index-based loops | `.iter()`, `.enumerate()` |
| `collect::<Vec<_>>()` then iterate | Chain iterators |
| Manual unsafe cell | `Cell`, `RefCell` |
| `mem::transmute` for casts | `as` or `TryFrom` |
| Custom linked list | `Vec`, `VecDeque` |
| `lazy_static!` | `std::sync::OnceLock` |
---
## Quick Review Checklist
- [ ] No `.clone()` without justification
- [ ] No `.unwrap()` in library code
- [ ] No `pub` fields with invariants
- [ ] No index loops when iterator works
- [ ] No `String` where `&str` suffices
- [ ] No ignored `#[must_use]` warnings
- [ ] No `unsafe` without SAFETY comment
- [ ] No giant functions (>50 lines)
---
## Related Skills
| When | See |
|------|-----|
| Ownership patterns | m01-ownership |
| Error handling | m06-error-handling |
| Mental models | m14-mental-model |
| Performance | m10-performance |
@@ -1,421 +0,0 @@
# Common Rust Anti-Patterns & Mistakes
## Ownership Anti-Patterns
### 1. Clone Everything
```rust
// ANTI-PATTERN: clone to avoid borrow checker
fn process(data: Vec<String>) {
for item in data.clone() { // unnecessary clone
println!("{}", item);
}
use_data(data);
}
// BETTER: borrow when you don't need ownership
fn process(data: Vec<String>) {
for item in &data { // borrow instead
println!("{}", item);
}
use_data(data);
}
```
### 2. Unnecessary Box
```rust
// ANTI-PATTERN: boxing everything
fn get_value() -> Box<String> {
Box::new(String::from("hello"))
}
// BETTER: return value directly
fn get_value() -> String {
String::from("hello")
}
```
### 3. Holding References Too Long
```rust
// ANTI-PATTERN: borrow prevents mutation
let mut data = vec![1, 2, 3];
let first = &data[0];
data.push(4); // ERROR: data is borrowed
println!("{}", first);
// BETTER: scope the borrow
let mut data = vec![1, 2, 3];
let first = data[0]; // copy the value
data.push(4); // OK
println!("{}", first);
```
---
## Error Handling Anti-Patterns
### 4. Unwrap Everywhere
```rust
// ANTI-PATTERN: crashes on error
fn process_file(path: &str) {
let content = std::fs::read_to_string(path).unwrap();
let config: Config = toml::from_str(&content).unwrap();
}
// BETTER: propagate errors
fn process_file(path: &str) -> Result<Config, Error> {
let content = std::fs::read_to_string(path)?;
let config: Config = toml::from_str(&content)?;
Ok(config)
}
```
### 5. Ignoring Errors
```rust
// ANTI-PATTERN: silent failure
let _ = file.write_all(data);
// BETTER: handle or propagate
file.write_all(data)?;
// or at minimum, log the error
if let Err(e) = file.write_all(data) {
eprintln!("Warning: failed to write: {}", e);
}
```
### 6. Panic in Library Code
```rust
// ANTI-PATTERN: library panics
pub fn parse(input: &str) -> Data {
if input.is_empty() {
panic!("input cannot be empty");
}
// ...
}
// BETTER: return Result
pub fn parse(input: &str) -> Result<Data, ParseError> {
if input.is_empty() {
return Err(ParseError::EmptyInput);
}
// ...
}
```
---
## String Anti-Patterns
### 7. String Instead of &str
```rust
// ANTI-PATTERN: forces allocation
fn greet(name: String) {
println!("Hello, {}", name);
}
greet("world".to_string()); // allocation
// BETTER: accept &str
fn greet(name: &str) {
println!("Hello, {}", name);
}
greet("world"); // no allocation
```
### 8. Format for Simple Concatenation
```rust
// ANTI-PATTERN: format overhead
let greeting = format!("{}{}", "Hello, ", name);
// BETTER for simple cases: push_str
let mut greeting = String::from("Hello, ");
greeting.push_str(name);
// Or use + for String + &str
let greeting = String::from("Hello, ") + name;
```
### 9. Repeated String Operations
```rust
// ANTI-PATTERN: O(n²) allocations
let mut result = String::new();
for word in words {
result = result + word + " ";
}
// BETTER: join
let result = words.join(" ");
// Or with_capacity + push_str
let mut result = String::with_capacity(total_len);
for word in words {
result.push_str(word);
result.push(' ');
}
```
---
## Collection Anti-Patterns
### 10. Index Instead of Iterator
```rust
// ANTI-PATTERN: bounds checking overhead
for i in 0..vec.len() {
process(vec[i]);
}
// BETTER: iterator
for item in &vec {
process(item);
}
```
### 11. Collect Then Iterate
```rust
// ANTI-PATTERN: unnecessary allocation
let filtered: Vec<_> = items.iter().filter(|x| x.valid).collect();
for item in filtered {
process(item);
}
// BETTER: chain iterators
for item in items.iter().filter(|x| x.valid) {
process(item);
}
```
### 12. Wrong Collection Type
```rust
// ANTI-PATTERN: Vec for frequent membership checks
let allowed: Vec<&str> = vec!["a", "b", "c"];
if allowed.contains(&input) { ... } // O(n)
// BETTER: HashSet for membership
use std::collections::HashSet;
let allowed: HashSet<&str> = ["a", "b", "c"].into();
if allowed.contains(input) { ... } // O(1)
```
---
## Concurrency Anti-Patterns
### 13. Mutex for Read-Heavy Data
```rust
// ANTI-PATTERN: Mutex when mostly reading
let data = Arc::new(Mutex::new(config));
// All readers block each other
// BETTER: RwLock for read-heavy workloads
let data = Arc::new(RwLock::new(config));
// Multiple readers can proceed in parallel
```
### 14. Holding Lock Across Await
```rust
// ANTI-PATTERN: lock held across await
async fn bad() {
let guard = mutex.lock().unwrap();
some_async_op().await; // lock held!
use(guard);
}
// BETTER: scope the lock
async fn good() {
let value = {
let guard = mutex.lock().unwrap();
guard.clone()
}; // lock released
some_async_op().await;
use(value);
}
```
### 15. Blocking in Async
```rust
// ANTI-PATTERN: blocking call in async
async fn bad() {
std::thread::sleep(Duration::from_secs(1)); // blocks executor!
}
// BETTER: async sleep
async fn good() {
tokio::time::sleep(Duration::from_secs(1)).await;
}
// For CPU work: spawn_blocking
async fn compute() {
tokio::task::spawn_blocking(|| heavy_work()).await
}
```
---
## Type System Anti-Patterns
### 16. Stringly Typed
```rust
// ANTI-PATTERN: strings for everything
fn connect(host: &str, port: &str, timeout: &str) { ... }
connect("8080", "localhost", "30"); // wrong order!
// BETTER: strong types
struct Host(String);
struct Port(u16);
struct Timeout(Duration);
fn connect(host: Host, port: Port, timeout: Timeout) { ... }
```
### 17. Boolean Parameters
```rust
// ANTI-PATTERN: what does true mean?
fn fetch(url: &str, use_cache: bool, validate_ssl: bool) { ... }
fetch("https://...", true, false); // unclear
// BETTER: builder or named parameters
struct FetchOptions {
use_cache: bool,
validate_ssl: bool,
}
fn fetch(url: &str, options: FetchOptions) { ... }
fetch("https://...", FetchOptions {
use_cache: true,
validate_ssl: false,
});
```
### 18. Option<Option<T>>
```rust
// ANTI-PATTERN: nested Option
fn find(id: u32) -> Option<Option<User>> { ... }
// What does None vs Some(None) mean?
// BETTER: use Result or custom enum
enum FindResult {
Found(User),
NotFound,
Error(String),
}
```
---
## API Design Anti-Patterns
### 19. Taking Ownership Unnecessarily
```rust
// ANTI-PATTERN: takes ownership but doesn't need it
fn validate(config: Config) -> bool {
config.timeout > 0 && config.retries >= 0
}
// BETTER: borrow
fn validate(config: &Config) -> bool {
config.timeout > 0 && config.retries >= 0
}
```
### 20. Returning References to Temporaries
```rust
// ANTI-PATTERN: impossible lifetime
fn get_default() -> &str {
let s = String::from("default");
&s // ERROR: s is dropped
}
// BETTER: return owned
fn get_default() -> String {
String::from("default")
}
// Or return static
fn get_default() -> &'static str {
"default"
}
```
### 21. Overly Generic Functions
```rust
// ANTI-PATTERN: complex generics for simple function
fn process<T, U, V>(input: T) -> V
where
T: Into<U>,
U: AsRef<str> + Clone,
V: From<String>,
{ ... }
// BETTER: concrete types if generics not needed
fn process(input: &str) -> String { ... }
```
---
## Macro Anti-Patterns
### 22. Macro When Function Works
```rust
// ANTI-PATTERN: macro for simple operation
macro_rules! add {
($a:expr, $b:expr) => { $a + $b };
}
// BETTER: just use a function
fn add(a: i32, b: i32) -> i32 { a + b }
```
### 23. Complex Macro Without Tests
```rust
// ANTI-PATTERN: complex macro with no tests
macro_rules! define_api {
// ... 100 lines of macro code ...
}
// BETTER: test macro outputs
#[test]
fn test_macro_expansion() {
// Use cargo-expand or trybuild
}
```
---
## Quick Reference
| Anti-Pattern | Better Alternative |
|--------------|-------------------|
| Clone everywhere | Borrow when possible |
| Unwrap everywhere | Propagate with `?` |
| `String` parameters | `&str` parameters |
| Index loops | Iterator loops |
| Collect then process | Chain iterators |
| Mutex for reads | RwLock for read-heavy |
| Lock across await | Scope the lock |
| Blocking in async | spawn_blocking |
| Stringly typed | Strong types |
| Boolean params | Builders or enums |
-55
View File
@@ -1,55 +0,0 @@
---
name: rust-daily
description: |
CRITICAL: Use for Rust news and daily/weekly/monthly reports. Triggers on:
rust news, rust daily, rust weekly, TWIR, rust blog,
Rust 日报, Rust 周报, Rust 新闻, Rust 动态
---
# Rust Daily Report
Fetch Rust community updates, filtered by time range.
## Data Sources
| Category | Sources |
|----------|---------|
| Ecosystem | Reddit r/rust, This Week in Rust |
| Official | blog.rust-lang.org, Inside Rust |
| Foundation | rustfoundation.org (news, blog, events) |
## Parameters
- `time_range`: day | week | month (default: week)
- `category`: all | ecosystem | official | foundation
## Execution
Read agent file then launch Task:
```
1. Read: ../../agents/rust-daily-reporter.md
2. Task(subagent_type: "general-purpose", run_in_background: false, prompt: <agent content>)
```
## Output Format
```markdown
# Rust {Weekly|Daily|Monthly} Report
**Time Range:** {start} - {end}
## Ecosystem
| Score | Title | Link |
## Official
| Date | Title | Summary |
## Foundation
| Date | Title | Summary |
```
## Validation
- Each source should have at least 1 result, otherwise mark "No updates"
- On fetch failure, retry with alternative tool
-171
View File
@@ -1,171 +0,0 @@
---
name: rust-learner
description: "Use when asking about Rust versions or crate info. Keywords: latest version, what's new, changelog, Rust 1.x, Rust release, stable, nightly, crate info, crates.io, lib.rs, docs.rs, API documentation, crate features, dependencies, which crate, what version, Rust edition, edition 2021, edition 2024, cargo add, cargo update, 最新版本, 版本号, 稳定版, 最新, 哪个版本, crate 信息, 文档, 依赖, Rust 版本, 新特性, 有什么特性"
---
# Rust Learner
> **Version:** 1.1.0 | **Last Updated:** 2026-01-16
You are an expert at fetching Rust and crate information. Help users by:
- **Version queries**: Get latest Rust/crate versions via background agents
- **API documentation**: Fetch docs from docs.rs
- **Changelog**: Get Rust version features from releases.rs
**Primary skill for fetching Rust/crate information. All agents run in background.**
## CRITICAL: How to Launch Agents
**All agents MUST be launched via Task tool with these parameters:**
```
Task(
subagent_type: "general-purpose",
run_in_background: true,
prompt: <read from ../../agents/*.md file>
)
```
**Workflow:**
1. Read the agent prompt file: `../../agents/<agent-name>.md` (relative to this skill)
2. Launch Task with `run_in_background: true`
3. Continue with other work or wait for completion
4. Read results when agent completes
## Agent Routing Table
| Query Type | Agent File | Source |
|------------|------------|--------|
| Rust version features | `../../agents/rust-changelog.md` | releases.rs |
| Crate info/version | `../../agents/crate-researcher.md` | lib.rs, crates.io |
| **Std library docs** (Send, Sync, Arc, etc.) | `../../agents/std-docs-researcher.md` | doc.rust-lang.org |
| Third-party crate docs (tokio, serde, etc.) | `../../agents/docs-researcher.md` | docs.rs |
| Clippy lints | `../../agents/clippy-researcher.md` | rust-clippy docs |
| **Rust news/daily report** | `../../agents/rust-daily-reporter.md` | Reddit, TWIR, blogs |
### Choosing docs-researcher vs std-docs-researcher
| Query Pattern | Use Agent |
|---------------|-----------|
| `std::*`, `Send`, `Sync`, `Arc`, `Rc`, `Box`, `Vec`, `String` | `std-docs-researcher` |
| `tokio::*`, `serde::*`, any third-party crate | `docs-researcher` |
## Tool Chain
All agents use this tool chain (in order):
1. **actionbook MCP** (first - get pre-computed selectors)
- `mcp__actionbook__search_actions("site_name")` → get action ID
- `mcp__actionbook__get_action_by_id(id)` → get URL + selectors
2. **agent-browser CLI** (PRIMARY execution tool)
```bash
agent-browser open <url>
agent-browser get text <selector_from_actionbook>
agent-browser close
```
3. **WebFetch** (LAST RESORT only if agent-browser unavailable)
### Fallback Principle (CRITICAL)
```
actionbook → agent-browser → WebFetch (only if agent-browser unavailable)
```
**DO NOT:**
- Skip agent-browser because it's slower
- Use WebFetch as primary when agent-browser is available
- Block on WebFetch without trying agent-browser first
## Example: Crate Version Query
```
User: "tokio latest version"
Claude:
1. Read ../../agents/crate-researcher.md
2. Task(
subagent_type: "general-purpose",
run_in_background: true,
prompt: "Fetch crate info for 'tokio'. Use actionbook MCP to get lib.rs selectors, then agent-browser to fetch. Return: name, version, description, features."
)
3. Wait for agent or continue with other work
4. Summarize results to user
```
## Example: Third-Party Crate Documentation
```
User: "tokio::spawn documentation"
Claude:
1. Read ../../agents/docs-researcher.md
2. Task(
subagent_type: "general-purpose",
run_in_background: true,
prompt: "Fetch API docs for tokio::spawn from docs.rs. Use agent-browser first. Return: signature, description, examples."
)
3. Wait for agent
4. Summarize API to user
```
## Example: Std Library Documentation
```
User: "Send trait documentation"
Claude:
1. Read ../../agents/std-docs-researcher.md (NOT docs-researcher!)
2. Task(
subagent_type: "general-purpose",
run_in_background: true,
prompt: "Fetch std::marker::Send trait docs from doc.rust-lang.org. Use agent-browser first. Return: description, implementors, examples."
)
3. Wait for agent
4. Summarize trait to user
```
## Example: Rust Changelog Query
```
User: "What's new in Rust 1.85?"
Claude:
1. Read ../../agents/rust-changelog.md
2. Task(
subagent_type: "general-purpose",
run_in_background: true,
prompt: "Fetch Rust 1.85 changelog from releases.rs. Use actionbook MCP for selectors, agent-browser to fetch. Return: language features, library changes, stabilized APIs."
)
3. Wait for agent
4. Summarize features to user
```
## Deprecated Patterns
| Deprecated | Use Instead | Reason |
|------------|-------------|--------|
| WebSearch for crate info | Task + crate-researcher | Structured data |
| Direct WebFetch | Task + actionbook | Pre-computed selectors |
| Foreground agent execution | `run_in_background: true` | Non-blocking |
| Guessing version numbers | Always use agents | Prevents misinformation |
## Error Handling
| Error | Cause | Solution |
|-------|-------|----------|
| actionbook unavailable | MCP not configured | Fall back to WebFetch |
| agent-browser not found | CLI not installed | Fall back to WebFetch |
| Agent timeout | Site slow/down | Retry or inform user |
| Empty results | Selector mismatch | Report and use WebFetch fallback |
## Proactive Triggering
This skill triggers AUTOMATICALLY when:
- Any Rust crate name mentioned (tokio, serde, axum, sqlx, etc.)
- Questions about "latest", "new", "version", "changelog"
- API documentation requests
- Dependency/feature questions
**DO NOT use WebSearch for Rust crate info. Launch background agents instead.**
-341
View File
@@ -1,341 +0,0 @@
---
name: rust-router
description: "CRITICAL: Use for ALL Rust questions including errors, design, and coding.
Triggers on: Rust, cargo, rustc, crate, Cargo.toml,
意图分析, 问题分析, 语义分析, analyze intent, question analysis,
compile error, borrow error, lifetime error, ownership error, type error, trait error,
value moved, cannot borrow, does not live long enough, mismatched types, not satisfied,
E0382, E0597, E0277, E0308, E0499, E0502, E0596,
async, await, Send, Sync, tokio, unsafe, FFI, concurrency, error handling,
编译错误, compile error, 所有权, ownership, 借用, borrow, 生命周期, lifetime, 类型错误, type error,
异步, async, 并发, concurrency, 错误处理, error handling,
问题, problem, question, 怎么用, how to use, 如何, how to, 为什么, why,
什么是, what is, 帮我写, help me write, 实现, implement, 解释, explain, 区别, difference, 最佳实践, best practice"
globs: ["**/Cargo.toml", "**/*.rs"]
---
# Rust Question Router
> **Version:** 2.0.0 | **Last Updated:** 2026-01-17
>
> **New in v2.0:** Meta-Cognition Routing - Three-layer cognitive model for deeper answers
## Meta-Cognition Framework
### Core Principle
**Don't answer directly. Trace through the cognitive layers first.**
```
Layer 3: Domain Constraints (WHY)
├── Business rules, regulatory requirements
├── domain-fintech, domain-web, domain-cli, etc.
└── "Why is it designed this way?"
Layer 2: Design Choices (WHAT)
├── Architecture patterns, DDD concepts
├── m09-m15 skills
└── "What pattern should I use?"
Layer 1: Language Mechanics (HOW)
├── Ownership, borrowing, lifetimes, traits
├── m01-m07 skills
└── "How do I implement this in Rust?"
```
### Routing by Entry Point
| User Signal | Entry Layer | Direction | First Skill |
|-------------|-------------|-----------|-------------|
| E0xxx error | Layer 1 | Trace UP ↑ | m01-m07 |
| Compile error | Layer 1 | Trace UP ↑ | Error table below |
| "How to design..." | Layer 2 | Check L3, then DOWN ↓ | m09-domain |
| "Building [domain] app" | Layer 3 | Trace DOWN ↓ | domain-* |
| "Best practice..." | Layer 2 | Both directions | m09-m15 |
| Performance issue | Layer 1 → 2 | UP then DOWN | m10-performance |
### CRITICAL: Dual-Skill Loading
**When domain keywords are present, you MUST load BOTH skills:**
| Domain Keywords | L1 Skill | L3 Skill |
|-----------------|----------|----------|
| Web API, HTTP, axum, handler | m07-concurrency | **domain-web** |
| 交易, 支付, trading, payment | m01-ownership | **domain-fintech** |
| CLI, terminal, clap | m07-concurrency | **domain-cli** |
| kubernetes, grpc, microservice | m07-concurrency | **domain-cloud-native** |
| embedded, no_std, MCU | m02-resource | **domain-embedded** |
**Example**: "Web API 报错 Rc cannot be sent"
- Load: m07-concurrency (L1 - Send/Sync error)
- Load: domain-web (L3 - Web state management)
- Answer must include both layers
### Trace Examples
```
User: "My trading system reports E0382"
1. Entry: Layer 1 (E0382 = ownership error)
2. Load: m01-ownership skill
3. Trace UP: What design led to this?
4. Check: domain-fintech (trading = immutable audit data)
5. Answer: Don't clone, use Arc<T> for shared immutable data
```
```
User: "How should I handle user authentication?"
1. Entry: Layer 2 (design question)
2. Trace UP to Layer 3: domain-web constraints
3. Load: domain-web skill (security, stateless HTTP)
4. Trace DOWN: m06-error-handling, m07-concurrency
5. Answer: JWT with proper error types, async handlers
```
---
## INSTRUCTIONS FOR CLAUDE
### Default Project Settings (When Creating Rust Code)
When creating new Rust projects or Cargo.toml files, ALWAYS use:
```toml
[package]
edition = "2024" # ALWAYS use latest stable edition
rust-version = "1.85"
[lints.rust]
unsafe_code = "warn"
[lints.clippy]
all = "warn"
pedantic = "warn"
```
**Rules:**
- ALWAYS use `edition = "2024"` (not 2021 or earlier)
- Include `rust-version` for MSRV clarity
- Enable clippy lints by default
- DO NOT use WebSearch for Rust questions - use skills and agents
### Meta-Cognition Routing Process
1. **Identify Entry Layer**
- E0xxx errors → Layer 1
- Design questions → Layer 2
- Domain-specific → Layer 3
2. **Load Appropriate Skill**
- Read the skill file for context
- Note the "Trace Up" and "Trace Down" sections
3. **Trace Through Layers**
- Don't stop at surface-level fix
- Ask "Why?" to trace up
- Ask "How?" to trace down
4. **Answer with Context**
- Include the reasoning chain
- Reference which layers/skills informed the answer
### When User Requests Intent Analysis
User may say: "analyze this question" / "what type of problem is this" / "analyze intent"
**Execute these steps:**
1. **Extract Keywords** - Identify Rust concepts, error codes, crate names
2. **Identify Entry Layer** - Which cognitive layer is this?
3. **Map to Skills** - Which m0x/m1x/domain skills apply?
4. **Report Analysis** - Tell user the layers and suggested trace
5. **Invoke Skill** - Load and apply the matched skill
---
## Layer 1 Skills (Language Mechanics)
| Pattern | Category | Route To |
|---------|----------|----------|
| move, borrow, lifetime, E0382, E0597 | m01 | m01-ownership |
| Box, Rc, Arc, RefCell, Cell | m02 | m02-resource |
| mut, interior mutability, E0499, E0502, E0596 | m03 | m03-mutability |
| generic, trait, inline, monomorphization | m04 | m04-zero-cost |
| type state, phantom, newtype | m05 | m05-type-driven |
| Result, Error, panic, ?, anyhow, thiserror | m06 | m06-error-handling |
| Send, Sync, thread, async, channel | m07 | m07-concurrency |
| unsafe, FFI, extern, raw pointer, transmute | - | **unsafe-checker** |
## Layer 2 Skills (Design Choices)
| Pattern | Category | Route To |
|---------|----------|----------|
| domain model, business logic | m09 | m09-domain |
| performance, optimization, benchmark | m10 | m10-performance |
| integration, interop, bindings | m11 | m11-ecosystem |
| resource lifecycle, RAII, Drop | m12 | m12-lifecycle |
| domain error, recovery strategy | m13 | m13-domain-error |
| mental model, how to think | m14 | m14-mental-model |
| anti-pattern, common mistake, pitfall | m15 | m15-anti-pattern |
## Layer 3 Skills (Domain Constraints)
| Domain Keywords | Route To |
|-----------------|----------|
| fintech, trading, decimal, currency | domain-fintech |
| ml, tensor, model, inference | domain-ml |
| kubernetes, docker, grpc, microservice | domain-cloud-native |
| embedded, sensor, mqtt, iot | domain-iot |
| web server, HTTP, REST, axum, actix | domain-web |
| CLI, command line, clap, terminal | domain-cli |
| no_std, microcontroller, firmware | domain-embedded |
---
## Error Code Routing
| Error Code | Layer | Route To | Common Cause |
|------------|-------|----------|--------------|
| E0382 | L1 | m01-ownership | Use of moved value |
| E0597 | L1 | m01-ownership | Lifetime too short |
| E0506 | L1 | m01-ownership | Cannot assign to borrowed |
| E0507 | L1 | m01-ownership | Cannot move out of borrowed |
| E0515 | L1 | m01-ownership | Return local reference |
| E0716 | L1 | m01-ownership | Temporary value dropped |
| E0106 | L1 | m01-ownership | Missing lifetime specifier |
| E0596 | L1 | m03-mutability | Cannot borrow as mutable |
| E0499 | L1 | m03-mutability | Multiple mutable borrows |
| E0502 | L1 | m03-mutability | Borrow conflict |
| E0277 | L1 | m04/m07 | Trait bound not satisfied |
| E0308 | L1 | m04-zero-cost | Type mismatch |
| E0599 | L1 | m04-zero-cost | No method found |
| E0038 | L1 | m04-zero-cost | Trait not object-safe |
| E0433 | L1 | m11-ecosystem | Cannot find crate/module |
---
## Unsafe-Specific Routing
For **detailed unsafe rules**, route to `unsafe-checker` skill:
| Pattern | Route To |
|---------|----------|
| unsafe code review, SAFETY comment | **unsafe-checker** |
| FFI, extern "C", C interop, libc | **unsafe-checker** |
| raw pointer, *mut, *const, NonNull | **unsafe-checker** |
| transmute, union, repr(C) | **unsafe-checker** |
| MaybeUninit, uninitialized memory | **unsafe-checker** |
| panic safety, double-free | **unsafe-checker** |
| Send impl, Sync impl, manual auto-traits | **unsafe-checker** |
---
## Functional Routing Table
| Pattern | Route To | Action |
|---------|----------|--------|
| latest version, what's new | **rust-learner** | Use agents |
| API, docs, documentation | **docs-researcher** | Use agent |
| Cargo.toml, dependencies | **dynamic-skills** | Suggest `/sync-crate-skills` |
| code style, naming, clippy | **coding-guidelines** | Read skill |
| unsafe code, FFI | **unsafe-checker** | Read skill |
| code review | **os-checker** | Suggest `/rust-review` |
---
## Priority Order
1. **Identify cognitive layer** (L1/L2/L3)
2. **Load entry skill** (m0x/m1x/domain)
3. **Trace through layers** (UP or DOWN)
4. **Cross-reference skills** as indicated in "Trace" sections
5. **Answer with reasoning chain**
## Skill File Paths
### Meta-Cognition Framework
```
_meta/reasoning-framework.md # How to trace through layers
_meta/layer-definitions.md # Layer definitions
_meta/externalization.md # Cognitive externalization
_meta/error-protocol.md # 3-Strike escalation rule
_meta/hooks-patterns.md # Automatic triggers
```
### Layer 1 Skills (Language Mechanics)
```
skills/m01-ownership/SKILL.md
skills/m02-resource/SKILL.md
skills/m03-mutability/SKILL.md
skills/m04-zero-cost/SKILL.md
skills/m05-type-driven/SKILL.md
skills/m06-error-handling/SKILL.md
skills/m07-concurrency/SKILL.md
```
### Layer 2 Skills (Design Choices)
```
skills/m09-domain/SKILL.md
skills/m10-performance/SKILL.md
skills/m11-ecosystem/SKILL.md
skills/m12-lifecycle/SKILL.md
skills/m13-domain-error/SKILL.md
skills/m14-mental-model/SKILL.md
skills/m15-anti-pattern/SKILL.md
```
### Layer 3 Skills (Domain Constraints)
```
skills/domain-fintech/SKILL.md
skills/domain-ml/SKILL.md
skills/domain-cloud-native/SKILL.md
skills/domain-iot/SKILL.md
skills/domain-web/SKILL.md
skills/domain-cli/SKILL.md
skills/domain-embedded/SKILL.md
```
---
## OS-Checker Integration
For code review and security auditing:
| Use Case | Command | Tools |
|----------|---------|-------|
| Daily check | `/rust-review` | clippy |
| Security audit | `/audit security` | cargo audit, geiger |
| Unsafe audit | `/audit safety` | miri, rudra |
| Concurrency audit | `/audit concurrency` | lockbud |
| Full audit | `/audit full` | all os-checker tools |
---
## Workflow Example with Meta-Cognition
```
User: "Why am I getting E0382 in my trading system?"
Analysis:
1. Entry: Layer 1 (E0382 = ownership/move error)
2. Load: m01-ownership skill
3. Context: "trading system" → domain-fintech
Trace UP ↑:
- E0382 in trading context
- Check domain-fintech: "immutable audit records"
- Finding: Trading data should be shared, not moved
Response:
"E0382 indicates a value was moved when still needed.
In a trading system (domain-fintech), transaction records
should be immutable and shareable for audit purposes.
Instead of cloning, consider:
- Arc<TradeRecord> for shared immutable access
- This aligns with financial audit requirements
See: m01-ownership (Trace Up section),
domain-fintech (Audit Requirements)"
```
-125
View File
@@ -1,125 +0,0 @@
---
name: rust-skill-creator
description: "Use when creating skills for Rust crates or std library documentation. Keywords: create rust skill, create crate skill, create std skill, 创建 rust skill, 创建 crate skill, 创建 std skill, 动态 rust skill, 动态 crate skill, skill for tokio, skill for serde, skill for axum, generate rust skill, rust 技能, crate 技能, 从文档创建skill, from docs create skill"
---
# Rust Skill Creator
> Create dynamic skills for Rust crates and std library documentation.
## When to Use
This skill handles requests to create skills for:
- Third-party crates (tokio, serde, axum, etc.)
- Rust standard library (std::sync, std::marker, etc.)
- Any Rust documentation URL
## Workflow
### 1. Identify the Target
| User Request | Target Type | URL Pattern |
|--------------|-------------|-------------|
| "create tokio skill" | Third-party crate | `docs.rs/tokio/latest/tokio/` |
| "create Send trait skill" | Std library | `doc.rust-lang.org/std/marker/trait.Send.html` |
| "create skill from URL" + URL | Custom URL | User-provided URL |
### 2. Execute the Command
Use the `/create-llms-for-skills` command:
```
/create-llms-for-skills <url> [requirements]
```
**Examples:**
```bash
# For third-party crate
/create-llms-for-skills https://docs.rs/tokio/latest/tokio/
# For std library
/create-llms-for-skills https://doc.rust-lang.org/std/marker/trait.Send.html
# With specific requirements
/create-llms-for-skills https://docs.rs/axum/latest/axum/ "Focus on routing and extractors"
```
### 3. Follow-up with Skill Creation
After llms.txt is generated, use:
```
/create-skills-via-llms <crate_name> <llms_path> [version]
```
## URL Construction Helper
| Target | URL Template |
|--------|--------------|
| Crate overview | `https://docs.rs/{crate}/latest/{crate}/` |
| Crate module | `https://docs.rs/{crate}/latest/{crate}/{module}/` |
| Std trait | `https://doc.rust-lang.org/std/{module}/trait.{Name}.html` |
| Std struct | `https://doc.rust-lang.org/std/{module}/struct.{Name}.html` |
| Std module | `https://doc.rust-lang.org/std/{module}/index.html` |
## Common Std Library Paths
| Item | Path |
|------|------|
| Send, Sync, Copy, Clone | `std/marker/trait.{Name}.html` |
| Arc, Mutex, RwLock | `std/sync/struct.{Name}.html` |
| Rc, Weak | `std/rc/struct.{Name}.html` |
| RefCell, Cell | `std/cell/struct.{Name}.html` |
| Box | `std/boxed/struct.Box.html` |
| Vec | `std/vec/struct.Vec.html` |
| String | `std/string/struct.String.html` |
| Option | `std/option/enum.Option.html` |
| Result | `std/result/enum.Result.html` |
## Example Interactions
### Example 1: Create Crate Skill
```
User: "Create a dynamic skill for tokio"
Claude:
1. Identify: Third-party crate "tokio"
2. Execute: /create-llms-for-skills https://docs.rs/tokio/latest/tokio/
3. Wait for llms.txt generation
4. Execute: /create-skills-via-llms tokio ~/tmp/{timestamp}-tokio-llms.txt
```
### Example 2: Create Std Library Skill
```
User: "Create a skill for Send and Sync traits"
Claude:
1. Identify: Std library traits
2. Execute: /create-llms-for-skills https://doc.rust-lang.org/std/marker/trait.Send.html https://doc.rust-lang.org/std/marker/trait.Sync.html
3. Wait for llms.txt generation
4. Execute: /create-skills-via-llms std-marker ~/tmp/{timestamp}-std-marker-llms.txt
```
### Example 3: Custom URL
```
User: "Create skill from https://docs.rs/sqlx/latest/sqlx/"
Claude:
1. Identify: User-provided URL
2. Execute: /create-llms-for-skills https://docs.rs/sqlx/latest/sqlx/
3. Follow standard workflow
```
## DO NOT
- Use `best-skill-creator` for Rust-related skill creation
- Skip the `/create-llms-for-skills` step
- Guess documentation URLs without verification
## Output Location
All generated skills are saved to: `~/.claude/skills/`
-136
View File
@@ -1,136 +0,0 @@
# Unsafe Checker - Quick Reference
**Auto-generated from rules/**
## Rule Summary by Section
### General Principles (3 rules)
| ID | Level | Title |
|----|-------|-------|
| general-01 | P | Do Not Abuse Unsafe to Escape Compiler Safety Checks |
| general-02 | P | Do Not Blindly Use Unsafe for Performance |
| general-03 | G | Do Not Create Aliases for Types/Methods Named "Unsafe" |
### Safety Abstraction (11 rules)
| ID | Level | Title |
|----|-------|-------|
| safety-01 | P | Be Aware of Memory Safety Issues from Panics |
| safety-02 | P | Unsafe Code Authors Must Verify Safety Invariants |
| safety-03 | P | Do Not Expose Uninitialized Memory in Public APIs |
| safety-04 | P | Avoid Double-Free from Panic Safety Issues |
| safety-05 | P | Consider Safety When Manually Implementing Auto Traits |
| safety-06 | P | Do Not Expose Raw Pointers in Public APIs |
| safety-07 | P | Provide Unsafe Counterparts for Performance Alongside Safe Methods |
| safety-08 | P | Mutable Return from Immutable Parameter is Wrong |
| safety-09 | P | Add SAFETY Comment Before Any Unsafe Block |
| safety-10 | G | Add Safety Section in Docs for Public Unsafe Functions |
| safety-11 | G | Use assert! Instead of debug_assert! in Unsafe Functions |
### Raw Pointers (6 rules)
| ID | Level | Title |
|----|-------|-------|
| ptr-01 | P | Do Not Share Raw Pointers Across Threads |
| ptr-02 | P | Prefer NonNull<T> Over *mut T |
| ptr-03 | P | Use PhantomData<T> for Variance and Ownership |
| ptr-04 | G | Do Not Dereference Pointers Cast to Misaligned Types |
| ptr-05 | G | Do Not Manually Convert Immutable Pointer to Mutable |
| ptr-06 | G | Prefer pointer::cast Over `as` for Pointer Casting |
### Union (2 rules)
| ID | Level | Title |
|----|-------|-------|
| union-01 | P | Avoid Union Except for C Interop |
| union-02 | P | Do Not Use Union Variants Across Different Lifetimes |
### Memory Layout (6 rules)
| ID | Level | Title |
|----|-------|-------|
| mem-01 | P | Choose Appropriate Data Layout for Struct/Tuple/Enum |
| mem-02 | P | Do Not Modify Memory Variables of Other Processes |
| mem-03 | P | Do Not Let String/Vec Auto-Drop Other Process's Memory |
| mem-04 | P | Prefer Reentrant Versions of C-API or Syscalls |
| mem-05 | P | Use Third-Party Crates for Bitfields |
| mem-06 | G | Use MaybeUninit<T> for Uninitialized Memory |
### FFI (18 rules)
| ID | Level | Title |
|----|-------|-------|
| ffi-01 | P | Avoid Passing Strings Directly to C |
| ffi-02 | P | Read Documentation Carefully for std::ffi Types |
| ffi-03 | P | Implement Drop for Wrapped C Pointers |
| ffi-04 | P | Handle Panics When Crossing FFI Boundaries |
| ffi-05 | P | Use Portable Type Aliases from std or libc |
| ffi-06 | P | Ensure C-ABI String Compatibility |
| ffi-07 | P | Do Not Implement Drop for Types Passed to External Code |
| ffi-08 | P | Handle Errors Properly in FFI |
| ffi-09 | P | Use References Instead of Raw Pointers in Safe Wrappers |
| ffi-10 | P | Exported Functions Must Be Thread-Safe |
| ffi-11 | P | Be Careful with repr(packed) Field References |
| ffi-12 | P | Document Invariant Assumptions for C Parameters |
| ffi-13 | P | Ensure Consistent Data Layout for Custom Types |
| ffi-14 | P | Types in FFI Should Have Stable Layout |
| ffi-15 | P | Validate Non-Robust External Values |
| ffi-16 | P | Separate Data and Code for Closures to C |
| ffi-17 | P | Use Opaque Types Instead of c_void |
| ffi-18 | P | Avoid Passing Trait Objects to C |
### I/O Safety (1 rule)
| ID | Level | Title |
|----|-------|-------|
| io-01 | P | Ensure I/O Safety When Using Raw Handles |
## Clippy Lint Mapping
| Clippy Lint | Rule | Category |
|-------------|------|----------|
| `undocumented_unsafe_blocks` | safety-09 | SAFETY comments |
| `missing_safety_doc` | safety-10 | Safety docs |
| `panic_in_result_fn` | safety-01, ffi-04 | Panic safety |
| `non_send_fields_in_send_ty` | safety-05 | Send/Sync |
| `uninit_assumed_init` | safety-03 | Initialization |
| `uninit_vec` | mem-06 | Initialization |
| `mut_from_ref` | safety-08 | Aliasing |
| `cast_ptr_alignment` | ptr-04 | Alignment |
| `cast_ref_to_mut` | ptr-05 | Aliasing |
| `ptr_as_ptr` | ptr-06 | Pointer casting |
| `unaligned_references` | ffi-11 | Packed structs |
| `debug_assert_with_mut_call` | safety-11 | Assertions |
## Quick Decision Tree
```
Writing unsafe code?
├─ FFI with C?
│ └─ See ffi-* rules
├─ Raw pointers?
│ └─ See ptr-* rules
├─ Manual Send/Sync?
│ └─ See safety-05
├─ MaybeUninit/uninitialized?
│ └─ See safety-03, mem-06
└─ Performance optimization?
└─ See general-02, safety-07
```
## Essential Checklist
Before every unsafe block:
- [ ] SAFETY comment present
- [ ] Invariants documented
- [ ] Pointer validity checked
- [ ] Aliasing rules followed
- [ ] Panic safety considered
- [ ] Tested with Miri
## Resources
- `checklists/before-unsafe.md` - Pre-writing checklist
- `checklists/review-unsafe.md` - Code review checklist
- `checklists/common-pitfalls.md` - Common bugs and fixes
- `examples/safe-abstraction.md` - Safe wrapper patterns
- `examples/ffi-patterns.md` - FFI best practices
-72
View File
@@ -1,72 +0,0 @@
---
name: unsafe-checker
description: "Use when reviewing unsafe code or writing FFI. Keywords: unsafe, raw pointer, FFI, extern, transmute, *mut, *const, union, #[repr(C)], libc, std::ffi, MaybeUninit, NonNull, PhantomData, Send, Sync, SAFETY comment, soundness, undefined behavior, UB, how to call C functions, safe wrapper for unsafe code, when is unsafe necessary, memory layout, bindgen, cbindgen, CString, CStr, invariant, 安全抽象, 裸指针, 外部函数接口, 内存布局, 不安全代码, FFI 绑定, 未定义行为"
globs: ["**/*.rs"]
---
# Unsafe Rust Checker
## When Unsafe is Valid
| Use Case | Example |
|----------|---------|
| FFI | Calling C functions |
| Low-level abstractions | Implementing `Vec`, `Arc` |
| Performance | Measured bottleneck with safe alternative too slow |
**NOT valid:** Escaping borrow checker without understanding why.
## Required Documentation
```rust
// SAFETY: <why this is safe>
unsafe { ... }
/// # Safety
/// <caller requirements>
pub unsafe fn dangerous() { ... }
```
## Quick Reference
| Operation | Safety Requirements |
|-----------|---------------------|
| `*ptr` deref | Valid, aligned, initialized |
| `&*ptr` | + No aliasing violations |
| `transmute` | Same size, valid bit pattern |
| `extern "C"` | Correct signature, ABI |
| `static mut` | Synchronization guaranteed |
| `impl Send/Sync` | Actually thread-safe |
## Common Errors
| Error | Fix |
|-------|-----|
| Null pointer deref | Check for null before deref |
| Use after free | Ensure lifetime validity |
| Data race | Add proper synchronization |
| Alignment violation | Use `#[repr(C)]`, check alignment |
| Invalid bit pattern | Use `MaybeUninit` |
| Missing SAFETY comment | Add `// SAFETY:` |
## Deprecated → Better
| Deprecated | Use Instead |
|------------|-------------|
| `mem::uninitialized()` | `MaybeUninit<T>` |
| `mem::zeroed()` for refs | `MaybeUninit<T>` |
| Raw pointer arithmetic | `NonNull<T>`, `ptr::add` |
| `CString::new().unwrap().as_ptr()` | Store `CString` first |
| `static mut` | `AtomicT` or `Mutex` |
| Manual extern | `bindgen` |
## FFI Crates
| Direction | Crate |
|-----------|-------|
| C → Rust | bindgen |
| Rust → C | cbindgen |
| Python | PyO3 |
| Node.js | napi-rs |
Claude knows unsafe Rust. Focus on SAFETY comments and soundness.
@@ -1,115 +0,0 @@
# Checklist: Before Writing Unsafe Code
Use this checklist before writing any `unsafe` block or `unsafe fn`.
## 1. Do You Really Need Unsafe?
- [ ] Have you tried all safe alternatives?
- [ ] Can you restructure the code to satisfy the borrow checker?
- [ ] Would interior mutability (`Cell`, `RefCell`, `Mutex`) solve the problem?
- [ ] Is there a safe crate that already does this?
- [ ] Is the performance gain (if any) worth the safety risk?
**If you answered "no" to all, proceed with unsafe.**
## 2. What Unsafe Operation Do You Need?
Identify which specific unsafe operation you're performing:
- [ ] Dereferencing a raw pointer (`*const T`, `*mut T`)
- [ ] Calling an `unsafe` function
- [ ] Accessing a mutable static variable
- [ ] Implementing an unsafe trait (`Send`, `Sync`, etc.)
- [ ] Accessing fields of a `union`
- [ ] Using `extern "C"` functions (FFI)
## 3. Safety Invariants
For each unsafe operation, document the invariants:
### For Pointer Dereference:
- [ ] Is the pointer non-null?
- [ ] Is the pointer properly aligned for the type?
- [ ] Does the pointer point to valid, initialized memory?
- [ ] Is the memory not being mutated by other code?
- [ ] Will the memory remain valid for the entire duration of use?
### For Mutable Aliasing:
- [ ] Are you creating multiple mutable references to the same memory?
- [ ] Is there any possibility of aliasing `&mut` and `&`?
- [ ] Have you verified no other code can access this memory?
### For FFI:
- [ ] Is the function signature correct (types, ABI)?
- [ ] Are you handling potential null pointers?
- [ ] Are you handling potential panics (catch_unwind)?
- [ ] Is memory ownership clear (who allocates, who frees)?
### For Send/Sync:
- [ ] Is concurrent access properly synchronized?
- [ ] Are there any data races possible?
- [ ] Does the type truly satisfy the trait requirements?
## 4. Panic Safety
- [ ] What happens if this code panics at any line?
- [ ] Are data structures left in a valid state on panic?
- [ ] Do you need a panic guard for cleanup?
- [ ] Could a destructor see invalid state?
## 5. Documentation
- [ ] Have you written a `// SAFETY:` comment explaining:
- What invariants must hold?
- Why those invariants are upheld here?
- [ ] For `unsafe fn`, have you written `# Safety` docs explaining:
- What the caller must guarantee?
- What happens if requirements are violated?
## 6. Testing and Verification
- [ ] Can you add debug assertions to verify invariants?
- [ ] Have you tested with Miri (`cargo miri test`)?
- [ ] Have you tested with address sanitizer (`RUSTFLAGS="-Zsanitizer=address"`)?
- [ ] Have you considered fuzzing the unsafe code?
## Quick Reference: Common SAFETY Comments
```rust
// SAFETY: We checked that index < len above, so this is in bounds.
// SAFETY: The pointer was created from a valid reference and hasn't been invalidated.
// SAFETY: We hold the lock, guaranteeing exclusive access.
// SAFETY: The type is #[repr(C)] and all fields are initialized.
// SAFETY: Caller guarantees the pointer is non-null and properly aligned.
```
## Decision Flowchart
```
Need unsafe?
|
v
Can you use safe Rust? --Yes--> Don't use unsafe
|
No
v
Can you use existing safe abstraction? --Yes--> Use it (std, crates)
|
No
v
Document all invariants
|
v
Add SAFETY comments
|
v
Write the unsafe code
|
v
Test with Miri
```
@@ -1,253 +0,0 @@
# Common Unsafe Pitfalls and Fixes
A reference of frequently encountered unsafe bugs and how to fix them.
## Pitfall 1: Dangling Pointer from Local
**Bug:**
```rust
fn bad() -> *const i32 {
let x = 42;
&x as *const i32 // Dangling after return!
}
```
**Fix:**
```rust
fn good() -> Box<i32> {
Box::new(42) // Heap allocation lives beyond function
}
// Or return the value itself
fn better() -> i32 {
42
}
```
## Pitfall 2: CString Lifetime
**Bug:**
```rust
fn bad() -> *const c_char {
let s = CString::new("hello").unwrap();
s.as_ptr() // Dangling! CString dropped
}
```
**Fix:**
```rust
fn good(s: &CString) -> *const c_char {
s.as_ptr() // Caller keeps CString alive
}
// Or take ownership
fn also_good(s: CString) -> *const c_char {
s.into_raw() // Caller must free with CString::from_raw
}
```
## Pitfall 3: Vec set_len with Uninitialized Data
**Bug:**
```rust
fn bad() -> Vec<String> {
let mut v = Vec::with_capacity(10);
unsafe { v.set_len(10); } // Strings are uninitialized!
v
}
```
**Fix:**
```rust
fn good() -> Vec<String> {
let mut v = Vec::with_capacity(10);
for _ in 0..10 {
v.push(String::new());
}
v
}
// Or use resize
fn also_good() -> Vec<String> {
let mut v = Vec::new();
v.resize(10, String::new());
v
}
```
## Pitfall 4: Reference to Packed Field
**Bug:**
```rust
#[repr(packed)]
struct Packed { a: u8, b: u32 }
fn bad(p: &Packed) -> &u32 {
&p.b // UB: misaligned reference!
}
```
**Fix:**
```rust
fn good(p: &Packed) -> u32 {
unsafe { std::ptr::addr_of!(p.b).read_unaligned() }
}
```
## Pitfall 5: Mutable Aliasing Through Raw Pointers
**Bug:**
```rust
fn bad() {
let mut x = 42;
let ptr1 = &mut x as *mut i32;
let ptr2 = &mut x as *mut i32; // Already have ptr1!
unsafe {
*ptr1 = 1;
*ptr2 = 2; // Aliasing mutable pointers!
}
}
```
**Fix:**
```rust
fn good() {
let mut x = 42;
let ptr = &mut x as *mut i32;
unsafe {
*ptr = 1;
*ptr = 2; // Same pointer, sequential access
}
}
```
## Pitfall 6: Transmute to Wrong Size
**Bug:**
```rust
fn bad() {
let x: u32 = 42;
let y: u64 = unsafe { std::mem::transmute(x) }; // UB: size mismatch!
}
```
**Fix:**
```rust
fn good() {
let x: u32 = 42;
let y: u64 = x as u64; // Use conversion
}
```
## Pitfall 7: Invalid Enum Discriminant
**Bug:**
```rust
#[repr(u8)]
enum Status { A = 0, B = 1, C = 2 }
fn bad(raw: u8) -> Status {
unsafe { std::mem::transmute(raw) } // UB if raw > 2!
}
```
**Fix:**
```rust
fn good(raw: u8) -> Option<Status> {
match raw {
0 => Some(Status::A),
1 => Some(Status::B),
2 => Some(Status::C),
_ => None,
}
}
```
## Pitfall 8: FFI Panic Unwinding
**Bug:**
```rust
#[no_mangle]
extern "C" fn callback(x: i32) -> i32 {
if x < 0 {
panic!("negative!"); // UB: unwinding across FFI!
}
x * 2
}
```
**Fix:**
```rust
#[no_mangle]
extern "C" fn callback(x: i32) -> i32 {
std::panic::catch_unwind(|| {
if x < 0 {
panic!("negative!");
}
x * 2
}).unwrap_or(-1) // Return error code on panic
}
```
## Pitfall 9: Double Free from Clone + into_raw
**Bug:**
```rust
struct Handle(*mut c_void);
impl Clone for Handle {
fn clone(&self) -> Self {
Handle(self.0) // Both now "own" same pointer!
}
}
impl Drop for Handle {
fn drop(&mut self) {
unsafe { free(self.0); } // Double free when both drop!
}
}
```
**Fix:**
```rust
struct Handle(*mut c_void);
// Don't implement Clone, or implement proper reference counting
impl Handle {
fn clone_ptr(&self) -> *mut c_void {
self.0 // Return raw pointer, no ownership
}
}
```
## Pitfall 10: Forget Doesn't Run Destructors
**Bug:**
```rust
fn bad() {
let guard = lock.lock();
std::mem::forget(guard); // Lock never released!
}
```
**Fix:**
```rust
fn good() {
let guard = lock.lock();
// Let guard drop naturally
// or explicitly: drop(guard);
}
```
## Quick Reference Table
| Pitfall | Detection | Fix |
|---------|-----------|-----|
| Dangling pointer | Miri | Extend lifetime or heap allocate |
| Uninitialized read | Miri | Use MaybeUninit properly |
| Misaligned access | Miri, UBsan | read_unaligned, copy by value |
| Data race | TSan | Use atomics or mutex |
| Double free | ASan | Track ownership carefully |
| Invalid enum | Manual review | Use TryFrom |
| FFI panic | Testing | catch_unwind |
| Type confusion | Miri | Match types exactly |
@@ -1,113 +0,0 @@
# Checklist: Reviewing Unsafe Code
Use this checklist when reviewing code containing `unsafe`.
## 1. Surface-Level Checks
- [ ] Does every `unsafe` block have a `// SAFETY:` comment?
- [ ] Does every `unsafe fn` have `# Safety` documentation?
- [ ] Are the safety comments specific and verifiable, not vague?
- [ ] Is the unsafe code minimized (smallest possible unsafe block)?
## 2. Pointer Validity
For each pointer dereference:
- [ ] **Non-null**: Is null checked before dereference?
- [ ] **Aligned**: Is alignment verified or guaranteed by construction?
- [ ] **Valid**: Does the pointer point to allocated memory?
- [ ] **Initialized**: Is the memory initialized before reading?
- [ ] **Lifetime**: Is the memory valid for the entire use duration?
- [ ] **Unique**: For `&mut`, is there only one mutable reference?
## 3. Memory Safety
- [ ] **No aliasing**: Are `&` and `&mut` never created to the same memory simultaneously?
- [ ] **No use-after-free**: Is memory not accessed after deallocation?
- [ ] **No double-free**: Is memory freed exactly once?
- [ ] **No data races**: Is concurrent access properly synchronized?
- [ ] **Bounds checked**: Are array/slice accesses in bounds?
## 4. Type Safety
- [ ] **Transmute**: Are transmuted types actually compatible?
- [ ] **Repr**: Do FFI types have `#[repr(C)]`?
- [ ] **Enum values**: Are enum discriminants validated from external sources?
- [ ] **Unions**: Is the correct union field accessed?
## 5. Panic Safety
- [ ] What state is the program in if this code panics?
- [ ] Are partially constructed objects properly cleaned up?
- [ ] Do Drop implementations see valid state?
- [ ] Is there a panic guard if needed?
## 6. FFI-Specific Checks
- [ ] **Types**: Do Rust types match C types exactly?
- [ ] **Strings**: Are strings properly null-terminated?
- [ ] **Ownership**: Is it clear who owns/frees memory?
- [ ] **Thread safety**: Are callbacks thread-safe?
- [ ] **Panic boundary**: Are panics caught before crossing FFI?
- [ ] **Error handling**: Are C-style errors properly handled?
## 7. Concurrency Checks
- [ ] **Send/Sync**: Are manual implementations actually sound?
- [ ] **Atomics**: Are memory orderings correct?
- [ ] **Locks**: Is there potential for deadlock?
- [ ] **Data races**: Is all shared mutable state synchronized?
## 8. Red Flags (Require Extra Scrutiny)
| Pattern | Concern |
|---------|---------|
| `transmute` | Type compatibility, provenance |
| `as` on pointers | Alignment, type punning |
| `static mut` | Data races |
| `*const T as *mut T` | Aliasing violation |
| Manual `Send`/`Sync` | Thread safety |
| `assume_init` | Initialization |
| `set_len` on Vec | Uninitialized memory |
| `from_raw_parts` | Lifetime, validity |
| `offset`/`add`/`sub` | Out of bounds |
| FFI callbacks | Panic safety |
## 9. Verification Questions
Ask the author:
- "What would happen if [X invariant] was violated?"
- "How do you know [pointer/reference] is valid here?"
- "What if this panics at [specific line]?"
- "Who is responsible for freeing this memory?"
## 10. Testing Requirements
- [ ] Has this been tested with Miri?
- [ ] Are there unit tests covering edge cases?
- [ ] Are there tests for error conditions?
- [ ] Has concurrent code been tested under stress?
## Review Severity Guide
| Severity | Requires |
|----------|----------|
| `transmute` | Two reviewers, Miri test |
| Manual `Send`/`Sync` | Thread safety expert review |
| FFI | Documentation of C interface |
| `static mut` | Justification for not using atomic/mutex |
| Pointer arithmetic | Bounds proof |
## Sample Review Comments
```
// Good SAFETY comment ✓
// SAFETY: index was checked to be < len on line 42
// Needs improvement ✗
// SAFETY: This is safe because we know it works
// Missing information ✗
// SAFETY: ptr is valid
// (Why is it valid? How do we know?)
```
@@ -1,353 +0,0 @@
# FFI Best Practices and Patterns
Examples of safe and idiomatic Rust-C interoperability.
## Pattern 1: Basic FFI Wrapper
```rust
use std::ffi::{CStr, CString};
use std::os::raw::{c_char, c_int, c_void};
use std::ptr::NonNull;
// Raw C API
mod ffi {
use super::*;
extern "C" {
pub fn lib_create(name: *const c_char) -> *mut c_void;
pub fn lib_destroy(handle: *mut c_void);
pub fn lib_process(handle: *mut c_void, data: *const u8, len: usize) -> c_int;
pub fn lib_get_error() -> *const c_char;
}
}
// Safe Rust wrapper
pub struct Library {
handle: NonNull<c_void>,
}
#[derive(Debug)]
pub struct LibraryError(String);
impl Library {
pub fn new(name: &str) -> Result<Self, LibraryError> {
let c_name = CString::new(name).map_err(|_| LibraryError("invalid name".into()))?;
let handle = unsafe { ffi::lib_create(c_name.as_ptr()) };
NonNull::new(handle)
.map(|handle| Self { handle })
.ok_or_else(|| Self::last_error())
}
pub fn process(&self, data: &[u8]) -> Result<(), LibraryError> {
let result = unsafe {
ffi::lib_process(self.handle.as_ptr(), data.as_ptr(), data.len())
};
if result == 0 {
Ok(())
} else {
Err(Self::last_error())
}
}
fn last_error() -> LibraryError {
let ptr = unsafe { ffi::lib_get_error() };
if ptr.is_null() {
LibraryError("unknown error".into())
} else {
let msg = unsafe { CStr::from_ptr(ptr) }
.to_string_lossy()
.into_owned();
LibraryError(msg)
}
}
}
impl Drop for Library {
fn drop(&mut self) {
unsafe { ffi::lib_destroy(self.handle.as_ptr()); }
}
}
// Prevent accidental copies
impl !Clone for Library {}
```
## Pattern 2: Callback Registration
```rust
use std::os::raw::{c_int, c_void};
use std::panic::{catch_unwind, AssertUnwindSafe};
type CCallback = extern "C" fn(value: c_int, user_data: *mut c_void) -> c_int;
extern "C" {
fn register_callback(cb: CCallback, user_data: *mut c_void);
fn unregister_callback();
}
/// Safely register a Rust closure as a C callback.
pub struct CallbackGuard<F> {
_closure: Box<F>,
}
impl<F: FnMut(i32) -> i32 + 'static> CallbackGuard<F> {
pub fn register(closure: F) -> Self {
let boxed = Box::new(closure);
let user_data = Box::into_raw(boxed) as *mut c_void;
extern "C" fn trampoline<F: FnMut(i32) -> i32>(
value: c_int,
user_data: *mut c_void,
) -> c_int {
let result = catch_unwind(AssertUnwindSafe(|| {
let closure = unsafe { &mut *(user_data as *mut F) };
closure(value as i32) as c_int
}));
result.unwrap_or(-1)
}
unsafe {
register_callback(trampoline::<F>, user_data);
}
Self {
// SAFETY: We just created this box and need to keep it alive
_closure: unsafe { Box::from_raw(user_data as *mut F) },
}
}
}
impl<F> Drop for CallbackGuard<F> {
fn drop(&mut self) {
unsafe { unregister_callback(); }
// Box in _closure is dropped automatically
}
}
// Usage
fn example() {
let multiplier = 2;
let _guard = CallbackGuard::register(move |x| x * multiplier);
// Callback is active until _guard is dropped
}
```
## Pattern 3: Opaque Handle Types
```rust
use std::marker::PhantomData;
// Opaque type markers - prevents mixing up handles
#[repr(C)]
pub struct DatabaseHandle {
_data: [u8; 0],
_marker: PhantomData<(*mut u8, std::marker::PhantomPinned)>,
}
#[repr(C)]
pub struct ConnectionHandle {
_data: [u8; 0],
_marker: PhantomData<(*mut u8, std::marker::PhantomPinned)>,
}
mod ffi {
use super::*;
extern "C" {
pub fn db_open(path: *const c_char) -> *mut DatabaseHandle;
pub fn db_close(db: *mut DatabaseHandle);
pub fn db_connect(db: *mut DatabaseHandle) -> *mut ConnectionHandle;
pub fn conn_close(conn: *mut ConnectionHandle);
pub fn conn_query(conn: *mut ConnectionHandle, sql: *const c_char) -> c_int;
}
}
// Type-safe wrappers
pub struct Database {
handle: NonNull<DatabaseHandle>,
}
pub struct Connection<'db> {
handle: NonNull<ConnectionHandle>,
_db: PhantomData<&'db Database>,
}
impl Database {
pub fn open(path: &str) -> Result<Self, ()> {
let c_path = CString::new(path).map_err(|_| ())?;
let handle = unsafe { ffi::db_open(c_path.as_ptr()) };
NonNull::new(handle).map(|h| Self { handle: h }).ok_or(())
}
pub fn connect(&self) -> Result<Connection<'_>, ()> {
let handle = unsafe { ffi::db_connect(self.handle.as_ptr()) };
NonNull::new(handle)
.map(|h| Connection { handle: h, _db: PhantomData })
.ok_or(())
}
}
impl Drop for Database {
fn drop(&mut self) {
// All Connections must be dropped first (enforced by lifetime)
unsafe { ffi::db_close(self.handle.as_ptr()); }
}
}
impl Connection<'_> {
pub fn query(&self, sql: &str) -> Result<(), ()> {
let c_sql = CString::new(sql).map_err(|_| ())?;
let result = unsafe { ffi::conn_query(self.handle.as_ptr(), c_sql.as_ptr()) };
if result == 0 { Ok(()) } else { Err(()) }
}
}
impl Drop for Connection<'_> {
fn drop(&mut self) {
unsafe { ffi::conn_close(self.handle.as_ptr()); }
}
}
```
## Pattern 4: Error Handling Across FFI
```rust
use std::os::raw::c_int;
// Error codes for C
pub const SUCCESS: c_int = 0;
pub const ERR_NULL_PTR: c_int = 1;
pub const ERR_INVALID_UTF8: c_int = 2;
pub const ERR_IO: c_int = 3;
pub const ERR_PANIC: c_int = -1;
// Thread-local error storage
thread_local! {
static LAST_ERROR: std::cell::RefCell<Option<Box<dyn std::error::Error>>> =
std::cell::RefCell::new(None);
}
fn set_last_error<E: std::error::Error + 'static>(err: E) {
LAST_ERROR.with(|e| {
*e.borrow_mut() = Some(Box::new(err));
});
}
/// Get the last error message. Caller must free with `free_string`.
#[no_mangle]
pub extern "C" fn get_last_error() -> *mut c_char {
LAST_ERROR.with(|e| {
e.borrow()
.as_ref()
.map(|err| {
CString::new(err.to_string())
.unwrap_or_else(|_| CString::new("error").unwrap())
.into_raw()
})
.unwrap_or(std::ptr::null_mut())
})
}
/// Free a string returned by this library.
#[no_mangle]
pub extern "C" fn free_string(s: *mut c_char) {
if !s.is_null() {
// SAFETY: String was created by CString::into_raw
unsafe { drop(CString::from_raw(s)); }
}
}
/// Example function with proper error handling.
#[no_mangle]
pub extern "C" fn do_operation(data: *const u8, len: usize) -> c_int {
let result = catch_unwind(AssertUnwindSafe(|| -> Result<(), c_int> {
if data.is_null() {
return Err(ERR_NULL_PTR);
}
let slice = unsafe { std::slice::from_raw_parts(data, len) };
std::str::from_utf8(slice)
.map_err(|e| {
set_last_error(e);
ERR_INVALID_UTF8
})?;
// Do actual work...
Ok(())
}));
match result {
Ok(Ok(())) => SUCCESS,
Ok(Err(code)) => code,
Err(_) => ERR_PANIC,
}
}
```
## Pattern 5: Struct with C Layout
```rust
use std::os::raw::{c_char, c_int};
/// A C-compatible configuration struct.
#[repr(C)]
pub struct Config {
pub version: c_int,
pub flags: u32,
pub name: [c_char; 64],
pub name_len: usize,
}
impl Config {
pub fn new(version: i32, flags: u32, name: &str) -> Option<Self> {
if name.len() >= 64 {
return None;
}
let mut config = Self {
version: version as c_int,
flags,
name: [0; 64],
name_len: name.len(),
};
// Copy name bytes
for (i, byte) in name.bytes().enumerate() {
config.name[i] = byte as c_char;
}
Some(config)
}
pub fn name(&self) -> &str {
let bytes = unsafe {
std::slice::from_raw_parts(
self.name.as_ptr() as *const u8,
self.name_len,
)
};
// SAFETY: We only store valid UTF-8 in new()
unsafe { std::str::from_utf8_unchecked(bytes) }
}
}
// Verify layout at compile time
const _: () = {
assert!(std::mem::size_of::<Config>() == 80); // 4 + 4 + 64 + 8
assert!(std::mem::align_of::<Config>() == 8);
};
```
## Key FFI Guidelines
1. **Always use `#[repr(C)]`** for types crossing FFI
2. **Handle null pointers** at the boundary
3. **Catch panics** before returning to C
4. **Document ownership** clearly
5. **Use opaque types** for type safety
6. **Keep unsafe minimal** and well-documented
@@ -1,272 +0,0 @@
# Safe Abstraction Examples
Examples of building safe APIs on top of unsafe code.
## Example 1: Simple Wrapper with Bounds Check
```rust
/// A slice wrapper that provides unchecked access internally
/// but safe access externally.
pub struct SafeSlice<'a, T> {
ptr: *const T,
len: usize,
_marker: std::marker::PhantomData<&'a T>,
}
impl<'a, T> SafeSlice<'a, T> {
/// Creates a SafeSlice from a regular slice.
pub fn new(slice: &'a [T]) -> Self {
Self {
ptr: slice.as_ptr(),
len: slice.len(),
_marker: std::marker::PhantomData,
}
}
/// Safe get - returns Option.
pub fn get(&self, index: usize) -> Option<&T> {
if index < self.len {
// SAFETY: We just verified index < len
Some(unsafe { &*self.ptr.add(index) })
} else {
None
}
}
/// Unsafe get - caller must ensure bounds.
///
/// # Safety
/// `index` must be less than `self.len()`.
pub unsafe fn get_unchecked(&self, index: usize) -> &T {
debug_assert!(index < self.len);
&*self.ptr.add(index)
}
pub fn len(&self) -> usize {
self.len
}
}
```
## Example 2: Resource Wrapper with Drop
```rust
use std::ptr::NonNull;
/// Safe wrapper around a C-allocated buffer.
pub struct CBuffer {
ptr: NonNull<u8>,
len: usize,
}
extern "C" {
fn c_alloc(size: usize) -> *mut u8;
fn c_free(ptr: *mut u8);
}
impl CBuffer {
/// Creates a new buffer. Returns None if allocation fails.
pub fn new(size: usize) -> Option<Self> {
let ptr = unsafe { c_alloc(size) };
NonNull::new(ptr).map(|ptr| Self { ptr, len: size })
}
/// Returns a slice view of the buffer.
pub fn as_slice(&self) -> &[u8] {
// SAFETY: ptr is valid for len bytes (from c_alloc contract)
unsafe { std::slice::from_raw_parts(self.ptr.as_ptr(), self.len) }
}
/// Returns a mutable slice view.
pub fn as_mut_slice(&mut self) -> &mut [u8] {
// SAFETY: We have &mut self, so exclusive access
unsafe { std::slice::from_raw_parts_mut(self.ptr.as_ptr(), self.len) }
}
}
impl Drop for CBuffer {
fn drop(&mut self) {
// SAFETY: ptr was allocated by c_alloc and not yet freed
unsafe { c_free(self.ptr.as_ptr()); }
}
}
// Prevent double-free
impl !Clone for CBuffer {}
// Safe to send between threads (assuming c_alloc is thread-safe)
unsafe impl Send for CBuffer {}
```
## Example 3: Interior Mutability with UnsafeCell
```rust
use std::cell::UnsafeCell;
use std::sync::atomic::{AtomicBool, Ordering};
/// A simple spinlock demonstrating safe abstraction over UnsafeCell.
pub struct SpinLock<T> {
locked: AtomicBool,
data: UnsafeCell<T>,
}
pub struct SpinLockGuard<'a, T> {
lock: &'a SpinLock<T>,
}
impl<T> SpinLock<T> {
pub const fn new(data: T) -> Self {
Self {
locked: AtomicBool::new(false),
data: UnsafeCell::new(data),
}
}
pub fn lock(&self) -> SpinLockGuard<'_, T> {
// Spin until we acquire the lock
while self.locked.compare_exchange_weak(
false,
true,
Ordering::Acquire,
Ordering::Relaxed,
).is_err() {
std::hint::spin_loop();
}
SpinLockGuard { lock: self }
}
}
impl<T> std::ops::Deref for SpinLockGuard<'_, T> {
type Target = T;
fn deref(&self) -> &T {
// SAFETY: We hold the lock, so we have exclusive access
unsafe { &*self.lock.data.get() }
}
}
impl<T> std::ops::DerefMut for SpinLockGuard<'_, T> {
fn deref_mut(&mut self) -> &mut T {
// SAFETY: We hold the lock, so we have exclusive access
unsafe { &mut *self.lock.data.get() }
}
}
impl<T> Drop for SpinLockGuard<'_, T> {
fn drop(&mut self) {
self.lock.locked.store(false, Ordering::Release);
}
}
// SAFETY: The lock ensures only one thread accesses data at a time
unsafe impl<T: Send> Sync for SpinLock<T> {}
unsafe impl<T: Send> Send for SpinLock<T> {}
```
## Example 4: Iterator with Lifetime Tracking
```rust
use std::marker::PhantomData;
/// An iterator over raw pointer range with proper lifetime tracking.
pub struct PtrIter<'a, T> {
current: *const T,
end: *const T,
_marker: PhantomData<&'a T>,
}
impl<'a, T> PtrIter<'a, T> {
/// Creates an iterator from a slice.
pub fn new(slice: &'a [T]) -> Self {
let ptr = slice.as_ptr();
Self {
current: ptr,
// SAFETY: Adding len to slice pointer is always valid
end: unsafe { ptr.add(slice.len()) },
_marker: PhantomData,
}
}
}
impl<'a, T> Iterator for PtrIter<'a, T> {
type Item = &'a T;
fn next(&mut self) -> Option<Self::Item> {
if self.current == self.end {
None
} else {
// SAFETY:
// - current < end (checked above)
// - PhantomData<&'a T> ensures the data lives for 'a
let item = unsafe { &*self.current };
self.current = unsafe { self.current.add(1) };
Some(item)
}
}
}
```
## Example 5: Builder Pattern with Delayed Initialization
```rust
use std::mem::MaybeUninit;
/// A builder that collects exactly N items, then produces an array.
pub struct ArrayBuilder<T, const N: usize> {
data: [MaybeUninit<T>; N],
count: usize,
}
impl<T, const N: usize> ArrayBuilder<T, N> {
pub fn new() -> Self {
Self {
// SAFETY: MaybeUninit doesn't require initialization
data: unsafe { MaybeUninit::uninit().assume_init() },
count: 0,
}
}
pub fn push(&mut self, value: T) -> Result<(), T> {
if self.count >= N {
return Err(value);
}
self.data[self.count].write(value);
self.count += 1;
Ok(())
}
pub fn build(self) -> Option<[T; N]> {
if self.count != N {
return None;
}
// SAFETY: All N elements have been initialized
let result = unsafe {
// Prevent drop of self.data (we're moving out)
let data = std::ptr::read(&self.data);
std::mem::forget(self);
// Transmute MaybeUninit array to initialized array
std::mem::transmute_copy::<[MaybeUninit<T>; N], [T; N]>(&data)
};
Some(result)
}
}
impl<T, const N: usize> Drop for ArrayBuilder<T, N> {
fn drop(&mut self) {
// Drop only initialized elements
for i in 0..self.count {
// SAFETY: Elements 0..count are initialized
unsafe { self.data[i].assume_init_drop(); }
}
}
}
```
## Key Patterns
1. **Encapsulation**: Hide unsafe behind safe public API
2. **Invariant maintenance**: Use private fields to maintain invariants
3. **PhantomData**: Track lifetimes and ownership for pointers
4. **RAII**: Use Drop for cleanup
5. **Type state**: Use types to encode valid states
@@ -1,17 +0,0 @@
{
"name": "unsafe-checker",
"version": "1.0.0",
"description": "Unsafe Rust code review and safety abstraction checker",
"source": "https://github.com/Rust-Coding-Guidelines/rust-coding-guidelines-zh",
"lastUpdated": "2026-01-16",
"ruleCount": 47,
"sections": [
{ "id": "general", "name": "General Principles", "count": 3 },
{ "id": "safety", "name": "Safety Abstraction", "count": 11 },
{ "id": "ptr", "name": "Raw Pointers", "count": 6 },
{ "id": "union", "name": "Union", "count": 2 },
{ "id": "mem", "name": "Memory Layout", "count": 6 },
{ "id": "ffi", "name": "FFI", "count": 18 },
{ "id": "io", "name": "I/O Safety", "count": 1 }
]
}
@@ -1,77 +0,0 @@
# Unsafe Checker - Section Definitions
## Section Overview
| # | Section | Prefix | Level | Count | Impact |
|---|---------|--------|-------|-------|--------|
| 1 | General Principles | `general-` | CRITICAL | 3 | Foundational unsafe usage guidance |
| 2 | Safety Abstraction | `safety-` | CRITICAL | 11 | Building sound safe APIs |
| 3 | Raw Pointers | `ptr-` | HIGH | 6 | Pointer manipulation safety |
| 4 | Union | `union-` | HIGH | 2 | Union type safety |
| 5 | Memory Layout | `mem-` | HIGH | 6 | Data representation correctness |
| 6 | FFI | `ffi-` | CRITICAL | 18 | C interoperability safety |
| 7 | I/O Safety | `io-` | MEDIUM | 1 | Handle/resource safety |
## Section Details
### 1. General Principles (`general-`)
**Focus**: When and why to use unsafe
- P.UNS.01: Don't abuse unsafe to escape borrow checker
- P.UNS.02: Don't use unsafe blindly for performance
- G.UNS.01: Don't create aliases for "unsafe" named items
### 2. Safety Abstraction (`safety-`)
**Focus**: Building sound safe abstractions over unsafe code
Key invariants:
- Panic safety
- Memory initialization
- Send/Sync correctness
- API soundness
### 3. Raw Pointers (`ptr-`)
**Focus**: Safe pointer manipulation patterns
- Aliasing rules
- Alignment requirements
- Null/dangling prevention
- Type casting
### 4. Union (`union-`)
**Focus**: Safe union usage (primarily for C interop)
- Initialization rules
- Lifetime considerations
- Type punning dangers
### 5. Memory Layout (`mem-`)
**Focus**: Correct data representation
- `#[repr(C)]` usage
- Alignment and padding
- Uninitialized memory
- Cross-process memory
### 6. FFI (`ffi-`)
**Focus**: Safe C interoperability
Subcategories:
- String handling (CString, CStr)
- Type compatibility
- Error handling across FFI
- Thread safety
- Resource management
### 7. I/O Safety (`io-`)
**Focus**: Handle and resource ownership
- Raw file descriptor safety
- Handle validity guarantees
@@ -1,53 +0,0 @@
# Rule Template
Use this template for all unsafe-checker rules.
---
```markdown
---
id: {prefix}-{number}
original_id: P.UNS.XXX.YY or G.UNS.XXX.YY
level: P|G
impact: CRITICAL|HIGH|MEDIUM
clippy: <clippy_lint_name> (if applicable)
---
# {Rule Title}
## Summary
One-sentence description of what this rule requires.
## Rationale
Why this rule matters for safety/soundness.
## Bad Example
```rust
// DON'T: Description of the anti-pattern
<code that violates the rule>
```
## Good Example
```rust
// DO: Description of the correct pattern
<code that follows the rule>
```
## Common Violations
1. Violation pattern 1
2. Violation pattern 2
## Checklist
- [ ] Check item 1
- [ ] Check item 2
## Related Rules
- `{other-rule-id}`: Brief description
```
@@ -1,122 +0,0 @@
---
id: ffi-01
original_id: P.UNS.FFI.01
level: P
impact: HIGH
---
# Avoid Passing Strings Directly to C from Public Rust API
## Summary
Use `CString` and `CStr` for string handling at FFI boundaries. Never pass Rust `String` or `&str` directly to C.
## Rationale
- Rust strings are UTF-8, not null-terminated
- C strings require null terminator
- Rust strings may contain interior null bytes
- Memory layout differs between Rust String and C char*
## Bad Example
```rust
extern "C" {
fn c_print(s: *const u8);
fn c_strlen(s: *const u8) -> usize;
}
// DON'T: Pass Rust string directly
fn bad_print(s: &str) {
unsafe {
c_print(s.as_ptr()); // Not null-terminated!
}
}
// DON'T: Assume length matches
fn bad_strlen(s: &str) -> usize {
unsafe {
c_strlen(s.as_ptr()) // May read past buffer
}
}
// DON'T: Use String in FFI signatures
extern "C" fn bad_callback(s: String) { // Wrong!
println!("{}", s);
}
```
## Good Example
```rust
use std::ffi::{CString, CStr};
use std::os::raw::c_char;
extern "C" {
fn c_print(s: *const c_char);
fn c_strlen(s: *const c_char) -> usize;
fn c_get_string() -> *const c_char;
}
// DO: Convert to CString for passing to C
fn good_print(s: &str) -> Result<(), std::ffi::NulError> {
let c_string = CString::new(s)?; // Adds null terminator, checks for interior nulls
unsafe {
c_print(c_string.as_ptr());
}
Ok(())
}
// DO: Use CStr for receiving C strings
fn good_receive() -> String {
unsafe {
let ptr = c_get_string();
let c_str = CStr::from_ptr(ptr);
c_str.to_string_lossy().into_owned()
}
}
// DO: Handle interior null bytes
fn handle_nulls(s: &str) {
match CString::new(s) {
Ok(c_string) => unsafe { c_print(c_string.as_ptr()) },
Err(e) => {
// String contains interior null at position e.nul_position()
eprintln!("String contains null byte at {}", e.nul_position());
}
}
}
// DO: Use proper types in callbacks
extern "C" fn good_callback(s: *const c_char) {
if !s.is_null() {
let c_str = unsafe { CStr::from_ptr(s) };
if let Ok(rust_str) = c_str.to_str() {
println!("{}", rust_str);
}
}
}
```
## String Type Comparison
| Type | Null-terminated | Encoding | Use |
|------|-----------------|----------|-----|
| `String` | No | UTF-8 | Rust owned |
| `&str` | No | UTF-8 | Rust borrowed |
| `CString` | Yes | Byte | Rust-to-C owned |
| `&CStr` | Yes | Byte | Rust-to-C borrowed |
| `*const c_char` | Yes | Byte | FFI pointer |
| `OsString` | Platform | Platform | Paths, env |
## Checklist
- [ ] Am I passing Rust strings to C? → Use CString
- [ ] Am I receiving C strings? → Use CStr
- [ ] Does my string contain null bytes? → Handle NulError
- [ ] Am I checking for null pointers from C?
## Related Rules
- `ffi-02`: Read documentation for std::ffi types
- `ffi-06`: Ensure C-ABI string compatibility
@@ -1,133 +0,0 @@
---
id: ffi-02
original_id: P.UNS.FFI.02
level: P
impact: MEDIUM
---
# Read Documentation Carefully When Using std::ffi Types
## Summary
The `std::ffi` module has many types with subtle differences. Read their documentation carefully to avoid misuse.
## Key Types in std::ffi
### CString vs CStr
```rust
use std::ffi::{CString, CStr};
use std::os::raw::c_char;
// CString: Owned, heap-allocated, null-terminated
// - Use when creating strings to pass to C
// - Owns the memory
let owned = CString::new("hello").unwrap();
let ptr: *const c_char = owned.as_ptr();
// ptr valid until `owned` is dropped
// CStr: Borrowed, null-terminated
// - Use when receiving strings from C
// - Does not own memory
let borrowed: &CStr = unsafe { CStr::from_ptr(ptr) };
// borrowed valid as long as ptr is valid
```
### OsString vs OsStr
```rust
use std::ffi::{OsString, OsStr};
use std::path::Path;
// OsString/OsStr: Platform-native strings
// - Windows: potentially ill-formed UTF-16
// - Unix: arbitrary bytes
// - Use for paths and environment variables
let path = Path::new("/some/path");
let os_str: &OsStr = path.as_os_str();
// Convert to Rust string (may fail)
if let Some(s) = os_str.to_str() {
println!("Valid UTF-8: {}", s);
}
```
### c_void and Opaque Types
```rust
use std::ffi::c_void;
extern "C" {
fn get_handle() -> *mut c_void;
fn use_handle(h: *mut c_void);
}
// c_void is for truly opaque pointers
// Better: use dedicated opaque types (see ffi-17)
```
## Common Pitfalls
```rust
use std::ffi::CString;
// PITFALL 1: CString::as_ptr() lifetime
fn bad_ptr() -> *const i8 {
let s = CString::new("hello").unwrap();
s.as_ptr() // Dangling! s dropped at end of function
}
fn good_ptr(s: &CString) -> *const i8 {
s.as_ptr() // OK: s outlives the pointer
}
// PITFALL 2: CString::new with interior nulls
let result = CString::new("hello\0world");
assert!(result.is_err()); // Interior null!
// PITFALL 3: CStr::from_ptr safety
unsafe {
let ptr: *const i8 = std::ptr::null();
// let cstr = CStr::from_ptr(ptr); // UB: null pointer!
// Always check for null first
if !ptr.is_null() {
let cstr = CStr::from_ptr(ptr);
}
}
// PITFALL 4: CStr assumes valid null-terminated string
unsafe {
let bytes = [104, 101, 108, 108, 111]; // "hello" without null
let ptr = bytes.as_ptr() as *const i8;
// let cstr = CStr::from_ptr(ptr); // UB: no null terminator!
// Use from_bytes_with_nul instead
let bytes_with_nul = b"hello\0";
let cstr = CStr::from_bytes_with_nul(bytes_with_nul).unwrap();
}
```
## Type Selection Guide
| Scenario | Type |
|----------|------|
| Create string for C | `CString` |
| Borrow string from C | `&CStr` |
| File paths | `OsString`, `Path` |
| Environment variables | `OsString` |
| Opaque C pointers | Newtype over `*mut c_void` |
| C integers | `c_int`, `c_long`, etc. |
## Checklist
- [ ] Have I read the docs for the std::ffi type I'm using?
- [ ] Am I aware of the lifetime constraints?
- [ ] Am I handling potential errors (NulError, UTF-8 errors)?
- [ ] Is there a better type for my use case?
## Related Rules
- `ffi-01`: Use CString/CStr for strings
- `ffi-17`: Use opaque types instead of c_void
@@ -1,162 +0,0 @@
---
id: ffi-03
original_id: P.UNS.FFI.03
level: P
impact: CRITICAL
---
# Implement Drop for Rust Types Wrapping Memory-Managing C Pointers
## Summary
When wrapping a C pointer that owns memory, implement `Drop` to call the appropriate C deallocation function.
## Rationale
- C allocated memory must be freed with the matching C function
- Rust's default drop won't clean up foreign memory
- Resource leaks and double-frees are common FFI bugs
## Bad Example
```rust
extern "C" {
fn create_resource() -> *mut Resource;
fn free_resource(r: *mut Resource);
}
// DON'T: Wrapper without Drop
struct ResourceHandle {
ptr: *mut Resource,
}
impl ResourceHandle {
fn new() -> Self {
Self {
ptr: unsafe { create_resource() }
}
}
// Memory leak! ptr is never freed
}
// DON'T: Forget to handle null
impl Drop for BadHandle {
fn drop(&mut self) {
unsafe {
free_resource(self.ptr); // Crash if ptr is null!
}
}
}
```
## Good Example
```rust
use std::ptr::NonNull;
extern "C" {
fn create_resource() -> *mut Resource;
fn free_resource(r: *mut Resource);
}
// DO: Proper wrapper with Drop
struct ResourceHandle {
ptr: NonNull<Resource>,
}
impl ResourceHandle {
fn new() -> Option<Self> {
let ptr = unsafe { create_resource() };
NonNull::new(ptr).map(|ptr| Self { ptr })
}
fn as_ptr(&self) -> *mut Resource {
self.ptr.as_ptr()
}
}
impl Drop for ResourceHandle {
fn drop(&mut self) {
// SAFETY: ptr was allocated by create_resource
// and hasn't been freed yet
unsafe {
free_resource(self.ptr.as_ptr());
}
}
}
// Prevent accidental copies that would cause double-free
impl !Clone for ResourceHandle {}
// DO: Document ownership transfer
impl ResourceHandle {
/// Consumes the handle and returns the raw pointer.
///
/// The caller is responsible for freeing the resource.
fn into_raw(self) -> *mut Resource {
let ptr = self.ptr.as_ptr();
std::mem::forget(self); // Don't run Drop
ptr
}
/// Creates a handle from a raw pointer.
///
/// # Safety
///
/// ptr must have been allocated by create_resource()
/// and not yet freed.
unsafe fn from_raw(ptr: *mut Resource) -> Option<Self> {
NonNull::new(ptr).map(|ptr| Self { ptr })
}
}
```
## Complete Pattern with Multiple Resources
```rust
struct Connection {
handle: NonNull<c_void>,
}
struct Statement<'conn> {
handle: NonNull<c_void>,
_conn: std::marker::PhantomData<&'conn Connection>,
}
impl Connection {
fn prepare(&self, sql: &str) -> Option<Statement<'_>> {
let handle = unsafe { db_prepare(self.handle.as_ptr(), sql.as_ptr()) };
NonNull::new(handle).map(|handle| Statement {
handle,
_conn: std::marker::PhantomData,
})
}
}
impl Drop for Connection {
fn drop(&mut self) {
// Statements must be dropped before Connection
// PhantomData ensures this at compile time
unsafe { db_close(self.handle.as_ptr()); }
}
}
impl Drop for Statement<'_> {
fn drop(&mut self) {
unsafe { db_finalize(self.handle.as_ptr()); }
}
}
```
## Checklist
- [ ] Does my wrapper own the C resource?
- [ ] Did I implement Drop with the correct C free function?
- [ ] Did I handle null pointers?
- [ ] Did I prevent Clone/Copy to avoid double-free?
- [ ] Did I consider ownership transfer methods (into_raw/from_raw)?
## Related Rules
- `mem-03`: Don't let String/Vec drop foreign memory
- `ffi-07`: Don't implement Drop for types passed to external code
@@ -1,145 +0,0 @@
---
id: ffi-04
original_id: P.UNS.FFI.04
level: P
impact: CRITICAL
clippy: panic_in_result_fn
---
# Handle Panics When Crossing FFI Boundaries
## Summary
Panics must not unwind across FFI boundaries. Use `catch_unwind` or mark functions as `extern "C-unwind"`.
## Rationale
- Unwinding across C code is undefined behavior
- C has no concept of Rust panics
- Can corrupt C stack frames and cause crashes
- Even with `panic=abort`, still UB to attempt unwinding in `extern "C"`
## Bad Example
```rust
// DON'T: Allow panics to escape to C
#[no_mangle]
pub extern "C" fn callback(data: *const u8, len: usize) -> i32 {
let slice = unsafe { std::slice::from_raw_parts(data, len) };
// If this panics, UB occurs!
let sum: i32 = slice.iter().map(|&x| x as i32).sum();
// If this panics due to overflow in debug, UB!
process(sum)
}
// DON'T: Unwrap in extern functions
#[no_mangle]
pub extern "C" fn parse_config(path: *const c_char) -> i32 {
let path = unsafe { CStr::from_ptr(path) };
let config = std::fs::read_to_string(path.to_str().unwrap()).unwrap(); // Can panic!
0
}
```
## Good Example
```rust
use std::panic::{catch_unwind, AssertUnwindSafe};
use std::ffi::CStr;
use std::os::raw::{c_char, c_int};
// DO: Catch panics at FFI boundary
#[no_mangle]
pub extern "C" fn safe_callback(data: *const u8, len: usize) -> c_int {
let result = catch_unwind(AssertUnwindSafe(|| {
if data.is_null() || len == 0 {
return -1;
}
let slice = unsafe { std::slice::from_raw_parts(data, len) };
let sum: i32 = slice.iter().map(|&x| x as i32).sum();
sum
}));
match result {
Ok(value) => value,
Err(_) => {
// Log error, return error code
eprintln!("Panic caught at FFI boundary");
-1
}
}
}
// DO: Use Result-based API internally
#[no_mangle]
pub extern "C" fn parse_config(path: *const c_char) -> c_int {
let result = catch_unwind(AssertUnwindSafe(|| -> Result<(), Box<dyn std::error::Error>> {
let path = unsafe { CStr::from_ptr(path) }.to_str()?;
let _config = std::fs::read_to_string(path)?;
Ok(())
}));
match result {
Ok(Ok(())) => 0,
Ok(Err(e)) => {
eprintln!("Error: {}", e);
-1
}
Err(_) => {
eprintln!("Panic in parse_config");
-2
}
}
}
// DO: For Rust-calling-Rust across C, use "C-unwind"
#[no_mangle]
pub extern "C-unwind" fn rust_callback_can_unwind() {
// This is OK to panic if called from Rust through C
// The "C-unwind" ABI allows unwinding
panic!("This is allowed");
}
```
## FFI Error Handling Pattern
```rust
// Define error codes
const SUCCESS: c_int = 0;
const ERR_NULL_PTR: c_int = -1;
const ERR_INVALID_UTF8: c_int = -2;
const ERR_IO: c_int = -3;
const ERR_PANIC: c_int = -99;
// Thread-local for detailed error
thread_local! {
static LAST_ERROR: std::cell::RefCell<Option<String>> = std::cell::RefCell::new(None);
}
fn set_error(msg: String) {
LAST_ERROR.with(|e| *e.borrow_mut() = Some(msg));
}
#[no_mangle]
pub extern "C" fn get_last_error() -> *const c_char {
LAST_ERROR.with(|e| {
e.borrow().as_ref().map(|s| s.as_ptr() as *const c_char)
.unwrap_or(std::ptr::null())
})
}
```
## Checklist
- [ ] Does my extern "C" function use catch_unwind?
- [ ] Am I avoiding unwrap/expect in FFI functions?
- [ ] Do I return error codes for error conditions?
- [ ] Have I considered using "C-unwind" for Rust-to-Rust through C?
## Related Rules
- `ffi-08`: Handle errors properly in FFI
- `safety-01`: Panic safety
@@ -1,113 +0,0 @@
---
id: ffi-05
original_id: P.UNS.FFI.05
level: P
impact: HIGH
---
# Use Portable Type Aliases from std or libc
## Summary
Use type aliases from `std::os::raw` or the `libc` crate for C-compatible types. Don't assume sizes of C types.
## Rationale
- C types have platform-dependent sizes (`int` is not always 32 bits)
- `long` is 32 bits on Windows, 64 bits on Unix
- Using Rust primitives directly causes portability bugs
## Bad Example
```rust
// DON'T: Use Rust types directly for C interop
extern "C" {
fn c_function(x: i32, y: i64) -> i32; // Might not match C types!
}
// DON'T: Assume sizes
#[repr(C)]
struct BadStruct {
count: i32, // C 'int' might not be 32 bits
size: i64, // C 'long' varies by platform!
ptr: usize, // size_t? intptr_t? Different!
}
```
## Good Example
```rust
use std::os::raw::{c_int, c_long, c_char, c_void};
// DO: Use std::os::raw types
extern "C" {
fn c_function(x: c_int, y: c_long) -> c_int;
}
// DO: Use libc for more types
use libc::{size_t, ssize_t, off_t, pid_t, time_t};
extern "C" {
fn read(fd: c_int, buf: *mut c_void, count: size_t) -> ssize_t;
fn lseek(fd: c_int, offset: off_t, whence: c_int) -> off_t;
fn getpid() -> pid_t;
}
// DO: Match C struct layout
#[repr(C)]
struct GoodStruct {
count: c_int,
size: c_long,
data: *mut c_void,
}
// DO: Use isize/usize for pointer-sized integers
#[repr(C)]
struct PointerSized {
offset: isize, // intptr_t equivalent
size: usize, // size_t in pointer arithmetic
}
```
## Type Mapping Reference
| C Type | Rust Type | Notes |
|--------|-----------|-------|
| `char` | `c_char` | May be signed or unsigned! |
| `signed char` | `i8` | |
| `unsigned char` | `u8` | |
| `short` | `c_short` | Usually i16 |
| `int` | `c_int` | Usually i32 |
| `long` | `c_long` | 32 or 64 bits! |
| `long long` | `c_longlong` | Usually i64 |
| `size_t` | `usize` or `libc::size_t` | |
| `ssize_t` | `isize` or `libc::ssize_t` | |
| `float` | `c_float` / `f32` | |
| `double` | `c_double` / `f64` | |
| `void*` | `*mut c_void` | |
| `const void*` | `*const c_void` | |
## Platform Differences
```rust
#[cfg(target_pointer_width = "64")]
type PtrDiff = i64;
#[cfg(target_pointer_width = "32")]
type PtrDiff = i32;
// Better: use isize
let diff: isize = ptr1 as isize - ptr2 as isize;
```
## Checklist
- [ ] Am I using std::os::raw or libc types for FFI?
- [ ] Have I avoided assuming c_long is 64 bits?
- [ ] Am I using size_t/usize for sizes?
- [ ] Have I tested on multiple platforms?
## Related Rules
- `ffi-13`: Ensure consistent data layout
- `ffi-14`: Types in FFI should have stable layout
@@ -1,151 +0,0 @@
---
id: ffi-06
original_id: P.UNS.FFI.06
level: P
impact: HIGH
---
# Ensure C-ABI Compatibility for Strings Between Rust and C
## Summary
When passing strings across FFI, ensure both sides agree on encoding, null-termination, and memory ownership.
## Rationale
- Rust strings are UTF-8, C strings are byte arrays
- C expects null termination, Rust strings don't have it
- Memory ownership must be explicit to avoid leaks/double-frees
## String Passing Patterns
### Rust to C (Caller Allocates)
```rust
use std::ffi::CString;
use std::os::raw::c_char;
extern "C" {
fn c_process_string(s: *const c_char);
}
fn rust_to_c(s: &str) -> Result<(), std::ffi::NulError> {
let c_string = CString::new(s)?;
// c_string lives until end of scope
unsafe {
c_process_string(c_string.as_ptr());
}
// c_string dropped here, memory freed
Ok(())
}
```
### C to Rust (C Allocates, Rust Borrows)
```rust
use std::ffi::CStr;
use std::os::raw::c_char;
extern "C" {
fn c_get_string() -> *const c_char;
}
fn c_to_rust() -> Option<String> {
let ptr = unsafe { c_get_string() };
if ptr.is_null() {
return None;
}
// Borrow from C, don't take ownership
let c_str = unsafe { CStr::from_ptr(ptr) };
Some(c_str.to_string_lossy().into_owned())
}
```
### C to Rust (Ownership Transfer)
```rust
extern "C" {
fn c_create_string() -> *mut c_char;
fn c_free_string(s: *mut c_char);
}
struct CAllocatedString {
ptr: *mut c_char,
}
impl CAllocatedString {
fn new() -> Option<Self> {
let ptr = unsafe { c_create_string() };
if ptr.is_null() {
None
} else {
Some(Self { ptr })
}
}
fn as_str(&self) -> &str {
let c_str = unsafe { CStr::from_ptr(self.ptr) };
c_str.to_str().unwrap_or("")
}
}
impl Drop for CAllocatedString {
fn drop(&mut self) {
unsafe { c_free_string(self.ptr); }
}
}
```
### Rust to C (Ownership Transfer)
```rust
extern "C" {
fn c_take_ownership(s: *mut c_char); // C will free
}
fn give_to_c(s: &str) -> Result<(), std::ffi::NulError> {
let c_string = CString::new(s)?;
let ptr = c_string.into_raw(); // Don't drop CString
unsafe {
c_take_ownership(ptr);
// C now owns this memory
// To free it back in Rust: let _ = CString::from_raw(ptr);
}
Ok(())
}
```
## Encoding Considerations
```rust
// UTF-8 to platform encoding
use std::ffi::OsString;
use std::os::unix::ffi::OsStrExt;
fn to_platform_string(s: &str) -> CString {
// On Unix, UTF-8 usually works
CString::new(s).unwrap()
}
#[cfg(windows)]
fn to_wide_string(s: &str) -> Vec<u16> {
use std::os::windows::ffi::OsStrExt;
std::ffi::OsStr::new(s)
.encode_wide()
.chain(std::iter::once(0))
.collect()
}
```
## Checklist
- [ ] Is the string null-terminated when passed to C?
- [ ] Who allocates the memory? Who frees it?
- [ ] Is the encoding (UTF-8, ASCII, platform) documented?
- [ ] Am I handling conversion errors (interior nulls, invalid UTF-8)?
## Related Rules
- `ffi-01`: Use CString/CStr at FFI boundaries
- `ffi-02`: Read std::ffi documentation
@@ -1,131 +0,0 @@
---
id: ffi-07
original_id: P.UNS.FFI.07
level: P
impact: HIGH
---
# Do Not Implement Drop for Types Passed to External Code
## Summary
If a type will be passed to external code that manages its lifetime, don't implement `Drop`. Otherwise, both Rust and the external code will try to free it.
## Rationale
- External code (C library) may take ownership of the data
- If Rust also tries to drop it, you get double-free
- Need clear ownership boundaries
## Bad Example
```rust
// DON'T: Drop on type that external code will free
#[repr(C)]
struct EventHandler {
callback: extern "C" fn(i32),
user_data: *mut c_void,
}
impl Drop for EventHandler {
fn drop(&mut self) {
// BAD: What if the C library already freed user_data?
unsafe { libc::free(self.user_data); }
}
}
extern "C" {
// C takes ownership and frees EventHandler when done
fn register_handler(h: *mut EventHandler);
}
fn bad_register() {
let handler = EventHandler { /* ... */ };
let ptr = Box::into_raw(Box::new(handler));
unsafe {
register_handler(ptr);
// If C code frees this, and Rust's Drop runs too = double-free
}
}
```
## Good Example
```rust
// DO: No Drop for types whose lifetime is managed externally
#[repr(C)]
struct EventHandler {
callback: extern "C" fn(i32),
user_data: *mut c_void,
}
// No Drop impl - C library manages lifetime
extern "C" {
fn register_handler(h: *mut EventHandler);
fn unregister_handler(h: *mut EventHandler);
}
// DO: Wrap in a Rust type that knows when it's safe to drop
struct RegisteredHandler {
ptr: *mut EventHandler,
registered: bool,
}
impl RegisteredHandler {
fn register(handler: EventHandler) -> Self {
let ptr = Box::into_raw(Box::new(handler));
unsafe { register_handler(ptr); }
Self { ptr, registered: true }
}
fn unregister(&mut self) {
if self.registered {
unsafe { unregister_handler(self.ptr); }
self.registered = false;
}
}
}
impl Drop for RegisteredHandler {
fn drop(&mut self) {
self.unregister();
// Only free if we still own it
if !self.registered {
unsafe { drop(Box::from_raw(self.ptr)); }
}
}
}
// DO: Use ManuallyDrop for explicit control
use std::mem::ManuallyDrop;
fn explicit_ownership() {
let handler = ManuallyDrop::new(EventHandler { /* ... */ });
let ptr = &*handler as *const EventHandler as *mut EventHandler;
unsafe {
register_handler(ptr);
// C now owns handler, don't drop it in Rust
}
}
```
## Ownership Patterns
| Pattern | Who Owns | Rust Drop? |
|---------|----------|------------|
| Rust creates, Rust frees | Rust | Yes |
| Rust creates, C frees | C | No |
| C creates, C frees | C | No (use wrapper) |
| C creates, Rust frees | Rust | Yes (in wrapper) |
## Checklist
- [ ] Who will free this type's memory?
- [ ] If external code frees it, am I avoiding Drop?
- [ ] If ownership is conditional, do I track it?
- [ ] Am I using ManuallyDrop or forget() when transferring ownership?
## Related Rules
- `ffi-03`: Implement Drop for wrapped C pointers (opposite case)
- `mem-03`: Don't let String/Vec drop foreign memory
@@ -1,146 +0,0 @@
---
id: ffi-08
original_id: P.UNS.FFI.08
level: P
impact: HIGH
---
# Handle Errors Properly in FFI
## Summary
FFI functions must use C-compatible error handling (return codes, errno, out parameters). Rust's Result/Option don't cross FFI boundaries.
## Rationale
- C doesn't have Result or Option
- Exceptions don't exist in C
- Must use patterns C code understands
## Bad Example
```rust
// DON'T: Return Result across FFI
#[no_mangle]
pub extern "C" fn bad_open(path: *const c_char) -> Result<Handle, Error> {
// Result is not C-compatible!
unimplemented!()
}
// DON'T: Return Option across FFI
#[no_mangle]
pub extern "C" fn bad_find(id: i32) -> Option<*mut Data> {
// Option<*mut T> might work but is confusing
unimplemented!()
}
```
## Good Example
```rust
use std::os::raw::{c_char, c_int};
// Error codes
const SUCCESS: c_int = 0;
const ERR_NULL_PTR: c_int = 1;
const ERR_INVALID_PATH: c_int = 2;
const ERR_FILE_NOT_FOUND: c_int = 3;
const ERR_PERMISSION: c_int = 4;
const ERR_UNKNOWN: c_int = -1;
// DO: Return error code, output via pointer
#[no_mangle]
pub extern "C" fn open_file(
path: *const c_char,
out_handle: *mut *mut Handle
) -> c_int {
if path.is_null() || out_handle.is_null() {
return ERR_NULL_PTR;
}
let path_str = match unsafe { CStr::from_ptr(path) }.to_str() {
Ok(s) => s,
Err(_) => return ERR_INVALID_PATH,
};
match File::open(path_str) {
Ok(file) => {
let handle = Box::into_raw(Box::new(Handle { file }));
unsafe { *out_handle = handle; }
SUCCESS
}
Err(e) => {
match e.kind() {
std::io::ErrorKind::NotFound => ERR_FILE_NOT_FOUND,
std::io::ErrorKind::PermissionDenied => ERR_PERMISSION,
_ => ERR_UNKNOWN,
}
}
}
}
// DO: Use errno for POSIX-style APIs
#[cfg(unix)]
#[no_mangle]
pub extern "C" fn posix_style_read(
fd: c_int,
buf: *mut u8,
count: usize
) -> isize {
if buf.is_null() {
unsafe { *libc::__errno_location() = libc::EINVAL; }
return -1;
}
// ... do read ...
// On error:
// unsafe { *libc::__errno_location() = error_code; }
// return -1;
count as isize
}
// DO: Provide error message function
thread_local! {
static LAST_ERROR: std::cell::RefCell<Option<String>> = std::cell::RefCell::new(None);
}
#[no_mangle]
pub extern "C" fn get_error_message(buf: *mut c_char, len: usize) -> c_int {
LAST_ERROR.with(|e| {
if let Some(msg) = e.borrow().as_ref() {
let bytes = msg.as_bytes();
let copy_len = std::cmp::min(bytes.len(), len.saturating_sub(1));
unsafe {
std::ptr::copy_nonoverlapping(bytes.as_ptr(), buf as *mut u8, copy_len);
*buf.add(copy_len) = 0;
}
SUCCESS
} else {
ERR_UNKNOWN
}
})
}
```
## Error Handling Patterns
| Pattern | Usage |
|---------|-------|
| Return code | Simple success/failure |
| Return code + out param | Return value on success |
| errno | POSIX-style APIs |
| Error message function | Detailed error info |
| Last-error thread-local | Windows-style APIs |
## Checklist
- [ ] Am I returning C-compatible error indicators?
- [ ] Are output parameters used for return values?
- [ ] Is there a way to get detailed error info?
- [ ] Am I documenting all possible error codes?
## Related Rules
- `ffi-04`: Handle panics at FFI boundary
- `safety-10`: Document safety requirements
@@ -1,136 +0,0 @@
---
id: ffi-09
original_id: P.UNS.FFI.09
level: P
impact: MEDIUM
---
# Use References Instead of Raw Pointers When Calling Safe C Functions
## Summary
When wrapping C functions that don't need null pointers, use Rust references in the safe wrapper to enforce non-null at compile time.
## Rationale
- References guarantee non-null
- References have lifetime tracking
- Raw pointers should stay in the unsafe FFI layer
- Safe Rust API should use safe types
## Bad Example
```rust
extern "C" {
fn c_process(data: *const u8, len: usize);
}
// DON'T: Expose raw pointers in safe API
pub fn process(data: *const u8, len: usize) {
// Caller might pass null!
unsafe { c_process(data, len); }
}
// DON'T: Unsafe function when it could be safe
pub unsafe fn process_unsafe(data: *const u8, len: usize) {
// Why force caller to use unsafe?
c_process(data, len);
}
```
## Good Example
```rust
extern "C" {
fn c_process(data: *const u8, len: usize);
fn c_modify(data: *mut Data);
fn c_optional(data: *const Data); // Can be null
}
// DO: Use slice reference for safe API
pub fn process(data: &[u8]) {
// Reference guarantees non-null
// Slice guarantees valid length
unsafe { c_process(data.as_ptr(), data.len()); }
}
// DO: Use &mut for exclusive access
pub fn modify(data: &mut Data) {
// Mutable reference guarantees:
// - Non-null
// - Exclusive access
// - Valid for duration
unsafe { c_modify(data as *mut Data); }
}
// DO: Use Option<&T> for nullable parameters
pub fn optional(data: Option<&Data>) {
let ptr = data.map(|d| d as *const Data).unwrap_or(std::ptr::null());
unsafe { c_optional(ptr); }
}
// DO: Wrap FFI types in safe Rust types
pub struct SafeHandle(*mut c_void);
impl SafeHandle {
pub fn new() -> Option<Self> {
let ptr = unsafe { create_handle() };
if ptr.is_null() {
None
} else {
Some(Self(ptr))
}
}
// Methods take &self or &mut self, not raw pointers
pub fn do_something(&self) {
unsafe { handle_operation(self.0); }
}
}
```
## Converting Between References and Pointers
```rust
// Reference to pointer
fn ref_to_ptr(r: &Data) -> *const Data {
r as *const Data
}
fn mut_ref_to_ptr(r: &mut Data) -> *mut Data {
r as *mut Data
}
// Slice to pointer
fn slice_to_ptr(s: &[u8]) -> (*const u8, usize) {
(s.as_ptr(), s.len())
}
// Pointer to reference (unsafe)
unsafe fn ptr_to_ref<'a>(p: *const Data) -> &'a Data {
&*p
}
unsafe fn ptr_to_mut<'a>(p: *mut Data) -> &'a mut Data {
&mut *p
}
```
## When to Use Raw Pointers
- FFI declarations (`extern "C"`)
- Implementing the unsafe boundary layer
- When null is a valid value
- When the pointee might not be valid Rust (e.g., uninitialized)
## Checklist
- [ ] Can this parameter be a reference instead of a pointer?
- [ ] Am I checking for null in the unsafe layer?
- [ ] Is the safe API free of raw pointers?
- [ ] Do I use Option<&T> for nullable references?
## Related Rules
- `safety-06`: Don't expose raw pointers in public APIs
- `ffi-02`: Read std::ffi documentation
@@ -1,132 +0,0 @@
---
id: ffi-10
original_id: P.UNS.FFI.10
level: P
impact: CRITICAL
---
# Exported Rust Functions Must Be Designed for Thread-Safety
## Summary
Functions exported to C with `#[no_mangle] extern "C"` may be called from multiple threads. Ensure they are thread-safe.
## Rationale
- C code doesn't know about Rust's thread safety guarantees
- C may call your function from any thread
- Global state must be synchronized
- Race conditions are undefined behavior
## Bad Example
```rust
// DON'T: Unsynchronized global state
static mut COUNTER: i32 = 0;
#[no_mangle]
pub extern "C" fn increment() -> i32 {
unsafe {
COUNTER += 1; // Data race if called from multiple threads!
COUNTER
}
}
// DON'T: Thread-local assuming single thread
thread_local! {
static CONFIG: RefCell<Config> = RefCell::new(Config::default());
}
#[no_mangle]
pub extern "C" fn set_config(value: i32) {
// Different threads get different configs!
// Is that what the C caller expects?
CONFIG.with(|c| c.borrow_mut().value = value);
}
// DON'T: Non-Send types in globals
static mut HANDLE: Option<Rc<Data>> = None; // Rc is not Send!
```
## Good Example
```rust
use std::sync::atomic::{AtomicI32, Ordering};
use std::sync::{Mutex, OnceLock};
// DO: Use atomics for simple counters
static COUNTER: AtomicI32 = AtomicI32::new(0);
#[no_mangle]
pub extern "C" fn increment() -> i32 {
COUNTER.fetch_add(1, Ordering::SeqCst) + 1
}
// DO: Use Mutex for complex state
static CONFIG: OnceLock<Mutex<Config>> = OnceLock::new();
fn get_config() -> &'static Mutex<Config> {
CONFIG.get_or_init(|| Mutex::new(Config::default()))
}
#[no_mangle]
pub extern "C" fn set_config_value(value: i32) -> i32 {
match get_config().lock() {
Ok(mut config) => {
config.value = value;
0 // Success
}
Err(_) => -1 // Lock poisoned
}
}
// DO: Document thread safety requirements
/// Initializes the library. NOT thread-safe.
/// Must be called once from main thread before any other function.
#[no_mangle]
pub extern "C" fn init() -> i32 {
// One-time initialization
0
}
/// Processes data. Thread-safe.
/// May be called from multiple threads concurrently.
#[no_mangle]
pub extern "C" fn process(data: *const u8, len: usize) -> i32 {
// Uses only local state or synchronized globals
0
}
// DO: Make non-thread-safe APIs explicit
/// Handle for single-threaded use only.
///
/// # Thread Safety
///
/// This handle must only be used from the thread that created it.
struct SingleThreadHandle {
data: *mut Data,
_not_send: std::marker::PhantomData<*const ()>, // !Send
}
```
## Synchronization Patterns
| Pattern | Use Case |
|---------|----------|
| `AtomicT` | Simple counters, flags |
| `Mutex<T>` | Complex shared state |
| `RwLock<T>` | Read-heavy shared state |
| `OnceLock<T>` | Lazy one-time init |
| `thread_local!` | Per-thread state (document!) |
## Checklist
- [ ] Does my exported function access global state?
- [ ] Is that state properly synchronized?
- [ ] Have I documented thread-safety guarantees?
- [ ] Are any types !Send/!Sync exposed across FFI?
## Related Rules
- `ptr-01`: Don't share raw pointers across threads
- `safety-05`: Send/Sync implementation safety
@@ -1,142 +0,0 @@
---
id: ffi-11
original_id: P.UNS.FFI.11
level: P
impact: HIGH
clippy: unaligned_references
---
# Be Careful with UB When Referencing #[repr(packed)] Struct Fields
## Summary
Creating references to fields in `#[repr(packed)]` structs is undefined behavior if the field is misaligned. Use raw pointers and `read_unaligned`/`write_unaligned` instead.
## Rationale
- Packed structs have no padding, so fields may be misaligned
- References must be aligned; misaligned references are UB
- Even implicit references (method calls, match) can cause UB
## Bad Example
```rust
#[repr(C, packed)]
struct Packet {
header: u8,
value: u32, // Misaligned! At offset 1, not 4
data: u64, // Misaligned! At offset 5, not 8
}
fn bad_reference(p: &Packet) -> &u32 {
&p.value // UB: Creates misaligned reference!
}
fn bad_match(p: &Packet) {
match p.value { // UB: Match creates a reference
0 => {},
_ => {},
}
}
fn bad_method(p: &Packet) {
p.value.to_string(); // UB: Method call creates reference
}
fn bad_borrow(p: &mut Packet) {
let v = &mut p.value; // UB: Misaligned mutable reference
*v = 42;
}
```
## Good Example
```rust
#[repr(C, packed)]
struct Packet {
header: u8,
value: u32,
data: u64,
}
// DO: Copy out the value
fn good_read(p: &Packet) -> u32 {
p.value // Copies the value, no reference created
}
// DO: Use addr_of! for raw pointer (Rust 2021+)
fn good_ptr_read(p: &Packet) -> u32 {
// SAFETY: read_unaligned handles misalignment
unsafe {
std::ptr::addr_of!(p.value).read_unaligned()
}
}
// DO: Use addr_of_mut! for writing
fn good_ptr_write(p: &mut Packet, value: u32) {
// SAFETY: write_unaligned handles misalignment
unsafe {
std::ptr::addr_of_mut!(p.value).write_unaligned(value);
}
}
// DO: Create accessor methods
impl Packet {
fn value(&self) -> u32 {
unsafe { std::ptr::addr_of!(self.value).read_unaligned() }
}
fn set_value(&mut self, value: u32) {
unsafe { std::ptr::addr_of_mut!(self.value).write_unaligned(value); }
}
fn data(&self) -> u64 {
unsafe { std::ptr::addr_of!(self.data).read_unaligned() }
}
}
// DO: Consider using byte arrays + from_ne_bytes
#[repr(C, packed)]
struct PacketBytes {
header: u8,
value: [u8; 4], // Store as bytes
data: [u8; 8],
}
impl PacketBytes {
fn value(&self) -> u32 {
u32::from_ne_bytes(self.value) // Safe, no alignment issue
}
}
```
## Safe Alternatives
```rust
// Alternative 1: Don't use packed
#[repr(C)]
struct AlignedPacket {
header: u8,
_pad: [u8; 3],
value: u32,
data: u64,
}
// Alternative 2: Use zerocopy crate
// use zerocopy::{AsBytes, FromBytes};
// Alternative 3: Use bytemuck
// use bytemuck::{Pod, Zeroable};
```
## Checklist
- [ ] Am I creating references to packed struct fields?
- [ ] Am I using addr_of! / addr_of_mut! for field access?
- [ ] Am I using read_unaligned / write_unaligned?
- [ ] Would a byte array representation be safer?
## Related Rules
- `ptr-04`: Don't dereference misaligned pointers
- `mem-01`: Choose appropriate data layout
@@ -1,164 +0,0 @@
---
id: ffi-12
original_id: P.UNS.FFI.12
level: P
impact: MEDIUM
---
# Document Invariant Assumptions for C-Provided Parameters
## Summary
When receiving parameters from C, document what invariants you assume (non-null, alignment, validity, lifetime) and verify them when possible.
## Rationale
- C doesn't enforce invariants at compile time
- Rust code needs to validate or document assumptions
- Debugging FFI bugs is hard without clear documentation
## Bad Example
```rust
// DON'T: Undocumented assumptions
extern "C" {
fn get_data() -> *mut Data;
}
fn bad_use() -> &'static Data {
let ptr = unsafe { get_data() };
// Assumes:
// - ptr is non-null (not documented)
// - ptr is aligned (not checked)
// - Data is valid (not verified)
// - Lifetime is 'static (just guessing)
unsafe { &*ptr }
}
// DON'T: Silent assumptions in function signature
#[no_mangle]
pub extern "C" fn process(data: *const Data, len: usize) {
// What if data is null?
// What if len is wrong?
// What if data contains invalid Data?
let slice = unsafe {
std::slice::from_raw_parts(data, len)
};
}
```
## Good Example
```rust
/// Retrieves data from the C library.
///
/// # Invariants Assumed from C
///
/// - Returns a non-null pointer on success, null on failure
/// - Returned pointer is valid for the lifetime of the library
/// - Returned pointer is aligned for `Data`
/// - The `Data` struct is fully initialized
extern "C" {
fn get_data() -> *mut Data;
}
fn documented_use() -> Option<&'static Data> {
let ptr = unsafe { get_data() };
// Verify what we can
if ptr.is_null() {
return None;
}
// Document what we can't verify
// SAFETY:
// - Non-null: checked above
// - Aligned: documented in C library docs
// - Valid: C library guarantees initialized Data
// - Lifetime: C library guarantees static lifetime
Some(unsafe { &*ptr })
}
/// Processes data provided by C caller.
///
/// # Parameters
///
/// - `data`: Must be non-null, aligned for `Data`, and point to `len` valid `Data` items
/// - `len`: Number of items. Must not exceed `isize::MAX / size_of::<Data>()`
///
/// # Returns
///
/// - `0` on success
/// - `-1` if `data` is null
/// - `-2` if `len` is invalid
///
/// # Thread Safety
///
/// This function is thread-safe. The `data` array must not be mutated during the call.
#[no_mangle]
pub extern "C" fn process_documented(data: *const Data, len: usize) -> i32 {
// Verify invariants we can check
if data.is_null() {
return -1;
}
if len > isize::MAX as usize / std::mem::size_of::<Data>() {
return -2;
}
// SAFETY:
// - Non-null: checked above
// - Aligned: documented requirement for caller
// - Valid for len items: documented requirement for caller
// - Not mutated: documented thread safety requirement
let slice = unsafe { std::slice::from_raw_parts(data, len) };
for item in slice {
// process...
}
0
}
```
## Documentation Template
```rust
/// Brief description.
///
/// # Parameters
///
/// - `param`: Description, constraints (non-null, aligned, etc.)
///
/// # Invariants Assumed
///
/// The following invariants are assumed and NOT verified:
/// - Invariant 1: explanation
/// - Invariant 2: explanation
///
/// The following invariants ARE verified at runtime:
/// - Verified 1: how it's checked
///
/// # Safety (for unsafe fn)
///
/// Caller must ensure:
/// - Requirement 1
/// - Requirement 2
///
/// # Errors
///
/// Returns error code when:
/// - Condition 1: error code
```
## Checklist
- [ ] Have I documented all assumptions about C parameters?
- [ ] Which invariants can I verify at runtime?
- [ ] Which must I trust the C caller to uphold?
- [ ] Have I documented error conditions and return values?
## Related Rules
- `safety-02`: Verify safety invariants
- `safety-10`: Document safety requirements
@@ -1,146 +0,0 @@
---
id: ffi-13
original_id: P.UNS.FFI.13
level: P
impact: HIGH
---
# Ensure Consistent Data Layout for Custom Types
## Summary
Types shared between Rust and C must have `#[repr(C)]` to ensure the memory layout matches what C expects.
## Rationale
- Rust's default layout is unspecified and may change
- C has specific, standardized layout rules
- Mismatched layouts cause memory corruption
## Bad Example
```rust
// DON'T: Rust layout for FFI types
struct BadStruct {
a: u8,
b: u32,
c: u8,
}
// Rust may reorder to: b, a, c (for better packing)
// C expects: a, padding, b, c, padding
extern "C" {
fn use_struct(s: *const BadStruct); // Layout mismatch!
}
// DON'T: Assume Rust enum layout matches C
enum BadEnum {
A,
B(i32),
C { x: u8, y: u8 },
}
// Rust enum layout is complex and not C-compatible
```
## Good Example
```rust
// DO: Use repr(C) for FFI structs
#[repr(C)]
struct GoodStruct {
a: u8, // offset 0
// 3 bytes padding
b: u32, // offset 4
c: u8, // offset 8
// 3 bytes padding
}
// Total size: 12, align: 4
// DO: Use repr(C) for enums with explicit discriminant
#[repr(C)]
enum GoodEnum {
A = 0,
B = 1,
C = 2,
}
// Equivalent to C: enum { A = 0, B = 1, C = 2 };
// DO: For complex enums, use tagged unions
#[repr(C)]
struct TaggedUnion {
tag: GoodEnum,
data: GoodUnionData,
}
#[repr(C)]
union GoodUnionData {
a: (), // For GoodEnum::A
b: i32, // For GoodEnum::B
c: [u8; 2], // For GoodEnum::C
}
// DO: Verify layout at compile time
const _: () = {
assert!(std::mem::size_of::<GoodStruct>() == 12);
assert!(std::mem::align_of::<GoodStruct>() == 4);
};
```
## Layout Verification
```rust
use std::mem::{size_of, align_of, offset_of};
#[repr(C)]
struct Verified {
a: u8,
b: u32,
c: u8,
}
// Compile-time layout verification
const _: () = {
assert!(size_of::<Verified>() == 12);
assert!(align_of::<Verified>() == 4);
// offset_of! requires nightly or crate
// assert!(offset_of!(Verified, a) == 0);
// assert!(offset_of!(Verified, b) == 4);
// assert!(offset_of!(Verified, c) == 8);
};
// Runtime verification
#[test]
fn verify_layout() {
assert_eq!(size_of::<Verified>(), 12);
assert_eq!(align_of::<Verified>(), 4);
let v = Verified { a: 0, b: 0, c: 0 };
let base = &v as *const _ as usize;
assert_eq!(&v.a as *const _ as usize - base, 0);
assert_eq!(&v.b as *const _ as usize - base, 4);
assert_eq!(&v.c as *const _ as usize - base, 8);
}
```
## repr Options
| Attribute | Effect |
|-----------|--------|
| `#[repr(C)]` | C-compatible layout |
| `#[repr(C, packed)]` | C layout, no padding |
| `#[repr(C, align(N))]` | C layout, minimum align N |
| `#[repr(transparent)]` | Same layout as single field |
| `#[repr(u8)]` etc. | Enum discriminant type |
## Checklist
- [ ] Is every FFI struct marked `#[repr(C)]`?
- [ ] Is every FFI enum using explicit discriminants?
- [ ] Have I verified the layout matches the C header?
- [ ] Have I added compile-time assertions?
## Related Rules
- `mem-01`: Choose appropriate data layout
- `ffi-14`: Types in FFI should have stable layout
@@ -1,135 +0,0 @@
---
id: ffi-14
original_id: P.UNS.FFI.14
level: P
impact: HIGH
---
# Types Used in FFI Should Have Stable Layout
## Summary
FFI types should not change layout between versions. Use `#[repr(C)]` and avoid types with unstable layout like generic `std` types.
## Rationale
- ABI compatibility requires stable layout
- Dynamic libraries may be loaded with different compiler versions
- Layout changes break binary compatibility
## Bad Example
```rust
// DON'T: Use Rust std types with unstable layout in FFI
extern "C" {
// Vec layout is not stable!
fn bad_vec(v: Vec<i32>);
// String layout is not stable!
fn bad_string(s: String);
// HashMap layout varies between versions
fn bad_map(m: std::collections::HashMap<i32, i32>);
}
// DON'T: Use Rust-specific types in C structs
#[repr(C)]
struct BadMixed {
id: i32,
data: Vec<u8>, // Vec is not C-compatible!
}
// DON'T: Use Option with non-null optimization assumptions
#[repr(C)]
struct BadOption {
value: Option<std::num::NonZeroU32>, // Layout may change!
}
```
## Good Example
```rust
use std::os::raw::{c_int, c_char, c_void};
// DO: Use C-compatible types
#[repr(C)]
struct GoodStruct {
id: c_int,
name: *const c_char, // C-style string
data: *const c_void, // Generic pointer
data_len: usize,
}
// DO: Use explicit struct for what Vec would provide
#[repr(C)]
struct GoodBuffer {
ptr: *mut u8,
len: usize,
cap: usize,
}
impl GoodBuffer {
fn from_vec(mut v: Vec<u8>) -> Self {
let buf = Self {
ptr: v.as_mut_ptr(),
len: v.len(),
cap: v.capacity(),
};
std::mem::forget(v);
buf
}
/// # Safety
/// Must have been created by from_vec()
unsafe fn into_vec(self) -> Vec<u8> {
Vec::from_raw_parts(self.ptr, self.len, self.cap)
}
}
// DO: Use fixed-size arrays for bounded data
#[repr(C)]
struct FixedName {
name: [c_char; 64],
name_len: usize,
}
// DO: Define your own stable option type
#[repr(C)]
struct OptionalU32 {
has_value: bool,
value: u32,
}
impl From<Option<u32>> for OptionalU32 {
fn from(opt: Option<u32>) -> Self {
match opt {
Some(v) => Self { has_value: true, value: v },
None => Self { has_value: false, value: 0 },
}
}
}
```
## Stable Types for FFI
| Use Instead Of | Stable Type |
|----------------|-------------|
| `Vec<T>` | `*mut T` + `len` + `cap` |
| `String` | `*const c_char` or `*mut c_char` + `len` |
| `&[T]` | `*const T` + `len` |
| `Option<T>` | Custom tagged struct |
| `Result<T, E>` | Error code + out parameter |
| `Box<T>` | `*mut T` |
| `bool` | `c_int` or explicit `u8` |
## Checklist
- [ ] Am I using only C-compatible primitive types?
- [ ] Am I avoiding std collection types in FFI signatures?
- [ ] Have I created stable wrappers for Rust types?
- [ ] Is the layout documented for other languages?
## Related Rules
- `ffi-13`: Ensure consistent data layout
- `ffi-05`: Use portable type aliases
@@ -1,145 +0,0 @@
---
id: ffi-15
original_id: P.UNS.FFI.15
level: P
impact: HIGH
---
# Validate Non-Robust External Values
## Summary
Data received from external sources (FFI, files, network) may be invalid. Validate before using it as Rust types with stricter invariants.
## Rationale
- External data can be malicious or corrupted
- Rust types have invariants (e.g., valid UTF-8 for str)
- Invalid data causes undefined behavior
## Bad Example
```rust
// DON'T: Trust external data
extern "C" {
fn get_status() -> u8;
}
#[derive(Debug)]
enum Status { Active = 0, Inactive = 1, Pending = 2 }
fn bad_convert() -> Status {
let raw = unsafe { get_status() };
// BAD: Assumes C returns valid enum value
unsafe { std::mem::transmute(raw) } // UB if raw > 2
}
// DON'T: Trust strings from C
fn bad_string(ptr: *const c_char) -> &str {
let cstr = unsafe { CStr::from_ptr(ptr) };
// BAD: Assumes valid UTF-8
cstr.to_str().unwrap()
}
// DON'T: Trust size values
fn bad_size(ptr: *const u8, len: usize) -> Vec<u8> {
// BAD: len could be huge, causing OOM
// BAD: len could exceed actual data
unsafe { std::slice::from_raw_parts(ptr, len) }.to_vec()
}
```
## Good Example
```rust
// DO: Validate enum values
#[derive(Debug, Clone, Copy)]
#[repr(u8)]
enum Status {
Active = 0,
Inactive = 1,
Pending = 2,
}
impl TryFrom<u8> for Status {
type Error = InvalidStatusError;
fn try_from(value: u8) -> Result<Self, Self::Error> {
match value {
0 => Ok(Status::Active),
1 => Ok(Status::Inactive),
2 => Ok(Status::Pending),
_ => Err(InvalidStatusError(value)),
}
}
}
fn good_convert() -> Result<Status, InvalidStatusError> {
let raw = unsafe { get_status() };
Status::try_from(raw) // Returns error for invalid values
}
// DO: Handle invalid UTF-8
fn good_string(ptr: *const c_char) -> Result<String, std::str::Utf8Error> {
if ptr.is_null() {
return Ok(String::new());
}
let cstr = unsafe { CStr::from_ptr(ptr) };
cstr.to_str().map(|s| s.to_owned())
}
fn good_string_lossy(ptr: *const c_char) -> String {
if ptr.is_null() {
return String::new();
}
let cstr = unsafe { CStr::from_ptr(ptr) };
cstr.to_string_lossy().into_owned() // Replaces invalid UTF-8
}
// DO: Validate sizes
const MAX_REASONABLE_SIZE: usize = 100 * 1024 * 1024; // 100 MB
fn good_size(ptr: *const u8, len: usize) -> Result<Vec<u8>, ValidationError> {
if ptr.is_null() {
return Err(ValidationError::NullPointer);
}
if len > MAX_REASONABLE_SIZE {
return Err(ValidationError::SizeTooLarge);
}
// Still need to trust that ptr points to len valid bytes
// Document this as a caller requirement
let slice = unsafe { std::slice::from_raw_parts(ptr, len) };
Ok(slice.to_vec())
}
// DO: Use num_enum for safe enum conversion
// use num_enum::TryFromPrimitive;
//
// #[derive(TryFromPrimitive)]
// #[repr(u8)]
// enum Status { Active = 0, Inactive = 1, Pending = 2 }
```
## Validation Patterns
| External Data | Validation |
|---------------|------------|
| Enum discriminant | Match against valid values |
| String | Check UTF-8 or use lossy conversion |
| Size/length | Check against maximum |
| Pointer | Check for null |
| Boolean | Explicit 0/1 check or treat any non-zero as true |
| Float | Check for NaN, infinity if problematic |
## Checklist
- [ ] Am I validating external enum values?
- [ ] Am I handling potential invalid UTF-8?
- [ ] Am I checking sizes against reasonable limits?
- [ ] Am I using TryFrom instead of transmute?
## Related Rules
- `ffi-12`: Document invariant assumptions
- `safety-02`: Verify safety invariants
@@ -1,141 +0,0 @@
---
id: ffi-16
original_id: P.UNS.FFI.16
level: P
impact: HIGH
---
# Separate Data and Code When Passing Rust Closures to C
## Summary
C callbacks are function pointers without captured state. To pass Rust closures to C, separate the function pointer from the closure data using a "trampoline" pattern.
## Rationale
- Rust closures can capture state (like lambdas)
- C function pointers are just addresses, no state
- Must pass state separately via `void*` user_data
## Bad Example
```rust
// DON'T: Try to pass closure directly
extern "C" {
fn set_callback(cb: fn(i32) -> i32); // Only works for non-capturing!
}
fn bad_closure() {
let multiplier = 2;
let closure = |x| x * multiplier; // Captures multiplier
// This won't compile - closure is not fn pointer
// set_callback(closure);
}
// DON'T: Transmute closure to function pointer
fn bad_transmute() {
let closure = |x: i32| x * 2;
let fp: fn(i32) -> i32 = unsafe { std::mem::transmute(closure) };
// UB: Closure may have non-zero size
}
```
## Good Example
```rust
use std::os::raw::c_void;
use std::ffi::c_int;
// C callback signature with user_data
type CCallback = extern "C" fn(value: c_int, user_data: *mut c_void) -> c_int;
extern "C" {
fn set_callback(cb: CCallback, user_data: *mut c_void);
fn remove_callback();
}
// DO: Use trampoline pattern
fn good_closure<F: FnMut(i32) -> i32>(mut closure: F) {
// Trampoline function that forwards to the closure
extern "C" fn trampoline<F: FnMut(i32) -> i32>(
value: c_int,
user_data: *mut c_void,
) -> c_int {
let closure = unsafe { &mut *(user_data as *mut F) };
closure(value as i32) as c_int
}
let user_data = &mut closure as *mut F as *mut c_void;
unsafe {
set_callback(trampoline::<F>, user_data);
// Important: closure must live until callback is removed!
}
}
// DO: Box the closure for 'static lifetime
struct CallbackHandle {
closure: Box<dyn FnMut(i32) -> i32>,
}
impl CallbackHandle {
fn new<F: FnMut(i32) -> i32 + 'static>(closure: F) -> Self {
Self { closure: Box::new(closure) }
}
fn register(&mut self) {
extern "C" fn trampoline(value: c_int, user_data: *mut c_void) -> c_int {
let closure = unsafe { &mut *(user_data as *mut Box<dyn FnMut(i32) -> i32>) };
closure(value as i32) as c_int
}
let user_data = &mut self.closure as *mut _ as *mut c_void;
unsafe { set_callback(trampoline, user_data); }
}
}
impl Drop for CallbackHandle {
fn drop(&mut self) {
unsafe { remove_callback(); }
// Now safe to drop closure
}
}
// Usage
fn example() {
let multiplier = 2;
let mut handle = CallbackHandle::new(move |x| x * multiplier);
handle.register();
// handle must live until callback is no longer needed
}
```
## Trampoline Pattern
```
Rust Closure: |x| x * captured_value
|
v
+-----------------+ +-----------------+
| trampoline fn | --> | closure data |
| (no captures) | | (captured_value)|
+-----------------+ +-----------------+
| ^
| user_data ptr |
+-------------------------+
C sees: function pointer + void* user_data
```
## Checklist
- [ ] Does my closure capture any state?
- [ ] Am I using the trampoline pattern?
- [ ] Does the closure data live long enough?
- [ ] Am I unregistering before dropping the closure?
## Related Rules
- `ffi-03`: Implement Drop for resource wrappers
- `ffi-10`: Thread safety for callbacks
@@ -1,152 +0,0 @@
---
id: ffi-17
original_id: P.UNS.FFI.17
level: P
impact: MEDIUM
---
# Use Dedicated Opaque Type Pointers Instead of c_void for C Opaque Types
## Summary
Instead of using `*mut c_void` for opaque C handles, create dedicated marker types that provide type safety.
## Rationale
- `*mut c_void` accepts any pointer, easy to mix up handles
- Dedicated types catch mistakes at compile time
- Self-documenting code
- Prevents accidental use of wrong free function
## Bad Example
```rust
use std::ffi::c_void;
extern "C" {
fn create_database() -> *mut c_void;
fn create_connection() -> *mut c_void;
fn execute(conn: *mut c_void, query: *const i8);
fn close_database(db: *mut c_void);
fn close_connection(conn: *mut c_void);
}
fn bad_usage() {
let db = unsafe { create_database() };
let conn = unsafe { create_connection() };
// BUG: Passed db where conn was expected - compiles fine!
unsafe { execute(db, b"SELECT 1\0".as_ptr() as *const i8) };
// BUG: Wrong close function - compiles fine!
unsafe { close_connection(db) };
unsafe { close_database(conn) };
}
```
## Good Example
```rust
use std::marker::PhantomData;
// DO: Define opaque marker types
#[repr(C)]
pub struct Database {
_private: [u8; 0],
_marker: PhantomData<(*mut u8, std::marker::PhantomPinned)>,
}
#[repr(C)]
pub struct Connection {
_private: [u8; 0],
_marker: PhantomData<(*mut u8, std::marker::PhantomPinned)>,
}
extern "C" {
fn create_database() -> *mut Database;
fn create_connection(db: *mut Database) -> *mut Connection;
fn execute(conn: *mut Connection, query: *const i8) -> i32;
fn close_database(db: *mut Database);
fn close_connection(conn: *mut Connection);
}
fn good_usage() {
let db = unsafe { create_database() };
let conn = unsafe { create_connection(db) };
// Compile error: expected *mut Connection, found *mut Database
// unsafe { execute(db, b"SELECT 1\0".as_ptr() as *const i8) };
// Correct usage
unsafe { execute(conn, b"SELECT 1\0".as_ptr() as *const i8) };
unsafe { close_connection(conn) };
unsafe { close_database(db) };
}
// DO: Wrap in safe Rust types
pub struct SafeDatabase {
ptr: *mut Database,
}
impl SafeDatabase {
pub fn new() -> Option<Self> {
let ptr = unsafe { create_database() };
if ptr.is_null() { None } else { Some(Self { ptr }) }
}
pub fn connect(&self) -> Option<SafeConnection<'_>> {
let ptr = unsafe { create_connection(self.ptr) };
if ptr.is_null() { None } else { Some(SafeConnection { ptr, _db: PhantomData }) }
}
}
impl Drop for SafeDatabase {
fn drop(&mut self) {
unsafe { close_database(self.ptr); }
}
}
pub struct SafeConnection<'db> {
ptr: *mut Connection,
_db: PhantomData<&'db SafeDatabase>,
}
impl SafeConnection<'_> {
pub fn execute(&self, query: &str) -> Result<(), ()> {
let query = std::ffi::CString::new(query).map_err(|_| ())?;
let result = unsafe { execute(self.ptr, query.as_ptr()) };
if result == 0 { Ok(()) } else { Err(()) }
}
}
impl Drop for SafeConnection<'_> {
fn drop(&mut self) {
unsafe { close_connection(self.ptr); }
}
}
```
## Opaque Type Pattern
```rust
// The zero-sized array makes it impossible to construct
// PhantomData ensures proper variance and !Send/!Sync if needed
#[repr(C)]
pub struct OpaqueHandle {
_private: [u8; 0],
_marker: PhantomData<(*mut u8, std::marker::PhantomPinned)>,
}
```
## Checklist
- [ ] Am I using `*mut c_void` for distinct handle types?
- [ ] Would dedicated types prevent bugs?
- [ ] Have I wrapped opaque pointers in safe Rust types?
- [ ] Do my types enforce correct handle/function pairing?
## Related Rules
- `ffi-02`: Read std::ffi documentation
- `ffi-03`: Implement Drop for wrapped pointers
@@ -1,165 +0,0 @@
---
id: ffi-18
original_id: P.UNS.FFI.18
level: P
impact: HIGH
---
# Avoid Passing Trait Objects to C Interfaces
## Summary
Trait objects (`dyn Trait`) have Rust-specific layout (fat pointers with vtable) that is not compatible with C.
## Rationale
- Trait objects are "fat pointers": data ptr + vtable ptr
- C expects thin pointers (single pointer)
- Vtable layout is not stable across Rust versions
- C cannot call Rust vtable methods
## Bad Example
```rust
// DON'T: Pass trait objects to C
trait Handler {
fn handle(&self, data: i32);
}
extern "C" {
// This won't work - dyn Handler is a fat pointer!
fn set_handler(h: *const dyn Handler);
}
// DON'T: Store trait objects in FFI structs
#[repr(C)]
struct BadCallback {
handler: *const dyn Handler, // Not C-compatible!
}
```
## Good Example
```rust
use std::os::raw::{c_int, c_void};
// DO: Use function pointers with user_data (trampoline pattern)
type HandlerFn = extern "C" fn(data: c_int, user_data: *mut c_void);
extern "C" {
fn set_handler(handler: HandlerFn, user_data: *mut c_void);
}
trait Handler {
fn handle(&self, data: i32);
}
fn register_handler<H: Handler + 'static>(handler: H) {
// Box the handler
let boxed: Box<H> = Box::new(handler);
let user_data = Box::into_raw(boxed) as *mut c_void;
extern "C" fn trampoline<H: Handler>(data: c_int, user_data: *mut c_void) {
let handler = unsafe { &*(user_data as *const H) };
handler.handle(data as i32);
}
unsafe {
set_handler(trampoline::<H>, user_data);
}
}
// DO: Use concrete types when possible
struct ConcreteHandler {
multiplier: i32,
}
impl Handler for ConcreteHandler {
fn handle(&self, data: i32) {
println!("{}", data * self.multiplier);
}
}
// DO: Create C-compatible vtable manually if needed
#[repr(C)]
struct HandlerVtable {
handle: extern "C" fn(this: *const c_void, data: c_int),
drop: extern "C" fn(this: *mut c_void),
}
#[repr(C)]
struct CCompatibleHandler {
data: *mut c_void,
vtable: *const HandlerVtable,
}
impl CCompatibleHandler {
fn new<H: Handler + 'static>(handler: H) -> Self {
extern "C" fn handle_impl<H: Handler>(this: *const c_void, data: c_int) {
let handler = unsafe { &*(this as *const H) };
handler.handle(data as i32);
}
extern "C" fn drop_impl<H: Handler>(this: *mut c_void) {
unsafe { drop(Box::from_raw(this as *mut H)); }
}
static VTABLE: HandlerVtable = HandlerVtable {
handle: handle_impl::<ConcreteHandler>, // Need concrete type
drop: drop_impl::<ConcreteHandler>,
};
Self {
data: Box::into_raw(Box::new(handler)) as *mut c_void,
vtable: &VTABLE,
}
}
fn handle(&self, data: i32) {
unsafe {
((*self.vtable).handle)(self.data, data as c_int);
}
}
}
impl Drop for CCompatibleHandler {
fn drop(&mut self) {
unsafe {
((*self.vtable).drop)(self.data);
}
}
}
```
## Why Trait Objects Don't Work
```
Rust trait object (*const dyn Handler):
[data pointer][vtable pointer] <- 16 bytes on 64-bit
C pointer (void*):
[pointer] <- 8 bytes on 64-bit
The sizes don't match!
```
## Alternatives to Trait Objects
| Instead of | Use |
|------------|-----|
| `dyn Trait` | Function pointer + user_data |
| `Box<dyn Trait>` | Boxed concrete type + trampoline |
| `&dyn Trait` | C-compatible vtable struct |
| `Arc<dyn Trait>` | Reference counting wrapper |
## Checklist
- [ ] Am I passing trait objects across FFI?
- [ ] Can I use concrete types instead?
- [ ] Have I used the trampoline pattern for callbacks?
- [ ] If vtable is needed, is it C-compatible?
## Related Rules
- `ffi-16`: Closure to C with trampoline pattern
- `ffi-14`: Types should have stable layout
@@ -1,71 +0,0 @@
---
id: general-01
original_id: P.UNS.01
level: P
impact: CRITICAL
---
# Do Not Abuse Unsafe to Escape Compiler Safety Checks
## Summary
Unsafe Rust should not be used as an escape hatch from the borrow checker or other compiler safety mechanisms.
## Rationale
The borrow checker exists to prevent memory safety bugs. Using `unsafe` to bypass it defeats Rust's safety guarantees and introduces potential undefined behavior.
## Bad Example
```rust
// DON'T: Using unsafe to bypass borrow checker
fn bad_alias() {
let mut data = vec![1, 2, 3];
let ptr = data.as_mut_ptr();
// Unsafe used to create aliasing mutable references
unsafe {
let ref1 = &mut *ptr;
let ref2 = &mut *ptr; // UB: Two mutable references!
*ref1 = 10;
*ref2 = 20;
}
}
```
## Good Example
```rust
// DO: Work with the borrow checker, not against it
fn good_sequential() {
let mut data = vec![1, 2, 3];
data[0] = 10;
data[0] = 20; // Sequential mutations are fine
}
// DO: Use interior mutability when needed
use std::cell::RefCell;
fn good_interior_mut() {
let data = RefCell::new(vec![1, 2, 3]);
data.borrow_mut()[0] = 10;
}
```
## Legitimate Uses of Unsafe
1. **FFI**: Calling C functions or implementing C-compatible interfaces
2. **Low-level abstractions**: Implementing collections, synchronization primitives
3. **Performance**: Only after profiling shows measurable improvement, and with careful safety analysis
## Checklist
- [ ] Have I tried all safe alternatives first?
- [ ] Is the borrow checker preventing a genuine design need?
- [ ] Can I restructure the code to satisfy the borrow checker?
- [ ] If unsafe is necessary, have I documented the safety invariants?
## Related Rules
- `general-02`: Don't blindly use unsafe for performance
- `safety-02`: Unsafe code authors must verify safety invariants
@@ -1,90 +0,0 @@
---
id: general-02
original_id: P.UNS.02
level: P
impact: CRITICAL
---
# Do Not Blindly Use Unsafe for Performance
## Summary
Do not assume that using `unsafe` will automatically improve performance. Always measure first and verify the safety invariants.
## Rationale
1. Modern Rust optimizers often eliminate bounds checks when they can prove safety
2. Unsafe code may prevent optimizations by breaking aliasing assumptions
3. Unmeasured "optimizations" often provide no real benefit while introducing risk
## Bad Example
```rust
// DON'T: Blind unsafe for "performance"
fn sum_bad(slice: &[i32]) -> i32 {
let mut sum = 0;
// Unnecessary unsafe - LLVM can optimize the safe version
for i in 0..slice.len() {
unsafe {
sum += *slice.get_unchecked(i);
}
}
sum
}
```
## Good Example
```rust
// DO: Use safe iteration - compiler optimizes bounds checks away
fn sum_good(slice: &[i32]) -> i32 {
slice.iter().sum()
}
// DO: If unsafe is justified, document why
fn sum_justified(slice: &[i32]) -> i32 {
let mut sum = 0;
// This is actually slower than iter().sum() in most cases
// Only use get_unchecked when:
// 1. Profiler shows bounds checks as bottleneck
// 2. Iterator patterns can't be used
// 3. Safety is proven by other means
for i in 0..slice.len() {
// SAFETY: i is always < slice.len() due to loop condition
unsafe {
sum += *slice.get_unchecked(i);
}
}
sum
}
```
## When Unsafe Might Be Justified for Performance
1. **Hot inner loops** where profiling shows bounds checks are a bottleneck
2. **SIMD operations** that require specific memory alignment
3. **Lock-free data structures** with carefully verified memory orderings
## Measurement Workflow
```bash
# 1. Benchmark the safe version first
cargo bench --bench my_bench
# 2. Profile to identify actual bottlenecks
cargo flamegraph --bench my_bench
# 3. Only then consider unsafe, with measurements
```
## Checklist
- [ ] Have I benchmarked the safe version?
- [ ] Does profiling show this specific code as a bottleneck?
- [ ] Have I measured the actual improvement from unsafe?
- [ ] Is the performance gain worth the safety risk?
## Related Rules
- `general-01`: Don't abuse unsafe to escape safety checks
- `safety-02`: Unsafe code authors must verify safety invariants
@@ -1,74 +0,0 @@
---
id: general-03
original_id: G.UNS.01
level: G
impact: MEDIUM
---
# Do Not Create Aliases for Types/Methods Named "Unsafe"
## Summary
Do not create type aliases, re-exports, or wrapper methods that hide the "unsafe" nature of operations.
## Rationale
The word "unsafe" in Rust is a signal to developers that extra scrutiny is required. Hiding this signal makes code review harder and can lead to accidental misuse.
## Bad Example
```rust
// DON'T: Hide unsafe behind an alias
type SafePointer = *mut u8; // Still unsafe to dereference!
// DON'T: Wrap unsafe in a "safe-looking" name
pub fn get_value(ptr: *const i32) -> i32 {
unsafe { *ptr } // Caller doesn't know this is unsafe!
}
// DON'T: Re-export unsafe functions with different names
pub use std::mem::transmute as convert;
```
## Good Example
```rust
// DO: Keep "unsafe" visible in the API
pub unsafe fn get_value_unchecked(ptr: *const i32) -> i32 {
*ptr
}
// DO: If providing a safe wrapper, make the safety contract clear
/// Returns the value at the pointer.
///
/// # Safety
/// This is safe because the pointer is validated internally.
pub fn get_value_checked(ptr: *const i32) -> Option<i32> {
if ptr.is_null() {
None
} else {
// SAFETY: We checked for null above
Some(unsafe { *ptr })
}
}
// DO: Use clear naming for raw pointer types
type RawHandle = *mut c_void; // "Raw" signals potential unsafety
```
## Common Violations
1. Creating type aliases that hide pointer types
2. Wrapping unsafe functions in safe-looking functions without proper safety analysis
3. Re-exporting unsafe functions with "friendlier" names
## Checklist
- [ ] Does my API preserve visibility of unsafe operations?
- [ ] If wrapping unsafe code in safe API, is the safety invariant enforced?
- [ ] Are type aliases clearly named to indicate their nature?
## Related Rules
- `safety-06`: Don't expose raw pointers in public APIs
- `safety-09`: Add SAFETY comment before any unsafe block
@@ -1,151 +0,0 @@
---
id: io-01
original_id: P.UNS.FIO.01
level: P
impact: HIGH
---
# Ensure I/O Safety When Using Raw Handles
## Summary
When working with raw file descriptors or handles, ensure they are valid for the duration of use and properly ownership-tracked.
## Rationale
- Raw handles can be closed by other code
- Using a closed handle is undefined behavior
- Handle reuse can cause data corruption
- Rust 1.63+ provides I/O safety traits
## Bad Example
```rust
#[cfg(unix)]
mod bad_example {
use std::os::unix::io::RawFd;
// DON'T: Accept raw handle without ownership
fn bad_read(fd: RawFd) -> std::io::Result<Vec<u8>> {
// What if fd was closed? What if it's reused?
let mut buf = vec![0u8; 1024];
let n = unsafe {
libc::read(fd, buf.as_mut_ptr() as *mut libc::c_void, buf.len())
};
if n < 0 {
Err(std::io::Error::last_os_error())
} else {
buf.truncate(n as usize);
Ok(buf)
}
}
// DON'T: Store raw handle without tracking ownership
struct BadFileRef {
fd: RawFd, // Who owns this? Who closes it?
}
}
```
## Good Example
```rust
#[cfg(unix)]
mod good_example {
use std::os::unix::io::{AsFd, BorrowedFd, OwnedFd, FromRawFd, AsRawFd};
use std::fs::File;
// DO: Use BorrowedFd for borrowed access (Rust 1.63+)
fn good_read(fd: BorrowedFd<'_>) -> std::io::Result<Vec<u8>> {
let mut buf = vec![0u8; 1024];
// BorrowedFd guarantees the fd is valid for this call
let n = unsafe {
libc::read(
fd.as_raw_fd(),
buf.as_mut_ptr() as *mut libc::c_void,
buf.len()
)
};
if n < 0 {
Err(std::io::Error::last_os_error())
} else {
buf.truncate(n as usize);
Ok(buf)
}
}
// DO: Use OwnedFd for owned handles
struct GoodFileOwner {
fd: OwnedFd, // Clearly owns the handle
}
impl Drop for GoodFileOwner {
fn drop(&mut self) {
// OwnedFd closes automatically
}
}
// DO: Use generic AsFd bound for flexibility
fn generic_read<F: AsFd>(f: &F) -> std::io::Result<Vec<u8>> {
good_read(f.as_fd())
}
// Usage
fn example() -> std::io::Result<()> {
let file = File::open("test.txt")?;
// Pass as BorrowedFd
let data = good_read(file.as_fd())?;
// Or use generic function
let data = generic_read(&file)?;
Ok(())
}
// DO: Take ownership from raw fd
fn from_raw(fd: i32) -> Option<GoodFileOwner> {
if fd < 0 {
return None;
}
// SAFETY: Caller guarantees fd is valid and ownership is transferred
let owned = unsafe { OwnedFd::from_raw_fd(fd) };
Some(GoodFileOwner { fd: owned })
}
}
```
## I/O Safety Types (Rust 1.63+)
| Type | Meaning |
|------|---------|
| `OwnedFd` | Owns a file descriptor, closes on drop |
| `BorrowedFd<'a>` | Borrows a fd for lifetime 'a |
| `RawFd` | Raw integer, no safety guarantees |
| `AsFd` | Trait for types that have a fd |
| `From<OwnedFd>` | Create from owned fd |
| `Into<OwnedFd>` | Convert to owned fd |
## Windows Equivalents
```rust
#[cfg(windows)]
use std::os::windows::io::{
OwnedHandle, BorrowedHandle, RawHandle,
AsHandle, FromRawHandle,
OwnedSocket, BorrowedSocket, RawSocket,
AsSocket, FromRawSocket,
};
```
## Checklist
- [ ] Am I using BorrowedFd/OwnedFd instead of RawFd?
- [ ] Is ownership of handles clear?
- [ ] Am I using the AsFd trait for generic code?
- [ ] Is the fd guaranteed valid for the duration of use?
## Related Rules
- `ffi-03`: Implement Drop for resource wrappers
- `safety-02`: Verify safety invariants
@@ -1,130 +0,0 @@
---
id: mem-01
original_id: P.UNS.MEM.01
level: P
impact: HIGH
---
# Choose Appropriate Data Layout for Struct/Tuple/Enum
## Summary
Use `#[repr(...)]` attributes to control data layout when interfacing with C, doing memory mapping, or needing specific guarantees.
## Rationale
Rust's default layout is unspecified and may change between compiler versions. For FFI, persistence, or low-level memory operations, you need predictable layout.
## Repr Attributes
| Attribute | Use Case |
|-----------|----------|
| `#[repr(C)]` | C-compatible layout, stable field order |
| `#[repr(transparent)]` | Single-field struct with same layout as field |
| `#[repr(packed)]` | No padding (alignment = 1), careful with references! |
| `#[repr(align(N))]` | Minimum alignment of N bytes |
| `#[repr(u8)]`, `#[repr(i32)]`, etc. | Enum discriminant type |
## Bad Example
```rust
// DON'T: Assume Rust struct layout matches C
struct BadFFI {
a: u8,
b: u32,
c: u8,
}
// Rust may reorder fields or add different padding than C
// DON'T: Use packed without understanding the risks
#[repr(packed)]
struct Dangerous {
a: u8,
b: u32,
}
fn bad_ref(d: &Dangerous) -> &u32 {
&d.b // UB: Creates unaligned reference!
}
```
## Good Example
```rust
// DO: Use repr(C) for FFI
#[repr(C)]
struct GoodFFI {
a: u8,
b: u32,
c: u8,
}
// Guaranteed: a at 0, padding 1-3, b at 4, c at 8, padding 9-11
// DO: Use repr(transparent) for newtypes
#[repr(transparent)]
struct Wrapper(u32);
// Guaranteed same layout as u32, can be transmuted
// DO: Use repr(packed) carefully, access via copy
#[repr(C, packed)]
struct PackedData {
header: u8,
value: u32,
}
impl PackedData {
fn value(&self) -> u32 {
// Copy out the value to avoid unaligned reference
let ptr = std::ptr::addr_of!(self.value);
// SAFETY: Reading unaligned is OK with read_unaligned
unsafe { ptr.read_unaligned() }
}
}
// DO: Use align for SIMD or cache line alignment
#[repr(C, align(64))]
struct CacheAligned {
data: [u8; 64],
}
// DO: Specify enum discriminant for FFI
#[repr(u8)]
enum Status {
Ok = 0,
Error = 1,
Unknown = 255,
}
```
## Layout Guarantees
```rust
use std::mem::{size_of, align_of};
#[repr(C)]
struct Example {
a: u8, // offset 0, size 1
// padding: 3 bytes
b: u32, // offset 4, size 4
c: u8, // offset 8, size 1
// padding: 3 bytes
}
assert_eq!(size_of::<Example>(), 12);
assert_eq!(align_of::<Example>(), 4);
// repr(Rust) might reorder to: b, a, c -> size 8
```
## Checklist
- [ ] Is this type used in FFI? → Use `#[repr(C)]`
- [ ] Is this a newtype wrapper? → Consider `#[repr(transparent)]`
- [ ] Do I need specific alignment? → Use `#[repr(align(N))]`
- [ ] Am I using packed? → Never create references to packed fields
## Related Rules
- `ffi-13`: Ensure consistent data layout for custom types
- `ffi-14`: Types in FFI should have stable layout
- `ptr-04`: Alignment considerations
@@ -1,113 +0,0 @@
---
id: mem-02
original_id: P.UNS.MEM.02
level: P
impact: CRITICAL
---
# Do Not Modify Memory Variables of Other Processes or Dynamic Libraries
## Summary
Do not directly manipulate memory belonging to other processes or dynamically loaded libraries. Use proper IPC or FFI mechanisms.
## Rationale
- Other processes have separate address spaces; direct access is impossible on modern OSes
- Shared memory requires explicit setup and synchronization
- Dynamic library memory has ownership rules that must be respected
- Violating these causes undefined behavior or security vulnerabilities
## Bad Example
```rust
// DON'T: Try to access another process's memory directly
fn bad_cross_process(ptr: *mut i32) {
// This pointer from another process is meaningless in our address space
unsafe { *ptr = 42; } // Undefined behavior or crash
}
// DON'T: Modify library internals
extern "C" {
static mut LIBRARY_INTERNAL: i32;
}
fn bad_library_access() {
// Modifying library internals breaks encapsulation
unsafe { LIBRARY_INTERNAL = 100; } // May corrupt library state
}
```
## Good Example
```rust
// DO: Use proper IPC for cross-process communication
use std::io::{Read, Write};
use std::os::unix::net::UnixStream;
fn ipc_communication() -> std::io::Result<()> {
let mut stream = UnixStream::connect("/tmp/socket")?;
stream.write_all(b"message")?;
Ok(())
}
// DO: Use shared memory with proper synchronization
#[cfg(unix)]
fn shared_memory_example() {
use std::sync::atomic::{AtomicI32, Ordering};
// Properly set up shared memory region
// let shm = mmap shared memory...
// Use atomic operations for synchronization
let shared: &AtomicI32 = /* ... */;
shared.store(42, Ordering::Release);
}
// DO: Use proper FFI for library interaction
mod ffi {
extern "C" {
pub fn library_set_value(value: i32);
pub fn library_get_value() -> i32;
}
}
fn proper_library_access() {
unsafe {
ffi::library_set_value(42);
let value = ffi::library_get_value();
}
}
// DO: Use Rust's libloading for dynamic libraries
fn dynamic_library() -> Result<(), Box<dyn std::error::Error>> {
let lib = unsafe { libloading::Library::new("mylib.so")? };
let func: libloading::Symbol<extern "C" fn(i32) -> i32> =
unsafe { lib.get(b"my_function")? };
let result = func(42);
Ok(())
}
```
## Memory Ownership Rules
| Memory Type | Owner | Safe Access |
|-------------|-------|-------------|
| Stack variables | Current function | Direct |
| Heap (Box, Vec) | Rust allocator | Through smart pointers |
| Static | Program | With proper synchronization |
| Shared memory | Multiple processes | Atomic ops, mutexes |
| Library memory | Library | Through library API |
| FFI-allocated | C allocator | Through C free functions |
## Checklist
- [ ] Who allocated this memory?
- [ ] Who is responsible for freeing it?
- [ ] Is proper synchronization in place for shared access?
- [ ] Am I using the correct API for cross-boundary access?
## Related Rules
- `mem-03`: Don't let String/Vec drop other process's memory
- `ffi-03`: Implement Drop for wrapped C pointers
@@ -1,127 +0,0 @@
---
id: mem-03
original_id: P.UNS.MEM.03
level: P
impact: CRITICAL
---
# Do Not Let String/Vec Auto-Drop Other Process's Memory
## Summary
Never create `String`, `Vec`, or `Box` from memory allocated outside Rust's allocator. They will try to free the memory with the wrong deallocator.
## Rationale
`String`, `Vec`, and `Box` assume memory was allocated by Rust's global allocator. When dropped, they call `dealloc`. If the memory came from C's `malloc`, a different allocator, or shared memory, this causes undefined behavior.
## Bad Example
```rust
// DON'T: Create String from C-allocated memory
extern "C" {
fn c_get_string() -> *mut std::os::raw::c_char;
}
fn bad_string() -> String {
unsafe {
let ptr = c_get_string();
// BAD: String will try to free with Rust allocator
String::from_raw_parts(ptr as *mut u8, len, cap)
}
}
// DON'T: Create Vec from foreign memory
fn bad_vec(ptr: *mut u8, len: usize) -> Vec<u8> {
// BAD: Vec will free this memory incorrectly
unsafe { Vec::from_raw_parts(ptr, len, len) }
}
// DON'T: Wrap shared memory in Box
fn bad_box(shared_ptr: *mut Data) -> Box<Data> {
// BAD: Box will try to deallocate shared memory!
unsafe { Box::from_raw(shared_ptr) }
}
```
## Good Example
```rust
use std::ffi::CStr;
extern "C" {
fn c_get_string() -> *mut std::os::raw::c_char;
fn c_free_string(s: *mut std::os::raw::c_char);
}
// DO: Copy data into Rust-owned allocation
fn good_string() -> String {
unsafe {
let ptr = c_get_string();
let cstr = CStr::from_ptr(ptr);
let result = cstr.to_string_lossy().into_owned();
c_free_string(ptr); // Free with correct deallocator
result
}
}
// DO: Use wrapper that calls correct deallocator
struct CString {
ptr: *mut std::os::raw::c_char,
}
impl Drop for CString {
fn drop(&mut self) {
unsafe { c_free_string(self.ptr); }
}
}
// DO: Use slice for borrowed view, don't take ownership
fn good_slice(ptr: *const u8, len: usize) -> &'static [u8] {
// Only borrow, don't own
unsafe { std::slice::from_raw_parts(ptr, len) }
}
// DO: For shared memory, use raw pointers or custom wrapper
struct SharedBuffer {
ptr: *mut u8,
len: usize,
}
impl SharedBuffer {
fn as_slice(&self) -> &[u8] {
unsafe { std::slice::from_raw_parts(self.ptr, self.len) }
}
}
impl Drop for SharedBuffer {
fn drop(&mut self) {
// Unmap shared memory, don't deallocate
// munmap(self.ptr, self.len);
}
}
```
## Memory Allocation Compatibility
| Allocator | Can use Rust Vec/String/Box? |
|-----------|------------------------------|
| Rust global allocator | Yes |
| C malloc | No - use wrapper with C free |
| C++ new | No - use wrapper with C++ delete |
| Custom allocator | No - use allocator_api |
| mmap/shared memory | No - use munmap |
| Stack/static | No - never "free" |
## Checklist
- [ ] Who allocated this memory?
- [ ] Is it from Rust's global allocator?
- [ ] If not, do I have a custom Drop that frees correctly?
- [ ] Am I copying data or taking ownership?
## Related Rules
- `mem-02`: Don't modify other process's memory
- `ffi-03`: Implement Drop for wrapped C pointers
- `ffi-07`: Don't implement Drop for types passed to external code
@@ -1,121 +0,0 @@
---
id: mem-04
original_id: P.UNS.MEM.04
level: P
impact: HIGH
---
# Prefer Reentrant Versions of C-API or Syscalls
## Summary
When calling C functions or system calls, use reentrant (`_r`) versions to avoid data races from global state.
## Rationale
Many C library functions use static buffers or global state, making them unsafe in multithreaded programs. Reentrant versions use caller-provided buffers instead.
## Bad Example
```rust
use std::ffi::CStr;
extern "C" {
fn strtok(s: *mut i8, delim: *const i8) -> *mut i8;
fn localtime(time: *const i64) -> *mut Tm;
fn rand() -> i32;
}
// DON'T: Use non-reentrant functions
fn bad_tokenize(s: &mut [i8]) {
unsafe {
let delim = b" \0".as_ptr() as *const i8;
// strtok uses static buffer - not thread-safe!
let token = strtok(s.as_mut_ptr(), delim);
}
}
fn bad_time() {
unsafe {
let now: i64 = 0;
// localtime returns pointer to static buffer
let tm = localtime(&now); // Data race if called from multiple threads!
}
}
fn bad_random() -> i32 {
// rand() uses global state - not thread-safe
unsafe { rand() }
}
```
## Good Example
```rust
extern "C" {
fn strtok_r(s: *mut i8, delim: *const i8, saveptr: *mut *mut i8) -> *mut i8;
fn localtime_r(time: *const i64, result: *mut Tm) -> *mut Tm;
fn rand_r(seed: *mut u32) -> i32;
}
// DO: Use reentrant versions
fn good_tokenize(s: &mut [i8]) {
unsafe {
let delim = b" \0".as_ptr() as *const i8;
let mut saveptr: *mut i8 = std::ptr::null_mut();
// strtok_r uses caller-provided saveptr
let token = strtok_r(s.as_mut_ptr(), delim, &mut saveptr);
}
}
fn good_time() {
unsafe {
let now: i64 = 0;
let mut result: Tm = std::mem::zeroed();
// localtime_r writes to caller-provided buffer
localtime_r(&now, &mut result);
}
}
fn good_random(seed: &mut u32) -> i32 {
// rand_r uses caller-provided seed
unsafe { rand_r(seed) }
}
// BETTER: Use Rust standard library
fn best_time() {
use std::time::SystemTime;
let now = SystemTime::now(); // Thread-safe!
}
fn best_random() -> u32 {
use rand::Rng;
rand::thread_rng().gen() // Thread-safe!
}
```
## Common Non-Reentrant Functions
| Non-Reentrant | Reentrant | Rust Alternative |
|---------------|-----------|------------------|
| `strtok` | `strtok_r` | `str::split` |
| `localtime` | `localtime_r` | `chrono` crate |
| `gmtime` | `gmtime_r` | `chrono` crate |
| `ctime` | `ctime_r` | `chrono` crate |
| `rand` | `rand_r` | `rand` crate |
| `strerror` | `strerror_r` | `std::io::Error` |
| `getenv` | None (inherent race) | `std::env::var` (not atomic) |
| `readdir` | `readdir_r` | `std::fs::read_dir` |
| `gethostbyname` | `getaddrinfo` | `std::net::ToSocketAddrs` |
## Checklist
- [ ] Am I calling a C function that might use global state?
- [ ] Is there a `_r` reentrant version available?
- [ ] Is there a Rust standard library alternative?
- [ ] If neither, do I need synchronization?
## Related Rules
- `ffi-10`: Exported functions must be thread-safe
- `ptr-01`: Don't share raw pointers across threads
@@ -1,147 +0,0 @@
---
id: mem-05
original_id: P.UNS.MEM.05
level: P
impact: MEDIUM
---
# Use Third-Party Crates for Bitfields
## Summary
Use crates like `bitflags`, `bitvec`, or `modular-bitfield` instead of manual bit manipulation for complex bitfield operations.
## Rationale
- Manual bit manipulation is error-prone
- Easy to get offsets, masks, or endianness wrong
- Crates provide type-safe, tested abstractions
- Proc-macro crates generate efficient code
## Bad Example
```rust
// DON'T: Manual bitfield manipulation
struct Flags(u32);
impl Flags {
const READ: u32 = 1 << 0;
const WRITE: u32 = 1 << 1;
const EXECUTE: u32 = 1 << 2;
fn has_read(&self) -> bool {
(self.0 & Self::READ) != 0
}
fn set_read(&mut self) {
self.0 |= Self::READ;
}
fn clear_read(&mut self) {
self.0 &= !Self::READ; // Easy to forget the !
}
}
// DON'T: Manual packed bitfields for FFI
#[repr(C)]
struct PackedHeader {
data: u32,
}
impl PackedHeader {
// Error-prone: wrong shift or mask values
fn version(&self) -> u8 {
((self.data >> 24) & 0xFF) as u8
}
fn flags(&self) -> u16 {
((self.data >> 8) & 0xFFFF) as u16
}
fn tag(&self) -> u8 {
(self.data & 0xFF) as u8
}
}
```
## Good Example
```rust
// DO: Use bitflags for flag sets
use bitflags::bitflags;
bitflags! {
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
struct Flags: u32 {
const READ = 1 << 0;
const WRITE = 1 << 1;
const EXECUTE = 1 << 2;
const RW = Self::READ.bits() | Self::WRITE.bits();
}
}
fn use_flags() {
let mut flags = Flags::READ | Flags::WRITE;
flags.insert(Flags::EXECUTE);
flags.remove(Flags::WRITE);
if flags.contains(Flags::READ) {
println!("Readable");
}
}
// DO: Use modular-bitfield for packed structures
use modular_bitfield::prelude::*;
#[bitfield]
#[repr(C)]
struct PackedHeader {
tag: B8, // 8 bits
flags: B16, // 16 bits
version: B8, // 8 bits
}
fn use_packed() {
let header = PackedHeader::new()
.with_version(1)
.with_flags(0x1234)
.with_tag(0xAB);
assert_eq!(header.version(), 1);
assert_eq!(header.flags(), 0x1234);
}
// DO: Use bitvec for arbitrary bit manipulation
use bitvec::prelude::*;
fn use_bitvec() {
let mut bits = bitvec![u8, Msb0; 0; 16];
bits.set(0, true);
bits.set(7, true);
let byte: u8 = bits[0..8].load_be();
assert_eq!(byte, 0b1000_0001);
}
```
## Recommended Crates
| Crate | Use Case | Features |
|-------|----------|----------|
| `bitflags` | Flag sets (like C enums) | Type-safe, const, derives |
| `modular-bitfield` | Packed struct fields | Proc macro, repr(C) |
| `bitvec` | Arbitrary bit arrays | Slicing, iteration |
| `packed_struct` | Binary protocol structs | Endianness, derive |
| `deku` | Binary parsing | Derive, read/write |
## Checklist
- [ ] Am I manipulating multiple bit flags? → Use `bitflags`
- [ ] Am I packing fields into bytes? → Use `modular-bitfield` or `packed_struct`
- [ ] Am I doing binary protocol work? → Consider `deku`
- [ ] Is the manual approach really simpler?
## Related Rules
- `mem-01`: Choose appropriate data layout
- `ffi-13`: Ensure consistent data layout
@@ -1,146 +0,0 @@
---
id: mem-06
original_id: G.UNS.MEM.01
level: G
impact: HIGH
clippy: uninit_assumed_init, uninit_vec
---
# Use MaybeUninit<T> for Uninitialized Memory
## Summary
Use `MaybeUninit<T>` instead of `mem::uninitialized()` or `mem::zeroed()` when working with uninitialized memory.
## Rationale
- `mem::uninitialized()` is deprecated and unsound
- `mem::zeroed()` is UB for types where zero is invalid (references, NonZero, bool)
- `MaybeUninit<T>` clearly marks memory as potentially uninitialized
- Compiler can optimize based on initialization state
## Bad Example
```rust
// DON'T: Use deprecated uninitialized
fn bad_uninit<T>() -> T {
unsafe { std::mem::uninitialized() } // Deprecated, UB
}
// DON'T: Use zeroed for types where zero is invalid
fn bad_zeroed() -> &'static str {
unsafe { std::mem::zeroed() } // UB: null reference
}
fn bad_zeroed_bool() -> bool {
unsafe { std::mem::zeroed() } // UB: 0 might not be valid bool
}
// DON'T: Transmute to "initialize"
fn bad_transmute() -> [String; 10] {
unsafe { std::mem::transmute([0u8; std::mem::size_of::<[String; 10]>()]) }
}
// DON'T: Set Vec length without initializing
fn bad_vec() -> Vec<String> {
let mut v = Vec::with_capacity(10);
unsafe { v.set_len(10); } // Elements are uninitialized!
v
}
```
## Good Example
```rust
use std::mem::MaybeUninit;
// DO: Use MaybeUninit for delayed initialization
fn good_array() -> [String; 10] {
let mut arr: [MaybeUninit<String>; 10] =
unsafe { MaybeUninit::uninit().assume_init() };
for (i, elem) in arr.iter_mut().enumerate() {
elem.write(format!("item {}", i));
}
// SAFETY: All elements initialized above
unsafe { std::mem::transmute::<_, [String; 10]>(arr) }
}
// DO: Use MaybeUninit with arrays (cleaner with array_assume_init)
fn good_array_nightly() -> [String; 10] {
let mut arr: [MaybeUninit<String>; 10] =
[const { MaybeUninit::uninit() }; 10];
for (i, elem) in arr.iter_mut().enumerate() {
elem.write(format!("item {}", i));
}
// On nightly: arr.map(|e| unsafe { e.assume_init() })
unsafe { MaybeUninit::array_assume_init(arr) }
}
// DO: Use zeroed only for types where it's valid
fn good_zeroed() -> [u8; 1024] {
// SAFETY: All-zero bytes is valid for u8
unsafe { std::mem::zeroed() }
}
// DO: Initialize buffer properly
fn good_vec() -> Vec<u8> {
let mut v = Vec::with_capacity(1024);
// Option 1: Resize with default value
v.resize(1024, 0);
// Option 2: Use spare_capacity_mut
let spare = v.spare_capacity_mut();
for elem in spare.iter_mut().take(1024) {
elem.write(0);
}
unsafe { v.set_len(1024); }
v
}
// DO: Use MaybeUninit::uninit_array (nightly) or const array
fn good_uninit_array<const N: usize>() -> [MaybeUninit<u8>; N] {
// Stable: create array of uninit
[const { MaybeUninit::uninit() }; N]
}
```
## MaybeUninit API
```rust
use std::mem::MaybeUninit;
// Creation
let uninit: MaybeUninit<T> = MaybeUninit::uninit();
let zeroed: MaybeUninit<T> = MaybeUninit::zeroed();
let init: MaybeUninit<T> = MaybeUninit::new(value);
// Writing
uninit.write(value); // Returns &mut T
// Reading (unsafe)
let value: T = unsafe { uninit.assume_init() };
let ref_: &T = unsafe { uninit.assume_init_ref() };
let mut_: &mut T = unsafe { uninit.assume_init_mut() };
// Pointer access
let ptr: *const T = uninit.as_ptr();
let mut_ptr: *mut T = uninit.as_mut_ptr();
```
## Checklist
- [ ] Am I using `mem::uninitialized()`? → Replace with `MaybeUninit`
- [ ] Am I using `mem::zeroed()` for non-POD types? → Use `MaybeUninit`
- [ ] Am I setting Vec length without initialization? → Use proper initialization
- [ ] Have I initialized all MaybeUninit before assume_init?
## Related Rules
- `safety-03`: Don't expose uninitialized memory in APIs
- `safety-01`: Panic safety with partial initialization
@@ -1,113 +0,0 @@
---
id: ptr-01
original_id: P.UNS.PTR.01
level: P
impact: CRITICAL
---
# Do Not Share Raw Pointers Across Threads
## Summary
Raw pointers (`*const T`, `*mut T`) are not `Send` or `Sync` by default. Do not share them across threads without ensuring proper synchronization.
## Rationale
Raw pointers have no synchronization guarantees. Sharing them across threads can lead to data races, which are undefined behavior.
## Bad Example
```rust
use std::thread;
// DON'T: Share raw pointers across threads
fn bad_sharing() {
let mut data = 42i32;
let ptr = &mut data as *mut i32;
let handle = thread::spawn(move || {
// This is undefined behavior!
unsafe { *ptr = 100; }
});
// Main thread also accesses - data race!
unsafe { *ptr = 200; }
handle.join().unwrap();
}
// DON'T: Wrap in struct and impl Send unsafely
struct UnsafePtr(*mut i32);
unsafe impl Send for UnsafePtr {} // Unsound without synchronization!
```
## Good Example
```rust
use std::sync::{Arc, Mutex, atomic::{AtomicPtr, Ordering}};
use std::thread;
// DO: Use Arc<Mutex<T>> for shared mutable access
fn good_mutex() {
let data = Arc::new(Mutex::new(42i32));
let data_clone = Arc::clone(&data);
let handle = thread::spawn(move || {
*data_clone.lock().unwrap() = 100;
});
*data.lock().unwrap() = 200;
handle.join().unwrap();
}
// DO: Use AtomicPtr for lock-free pointer sharing
fn good_atomic() {
let data = Box::into_raw(Box::new(42i32));
let atomic_ptr = Arc::new(AtomicPtr::new(data));
let atomic_clone = Arc::clone(&atomic_ptr);
let handle = thread::spawn(move || {
let ptr = atomic_clone.load(Ordering::Acquire);
// SAFETY: We have exclusive access through atomic operations
unsafe { println!("Value: {}", *ptr); }
});
handle.join().unwrap();
// SAFETY: All threads done, we own the memory
unsafe { drop(Box::from_raw(atomic_ptr.load(Ordering::Relaxed))); }
}
// DO: If you must use raw pointers, ensure exclusive access
fn good_exclusive() {
let mut data = vec![1, 2, 3];
// Send data ownership to thread, not pointer
let handle = thread::spawn(move || {
data.push(4);
data
});
let data = handle.join().unwrap();
println!("{:?}", data);
}
```
## When Raw Pointers Across Threads Are Valid
Only with proper synchronization:
- Through `AtomicPtr` with appropriate memory orderings
- Protected by a `Mutex` (don't share the pointer, share the Mutex)
- Using lock-free algorithms with careful memory ordering
## Checklist
- [ ] Does my pointer cross thread boundaries?
- [ ] Is there synchronization preventing concurrent access?
- [ ] Can I use a higher-level abstraction (Arc, Mutex)?
- [ ] If implementing Send/Sync, is thread safety proven?
## Related Rules
- `safety-05`: Consider safety when implementing Send/Sync
- `safety-02`: Verify safety invariants
@@ -1,114 +0,0 @@
---
id: ptr-02
original_id: P.UNS.PTR.02
level: P
impact: MEDIUM
---
# Prefer NonNull<T> Over *mut T
## Summary
Use `NonNull<T>` instead of `*mut T` when the pointer should never be null. This enables null pointer optimization and makes the intent clear.
## Rationale
- `NonNull<T>` guarantees non-null at the type level
- Enables niche optimization: `Option<NonNull<T>>` is the same size as `*mut T`
- Makes invariants explicit in the type system
- Covariant over `T` (like `&T`), which is usually what you want
## Bad Example
```rust
// DON'T: Use *mut when pointer is always non-null
struct MyBox<T> {
ptr: *mut T, // Invariant: never null, but not enforced
}
impl<T> MyBox<T> {
pub fn new(value: T) -> Self {
let ptr = Box::into_raw(Box::new(value));
// ptr is guaranteed non-null, but type doesn't show it
Self { ptr }
}
pub fn get(&self) -> &T {
// Must add null check or document the invariant
unsafe { &*self.ptr }
}
}
```
## Good Example
```rust
use std::ptr::NonNull;
// DO: Use NonNull when pointer is never null
struct MyBox<T> {
ptr: NonNull<T>, // Type guarantees non-null
}
impl<T> MyBox<T> {
pub fn new(value: T) -> Self {
let ptr = Box::into_raw(Box::new(value));
// SAFETY: Box::into_raw never returns null
let ptr = unsafe { NonNull::new_unchecked(ptr) };
Self { ptr }
}
pub fn get(&self) -> &T {
// SAFETY: NonNull guarantees ptr is valid
unsafe { self.ptr.as_ref() }
}
}
impl<T> Drop for MyBox<T> {
fn drop(&mut self) {
// SAFETY: ptr was created from Box::into_raw
unsafe { drop(Box::from_raw(self.ptr.as_ptr())); }
}
}
// DO: Niche optimization with Option
struct OptionalBox<T> {
ptr: Option<NonNull<T>>, // Same size as *mut T!
}
```
## NonNull API
```rust
use std::ptr::NonNull;
// Creating NonNull
let ptr: NonNull<i32> = NonNull::new(raw_ptr).expect("null pointer");
let ptr: NonNull<i32> = unsafe { NonNull::new_unchecked(raw_ptr) };
let ptr: NonNull<i32> = NonNull::dangling(); // For ZSTs or uninitialized
// Using NonNull
let raw: *mut i32 = ptr.as_ptr();
let reference: &i32 = unsafe { ptr.as_ref() };
let mut_ref: &mut i32 = unsafe { ptr.as_mut() };
// Casting
let ptr: NonNull<u8> = ptr.cast::<u8>();
```
## When to Use *mut T Instead
- When null is a valid/expected value
- FFI with C code that may return null
- When variance matters (NonNull is covariant, sometimes you need invariance)
## Checklist
- [ ] Is my pointer ever null? If no, use NonNull
- [ ] Do I need null pointer optimization?
- [ ] Is the variance correct for my use case?
## Related Rules
- `ptr-03`: Use PhantomData for variance and ownership
- `safety-06`: Don't expose raw pointers in public APIs
@@ -1,125 +0,0 @@
---
id: ptr-03
original_id: P.UNS.PTR.03
level: P
impact: HIGH
---
# Use PhantomData<T> for Variance and Ownership with Pointer Generics
## Summary
When a struct contains raw pointers but logically owns or borrows the pointed-to data, use `PhantomData<T>` to tell the compiler about the relationship.
## Rationale
Raw pointers don't carry ownership or lifetime information. `PhantomData` lets you:
- Indicate ownership (for `Drop` check)
- Control variance (covariant, contravariant, invariant)
- Participate in lifetime elision
## Bad Example
```rust
// DON'T: Raw pointer without PhantomData
struct MyVec<T> {
ptr: *mut T,
len: usize,
cap: usize,
}
// Problems:
// 1. Compiler doesn't know we "own" the T values
// 2. T might be incorrectly determined as unused
// 3. Drop check may allow dangling references
```
## Good Example
```rust
use std::marker::PhantomData;
use std::ptr::NonNull;
// DO: Use PhantomData to express ownership
struct MyVec<T> {
ptr: NonNull<T>,
len: usize,
cap: usize,
_marker: PhantomData<T>, // We own T values
}
// For owned data: PhantomData<T>
// For borrowed data: PhantomData<&'a T>
// For mutably borrowed: PhantomData<&'a mut T>
// For function pointers: PhantomData<fn(T)> (contravariant)
// DO: Express lifetime relationships
struct Iter<'a, T> {
ptr: *const T,
end: *const T,
_marker: PhantomData<&'a T>, // Borrows T for 'a
}
impl<'a, T> Iterator for Iter<'a, T> {
type Item = &'a T;
fn next(&mut self) -> Option<Self::Item> {
if self.ptr == self.end {
None
} else {
// SAFETY: ptr < end, so ptr is valid
// Lifetime is tied to 'a through PhantomData
let current = unsafe { &*self.ptr };
self.ptr = unsafe { self.ptr.add(1) };
Some(current)
}
}
}
```
## PhantomData Patterns
| Phantom Type | Meaning | Variance |
|--------------|---------|----------|
| `PhantomData<T>` | Owns T | Covariant |
| `PhantomData<&'a T>` | Borrows T for 'a | Covariant in T, covariant in 'a |
| `PhantomData<&'a mut T>` | Mutably borrows T | Invariant in T, covariant in 'a |
| `PhantomData<*const T>` | Just has pointer | Covariant |
| `PhantomData<*mut T>` | Just has pointer | Invariant |
| `PhantomData<fn(T)>` | Consumes T | Contravariant |
| `PhantomData<fn() -> T>` | Produces T | Covariant |
## Drop Check
```rust
use std::marker::PhantomData;
// This tells the compiler that dropping MyVec may drop T values
struct MyVec<T> {
ptr: NonNull<T>,
_marker: PhantomData<T>,
}
impl<T> Drop for MyVec<T> {
fn drop(&mut self) {
// Drop all T values...
}
}
// Without PhantomData<T>, this might compile incorrectly:
// let x = MyVec::new(&local);
// drop(local); // Would be UB if allowed
// drop(x); // Tries to access dropped local
```
## Checklist
- [ ] Does my pointer type logically own the pointed-to data?
- [ ] Do I need to express a lifetime relationship?
- [ ] What variance do I need for my generic parameter?
- [ ] Will the type be dropped, and does it need drop check?
## Related Rules
- `ptr-02`: Prefer NonNull over *mut T
- `safety-05`: Send/Sync implementation safety

Some files were not shown because too many files have changed in this diff Show More