Commit Graph

252 Commits

Author SHA1 Message Date
Arthur Carcano 5a4da29f92 Migrate BlockRc and BlockResyncManager to TypedTree 2026-06-30 21:04:06 +00:00
Wesley Hershberger 76aae3cb96 Drop parse_duration for fundu-systemd (#1246) (#1468)
`parse_duration` is no longer maintained upstream; `fundu-system` seems like the best option to provide most of the functionality provided by parse_duration and minimize deps.

Fixes #1246

Dep diffstat: +3 -6

## Caveats
- I've done basically no testing of this PR beyond `cargo test`
- See my comment in #1246 for regression risk; this is a breaking change. Is there a document I should update to make note of this? Should I rebase this for `next-v3` instead?
- In theory CVE-2021-29932 is fixed by this PR as `fundu-systemd` doesn't support exponents, but I've done no verification that this is really the case beyond the info in #1246

Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1468
Reviewed-by: Alex <lx@deuxfleurs.fr>
2026-06-17 13:57:21 +00:00
jo 1d1456f1d6 Add details to read file errors (#1475)
If the files configured in `api_token_file` or `rpc_secret_file` are not found, garage exits with no details (at least not enough) of what went wrong:

    2026-06-14T23:40:34.517429Z  INFO garage::server: Loading configuration from tmp/config1.toml...
    Error: IO error: No such file or directory (os error 2)

This add the kind of file and the file path being read, to the returned error message:

    2026-06-14T23:41:41.136213Z  INFO garage::server: Loading configuration from tmp/config1.toml...
    Error: Failed to read secret file /run/secrets/rpc_secret: No such file or directory (os error 2)

Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1475
Reviewed-by: Alex <lx@deuxfleurs.fr>
2026-06-17 13:53:43 +00:00
Alex Auvolat bf0a24ea69 replace Option CRDT by explicit CancelingOption and MergingOption types 2026-05-13 11:20:06 +02:00
Arthur Carcano 0da317e3d5 Fuzz Bucket CRDT (#1442)
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1442
Reviewed-by: Alex <lx@deuxfleurs.fr>
2026-05-06 18:55:47 +00:00
Arthur Carcano 322da7242b Post review fixes 2026-05-01 21:36:17 +00:00
Arthur Carcano ade4d07bb5 Set up fuzz infrastructure 2026-05-01 21:36:17 +00:00
Alex Auvolat 7b119c0b4f bump version number to v2.3.0 2026-04-16 18:34:27 +02:00
Gauthier Zirnhelt 2798667345 Fix the LifecycleWorker being uncooperative (#1396)
## Summary

This PR ensures that the `LifecycleWorker` yields at least once to the Tokio scheduler in between each batch of 100 objects.

## Problem being solved

I'm administrating a Garage cluster which has been experiencing timeouts on all endpoints while the lifecycle worker is running at midnight UTC : `Ping timeout` error messages and even requests eventually failing due to `Could not reach quorum ...`.

I have found that this happens while the lifecycle worker is working on a big bucket (containing millions of objects) with a lifecycle rule that applies to very few objects.
The `process_object()` function does not hit any `await`:
- `last_bucket` is always the same, so the `bucket_table` is not read asynchronously
- no transaction is made on the `object_table` because my lifecycle rule (almost) never applies to any object

The first commit in this PR adds an executable which reproduces the problem that I've been experiencing in a self-contained way : the lifecycle worker starves the Tokio scheduler so much that no other task is able to run (or very rarely).
To run it : `cargo run -p garage_model --bin lifecycle-starvation-test`.
This commit can be dropped post-review, as it's only useful to demonstrate the starvation.

The error messages completely stopped after adding the extra yield to the nodes of my cluster.
The duration of the lifecycle worker task does not appear to have changed at all from what I can see (looking at the timestamps produced either by the self-contained binary or by each of my nodes with the `Lifecycle worker finished` message).

## Note

An other potential fix would have been to force the `WorkerProcessor` to yield before re-enqueuing a busy task, but this would have affected all Garage workers even though it's only the `LifecycleWorker` being uncooperative.

Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1396
Reviewed-by: Alex <lx@deuxfleurs.fr>
Co-authored-by: Gauthier Zirnhelt <gauthier.zirnhelt@insimo.fr>
Co-committed-by: Gauthier Zirnhelt <gauthier.zirnhelt@insimo.fr>
2026-04-15 12:13:18 +02:00
Gwen Lg 83f8bdbacd refactor: remove unnecessarily wrap value into a Result
this simplify code and remove some unwrap.
warning: this function's return value is unnecessarily wrapped by `Result`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.93.0/index.html#unnecessary_wraps
2026-02-17 18:38:44 +00:00
Gwen Lg 08fd6e659f docs: add missing backticks in documentation
this improve readability of documentation.
enable associated clippy lint `doc_markdown`
2026-02-17 18:38:44 +00:00
Gwen Lg 6cde00073f chore: enable workspace configuration of lints in Cargo.toml
and use workspace configuration in each package.
This allow to customize clippy and rust lint configuration for project.
No particular configuration in this commit.
2026-02-17 18:38:44 +00:00
Gwen Lg 934c4c31f1 chore: update dependency rand to 0.9 2026-02-10 19:16:49 +00:00
Gwen Lg 4a0be692b3 docs: various fixes in Rust code documentation
- add backticks on struct doc comments
lint message: unclosed HTML tag `M`
- add a blanck line for proper doc formating
lint message: doc list item without indentation
help: if this is supposed to be its own paragraph, add a blank line
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#doc_lazy_continuation
- fix hyperlink in doc + one url invalid
lint message: this URL is not a hyperlink
note: bare URLs are not automatically turned into clickable links
- ajust space in doc
lint message: doc list item overindented
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#doc_overindented_list_items
2026-01-29 20:49:28 +01:00
Gwen Lg 21db7a4d4b chore: remove various useless code
- call expect directly on Result
lint message: called `ok().expect()` on a `Result` value
help: you can call `expect()` directly on the `Result`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#ok_expect
- use assign operation instead of manual implementation
lint message: manual implementation of an assign operation
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#assign_op_pattern
- remove useless call to format
lint message: useless use of `format!`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#useless_format
- remove useless `?Sized`
lint message: `?Sized` bound is ignored because of a `Sized` requirement
note: ...because `Deserialize` has the bound `Sized`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#needless_maybe_sized
- use is_some instead of pattern maching with Some(_)
lint message: redundant pattern matching, consider using `is_some()`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#redundant_pattern_matching
- remove unneeded unit return type
lint message: unneeded unit return type
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#unused_unit
- remove redundant closure
lint message: redundant closure
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#redundant_closure
- use derive Default instead of manual implementation
lint message: this `impl` can be derived
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#derivable_impls
- remove unneeded `return` statement
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#needless_return
- remove empty string from println call
lint message: empty string literal in `println!`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#println_empty_string
- remove clone() on type than implement Copy
lint message: using `clone` on type `Option<ChecksumValue>` which implements the `Copy` trait
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#clone_on_copy
- remove useless let binding
lint message: this let-binding has unit value
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#let_unit_value
- remove useless len comparison to zero, already test of empty
lint message: length comparison to zero
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#len_zero
- remove useless `as_deref` call
lint message: derefed type is same as origin
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#needless_option_as_deref
- remove useless conversion of the same type
lint message: useless conversion to the same type: `replication_mode::ReplicationFactor`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#useless_conversion
- remove useless bool_comparison
lint message: equality checks against false can be replaced by a negation
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#bool_comparison
- remove useless to_string
lint message: unnecessary use of `to_string`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#unnecessary_to_owned

Signed-off-by: Gwen Lg <me@gwenlg.fr>
2026-01-29 20:49:28 +01:00
Alex Auvolat 582b168b6a bump version to v2.2.0 2026-01-24 12:32:22 +01:00
Alex Auvolat cd641a9ed2 Merge branch 'main-v1' into merge-v1 2026-01-24 09:56:02 +01:00
Alex Auvolat dc8d93698b small documentation fixes and simplify config struct 2026-01-06 14:33:20 +01:00
JaminMartin c4836916b0 feat: testing wan consul discovery 2025-12-19 10:24:11 +13:00
perrynzhou@gmail.com e3a5ec6ef6 rename put_blocks_max_parallel to block_max_concurrent_writes_per_request and update configuration.md 2025-12-12 07:09:38 +08:00
perrynzhou@gmail.com 4d124e1c76 Add the parameter, which replaces . This is to accommodate different storage media such as HDD and NVMe. 2025-12-10 06:43:51 +08:00
trinity-1686a 1fe932d07f migrate to this error
it doesn't generate a bazillion warning at compile time
2025-11-01 17:25:23 +01:00
trinity-1686a 82297371bf migrate to this error
it doesn't generate a bazillion warning at compile time
2025-11-01 17:20:39 +01:00
Alex Auvolat b43f309ec7 bump version to v2.1.0 2025-09-15 15:50:02 +02:00
Alex Auvolat 4b1fdbef55 bump version to v1.3.0 2025-09-14 21:36:33 +02:00
Alex Auvolat 909359ca4c Merge branch 'main-v1' into sync-v2-to-v1 2025-09-14 17:02:34 +02:00
Alex Auvolat 4c139bcbca add error case for layout not ready, and fail earlier in many places 2025-09-13 20:52:51 +02:00
Alex Auvolat 5cf354acb4 block: maximum number of simultaneous reads 2025-09-13 17:38:06 +02:00
Julien Kritter 96d7713915 Add support for an LSM-tree-based backend with Fjall 2025-08-27 18:54:34 +02:00
Alex Auvolat fbf03e9378 bump version to v1.2.0 2025-06-13 14:21:28 +02:00
Alex Auvolat 7ab1d176d4 Merge branch 'main' into next-v2 2025-05-23 16:33:07 +02:00
trinity-1686a bba9202f31 add test for punycode 2025-05-19 20:36:03 +02:00
trinity-1686a 539af12d21 allow punnycode in bucket name 2025-05-19 18:07:04 +02:00
Alex Auvolat d38d62f4d7 bump version to v2.0.0 2025-04-17 12:36:41 +02:00
Alex Auvolat ff6ec62d54 admin api: add metrics_require_token config option and update doc 2025-03-11 14:17:22 +01:00
Alex Auvolat 29ce490dd6 Merge branch 'main' into next-v2 2025-03-05 14:50:22 +01:00
Alex Auvolat 4689b10448 bump version to v1.1.0 2025-03-05 10:19:51 +01:00
Alex Auvolat 2f55889835 add configuration option to enable/disable monitoring bucket in web metrics 2025-02-14 14:59:00 +01:00
Alex Auvolat 3151695011 Merge branch 'next-v2' into 1686a/s3-redirects 2025-02-14 13:11:33 +01:00
trinity-1686a f034e834fa Merge branch 'main' into 1686a/s3-redirects 2025-02-08 16:01:11 +01:00
Alex Auvolat 10bbb26b30 cli_v2: implement ListWorkers and GetWorkerInfo 2025-02-03 18:54:51 +01:00
Alex Auvolat 4563313f87 use cargo-shear to remove many unused dependencies between crates 2025-01-31 18:47:30 +01:00
Baptiste Jonglez 59c153d280 db-snapshot: allow to set directory where snapshots are stored
Fix #926
2025-01-27 18:33:55 +01:00
Stefan Majer 2eb9fcae20 Fix all typos 2025-01-16 13:22:00 +01:00
Alex Auvolat 44ce6ae5b4 properly implement new bucket model using a migration 2025-01-04 18:50:49 +01:00
Renjaya Raga Zenta 4c1bf42192 feat: add use_local_tz configuration
Used in lifecycle_worker to determine midnight time
2024-11-23 05:51:12 +07:00
Alex Auvolat 7a143f46fc Bump to version 1.0.1 2024-09-22 14:25:32 +02:00
Florian Klink a0f6bc5b7f add rpc_public_addr_subnet config option
In case `rpc_public_addr` is not set, but autodiscovery is used, this
allows filtering the list of automatically discovered IPs to a specific
subnet.

For example, if nodes should pick *their* IP inside a specific subnet,
but you don't want to explicitly write the IP down (as it's dynamic, or
you want to share configs across nodes), you can use this option.
2024-06-05 08:41:36 +02:00
Alex Auvolat afad62939e [next-0.10] bump version number to 1.0 2024-03-28 15:19:44 +01:00
Alex Auvolat 8bfc16ba7d Merge branch 'main' into next-0.10 2024-03-28 15:01:05 +01:00