Commit Graph

1631 Commits

Author SHA1 Message Date
i5-650 c85597fd18 Better handling of wildcards in CORS rules (fix #1105) (#1516)
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1516
2026-09-03 07:50:59 +00:00
i5-650 d0176f8e30 deregister from consul on shutdown (fix #1062) (#1507)
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1507
2026-08-22 17:19:51 +00:00
i5-650 d9b1dba137 1250 - add NO_COLOR env var to disable logs colors (#1503)
Fix #1250

Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1503
Reviewed-by: Alex <lx@deuxfleurs.fr>
2026-08-19 08:03:44 +00:00
Alex Auvolat aa4e4656b1 GetKeyInfo: properly list locally-aliased buckets even if they have no permissions 2026-07-25 14:24:58 +02:00
Alex Auvolat 5549077d6f GetBucketInfo: properly return keys that have local aliases even if they have no permissions 2026-07-25 14:24:49 +02:00
smattymatty 663fc5ae48 fix(s3): allow UTF-8 in PostObject form field values (fix #1489) (#1492)
PostObject (presigned POST) returns `400 InvalidHeaderValue` when the upload involves a non-ASCII filename. The same key uploads fine vie PUT, as the issue #1489 noted.

The problem was that `handle_post_object` stores the form field values in an `http::HeaderMap`. values go in through `HeaderValue::from_str` but are read back with the strict `HeaderValue::to_str()` (ASCII-only) which fails on any UTF-8 value.

This is the same problem fixed in eab2b81b for `x-amz-meta-*` headers, and the fix is the same:
`std::str::from_utf8(value.as_bytes())` instead of `to_str()`. The `key` field in `post_object.rs` and the standard headers (`content-disposition` etc.) in `extract_metadata_headers`.

Added integration tests for PostObject (there were none): UTF-8 key, `${filename}` substitution, and UTF-8 `Content-Disposition` metadata. The substitution test passes even without the fix, proving that the filename path was never broken, only the form-params round-trip.

Co-authored-by: Mathew Storm <mathew@stormdevelopments.ca>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1492
Reviewed-by: trinity-1686a <trinity-1686a@noreply.localhost>
2026-07-20 18:12:39 +00:00
Andreas Schneider 7e246b20e9 fix: avoid rewriting peer list file when unchanged (#1488)
save_peer_list() ran unconditionally every 60s from the discovery
loop, writing the same data to disk even when the peer set hadn't
changed. Compare the newly encoded peer list against the bytes
already on disk and skip the write when they match.

Fixes #1457

Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1488
Reviewed-by: Alex <lx@deuxfleurs.fr>
2026-07-14 10:54:33 +00:00
deanqx 936e53fe2f docs: update old "key new" to new "key create" command (#1487)
Fix invalid references to the `garage key new` commands. Mainly on documentation side except one script: `script/dev-bucket.sh`.

This is my first pull request ever. I'm open for feedback, let me know if I should change anything about this PR.

Co-authored-by: deanqx <110404616+deanqx@users.noreply.github.com>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1487
2026-07-07 10:34:42 +00:00
Arthur Carcano 7cdcaa8ebe Panic on (reoccuring) decode error 2026-06-30 21:04:06 +00:00
Arthur Carcano 4133476ec9 Add TODO comment for migration to typedtree 2026-06-30 21:04:06 +00:00
Arthur Carcano b277d49ad6 Add more type invariants to RcEntry and new RcState 2026-06-30 21:04:06 +00:00
Arthur Carcano 5a4da29f92 Migrate BlockRc and BlockResyncManager to TypedTree 2026-06-30 21:04:06 +00:00
Arthur Carcano eb91f463f5 Add TypedTree 2026-06-30 21:04:06 +00:00
Armaël Guéneau 0ebf60f709 k2v-client: re-export crates hyper_rustls and hyper_util 2026-06-30 21:02:44 +00:00
Armaël Guéneau d588a96125 k2v-client: implement Debug for K2vClient 2026-06-30 21:02:44 +00:00
Armaël Guéneau e665a3432c k2v-client: derive Clone for the K2vClient type 2026-06-30 21:02:44 +00:00
Armaël Guéneau 5884b4e9c7 k2v-client: repair build; serializers depend on the serde "std" features 2026-06-30 21:02:44 +00:00
Mathew Storm 8eb7628bf4 test(s3): cover bulk DeleteObjects of a non-existent key 2026-06-17 14:05:14 +00:00
Mathew Storm cd26594673 fix(s3): treat NoSuchKey as success in bulk DeleteObjects 2026-06-17 14:05:14 +00:00
Armaël Guéneau 4f9faeb282 minor fixes 2026-06-17 14:00:13 +00:00
Armaël Guéneau a159c1c483 add missing repair-on-read for k2v range reads 2026-06-17 14:00:13 +00:00
Armaël Guéneau 555e0826a2 define a separate enum for the monotonic/non-monotonic read flag 2026-06-17 14:00:13 +00:00
Armaël Guéneau d0f89068c6 repair_on_read: send multiple items to update in a single RPC 2026-06-17 14:00:13 +00:00
Armaël Guéneau 3e25914210 K2V: provide monotonic reads by default, with a flag to opt-out
This performs synchronous repair-on-read for K2V reads:
- uses the new get_*_monotonic operations in the table module
- implements repair-on-read for the K2V-specific poll operations
2026-06-17 14:00:13 +00:00
Armaël Guéneau 5500f1c412 table: disable asynchronous repair-on-read; add get_*_monotonic functions for synchronous repair-on-read 2026-06-17 14:00:13 +00:00
Armaël Guéneau 2d7c0a6087 table: also apply repair-on-read the first time a value is set 2026-06-17 14:00:13 +00:00
Wesley Hershberger 76aae3cb96 Drop parse_duration for fundu-systemd (#1246) (#1468)
`parse_duration` is no longer maintained upstream; `fundu-system` seems like the best option to provide most of the functionality provided by parse_duration and minimize deps.

Fixes #1246

Dep diffstat: +3 -6

## Caveats
- I've done basically no testing of this PR beyond `cargo test`
- See my comment in #1246 for regression risk; this is a breaking change. Is there a document I should update to make note of this? Should I rebase this for `next-v3` instead?
- In theory CVE-2021-29932 is fixed by this PR as `fundu-systemd` doesn't support exponents, but I've done no verification that this is really the case beyond the info in #1246

Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1468
Reviewed-by: Alex <lx@deuxfleurs.fr>
2026-06-17 13:57:21 +00:00
jo 1d1456f1d6 Add details to read file errors (#1475)
If the files configured in `api_token_file` or `rpc_secret_file` are not found, garage exits with no details (at least not enough) of what went wrong:

    2026-06-14T23:40:34.517429Z  INFO garage::server: Loading configuration from tmp/config1.toml...
    Error: IO error: No such file or directory (os error 2)

This add the kind of file and the file path being read, to the returned error message:

    2026-06-14T23:41:41.136213Z  INFO garage::server: Loading configuration from tmp/config1.toml...
    Error: Failed to read secret file /run/secrets/rpc_secret: No such file or directory (os error 2)

Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1475
Reviewed-by: Alex <lx@deuxfleurs.fr>
2026-06-17 13:53:43 +00:00
trinity-1686a da568cf56f fix reflected xss when returning errors on web endpoint
Reported by Filippo Decortes from ares-cyber.ai
2026-06-10 19:44:27 +02:00
Dave St.Germain 2bde733e09 fix: enable compilation on OpenBSD by removing keepalive interval (fix #1413) (#1453)
This fixes #1413 by conditionally compiling the section that sets a keepalive interval, which isn't supported on OpenBSD.

Tested on OpenBSD 7.8

Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1453
Reviewed-by: Alex <lx@deuxfleurs.fr>
2026-05-14 15:17:20 +00:00
Alex Auvolat bacc6c98b2 replace expiration field with custom type that merges to min value 2026-05-13 11:20:10 +02:00
Alex Auvolat bf0a24ea69 replace Option CRDT by explicit CancelingOption and MergingOption types 2026-05-13 11:20:06 +02:00
Arthur Carcano eb37a3e11a Fuzzing for K2VItem Crdt (#1438)
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1438
Reviewed-by: Alex <lx@deuxfleurs.fr>
2026-05-12 14:44:54 +00:00
smattymatty 54c63387cb fix(cors): include Access-Control-Allow-Headers in permissive OPTIONS placeholder (#1450)
The OPTIONS placeholder for buckets without a resolvable global alias returns` Access-Control-Allow-Origin: *` and `Access-Control-Allow-Methods: *` but omits `Access-Control-Allow-Headers`.

Bug verified against Garage v2.2.0 with a local-aliased bucket: OPTIONS placeholder doesn't have `Access-Control-Allow-Headers`, causes the browser to reject signed PUT preflights

The current placeholder fails open for unsigned simple requests but blocks every signed request, undermining the design intent flagged in the FIXME:

```rs
// We take the permissive approach of allowing everything,
// because we don't want to prevent web apps that use
// local bucket names from making API calls.
```

Adds `Access-Control-Allow-Headers: *` so the permissive default is actually permissive for the request shapes that exist in practice.

Refs #258. Does not address the broader FIXME (CORS rule resolution for local-aliased buckets); the placeholder approach is preserved.

All tests are fine locally:

```bash
 ▲ ~/opensource/garage cargo test -p garage_api_common cors::

running 5 tests
test cors::tests::preflight_with_single_allowed_origin_returns_request_origin ... ok
test cors::tests::preflight_with_multiple_allowed_origins_reflects_request_origin ... ok
test cors::tests::preflight_with_wildcard_allowed_origin_returns_wildcard ... ok
test xml::cors::tests::test_deserialize_norules ... ok
test xml::cors::tests::test_deserialize ... ok

test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 16 filtered out; finished in 0.00s
```

Co-authored-by: smattymatty <smattymatt@gmail.com>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1450
Reviewed-by: Alex <lx@deuxfleurs.fr>
2026-05-12 08:17:48 +00:00
Arthur Carcano a0887afc4f Add fuzing for AdminApiToken CRDT (#1443)
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1443
Reviewed-by: Alex <lx@deuxfleurs.fr>
2026-05-07 11:43:25 +00:00
Arthur Carcano 0da317e3d5 Fuzz Bucket CRDT (#1442)
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1442
Reviewed-by: Alex <lx@deuxfleurs.fr>
2026-05-06 18:55:47 +00:00
Arthur Carcano 322da7242b Post review fixes 2026-05-01 21:36:17 +00:00
Arthur Carcano 6a097e7de3 Add MPU table 2026-05-01 21:36:17 +00:00
Arthur Carcano 6ddae5397c Add version table fuzz 2026-05-01 21:36:17 +00:00
Arthur Carcano ade4d07bb5 Set up fuzz infrastructure 2026-05-01 21:36:17 +00:00
Alex Auvolat 12012916b7 simplify the garage health subcommand 2026-05-01 21:32:50 +02:00
Paul FLORENCE 9fa4e03748 Add health-check command to garage CLI
This command is used to check the local node health.

Related to https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/1354
2026-05-01 21:32:50 +02:00
Alex Auvolat 62349a6559 admin api: return full layout computation statistics as json (fix #1428) 2026-05-01 17:53:39 +00:00
Alex Auvolat ada0c8ab70 admin api: add fields to GetNodeInfo result (fix #1429) (#1434)
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1434
2026-05-01 16:57:27 +00:00
Alex Auvolat 7bc7f33f43 bg vars: return "never" when scrub never ran (fix #1421) (#1430)
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1430
2026-05-01 15:06:02 +00:00
Alex Auvolat be203494c5 set some flaky tests as #[ignore] (#1432)
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1432
2026-05-01 14:44:28 +00:00
Alex Auvolat 3c983ac5e0 admin api: properly eliminate irrelevant role deletions (fix #1427) (#1431)
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1431
2026-05-01 14:40:34 +00:00
Minkyu Kim a2c797000f fix(cors): return single matching origin instead of multiple values in Access-Control-Allow-Origin (#1419)
## Title
fix(cors): return single matching origin instead of multiple values in `Access-Control-Allow-Origin`

## Summary
This PR fixes bucket CORS responses when a single CORS rule contains multiple `AllowedOrigins`.

Previously, Garage returned the configured origins as a comma-separated list in `Access-Control-Allow-Origin`, for example:

```http
Access-Control-Allow-Origin: https://app.example.test, https://admin.example.test
```

This is not the expected browser-facing behavior.
When a request origin matches a configured rule, the response should reflect **only the matching request origin**, unless the rule contains `*`.

## What changed
- `Access-Control-Allow-Origin` now behaves as follows:
  - returns `*` when the matched rule contains a wildcard origin
  - otherwise returns the request `Origin` as a **single value**
- added `Vary: Origin` when ACAO reflects the request origin
- added preflight-specific `Vary` handling in the preflight path for:
  - `Origin`
  - `Access-Control-Request-Method`
  - `Access-Control-Request-Headers`

## Scope
This change applies to shared bucket CORS handling paths, including:
- S3 API responses
- K2V API responses
- S3 POST object responses
- web bucket responses
- preflight (`OPTIONS`) bucket CORS responses

This does **not** change admin API fixed CORS behavior.

## Reproduction
A direct repro script is included:

```bash
./script/test-cors-multi-origin.sh
```

It exercises two cases against a direct single-node Garage instance:

1. **single-origin control**
2. **multi-origin repro**

Before this fix, the multi-origin case returned a comma-separated ACAO value.

After this fix, both cases reflect only the request origin.

## Example behavior

### Before
```http
Access-Control-Allow-Origin: https://app.example.test, https://admin.example.test
```

### After
```http
Access-Control-Allow-Origin: https://app.example.test
```

## Tests
Added/updated tests in `src/api/common/cors.rs` for:
- single-origin control
- multiple allowed origins reflecting the request origin
- wildcard origin preserving `*`
- preserving existing `Vary` values while appending `Origin`

## Validation
Used for validation:

```bash
cargo test -p garage_api_common cors::tests -- --nocapture
cargo build -p garage --bin garage
./script/test-cors-multi-origin.sh
```

## Reproducibility
For reviewers who want to validate behavior by commit:

- Before fix: `aa368e4b`
  - includes the direct repro script and the regression test setup
  - multi-origin ACAO is reproduced as a comma-separated value

- After fix: `f630eb92`
  - reflects only the matching request origin
  - preserves wildcard behavior
  - adds `Vary: Origin` and preflight-specific `Vary` handling

Branch:
- `fix/cors-multiple-allow-origin`

Base used during validation:
- `74ad3bf8` (`main-v2`)

Closes Deuxfleurs/garage#1149

Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1419
2026-04-28 14:48:02 +00:00
Austin Drummond 80f9335950 collapse sequential whitespace in canonical SigV4 header values (#1424)
## Summary

Garage's SigV4 canonical-request builder trims leading/trailing whitespace from signed header values but does not collapse sequential internal whitespace, which the SigV4 spec requires:

> Convert sequential spaces to a single space.

— https://docs.aws.amazon.com/IAM/latest/UserGuide/create-signed-request.html

AWS SDKs apply this normalization before computing the signature, but transmit the raw value on the wire. The receiver must therefore apply the same normalization when reconstructing the canonical request, otherwise the recomputed hash differs and the request is rejected as `Invalid signature`.

Same class of canonicalization-drift bug as #1155 / !1382, but on the canonical-headers axis rather than the canonical-URI axis.

## Reproduction

Surfaces in practice with `gitlab-runner`'s S3 cache uploader. I was in the midst of migrating my runner cache from AWS S3 to garage, but I noticed some shared runner caches were no longer uploading.

I was using `sha256sum | sha256sum` to compute my cache keys, which leaves a trailing `  -` on the value. Once GitLab appends `-protected` for protected branches the resulting `x-amz-meta-cachekey` header value contains internal sequential whitespace and triggers the mismatch:

```
x-amz-meta-cachekey:php-  --protected
                              ^^
                              two spaces, preserved by Garage
```

Without the fix the included regression test (`test_presigned_put_with_user_metadata`) fails with HTTP 403; with the fix it returns 200.

`aws-cli` is unaffected because it signs `Content-Type` rather than user metadata, so the specific code path with whitespace-bearing signed header values isn't exercised.

## Fix

In `canonical_request` (`src/api/common/signature/payload.rs`), replace the `.trim()` call on the joined header value with the full SigV4 normalization — `split_whitespace().collect::<Vec<_>>().join(" ")` — which both trims edges and collapses internal runs.

## Tests

* New regression test `test_presigned_put_with_user_metadata` covering a  presigned PUT whose `x-amz-meta-*` value contains internal sequential whitespace.
* Full integration suite passes: `40 passed; 0 failed; 2 ignored`.
* `garage_api_common` unit tests pass: `18 passed; 0 failed`.

## Notes

* Backwards-compatible: any signature that validated before still validates, because clients are spec-required to collapse on their side; Garage was only rejecting requests where the client had collapsed correctly but Garage hadn't.
* No config or migration changes.
* Fix applies to both presigned-URL and Authorization-header code paths since they share the canonical-request builder.

Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1424
Reviewed-by: Alex <lx@deuxfleurs.fr>
2026-04-27 21:15:23 +00:00
Alex Auvolat d977ca4a24 fix new cargo clippy lints 2026-04-23 22:21:15 +02:00