1488 Commits

Author SHA1 Message Date
Alex Auvolat c5df820e2c api: start refactor of signature to calculate checksums earlier 2025-02-17 18:47:06 +01:00
Alex Auvolat a04d6cd5b8 api: streaming: parse unsigned streaming bodies and payload trailers 2025-02-17 16:23:24 +01:00
Alex Auvolat 44a896f9b5 api: add logic to parse x-amz-content-sha256 2025-02-16 18:25:35 +01:00
Alex Auvolat cee7560fc1 api: refactor: move checksum algorithms to common 2025-02-16 17:25:55 +01:00
Alex Auvolat 2f0c5ca220 signature: refactor: move constant defs to mod.rs 2025-02-16 16:34:18 +01:00
Alex 859b38b0d2 Merge pull request 'fix compilation warnings' (#959) from fixes into main
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/959
2025-02-14 17:32:30 +00:00
Alex Auvolat 2729a71d9d fix warning in garage test 2025-02-14 18:27:00 +01:00
Alex Auvolat c9d00f5f7b garage_api_s3: remove unused field in ListPartsQuery 2025-02-14 18:25:23 +01:00
Alex Auvolat 24470377c9 garage_model: fix warning about dead code 2025-02-14 18:12:14 +01:00
Alex Auvolat 5b26545abf fix deprecated uses of chrono in lifecycle worker 2025-02-14 18:08:23 +01:00
Alex Auvolat 165f9316e2 s3api: return Location in CompleteMultipartUpload (fix #852)
NB. The location returned is not guaranteed to work in all cases.
This already fixes the parse issue in #852.
2025-02-14 18:05:07 +01:00
Alex Auvolat e4c9a8cd53 block manager: avoid deadlock in fix_block_location (fix #845) 2025-02-14 17:41:50 +01:00
Alex Auvolat fdf4dad728 block resync: avoid saving blocks to draining nodes 2025-02-14 16:45:55 +01:00
Alex Auvolat 6820b69f30 block manager: improve read strategy to find blocks faster 2025-02-14 16:45:55 +01:00
Alex Auvolat d0104b9f9b block manager: write blocks only to currently active layout version (fix #815)
avoid wastefully writing blocks to nodes that will discard them as soon
as the layout migration is finished
2025-02-14 16:45:55 +01:00
Alex 3fe8db9e52 Merge pull request 'web_server.rs: Added bucket domain to observability' (#608) from jpds/garage:domain-web-requests into main
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/608
2025-02-14 14:26:08 +00:00
Alex 627a37fe9f Merge pull request 's3 api: parse x-id query parameter and warn of any inconsistency (fix #822)' (#954) from fix-822 into main
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/954
2025-02-14 14:07:01 +00:00
Alex Auvolat 2f55889835 add configuration option to enable/disable monitoring bucket in web metrics 2025-02-14 14:59:00 +01:00
Jonathan Davies 8b9cc5ca3f web_server.rs: Added bucket domain to observability. 2025-02-14 14:36:20 +01:00
Alex Auvolat c1b39d9ba1 s3 api: parse x-id query parameter and warn of any inconsistency (fix #822) 2025-02-14 14:30:58 +01:00
Alex Auvolat d84308c413 cli: return info of all nodes when doing garage stats -a (fix #814) 2025-02-14 14:11:41 +01:00
Alex Auvolat 62a3003cca rename Condition into RedirectCondition in internal model 2025-02-14 13:45:32 +01:00
Alex Auvolat 3151695011 Merge branch 'next-v2' into 1686a/s3-redirects 2025-02-14 13:11:33 +01:00
trinity-1686a f034e834fa Merge branch 'main' into 1686a/s3-redirects 2025-02-08 16:01:11 +01:00
trinity-1686a bf0f792418 add redirect_all to WebsiteConfig model 2025-02-08 15:36:48 +01:00
Baptiste Jonglez a2e134f036 db-snapshot: propagate any node snapshot error through RPC call
In particular, it means that "garage meta snapshot --all" will get an exit
code of 1 if any node fails to snapshot.

This makes sure that any external tool trying to snapshot nodes (e.g. from
cron) will be aware of the failure.

Fix #920
2025-02-07 00:29:43 +01:00
Baptiste Jonglez 06aa4b604f db-snapshot: Fix error reporting when using "garage meta snapshot --all"
Snapshot errors on remote nodes were not reported at all.

We now get proper error output such as:

    0fa0f35be69528ab  error: Internal error: DB error: LMDB: No space left on device (os error 28)
    88d92e2971d14bae  ok

Fix #920
2025-02-07 00:18:01 +01:00
Alex Auvolat 61f3de6496 Merge branch 'main' into next-v2 2025-02-05 21:00:19 +01:00
Alex Auvolat af67626ab2 remove async_trait for TableRepair 2025-02-05 20:45:07 +01:00
Alex Auvolat 5475da8ea8 remove async_trait used in generic_server.rs 2025-02-05 20:31:34 +01:00
Alex Auvolat 620dc58560 remove async_trait for traits declared in garage_net 2025-02-05 20:22:16 +01:00
Alex Auvolat 7c8fc04b96 massively speed up compilation of garage_api_admin by not using async_trait 2025-02-05 19:37:38 +01:00
Alex Auvolat f914db057a cli_v2: implement LaunchRepairOperation and remove old stuff 2025-02-05 15:36:47 +01:00
Alex Auvolat 406b6da163 cli_v2: implement Get{Node,Cluster}Statistics 2025-02-05 15:06:10 +01:00
Alex Auvolat 9f468b4439 cli_v2: implement CreateMetadataSnapshot 2025-02-05 14:22:10 +01:00
Alex Auvolat 97be7b38fa admin api: reorder things 2025-02-03 18:54:51 +01:00
Alex Auvolat 6a1079c412 admin api: impl RequestHandler for MetricsRequest 2025-02-03 18:54:51 +01:00
Alex Auvolat b1629dd355 cli_v2: implement RetryBlockResync and PurgeBlocks 2025-02-03 18:54:51 +01:00
Alex Auvolat d405a9f839 cli_v2: implement ListBlockErrors and GetBlockInfo 2025-02-03 18:54:51 +01:00
Alex Auvolat 7b9c047b11 cli_v2: add local_api_request with crazy type bound 2025-02-03 18:54:51 +01:00
Alex Auvolat 10bbb26b30 cli_v2: implement ListWorkers and GetWorkerInfo 2025-02-03 18:54:51 +01:00
Alex Auvolat 89ff9f5576 admin api: base infrastructure for local endpoints
admin api: rename EndpointHandler into RequestHandler to avoid confusion with RPC

wip: infrastructure for local api calls

admin api: fix things

admin api: first local endpoint to work with new scheme

admin api: implement SetWorkerVariable
2025-02-03 18:54:51 +01:00
Alex Auvolat bdaf55ab3f cli_v2: migrate cleanupincompleteuploads to Admin API
admin api: add CleanupIncompleteUploads spec
2025-02-03 18:54:51 +01:00
Alex Auvolat ec1a475923 build with rust 1.82.0 2025-02-03 17:46:48 +01:00
Alex Auvolat 390a5d97fe nix, ci: build with Crane
This removes our dependency on cargo2nix, which was causing us some
issues. Whereas cargo2nix creates one Nix derivation per crate, Crane
uses only two derivations:

1. Build dependencies only
2. Build the final binary

This means that during the second step, no caching can be done. For
instance, if we do a change in garage_model, we need to recompile all of
the Garage crates including those that do not depend on garage_model.
On the upside, this allows all of the Garage crates to be built at once
using cargo build logic, which is optimized for high parallelism and
better pipelining between all of the steps of the build. All in all,
this makes most builds faster than cargo2nix.

A few other changes have been made to the build scripts and CI:

- Unit tests are now run within a Nix derivation. In fact, we have
  different derivations to run the tests using LMDB and Sqlite as
  metadata db engines.

- For debug builds, most CI steps now run in parallel (with the notable
  exception of the smoke test that runs after the build, which is
  inevitable).

- We no longer pass the GIT_VERSION argument when building debug builds
  and running the tests. This means that dev binaries and test
  binaries don't know the exact version of Garage they are from. That
  shouldn't be an issue in most cases.

- The not-dynamic.sh scripts has been fixed to fail if the file does not
  exist.
2025-02-03 16:39:50 +01:00
Alex Auvolat fe937c2901 Merge branch 'main' into next-v2 2025-02-01 19:07:17 +01:00
Alex Auvolat d18c5ad0ff fix tests 2025-01-31 19:12:51 +01:00
Alex Auvolat 3d5e9a027e cargo defs: simplify and fix descriptions 2025-01-31 18:54:29 +01:00
Alex Auvolat 4563313f87 use cargo-shear to remove many unused dependencies between crates 2025-01-31 18:47:30 +01:00
Alex Auvolat afa28706e5 split s3/cors.rs into also common/cors.rs 2025-01-31 18:42:14 +01:00