Commit Graph

146 Commits

Author SHA1 Message Date
Alex Auvolat d38d62f4d7 bump version to v2.0.0 2025-04-17 12:36:41 +02:00
Alex Auvolat c6bed26347 relocalize logic into block manager 2025-03-25 16:41:19 +01:00
Alex Auvolat 2c9e849bbf remove dependency from garage_block to garage_table 2025-03-25 11:43:41 +01:00
Alex Auvolat 4689b10448 bump version to v1.1.0 2025-03-05 10:19:51 +01:00
Alex Auvolat e4c9a8cd53 block manager: avoid deadlock in fix_block_location (fix #845) 2025-02-14 17:41:50 +01:00
Alex Auvolat fdf4dad728 block resync: avoid saving blocks to draining nodes 2025-02-14 16:45:55 +01:00
Alex Auvolat d0104b9f9b block manager: write blocks only to currently active layout version (fix #815)
avoid wastefully writing blocks to nodes that will discard them as soon
as the layout migration is finished
2025-02-14 16:45:55 +01:00
Alex Auvolat 620dc58560 remove async_trait for traits declared in garage_net 2025-02-05 20:22:16 +01:00
Alex Auvolat ec1a475923 build with rust 1.82.0 2025-02-03 17:46:48 +01:00
Alex Auvolat 390a5d97fe nix, ci: build with Crane
This removes our dependency on cargo2nix, which was causing us some
issues. Whereas cargo2nix creates one Nix derivation per crate, Crane
uses only two derivations:

1. Build dependencies only
2. Build the final binary

This means that during the second step, no caching can be done. For
instance, if we do a change in garage_model, we need to recompile all of
the Garage crates including those that do not depend on garage_model.
On the upside, this allows all of the Garage crates to be built at once
using cargo build logic, which is optimized for high parallelism and
better pipelining between all of the steps of the build. All in all,
this makes most builds faster than cargo2nix.

A few other changes have been made to the build scripts and CI:

- Unit tests are now run within a Nix derivation. In fact, we have
  different derivations to run the tests using LMDB and Sqlite as
  metadata db engines.

- For debug builds, most CI steps now run in parallel (with the notable
  exception of the smoke test that runs after the build, which is
  inevitable).

- We no longer pass the GIT_VERSION argument when building debug builds
  and running the tests. This means that dev binaries and test
  binaries don't know the exact version of Garage they are from. That
  shouldn't be an issue in most cases.

- The not-dynamic.sh scripts has been fixed to fail if the file does not
  exist.
2025-02-03 16:39:50 +01:00
Alex Auvolat 4563313f87 use cargo-shear to remove many unused dependencies between crates 2025-01-31 18:47:30 +01:00
Stefan Majer 2eb9fcae20 Fix all typos 2025-01-16 13:22:00 +01:00
Alex Auvolat 295237476e fix formatting to comply with latest rustfmt 2025-01-12 17:36:25 +01:00
Alex Auvolat 7a143f46fc Bump to version 1.0.1 2024-09-22 14:25:32 +02:00
Alex Auvolat afad62939e [next-0.10] bump version number to 1.0 2024-03-28 15:19:44 +01:00
Alex Auvolat 8bfc16ba7d Merge branch 'main' into next-0.10 2024-03-28 15:01:05 +01:00
Alex Auvolat 85f580cbde [fix-buffering] change request sending strategy and fix priorities
remove LAS, priorize new requests but otherwise just do standard queuing
2024-03-27 16:22:40 +01:00
Alex Auvolat 0d3e285d13 [fix-buffering] implement block_ram_buffer_max to avoid excessive RAM usage 2024-03-27 16:22:40 +01:00
Alex Auvolat f7cd4eb600 Merge branch 'main' into next-0.10 2024-03-26 16:34:40 +01:00
Alex Auvolat ba33bb31f1 [check-data-dir] add marker files in data directories (fix #601) 2024-03-20 15:20:25 +01:00
Alex Auvolat 3165ab926c [block-ref-repair] rename rc's rc field to rc_table 2024-03-19 16:20:22 +01:00
Alex Auvolat dc0b78cdb8 [block-ref-repair] Block refcount recalculation and repair
- We always recalculate the reference count of a block before deleting
  it locally, to make sure that it is indeed zero.

- If we had to fetch a remote block but we were not able to get it,
  check that refcount is indeed > 0.

- Repair procedure that checks everything
2024-03-19 16:20:22 +01:00
Alex Auvolat 0038ca8a78 Merge branch 'main' into next-0.10 2024-03-18 20:19:30 +01:00
Alex Auvolat 7c86ff6c37 [disable-scrub] implement a disable_scrub configuration option 2024-03-14 17:01:16 +01:00
Alex Auvolat 05c92204ec [rm-sled] Remove counted_tree_hack 2024-03-08 15:09:57 +01:00
Alex Auvolat 44454aac01 [rm-sled] Remove the Sled database engine 2024-03-08 14:11:02 +01:00
Alex Auvolat 57acc60082 [sse-c] Implement SSE-C encryption 2024-03-07 15:43:47 +01:00
Alex Auvolat 8670140358 [rel-0.9.3] Bump version to 0.9.3 2024-03-04 14:00:55 +01:00
Alex Auvolat 6a7623e90d [rel-0.9.2] Bump version to v0.9.2 2024-03-01 16:54:39 +01:00
Alex Auvolat 4b978b7533 Merge branch 'main' into next-0.10 2024-02-26 18:55:24 +01:00
Alex Auvolat b76c0c102e [refactor-put] add ordering tag to blocks being sent to storage nodes 2024-02-26 18:35:11 +01:00
Alex Auvolat 916c67ccf4 Merge branch 'main' into next-0.10 2024-02-23 16:50:34 +01:00
Alex Auvolat 6ee691e65f [refactor-block] simplify some more 2024-02-23 12:50:10 +01:00
Alex Auvolat e9c42bca34 [refactor-block] add DataBlockStream type 2024-02-23 12:22:29 +01:00
Alex Auvolat cd1069c1d4 [refactor-block] refactor DataBlock and DataBlockPath 2024-02-23 12:15:52 +01:00
Alex Auvolat 07c7895948 [refactor-block] simplify rpc_get_block 2024-02-23 11:54:40 +01:00
Alex Auvolat 9b41f4ff20 [refactor-block] move read_stream_to_end to garage_net 2024-02-23 11:46:57 +01:00
Alex Auvolat 93552b9275 [refactor-block] Remove redundant BlockStream type 2024-02-23 11:33:38 +01:00
Alex Auvolat eb4a6ce106 Merge branch 'main' into next-0.10 2024-02-15 14:06:34 +01:00
Alex Auvolat 5ea24254a9 [import-netapp] import Netapp code into Garage codebase 2024-02-15 12:15:07 +01:00
Alex Auvolat cf2af186fc Merge branch 'main' into next-0.10 2024-02-13 11:36:28 +01:00
Alex Auvolat 22332e6c35 [dep-upgrade-202402] simplify/refactor GetObject 2024-02-05 20:26:33 +01:00
Alex Auvolat fe1af5d98b [dep-upgrade-202402] refactor dependencies: move all as workspace deps 2024-02-05 13:02:02 +01:00
Alex Auvolat ee57dd922b Bump version to 0.9.1 2024-01-16 16:28:17 +01:00
Alex Auvolat db48dd3d6c bump crate versions to 0.10.0 2024-01-11 12:05:51 +01:00
Alex Auvolat 5dd200c015 layout: move block_read_nodes_of to rpc_helper to avoid double-locking
(in theory, this could have caused a deadlock)
2023-12-08 12:02:24 +01:00
Alex Auvolat d6d239fc79 block manager: read_block using old layout versions if necessary 2023-11-27 11:52:57 +01:00
Alex Auvolat 33c8a489b0 layou: implement ack locking 2023-11-15 15:40:44 +01:00
Alex Auvolat 90e1619b1e table: take into account multiple write sets in inserts 2023-11-14 15:40:46 +01:00
Alex Auvolat 3b361d2959 layout: prepare for write sets 2023-11-14 14:28:16 +01:00