bump version to 0.8.3

fix test_website_check_domain
integration test: move json_body to root of crate
2026-08-13 15:46:53 +00:00 · 2023-08-28 13:17:26 +02:00 · 2023-08-28 12:40:28 +02:00 · 2023-08-28 12:32:57 +02:00 · 2023-08-28 12:20:34 +02:00 · 2023-08-28 10:18:33 +00:00
104 changed files with 6007 additions and 4957 deletions
@@ -11,10 +11,23 @@ members = [
 	"src/web",
 	"src/garage",
 	"src/k2v-client",
+	"src/format-table",
 ]

 default-members = ["src/garage"]

+[workspace.dependencies]
+format_table = { version = "0.1.1", path = "src/format-table" }
+garage_api = { version = "0.8.3", path = "src/api" }
+garage_block = { version = "0.8.3", path = "src/block" }
+garage_db = { version = "0.8.3", path = "src/db", default-features = false }
+garage_model = { version = "0.8.3", path = "src/model", default-features = false }
+garage_rpc = { version = "0.8.3", path = "src/rpc" }
+garage_table = { version = "0.8.3", path = "src/table" }
+garage_util = { version = "0.8.3", path = "src/util" }
+garage_web = { version = "0.8.3", path = "src/web" }
+k2v-client = { version = "0.0.4", path = "src/k2v-client" }
+
 [profile.dev]
 lto = "off"

@@ -632,7 +632,7 @@ paths:
      operationId: "UpdateBucket"
      summary: "Update a bucket"
      description: |
-        All fields (`websiteAccess` and `quotas`) are optionnal.
+        All fields (`websiteAccess` and `quotas`) are optional.
        If they are present, the corresponding modifications are applied to the bucket, otherwise nothing is changed.

        In `websiteAccess`: if `enabled` is `true`, `indexDocument` must be specified.
@@ -1,6 +1,6 @@
 +++
 title = "Build your own app"
-weight = 4
+weight = 40
 sort_by = "weight"
 template = "documentation.html"
 +++
@@ -1,6 +1,6 @@
 +++
 title = "Existing integrations"
-weight = 3
+weight = 30
 sort_by = "weight"
 template = "documentation.html"
 +++
@@ -11,6 +11,7 @@ In this section, we cover the following web applications:
 | [Peertube](#peertube)     | ✅       | Supported with the website endpoint, proxifying private videos unsupported     |
 | [Mastodon](#mastodon)     | ✅       | Natively supported    |
 | [Matrix](#matrix)     | ✅       |  Tested with `synapse-s3-storage-provider`    |
+| [ejabberd](#ejabberd)     | ✅       |  `mod_s3_upload`    |
 | [Pixelfed](#pixelfed)     | ❓       |  Not yet tested    |
 | [Pleroma](#pleroma)     | ❓       |  Not yet tested    |
 | [Lemmy](#lemmy)     | ✅        |  Supported with pict-rs    |
@@ -474,6 +475,52 @@ And add a new line. For example, to run it every 10 minutes:

 *External link:* [matrix-media-repo Documentation > S3](https://docs.t2bot.io/matrix-media-repo/configuration/s3-datastore.html)

+## ejabberd
+
+ejabberd is an XMPP server implementation which, with the `mod_s3_upload`
+module in the [ejabberd-contrib](https://github.com/processone/ejabberd-contrib)
+repository, can be integrated to store chat media files in Garage.
+
+For uploads, this module leverages presigned URLs - this allows XMPP clients to
+directly send media to Garage. Receiving clients then retrieve this media
+through the [static website](@/documentation/cookbook/exposing-websites.md)
+functionality.
+
+As the data itself is publicly accessible to someone with knowledge of the
+object URL - users are recommended to use
+[E2EE](@/documentation/cookbook/encryption.md) to protect this data-at-rest
+from unauthorized access.
+
+Install the module with:
+
+```bash
+ejabberdctl module_install mod_s3_upload
+```
+
+Create the required key and bucket with:
+
+```bash
+garage key new --name ejabberd
+garage bucket create objects.xmpp-server.fr
+garage bucket allow objects.xmpp-server.fr --read --write --key ejabberd
+garage bucket website --allow objects.xmpp-server.fr
+```
+
+The module can then be configured with:
+
+```
+  mod_s3_upload:
+    #bucket_url: https://objects.xmpp-server.fr.my-garage-instance.mydomain.tld
+    bucket_url: https://my-garage-instance.mydomain.tld/objects.xmpp-server.fr
+    access_key_id: GK...
+    access_key_secret: ...
+    region: garage
+    download_url: https://objects.xmpp-server.fr
+```
+
+Other configuration options can be found in the
+[configuration YAML file](https://github.com/processone/ejabberd-contrib/blob/master/mod_s3_upload/conf/mod_s3_upload.yml).
+
 ## Pixelfed

 [Pixelfed Technical Documentation > Configuration](https://docs.pixelfed.org/technical-documentation/env.html#filesystem)
@@ -539,7 +586,7 @@ secret_key = 'abcdef0123456789...'

 ```
 PICTRS__STORE__TYPE=object_storage
-PICTRS__STORE__ENDPOINT=http:/my-garage-instance.mydomain.tld:3900
+PICTRS__STORE__ENDPOINT=http://my-garage-instance.mydomain.tld:3900
 PICTRS__STORE__BUCKET_NAME=pictrs-data
 PICTRS__STORE__REGION=garage
 PICTRS__STORE__ACCESS_KEY=GK...
@@ -105,6 +105,7 @@ restic restore 79766175 --target /var/lib/postgresql
 Restic has way more features than the ones presented here.
 You can discover all of them by accessing its documentation from the link below.

+Files on Android devices can also be backed up with [restic-android](https://github.com/lhns/restic-android).

 *External links:* [Restic Documentation > Amazon S3](https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html#amazon-s3)

@@ -202,56 +202,11 @@ See its usage output for other commands available.
 ## Cyberduck & duck {#cyberduck}

 Both Cyberduck (the GUI) and duck (the CLI) have a concept of "Connection Profiles" that contain some presets for a specific provider.
-We wrote the following connection profile for Garage:

-```xml
-<?xml version="1.0" encoding="UTF-8"?>
-<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
-<plist version="1.0">
-    <dict>
-        <key>Protocol</key>
-        <string>s3</string>
-        <key>Vendor</key>
-        <string>garage</string>
-        <key>Scheme</key>
-        <string>https</string>
-        <key>Description</key>
-        <string>GarageS3</string>
-        <key>Default Hostname</key>
-        <string>127.0.0.1</string>
-        <key>Default Port</key>
-        <string>4443</string>
-        <key>Hostname Configurable</key>
-        <false/>
-        <key>Port Configurable</key>
-        <false/>
-        <key>Username Configurable</key>
-        <true/>
-        <key>Username Placeholder</key>
-        <string>Access Key ID (GK...)</string>
-        <key>Password Placeholder</key>
-        <string>Secret Key</string>
-        <key>Properties</key>
-        <array>
-            <string>s3service.disable-dns-buckets=true</string>
-        </array>
-        <key>Region</key>
-        <string>garage</string>
-        <key>Regions</key>
-        <array>
-            <string>garage</string>
-        </array>
-    </dict>
-</plist>
-```
-
-*Note: If your garage instance is configured with vhost access style, you can remove `s3service.disable-dns-buckets=true`.*
-
-### Instructions for the GUI
-
-Copy the connection profile, and save it anywhere as `garage.cyberduckprofile`.
-Then find this file with your file explorer and double click on it: Cyberduck will open a connection wizard for this profile.
-Simply follow the wizard and you should be done!
+Within Cyberduck, a
+[Garage connection profile](https://docs.cyberduck.io/protocols/s3/garage/) is
+available within the `Preferences -> Profiles` section. This can enabled and
+then connections to Garage may be configured.

 ### Instuctions for the CLI

@@ -1,7 +1,7 @@
 +++
 title="Cookbook"
 template = "documentation.html"
-weight = 2
+weight = 20
 sort_by = "weight"
 +++

@@ -37,7 +37,3 @@ This chapter could also be referred as "Tutorials" or "Best practices".
 - **[Monitoring Garage](@/documentation/cookbook/monitoring.md)** This page
  explains the Prometheus metrics available for monitoring the Garage
  cluster/nodes.
-
- **[Recovering from failures](@/documentation/cookbook/recovering.md):** Garage's first selling point is resilience
-  to hardware failures. This section explains how to recover from such a failure in the
-  best possible way.
@@ -7,10 +7,23 @@ Garage is also available in binary packages on:

 ## Alpine Linux

+If you use Alpine Linux, you can simply install the
+[garage](https://pkgs.alpinelinux.org/packages?name=garage) package from the
+Alpine Linux repositories (available since v3.17):
+
 ```bash
-apk install garage
+apk add garage
 ```

+The default configuration file is installed to `/etc/garage.toml`. You can run
+Garage using: `rc-service garage start`. If you don't specify `rpc_secret`, it
+will be automatically replaced with a random string on the first start.
+
+Please note that this package is built without Consul discovery, Kubernetes
+discovery, OpenTelemetry exporter, and K2V features (K2V will be enabled once
+it's stable).
+
+
 ## Arch Linux

 Garage is available in the [AUR](https://aur.archlinux.org/packages/garage).
@@ -0,0 +1,116 @@
+++
+title = "Encryption"
+weight = 50
+++
+
+Encryption is a recurring subject when discussing Garage. 
+Garage does not handle data encryption by itself, but many things can
+already be done with Garage's current feature set and the existing ecosystem.
+
+This page takes a high level approach to security in general and data encryption
+in particular.
+
+
+# Examining your need for encryption
+
+- Why do you want encryption in Garage?
+
+- What is your threat model? What are you fearing?
+  - A stolen HDD?
+  - A curious administrator?
+  - A malicious administrator?
+  - A remote attacker?
+  - etc.
+
+- What services do you want to protect with encryption?
+  - An existing application? Which one? (eg. Nextcloud)
+  - An application that you are writing
+
+- Any expertise you may have on the subject
+
+This page explains what Garage provides, and how you can improve the situation by yourself
+by adding encryption at different levels.
+
+We would be very curious to know your needs and thougs about ideas such as
+encryption practices and things like key management, as we want Garage to be a
+serious base platform for the developpment of secure, encrypted applications.
+Do not hesitate to come talk to us if you have any thoughts or questions on the
+subject.
+
+
+# Capabilities provided by Garage
+
+## Traffic is encrypted between Garage nodes
+
+RPCs between Garage nodes are encrypted. More specifically, contrary to many
+distributed software, it is impossible in Garage to have clear-text RPC.  We
+use the [kuska handshake](https://github.com/Kuska-ssb/handshake) library which
+implements a protocol that has been clearly reviewed, Secure ScuttleButt's
+Secret Handshake protocol.  This is why setting a `rpc_secret` is mandatory,
+and that's also why your nodes have super long identifiers.
+
+## HTTP API endpoints provided by Garage are in clear text
+
+Adding TLS support built into Garage is not currently planned.
+
+## Garage stores data in plain text on the filesystem
+
+Garage does not handle data encryption at rest by itself, and instead delegates
+to the user to add encryption, either at the storage layer (LUKS, etc) or on
+the client side (or both). There are no current plans to add data encryption
+directly in Garage.
+
+Implementing data encryption directly in Garage might make things simpler for
+end users, but also raises many more questions, especially around key
+management: for encryption of data, where could Garage get the encryption keys
+from ? If we encrypt data but keep the keys in a plaintext file next to them,
+it's useless. We probably don't want to have to manage secrets in garage as it
+would be very hard to do in a secure way. Maybe integrate with an external
+system such as Hashicorp Vault?
+
+
+# Adding data encryption using external tools
+
+## Encrypting traffic between a Garage node and your client
+
+You have multiple options to have encryption between your client and a node:
+
+  - Setup a reverse proxy with TLS / ACME / Let's encrypt
+  - Setup a Garage gateway locally, and only contact the garage daemon on `localhost`
+  - Only contact your Garage daemon over a secure, encrypted overlay network such as Wireguard
+
+## Encrypting data at rest
+
+Protects against the following threats:
+
+- Stolen HDD
+
+Crucially, does not protect againt malicious sysadmins or remote attackers that
+might gain access to your servers.
+
+Methods include full-disk encryption with tools such as LUKS.
+
+## Encrypting data on the client side
+
+Protects againt the following threats:
+
+- A honest-but-curious administrator
+- A malicious administrator that tries to corrupt your data
+- A remote attacker that can read your server's data
+
+Implementations are very specific to the various applications. Examples:
+
+- Matrix: uses the OLM protocol for E2EE of user messages. Media files stored
+  in Matrix are probably encrypted using symmetric encryption, with a key that is
+  distributed in the end-to-end encrypted message that contains the link to the object.
+
+- XMPP: clients normally support either OMEMO / OpenPGP for the E2EE of user
+  messages. Media files are encrypted per
+  [XEP-0454](https://xmpp.org/extensions/xep-0454.html).
+
+- Aerogramme: use the user's password as a key to decrypt data in the user's bucket
+
+- Cyberduck: comes with support for
+  [Cryptomator](https://docs.cyberduck.io/cryptomator/) which allows users to
+  create client-side vaults to encrypt files in before they are uploaded to a
+  cloud storage endpoint.
@@ -49,9 +49,5 @@ add the following lines in your Prometheus scrape config:
 To visualize the scraped data in Grafana,
 you can either import our [Grafana dashboard for Garage](https://git.deuxfleurs.fr/Deuxfleurs/garage/raw/branch/main/script/telemetry/grafana-garage-dashboard-prometheus.json)
 or make your own.
-We detail below the list of exposed metrics and their meaning.

-
-## List of exported metrics
-
-See our [dedicated page](@/documentation/reference-manual/monitoring.md) in the Reference manual section.
+The list of exported metrics is available on our [dedicated page](@/documentation/reference-manual/monitoring.md) in the Reference manual section.
@@ -197,6 +197,12 @@ The `garage` binary has two purposes:
 Ensure an appropriate `garage` binary (the same version as your Docker image) is available in your path.
 If your configuration file is at `/etc/garage.toml`, the `garage` binary should work with no further change.

+You can also use an alias as follows to use the Garage binary inside your docker container:
+
+```bash
+alias garage="docker exec -ti <container name> /garage"
+```
+
 You can test your `garage` CLI utility by running a simple command such as:

 ```bash
@@ -339,7 +345,7 @@ garage layout apply
 ```

 **WARNING:** if you want to use the layout modification commands in a script,
-make sure to read [this page](@/documentation/reference-manual/layout.md) first.
+make sure to read [this page](@/documentation/operations/layout.md) first.


 ## Using your Garage cluster
@@ -349,5 +355,5 @@ and is covered in the [quick start guide](@/documentation/quick-start/_index.md)
 Remember also that the CLI is self-documented thanks to the `--help` flag and
 the `help` subcommand (e.g. `garage help`, `garage key --help`).

-Configuring S3-compatible applicatiosn to interact with Garage
+Configuring S3-compatible applications to interact with Garage
 is covered in the [Integrations](@/documentation/connect/_index.md) section.
@@ -378,6 +378,47 @@ admin.garage.tld {
 But at the same time, the `reverse_proxy` is very flexible.
 For a production deployment, you should [read its documentation](https://caddyserver.com/docs/caddyfile/directives/reverse_proxy) as it supports features like DNS discovery of upstreams, load balancing with checks, streaming parameters, etc.

+### Caching
+
+Caddy can compiled with a
+[cache plugin](https://github.com/caddyserver/cache-handler) which can be used
+to provide a hot-cache at the webserver-level for static websites hosted by
+Garage.
+
+This can be configured as follows:
+
+```caddy
+# Caddy global configuration section
+{
+	# Bare minimum configuration to enable cache.
+	order cache before rewrite
+
+	cache
+
+	#cache
+	#	allowed_http_verbs GET
+	#	default_cache_control public
+	#	ttl 8h
+	#}
+}
+
+# Site specific section
+https:// {
+	cache
+
+	#cache {
+	#	timeout {
+	#		backend 30s
+	#	}
+	#}
+
+	reverse_proxy ...
+}
+```
+
+Caching is a complicated subject, and the reader is encouraged to study the
+available options provided by the plugin.
+
 ### On-demand TLS

 Caddy supports a technique called
@@ -428,3 +469,6 @@ https:// {
 	reverse_proxy localhost:3902 192.168.1.2:3902 example.tld:3902
 }
 ```
+
+More information on how this endpoint is implemented in Garage is available
+in the [Admin API Reference](@/documentation/reference-manual/admin-api.md) page.
@@ -33,7 +33,20 @@ NoNewPrivileges=true
 WantedBy=multi-user.target
 ```

-*A note on hardening: garage will be run as a non privileged user, its user id is dynamically allocated by systemd. It cannot access (read or write) home folders (/home, /root and /run/user), the rest of the filesystem can only be read but not written, only the path seen as /var/lib/garage is writable as seen by the service (mapped to /var/lib/private/garage on your host). Additionnaly, the process can not gain new privileges over time.*
+**A note on hardening:** Garage will be run as a non privileged user, its user
+id is dynamically allocated by systemd (set with `DynamicUser=true`). It cannot
+access (read or write) home folders (`/home`, `/root` and `/run/user`), the
+rest of the filesystem can only be read but not written, only the path seen as
+`/var/lib/garage` is writable as seen by the service. Additionnaly, the process
+can not gain new privileges over time.
+
+For this to work correctly, your `garage.toml` must be set with
+`metadata_dir=/var/lib/garage/meta` and `data_dir=/var/lib/garage/data`. This
+is mandatory to use the DynamicUser hardening feature of systemd, which
+autocreates these directories as virtual mapping. If the directory
+`/var/lib/garage` already exists before starting the server for the first time,
+the systemd service might not start correctly.  Note that in your host
+filesystem, Garage data will be held in `/var/lib/private/garage`.

 To start the service then automatically enable it at boot:

@@ -1,6 +1,6 @@
 +++
 title = "Design"
-weight = 6
+weight = 70
 sort_by = "weight"
 template = "documentation.html"
 +++
@@ -42,15 +42,13 @@ locations. They use Garage themselves for the following tasks:

 - As a [Matrix media backend](https://github.com/matrix-org/synapse-s3-storage-provider)

- To store personal data and shared documents through [Bagage](https://git.deuxfleurs.fr/Deuxfleurs/bagage), a homegrown WebDav-to-S3 proxy
+- As a Nix binary cache
+
+- To store personal data and shared documents through [Bagage](https://git.deuxfleurs.fr/Deuxfleurs/bagage), a homegrown WebDav-to-S3 and SFTP-to-S3 proxy
+
+- As a backup target using `rclone` and `restic`

 - In the Drone continuous integration platform to store task logs

- As a Nix binary cache
-
- As a backup target using `rclone`
-
 The Deuxfleurs Garage cluster is a multi-site cluster currently composed of
-4 nodes in 2 physical locations. In the future it will be expanded to at
-least 3 physical locations to fully exploit Garage's potential for high
-availability.
+9 nodes in 3 physical locations.
@@ -61,7 +61,7 @@ Garage prioritizes which nodes to query according to a few criteria:


 For further reading on the cluster structure look at the [gateway](@/documentation/cookbook/gateways.md) 
-and [cluster layout management](@/documentation/reference-manual/layout.md) pages.
+and [cluster layout management](@/documentation/operations/layout.md) pages.

 ## Garbage collection

@@ -1,6 +1,6 @@
 +++
 title = "Development"
-weight = 7
+weight = 80
 sort_by = "weight"
 template = "documentation.html"
 +++
@@ -25,7 +25,7 @@ git clone https://git.deuxfleurs.fr/Deuxfleurs/garage
 cd garage
 ```

-*Optionnaly, you can use our nix.conf file to speed up compilations:*
+*Optionally, you can use our nix.conf file to speed up compilations:*

 ```bash
 sudo mkdir -p /etc/nix
@@ -0,0 +1,23 @@
+++
+title = "Operations & Maintenance"
+weight = 50
+sort_by = "weight"
+template = "documentation.html"
+++
+
+This section contains a number of important information on how to best operate a Garage cluster,
+to ensure integrity and availability of your data:
+
+- **[Upgrading Garage](@/documentation/operations/upgrading.md):** General instructions on how to
+  upgrade your cluster from one version to the next. Instructions specific for each version upgrade
+  can bef ound in the [working documents](@/documentation/working-documents/_index.md) section.
+
+- **[Layout management](@/documentation/operations/layout.md):** Best practices for using the `garage layout`
+  commands when adding or removing nodes from your cluster.
+
+- **[Durability and repairs](@/documentation/operations/durability-repairs.md):** How to check for small things
+  that might be going wrong, and how to recover from such failures.
+
+- **[Recovering from failures](@/documentation/operations/recovering.md):** Garage's first selling point is resilience
+  to hardware failures. This section explains how to recover from such a failure in the
+  best possible way.
@@ -0,0 +1,117 @@
+++
+title = "Durability & Repairs"
+weight = 30
+++
+
+To ensure the best durability of your data and to fix any inconsistencies that may
+pop up in a distributed system, Garage provides a series of repair operations.
+This guide will explain the meaning of each of them and when they should be applied.
+
+
+# General syntax of repair operations
+
+Repair operations described below are of the form `garage repair <repair_name>`.
+These repairs will not launch without the `--yes` flag, which should
+be added as follows: `garage repair --yes <repair_name>`.
+By default these repair procedures will only run on the Garage node your CLI is
+connecting to. To run on all nodes, add the `-a` flag as follows:
+`garage repair -a --yes <repair_name>`.
+
+# Data block operations
+
+## Data store scrub
+
+Scrubbing the data store means examining each individual data block to check that
+their content is correct, by verifying their hash. Any block found to be corrupted
+(e.g. by bitrot or by an accidental manipulation of the datastore) will be
+restored from another node that holds a valid copy.
+
+Scrubs are automatically scheduled by Garage to run every 25-35 days (the
+actual time is randomized to spread load across nodes). The next scheduled run
+can be viewed with `garage worker get`.
+
+A scrub can also be launched manually using `garage repair scrub start`.
+
+To view the status of an ongoing scrub, first find the task ID of the scrub worker
+using `garage worker list`. Then, run `garage worker info <scrub_task_id>` to
+view detailed runtime statistics of the scrub. To gather cluster-wide information,
+this command has to be run on each individual node.
+
+A scrub is a very disk-intensive operation that might slow down your cluster.
+You may pause an ongoing scrub using `garage repair scrub pause`, but note that
+the scrub will resume automatically 24 hours later as Garage will not let your
+cluster run without a regular scrub. If the scrub procedure is too intensive
+for your servers and is slowing down your workload, the recommended solution
+is to increase the "scrub tranquility" using `garage repair scrub set-tranquility`.
+A higher tranquility value will make Garage take longer pauses between two block
+verifications. Of course, scrubbing the entire data store will also take longer.
+
+## Block check and resync
+
+In some cases, nodes hold a reference to a block but do not actually have the block
+stored on disk. Conversely, they may also have on disk blocks that are not referenced
+any more. To fix both cases, a block repair may be run with `garage repair blocks`.
+This will scan the entire block reference counter table to check that the blocks
+exist on disk, and will scan the entire disk store to check that stored blocks
+are referenced.
+
+It is recommended to run this procedure when changing your cluster layout,
+after the metadata tables have finished synchronizing between nodes
+(usually a few hours after `garage layout apply`).
+
+## Inspecting lost blocks
+
+In extremely rare situations, data blocks may be unavailable from the entire cluster.
+This means that even using `garage repair blocks`, some nodes may be unable
+to fetch data blocks for which they hold a reference.
+
+These errors are stored on each node in a list of "block resync errors", i.e.
+blocks for which the last resync operation failed.
+This list can be inspected using `garage block list-errors`.
+These errors usually fall into one of the following categories:
+
+1. a block is still referenced but the object was deleted, this is a case
+   of metadata reference inconsistency (see below for the fix)
+2. a block is referenced by a non-deleted object, but could not be fetched due
+   to a transient error such as a network failure
+3. a block is referenced by a non-deleted object, but could not be fetched due
+   to a permanent error such as there not being any valid copy of the block on the
+   entire cluster
+
+To help make the difference between cases 1 and cases 2 and 3, you may use the
+`garage block info` command to see which objects hold a reference to each block.
+
+In the second case (transient errors), Garage will try to fetch the block again
+after a certain time, so the error should disappear naturally. You can also
+request Garage to try to fetch the block immediately using `garage block retry-now`
+if you have fixed the transient issue.
+
+If you are confident that you are in the third scenario and that your data block
+is definitely lost, then there is no other choice than to declare your S3 objects
+as unrecoverable, and to delete them properly from the data store. This can be done
+using the `garage block purge` command.
+
+
+# Metadata operations
+
+## Metadata table resync
+
+Garage automatically resyncs all entries stored in the metadata tables every hour,
+to ensure that all nodes have the most up-to-date version of all the information
+they should be holding.
+The resync procedure is based on a Merkle tree that allows to efficiently find
+differences between nodes.
+
+In some special cases, e.g. before an upgrade, you might want to run a table
+resync manually. This can be done using `garage repair tables`.
+
+## Metadata table reference fixes
+
+In some very rare cases where nodes are unavailable, some references between objects
+are broken. For instance, if an object is deleted, the underlying versions or data
+blocks may still be held by Garage. If you suspect that such corruption has occurred
+in your cluster, you can run one of the following repair procedures:
+
+- `garage repair versions`: checks that all versions belong to a non-deleted object, and purges any orphan version
+- `garage repair block_refs`: checks that all block references belong to a non-deleted object version, and purges any orphan block reference (this will then allow the blocks to be garbage-collected)
+
@@ -1,6 +1,6 @@
 +++
 title = "Cluster layout management"
-weight = 50
+weight = 20
 +++

 The cluster layout in Garage is a table that assigns to each node a role in
@@ -1,6 +1,6 @@
 +++
 title = "Recovering from failures"
-weight = 50
+weight = 40
 +++

 Garage is meant to work on old, second-hand hardware.
@@ -1,6 +1,6 @@
 +++
 title = "Upgrading Garage"
-weight = 60
+weight = 10
 +++

 Garage is a stateful clustered application, where all nodes are communicating together and share data structures.
@@ -58,7 +58,7 @@ From a high level perspective, a major upgrade looks like this:

 ### Major upgarades with minimal downtime

-There is only one operation that has to be coordinated cluster-wide: the passage of one version of the internal RPC protocol to the next.
+There is only one operation that has to be coordinated cluster-wide: the switch of one version of the internal RPC protocol to the next.
 This means that an upgrade with very limited downtime can simply be performed from one major version to the next by restarting all nodes
 simultaneously in the new version.
 The downtime will simply be the time required for all nodes to stop and start again, which should be less than a minute.
@@ -1,6 +1,6 @@
 +++
 title = "Quick Start"
-weight = 0
+weight = 10
 sort_by = "weight"
 template = "documentation.html"
 +++
@@ -35,6 +35,9 @@ Place this binary somewhere in your `$PATH` so that you can invoke the `garage`
 command directly (for instance you can copy the binary in `/usr/local/bin`
 or in `~/.local/bin`).

+You may also check whether your distribution already includes a
+[binary package for Garage](@/documentation/cookbook/binary-packages.md).
+
 If a binary of the last version is not available for your architecture,
 or if you want a build customized for your system,
 you can [build Garage from source](@/documentation/cookbook/from-source.md).
@@ -290,13 +293,13 @@ sourcing the right file.*
 aws s3 ls

 # list objects of a bucket
-aws s3 ls s3://my_files
+aws s3 ls s3://nextcloud-bucket

 # copy from your filesystem to garage
-aws s3 cp /proc/cpuinfo s3://my_files/cpuinfo.txt
+aws s3 cp /proc/cpuinfo s3://nextcloud-bucket/cpuinfo.txt

 # copy from garage to your filesystem
-aws s3 cp s3/my_files/cpuinfo.txt /tmp/cpuinfo.txt
+aws s3 cp s3://nextcloud-bucket/cpuinfo.txt /tmp/cpuinfo.txt
 ```

 Note that you can use `awscli` for more advanced operations like
@@ -1,6 +1,6 @@
 +++
 title = "Reference Manual"
-weight = 5
+weight = 60
 sort_by = "weight"
 template = "documentation.html"
 +++
@@ -39,11 +39,95 @@ Authorization: Bearer <token>

 ## Administration API endpoints

-### Metrics-related endpoints
-
-#### Metrics `GET /metrics`
+### Metrics `GET /metrics`

 Returns internal Garage metrics in Prometheus format.
+The metrics are directly documented when returned by the API.
+
+**Example:**
+
+```
+$ curl -i http://localhost:3903/metrics
+HTTP/1.1 200 OK
+content-type: text/plain; version=0.0.4
+content-length: 12145
+date: Tue, 08 Aug 2023 07:25:05 GMT
+
+# HELP api_admin_error_counter Number of API calls to the various Admin API endpoints that resulted in errors
+# TYPE api_admin_error_counter counter
+api_admin_error_counter{api_endpoint="CheckWebsiteEnabled",status_code="400"} 1
+api_admin_error_counter{api_endpoint="CheckWebsiteEnabled",status_code="404"} 3
+# HELP api_admin_request_counter Number of API calls to the various Admin API endpoints
+# TYPE api_admin_request_counter counter
+api_admin_request_counter{api_endpoint="CheckWebsiteEnabled"} 7
+api_admin_request_counter{api_endpoint="Health"} 3
+# HELP api_admin_request_duration Duration of API calls to the various Admin API endpoints
+...
+```
+
+### Health `GET /health`
+
+Returns `200 OK` if enough nodes are up to have a quorum (ie. serve requests),
+otherwise returns `503 Service Unavailable`.
+
+**Example:**
+
+```
+$ curl -i http://localhost:3903/health
+HTTP/1.1 200 OK
+content-type: text/plain
+content-length: 102
+date: Tue, 08 Aug 2023 07:22:38 GMT
+
+Garage is fully operational
+Consult the full health check API endpoint at /v0/health for more details
+```
+
+### On-demand TLS `GET /check`
+
+To prevent abuses for on-demand TLS, Caddy developpers have specified an endpoint that can be queried by the reverse proxy
+to know if a given domain is allowed to get a certificate. Garage implements this endpoints to tell if a given domain is handled by Garage or is garbage.
+
+Garage responds with the following logic:
+ - If the domain matches the pattern `<bucket-name>.<s3_api.root_domain>`, returns 200 OK
+ - If the domain matches the pattern `<bucket-name>.<s3_web.root_domain>` and website is configured for `<bucket>`, returns 200 OK
+ - If the domain matches the pattern `<bucket-name>` and website is configured for `<bucket>`, returns 200 OK
+ - Otherwise, returns 404 Not Found, 400 Bad Request or 5xx requests.
+
+*Note 1: because in the path-style URL mode, there is only one domain that is not known by Garage, hence it is not supported by this API endpoint.
+You must manually declare the domain in your reverse-proxy. Idem for K2V.*
+
+*Note 2: buckets in a user's namespace are not supported yet by this endpoint. This is a limitation of this endpoint currently.*
+
+**Example:** Suppose a Garage instance configured with `s3_api.root_domain = .s3.garage.localhost` and `s3_web.root_domain = .web.garage.localhost`.
+
+With a private `media` bucket (name in the global namespace, website is disabled), the endpoint will feature the following behavior:
+
+```
+$ curl -so /dev/null -w "%{http_code}" http://localhost:3903/check?domain=media.s3.garage.localhost
+200
+$ curl -so /dev/null -w "%{http_code}" http://localhost:3903/check?domain=media
+400
+$ curl -so /dev/null -w "%{http_code}" http://localhost:3903/check?domain=media.web.garage.localhost
+400
+```
+
+With a public `example.com` bucket (name in the global namespace, website is activated), the endpoint will feature the following behavior:
+
+```
+$ curl -so /dev/null -w "%{http_code}" http://localhost:3903/check?domain=example.com.s3.garage.localhost
+200
+$ curl -so /dev/null -w "%{http_code}" http://localhost:3903/check?domain=example.com
+200
+$ curl -so /dev/null -w "%{http_code}" http://localhost:3903/check?domain=example.com.web.garage.localhost
+200
+```
+
+
+**References:**
+ - [Using On-Demand TLS](https://caddyserver.com/docs/automatic-https#using-on-demand-tls)
+ - [Add option for a backend check to approve use of on-demand TLS](https://github.com/caddyserver/caddy/pull/1939)
+ - [Serving tens of thousands of domains over HTTPS with Caddy](https://caddy.community/t/serving-tens-of-thousands-of-domains-over-https-with-caddy/11179)

 ### Cluster operations

@@ -35,12 +35,18 @@ bootstrap_peers = [


 [consul_discovery]
+api = "catalog"
 consul_http_addr = "http://127.0.0.1:8500"
 service_name = "garage-daemon"
 ca_cert = "/etc/consul/consul-ca.crt"
 client_cert = "/etc/consul/consul-client.crt"
 client_key = "/etc/consul/consul-key.crt"
+# for `agent` API mode, unset client_cert and client_key, and optionally enable `token`
+# token = "abcdef-01234-56789"
 tls_skip_verify = false
+tags = [ "dns-enabled" ]
+meta = { dns-acl = "allow trusted" }
+

 [kubernetes_discovery]
 namespace = "garage"
@@ -201,7 +207,7 @@ Garage supports the following replication modes:
    that should probably never be used.

 Note that in modes `2` and `3`,
-if at least the same number of zones are available, an arbitrary number of failures in 
+if at least the same number of zones are available, an arbitrary number of failures in
 any given zone is tolerated as copies of data will be spread over several zones.

 **Make sure `replication_mode` is the same in the configuration files of all nodes.
@@ -245,7 +251,7 @@ Values between `1` (faster compression) and `19` (smaller file) are standard com
 levels for zstd. From `20` to `22`, compression levels are referred as "ultra" and must be
 used with extra care as it will use lot of memory. A value of `0` will let zstd choose a
 default value (currently `3`). Finally, zstd has also compression designed to be faster
-than default compression levels, they range from `-1` (smaller file) to `-99` (faster 
+than default compression levels, they range from `-1` (smaller file) to `-99` (faster
 compression).

 If you do not specify a `compression_level` entry, Garage will set it to `1` for you. With
@@ -316,6 +322,12 @@ reached by other nodes of the cluster, which should be set in `rpc_public_addr`.

 The `consul_http_addr` parameter should be set to the full HTTP(S) address of the Consul server.

+### `api`
+
+Two APIs for service registration are supported: `catalog`  and `agent`. `catalog`, the default, will register a service using
+the `/v1/catalog` endpoints, enabling mTLS if `client_cert` and `client_key` are provided. The `agent` API uses the
+`v1/agent` endpoints instead, where an optional `token` may be provided.
+
 ### `service_name`

 `service_name` should be set to the service name under which Garage's
@@ -324,6 +336,7 @@ RPC ports are announced.
 ### `client_cert`, `client_key`

 TLS client certificate and client key to use when communicating with Consul over TLS. Both are mandatory when doing so.
+Only available when `api = "catalog"`.

 ### `ca_cert`

@@ -334,6 +347,29 @@ TLS CA certificate to use when communicating with Consul over TLS.
 Skip server hostname verification in TLS handshake.
 `ca_cert` is ignored when this is set.

+### `token`
+
+Uses the provided token for communication with Consul. Only available when `api = "agent"`.
+The policy assigned to this token should at least have these rules:
+
+```hcl
+// the `service_name` specified above
+service "garage" {
+  policy = "write"
+}
+
+service_prefix "" {
+  policy = "read"
+}
+
+node_prefix "" {
+  policy = "read"
+}
+```
+
+### `tags` and `meta`
+
+Additional list of tags and map of service meta to add during service registration.

 ## The `[kubernetes_discovery]` section

@@ -373,7 +409,7 @@ message that redirects the client to the correct region.

 ### `root_domain` {#root_domain}

-The optionnal suffix to access bucket using vhost-style in addition to path-style request.
+The optional suffix to access bucket using vhost-style in addition to path-style request.
 Note path-style requests are always enabled, whether or not vhost-style is configured.
 Configuring vhost-style S3 required a wildcard DNS entry, and possibly a wildcard TLS certificate,
 but might be required by softwares not supporting path-style requests.
@@ -396,7 +432,7 @@ This endpoint does not suport TLS: a reverse proxy should be used to provide it.

 ### `root_domain`

-The optionnal suffix appended to bucket names for the corresponding HTTP Host.
+The optional suffix appended to bucket names for the corresponding HTTP Host.

 For instance, if `root_domain` is `web.garage.eu`, a bucket called `deuxfleurs.fr`
 will be accessible either with hostname `deuxfleurs.fr.web.garage.eu`
@@ -35,7 +35,7 @@ This makes setting up and administering storage clusters, we hope, as easy as it

 A Garage cluster can very easily evolve over time, as storage nodes are added or removed.
 Garage will automatically rebalance data between nodes as needed to ensure the desired number of copies.
-Read about cluster layout management [here](@/documentation/reference-manual/layout.md).
+Read about cluster layout management [here](@/documentation/operations/layout.md).

 ### No RAFT slowing you down

@@ -3,7 +3,7 @@ title = "K2V"
 weight = 100
 +++

-Starting with version 0.7.2, Garage introduces an optionnal feature, K2V,
+Starting with version 0.7.2, Garage introduces an optional feature, K2V,
 which is an alternative storage API designed to help efficiently store
 many small values in buckets (in opposition to S3 which is more designed
 to store large blobs).
@@ -1,6 +1,6 @@
 +++
 title = "Working Documents"
-weight = 8
+weight = 90
 sort_by = "weight"
 template = "documentation.html"
 +++
@@ -453,7 +453,7 @@ Request body format:
 }
 ```

-All fields (`name`, `allow` and `deny`) are optionnal.
+All fields (`name`, `allow` and `deny`) are optional.
 If they are present, the corresponding modifications are applied to the key, otherwise nothing is changed.
 The possible flags in `allow` and `deny` are: `createBucket`.

@@ -609,7 +609,7 @@ Request body format:
 }
 ```

-All fields (`websiteAccess` and `quotas`) are optionnal.
+All fields (`websiteAccess` and `quotas`) are optional.
 If they are present, the corresponding modifications are applied to the bucket, otherwise nothing is changed.

 In `websiteAccess`: if `enabled` is `true`, `indexDocument` must be specified.
@@ -2,7 +2,9 @@
  "nodes": {
    "cargo2nix": {
      "inputs": {
-        "flake-compat": "flake-compat",
+        "flake-compat": [
+          "flake-compat"
+        ],
        "flake-utils": "flake-utils",
        "nixpkgs": [
          "nixpkgs"
@@ -25,28 +27,30 @@
      }
    },
    "flake-compat": {
-      "flake": false,
      "locked": {
-        "lastModified": 1650374568,
-        "narHash": "sha256-Z+s0J8/r907g149rllvwhb4pKi8Wam5ij0st8PwAh+E=",
-        "owner": "edolstra",
+        "lastModified": 1688025799,
+        "narHash": "sha256-ktpB4dRtnksm9F5WawoIkEneh1nrEvuxb5lJFt1iOyw=",
+        "owner": "nix-community",
        "repo": "flake-compat",
-        "rev": "b4a34015c698c7793d592d66adbab377907a2be8",
+        "rev": "8bf105319d44f6b9f0d764efa4fdef9f1cc9ba1c",
        "type": "github"
      },
      "original": {
-        "owner": "edolstra",
+        "owner": "nix-community",
        "repo": "flake-compat",
        "type": "github"
      }
    },
    "flake-utils": {
+      "inputs": {
+        "systems": "systems"
+      },
      "locked": {
-        "lastModified": 1659877975,
-        "narHash": "sha256-zllb8aq3YO3h8B/U0/J1WBgAL8EX5yWf5pMj3G0NAmc=",
+        "lastModified": 1681202837,
+        "narHash": "sha256-H+Rh19JDwRtpVPAWp64F+rlEtxUWBAQW28eAi3SRSzg=",
        "owner": "numtide",
        "repo": "flake-utils",
-        "rev": "c0e246b9b83f637f4681389ecabcb2681b4f3af0",
+        "rev": "cfacdce06f30d2b68473a46042957675eebb3401",
        "type": "github"
      },
      "original": {
@@ -56,12 +60,15 @@
      }
    },
    "flake-utils_2": {
+      "inputs": {
+        "systems": "systems_2"
+      },
      "locked": {
-        "lastModified": 1667395993,
-        "narHash": "sha256-nuEHfE/LcWyuSWnS8t12N1wc105Qtau+/OdUAjtQ0rA=",
+        "lastModified": 1681202837,
+        "narHash": "sha256-H+Rh19JDwRtpVPAWp64F+rlEtxUWBAQW28eAi3SRSzg=",
        "owner": "numtide",
        "repo": "flake-utils",
-        "rev": "5aed5285a952e0b949eb3ba02c12fa4fcfef535f",
+        "rev": "cfacdce06f30d2b68473a46042957675eebb3401",
        "type": "github"
      },
      "original": {
@@ -72,44 +79,58 @@
    },
    "nixpkgs": {
      "locked": {
-        "lastModified": 1665657542,
-        "narHash": "sha256-mojxNyzbvmp8NtVtxqiHGhRfjCALLfk9i/Uup68Y5q8=",
+        "lastModified": 1682109806,
+        "narHash": "sha256-d9g7RKNShMLboTWwukM+RObDWWpHKaqTYXB48clBWXI=",
        "owner": "NixOS",
        "repo": "nixpkgs",
-        "rev": "a3073c49bc0163fea6a121c276f526837672b555",
+        "rev": "2362848adf8def2866fabbffc50462e929d7fffb",
+        "type": "github"
+      },
+      "original": {
+        "owner": "NixOS",
+        "ref": "nixpkgs-unstable",
+        "repo": "nixpkgs",
+        "type": "github"
+      }
+    },
+    "nixpkgs_2": {
+      "locked": {
+        "lastModified": 1682423271,
+        "narHash": "sha256-WHhl1GiOij1ob4cTLL+yhqr+vFOUH8E5wAX8Ir8fvjE=",
+        "owner": "NixOS",
+        "repo": "nixpkgs",
+        "rev": "94517a501434a627c5d9e72ac6e7f26174b978d3",
        "type": "github"
      },
      "original": {
        "owner": "NixOS",
        "repo": "nixpkgs",
-        "rev": "a3073c49bc0163fea6a121c276f526837672b555",
+        "rev": "94517a501434a627c5d9e72ac6e7f26174b978d3",
        "type": "github"
      }
    },
    "root": {
      "inputs": {
        "cargo2nix": "cargo2nix",
-        "flake-utils": "flake-utils_2",
-        "nixpkgs": "nixpkgs"
-      }
-    },
-    "rust-overlay": {
-      "inputs": {
+        "flake-compat": "flake-compat",
        "flake-utils": [
          "cargo2nix",
          "flake-utils"
        ],
-        "nixpkgs": [
-          "cargo2nix",
-          "nixpkgs"
-        ]
+        "nixpkgs": "nixpkgs_2"
+      }
+    },
+    "rust-overlay": {
+      "inputs": {
+        "flake-utils": "flake-utils_2",
+        "nixpkgs": "nixpkgs"
      },
      "locked": {
-        "lastModified": 1664247556,
-        "narHash": "sha256-J4vazHU3609ekn7dr+3wfqPo5WGlZVAgV7jfux352L0=",
+        "lastModified": 1682389182,
+        "narHash": "sha256-8t2nmFnH+8V48+IJsf8AK51ebXNlVbOSVYOpiqJKvJE=",
        "owner": "oxalica",
        "repo": "rust-overlay",
-        "rev": "524db9c9ea7bc7743bb74cdd45b6d46ea3fcc2ab",
+        "rev": "74f1a64dd28faeeb85ef081f32cad2989850322c",
        "type": "github"
      },
      "original": {
@@ -117,6 +138,36 @@
        "repo": "rust-overlay",
        "type": "github"
      }
+    },
+    "systems": {
+      "locked": {
+        "lastModified": 1681028828,
+        "narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
+        "owner": "nix-systems",
+        "repo": "default",
+        "rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
+        "type": "github"
+      },
+      "original": {
+        "owner": "nix-systems",
+        "repo": "default",
+        "type": "github"
+      }
+    },
+    "systems_2": {
+      "locked": {
+        "lastModified": 1681028828,
+        "narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
+        "owner": "nix-systems",
+        "repo": "default",
+        "rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
+        "type": "github"
+      },
+      "original": {
+        "owner": "nix-systems",
+        "repo": "default",
+        "type": "github"
+      }
    }
  },
  "root": "root",
@@ -2,20 +2,37 @@
  description =
    "Garage, an S3-compatible distributed object store for self-hosted deployments";

+  # Nixpkgs unstable as of 2023-04-25, has rustc v1.68
  inputs.nixpkgs.url =
-    "github:NixOS/nixpkgs/a3073c49bc0163fea6a121c276f526837672b555";
+    "github:NixOS/nixpkgs/94517a501434a627c5d9e72ac6e7f26174b978d3";
+
+  inputs.flake-compat.url = "github:nix-community/flake-compat";
+
  inputs.cargo2nix = {
    # As of 2022-10-18: two small patches over unstable branch, one for clippy and one to fix feature detection
    url = "github:Alexis211/cargo2nix/a7a61179b66054904ef6a195d8da736eaaa06c36";
-    inputs.nixpkgs.follows = "nixpkgs";
-  };
-  inputs.flake-utils.url = "github:numtide/flake-utils";

-  outputs = { self, nixpkgs, cargo2nix, flake-utils }:
+    # As of 2023-04-25:
+    # - my two patches were merged into unstable (one for clippy and one to "fix" feature detection)
+    # - rustc v1.66
+    # url = "github:cargo2nix/cargo2nix/8fb57a670f7993bfc24099c33eb9c5abb51f29a2";
+
+    # Rust overlay as of 2023-04-25
+    inputs.rust-overlay.url =
+      "github:oxalica/rust-overlay/74f1a64dd28faeeb85ef081f32cad2989850322c";
+
+    inputs.nixpkgs.follows = "nixpkgs";
+    inputs.flake-compat.follows = "flake-compat";
+  };
+
+  inputs.flake-utils.follows = "cargo2nix/flake-utils";
+
+  outputs = { self, nixpkgs, cargo2nix, flake-utils, ... }:
    let
      git_version = self.lastModifiedDate;
      compile = import ./nix/compile.nix;
-    in flake-utils.lib.eachDefaultSystem (system:
+    in
+    flake-utils.lib.eachDefaultSystem (system:
      let pkgs = nixpkgs.legacyPackages.${system};
      in {
        packages = {
@@ -1,19 +1,17 @@
-rec {
-  # * Fixed dependencies
-  pkgsSrc = fetchTarball {
-    # As of 2022-10-13
-    url =
-      "https://github.com/NixOS/nixpkgs/archive/a3073c49bc0163fea6a121c276f526837672b555.zip";
-    sha256 = "1bz632psfbpmicyzjb8b4265y50shylccvfm6ry6mgnv5hvz324s";
-  };
-  cargo2nixSrc = fetchGit {
-    # As of 2022-10-18: two small patches over unstable branch, one for clippy and one to fix feature detection
-    url = "https://github.com/Alexis211/cargo2nix";
-    ref = "custom_unstable";
-    rev = "a7a61179b66054904ef6a195d8da736eaaa06c36";
+let
+  lock = builtins.fromJSON (builtins.readFile ../flake.lock);
+
+  inherit (lock.nodes.flake-compat.locked) owner repo rev narHash;
+
+  flake-compat = fetchTarball {
+    url = "https://github.com/${owner}/${repo}/archive/${rev}.tar.gz";
+    sha256 = narHash;
  };

-  # * Shared objects
-  cargo2nix = import cargo2nixSrc;
+  flake = (import flake-compat { system = builtins.currentSystem; src = ../.; });
+in
+rec {
+  pkgsSrc = flake.defaultNix.inputs.nixpkgs;
+  cargo2nix = flake.defaultNix.inputs.cargo2nix;
  cargo2nixOverlay = cargo2nix.overlays.default;
 }
@@ -32,7 +32,7 @@ let
  */
  toolchainOptions = if target == null || target == "x86_64-unknown-linux-musl"
  || target == "aarch64-unknown-linux-musl" then {
-    rustVersion = "1.63.0";
+    rustVersion = "1.68.0";
    extraRustComponents = [ "clippy" ];
  } else {
    rustToolchain = pkgs.symlinkJoin {
@@ -1,13 +1,13 @@
 pkgs:
 pkgs.buildGoModule rec {
  pname = "kaniko";
-  version = "1.6.0";
+  version = "1.9.2";

  src = pkgs.fetchFromGitHub {
    owner = "GoogleContainerTools";
    repo = "kaniko";
    rev = "v${version}";
-    sha256 = "1fnclr556avxay6pvgw5ya3xbxfnf2gv4njq2hr4fd6fcjyslq5h";
+    sha256 = "dXQ0/o1qISv+sjNVIpfF85bkbM9sGOGwqVbWZpMWfMY=";
  };

  vendorSha256 = null;
@@ -15,10 +15,10 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 0.3.0
+version: 0.4.1

 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to
 # follow Semantic Versioning. They should reflect the version the application is using.
 # It is recommended to use it with quotes.
-appVersion: "v0.7.2.1"
+appVersion: "v0.8.3"
@@ -4,9 +4,28 @@

 # Garage configuration. These values go to garage.toml
 garage:
+  # Can be changed for better performance on certain systems
+  # https://garagehq.deuxfleurs.fr/documentation/reference-manual/configuration/#db-engine-since-v0-8-0
+  dbEngine: "sled"
+
+  # Defaults is 1MB
+  # An increase can result in better performance in certain scenarios
+  # https://garagehq.deuxfleurs.fr/documentation/reference-manual/configuration/#block-size
+  blockSize: "1048576"
+
+  # Tuning parameters for the sled DB engine
+  # https://garagehq.deuxfleurs.fr/documentation/reference-manual/configuration/#sled-cache-capacity
+  sledCacheCapacity: "134217728"
+  sledFlushEveryMs: "2000"
+
  # Default to 3 replicas, see the replication_mode section at
-  # https://garagehq.deuxfleurs.fr/documentation/reference-manual/configuration/
+  # https://garagehq.deuxfleurs.fr/documentation/reference-manual/configuration/#replication-mode
  replicationMode: "3"
+
+  # zstd compression level of stored blocks
+  # https://garagehq.deuxfleurs.fr/documentation/reference-manual/configuration/#compression-level
+  compressionLevel: "1"
+
  rpcBindAddr: "[::]:3901"
  # If not given, a random secret will be generated and stored in a Secret object
  rpcSecret: ""
@@ -27,8 +46,19 @@ garage:
    metadata_dir = "/mnt/meta"
    data_dir = "/mnt/data"

+    db_engine = "{{ .Values.garage.dbEngine }}"
+
+    block_size = {{ .Values.garage.blockSize }}
+
+    {{- if eq .Values.garage.dbEngine "sled"}}
+    sled_cache_capacity = {{ .Values.garage.sledCacheCapacity }}
+    sled_flush_every_ms = {{ .Values.garage.sledFlushEveryMs }}
+    {{- end }}
+
    replication_mode = "{{ .Values.garage.replicationMode }}"

+    compression_level = {{ .Values.garage.compressionLevel }}
+
    rpc_bind_addr = "{{ .Values.garage.rpcBindAddr }}"
    # rpc_secret will be populated by the init container from a k8s secret object
    rpc_secret = "__RPC_SECRET_REPLACE__"
@@ -49,7 +79,7 @@ garage:
    bind_addr = "[::]:3902"
    root_domain = "{{ .Values.garage.s3.web.rootDomain }}"
    index = "{{ .Values.garage.s3.web.index }}"
-    
+
    [admin]
    api_bind_addr = "[::]:3903"
    {{- if .Values.monitoring.tracing.sink }}
@@ -211,4 +241,4 @@ monitoring:
      scrapeTimeout: 10s
      relabelings: []
  tracing:
-    sink: ""
+    sink: ""
@@ -1,6 +1,6 @@
 [package]
 name = "garage_api"
-version = "0.8.2"
+version = "0.8.3"
 authors = ["Alex Auvolat <alex@adnab.me>"]
 edition = "2018"
 license = "AGPL-3.0"
@@ -14,11 +14,11 @@ path = "lib.rs"
 # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

 [dependencies]
-garage_model = { version = "0.8.2", path = "../model" }
-garage_table = { version = "0.8.2", path = "../table" }
-garage_block = { version = "0.8.2", path = "../block" }
-garage_util = { version = "0.8.2", path = "../util" }
-garage_rpc = { version = "0.8.2", path = "../rpc" }
+garage_model.workspace = true
+garage_table.workspace = true
+garage_block.workspace = true
+garage_util.workspace = true
+garage_rpc.workspace = true

 async-trait = "0.1.7"
 base64 = "0.21"
@@ -28,7 +28,7 @@ crypto-common = "0.1"
 err-derive = "0.3"
 hex = "0.4"
 hmac = "0.12"
-idna = "0.3"
+idna = "0.4"
 tracing = "0.1"
 md-5 = "0.10"
 nom = "7.1"
@@ -47,11 +47,11 @@ http-range = "0.1"
 hyper = { version = "0.14", features = ["server", "http1", "runtime", "tcp", "stream"] }
 multer = "2.0"
 percent-encoding = "2.1.0"
-roxmltree = "0.14"
+roxmltree = "0.18"
 serde = { version = "1.0", features = ["derive"] }
 serde_bytes = "0.11"
 serde_json = "1.0"
-quick-xml = { version = "0.21", features = [ "serialize" ] }
+quick-xml = { version = "0.26", features = [ "serialize" ] }
 url = "2.3"

 opentelemetry = "0.17"
@@ -26,6 +26,7 @@ use crate::admin::cluster::*;
 use crate::admin::error::*;
 use crate::admin::key::*;
 use crate::admin::router::{Authorization, Endpoint};
+use crate::helpers::host_to_bucket;

 pub struct AdminApiServer {
 	garage: Arc<Garage>,
@@ -78,10 +79,7 @@ impl AdminApiServer {
 			.body(Body::empty())?)
 	}

-	async fn handle_check_website_enabled(
-		&self,
-		req: Request<Body>,
-	) -> Result<Response<Body>, Error> {
+	async fn handle_check_domain(&self, req: Request<Body>) -> Result<Response<Body>, Error> {
 		let query_params: HashMap<String, String> = req
 			.uri()
 			.query()
@@ -102,12 +100,56 @@ impl AdminApiServer {
 			.get("domain")
 			.ok_or_internal_error("Could not parse domain query string")?;

-		let bucket_id = self
+		if self.check_domain(domain).await? {
+			Ok(Response::builder()
+				.status(StatusCode::OK)
+				.body(Body::from(format!(
+					"Domain '{domain}' is managed by Garage"
+				)))?)
+		} else {
+			Err(Error::bad_request(format!(
+				"Domain '{domain}' is not managed by Garage"
+			)))
+		}
+	}
+
+	async fn check_domain(&self, domain: &str) -> Result<bool, Error> {
+		// Resolve bucket from domain name, inferring if the website must be activated for the
+		// domain to be valid.
+		let (bucket_name, must_check_website) = if let Some(bname) = self
+			.garage
+			.config
+			.s3_api
+			.root_domain
+			.as_ref()
+			.and_then(|rd| host_to_bucket(domain, rd))
+		{
+			(bname.to_string(), false)
+		} else if let Some(bname) = self
+			.garage
+			.config
+			.s3_web
+			.as_ref()
+			.and_then(|sw| host_to_bucket(domain, sw.root_domain.as_str()))
+		{
+			(bname.to_string(), true)
+		} else {
+			(domain.to_string(), true)
+		};
+
+		let bucket_id = match self
 			.garage
 			.bucket_helper()
-			.resolve_global_bucket_name(&domain)
+			.resolve_global_bucket_name(&bucket_name)
 			.await?
-			.ok_or(HelperError::NoSuchBucket(domain.to_string()))?;
+		{
+			Some(bucket_id) => bucket_id,
+			None => return Ok(false),
+		};
+
+		if !must_check_website {
+			return Ok(true);
+		}

 		let bucket = self
 			.garage
@@ -119,16 +161,8 @@ impl AdminApiServer {
 		let bucket_website_config = bucket_state.website_config.get();

 		match bucket_website_config {
-			Some(_v) => {
-				Ok(Response::builder()
-					.status(StatusCode::OK)
-					.body(Body::from(format!(
-						"Bucket '{domain}' is authorized for website hosting"
-					)))?)
-			}
-			None => Err(Error::bad_request(format!(
-				"Bucket '{domain}' is not authorized for website hosting"
-			))),
+			Some(_v) => Ok(true),
+			None => Ok(false),
 		}
 	}

@@ -229,7 +263,7 @@ impl ApiHandler for AdminApiServer {

 		match endpoint {
 			Endpoint::Options => self.handle_options(&req),
-			Endpoint::CheckWebsiteEnabled => self.handle_check_website_enabled(req).await,
+			Endpoint::CheckDomain => self.handle_check_domain(req).await,
 			Endpoint::Health => self.handle_health(),
 			Endpoint::Metrics => self.handle_metrics(),
 			Endpoint::GetClusterStatus => handle_get_cluster_status(&self.garage).await,
@@ -183,8 +183,8 @@ async fn bucket_info_results(
 				}
 			}),
 			keys: relevant_keys
-				.into_iter()
-				.map(|(_, key)| {
+				.into_values()
+				.map(|key| {
 					let p = key.state.as_option().unwrap();
 					GetBucketInfoKey {
 						access_key_id: key.key_id,
@@ -183,8 +183,8 @@ async fn key_info_results(garage: &Arc<Garage>, key: Key) -> Result<Response<Bod
 			create_bucket: *key_state.allow_create_bucket.get(),
 		},
 		buckets: relevant_buckets
-			.into_iter()
-			.map(|(_, bucket)| {
+			.into_values()
+			.map(|bucket| {
 				let state = bucket.state.as_option().unwrap();
 				KeyInfoBucketResult {
 					id: hex::encode(bucket.id),
@@ -17,7 +17,7 @@ router_match! {@func
 #[derive(Debug, Clone, PartialEq, Eq)]
 pub enum Endpoint {
 	Options,
-	CheckWebsiteEnabled,
+	CheckDomain,
 	Health,
 	Metrics,
 	GetClusterStatus,
@@ -92,7 +92,7 @@ impl Endpoint {

 		let res = router_match!(@gen_path_parser (req.method(), path, query) [
 			OPTIONS _ => Options,
-			GET "/check" => CheckWebsiteEnabled,
+			GET "/check" => CheckDomain,
 			GET "/health" => Health,
 			GET "/metrics" => Metrics,
 			GET "/v0/status" => GetClusterStatus,
@@ -138,7 +138,7 @@ impl Endpoint {
 	pub fn authorization_type(&self) -> Authorization {
 		match self {
 			Self::Health => Authorization::None,
-			Self::CheckWebsiteEnabled => Authorization::None,
+			Self::CheckDomain => Authorization::None,
 			Self::Metrics => Authorization::MetricsToken,
 			_ => Authorization::AdminToken,
 		}
@@ -128,7 +128,7 @@ impl<A: ApiHandler> ApiServer<A> {
 		let uri = req.uri().clone();

 		if let Ok(forwarded_for_ip_addr) =
-			forwarded_headers::handle_forwarded_for_headers(&req.headers())
+			forwarded_headers::handle_forwarded_for_headers(req.headers())
 		{
 			info!(
 				"{} (via {}) {} {}",
@@ -282,8 +282,8 @@ pub(crate) async fn handle_poll_range(
 	if let Some((items, seen_marker)) = resp {
 		let resp = PollRangeResponse {
 			items: items
-				.into_iter()
-				.map(|(_k, i)| ReadBatchResponseItem::from(i))
+				.into_values()
+				.map(ReadBatchResponseItem::from)
 				.collect::<Vec<_>>(),
 			seen_marker,
 		};
@@ -443,7 +443,7 @@ fn body_from_blocks_range(
 	// block.part_number, which is not the same in the case of a multipart upload)
 	let mut blocks: Vec<(VersionBlock, u64)> = Vec::with_capacity(std::cmp::min(
 		all_blocks.len(),
-		4 + ((end - begin) / std::cmp::max(all_blocks[0].1.size as u64, 1024)) as usize,
+		4 + ((end - begin) / std::cmp::max(all_blocks[0].1.size, 1024)) as usize,
 	));
 	let mut block_offset: u64 = 0;
 	for (_, b) in all_blocks.iter() {
@@ -454,7 +454,7 @@ fn body_from_blocks_range(
 		if block_offset < end && block_offset + b.size > begin {
 			blocks.push((*b, block_offset));
 		}
-		block_offset += b.size as u64;
+		block_offset += b.size;
 	}

 	let order_stream = OrderTag::stream();
@@ -30,7 +30,7 @@ pub async fn handle_post_object(
 		.get(header::CONTENT_TYPE)
 		.and_then(|ct| ct.to_str().ok())
 		.and_then(|ct| multer::parse_boundary(ct).ok())
-		.ok_or_bad_request("Counld not get multipart boundary")?;
+		.ok_or_bad_request("Could not get multipart boundary")?;

 	// 16k seems plenty for a header. 5G is the max size of a single part, so it seems reasonable
 	// for a PostObject
@@ -64,15 +64,13 @@ pub async fn handle_post_object(
 				"tag" => (/* tag need to be reencoded, but we don't support them yet anyway */),
 				"acl" => {
 					if params.insert("x-amz-acl", content).is_some() {
-						return Err(Error::bad_request(
-							"Field 'acl' provided more than one time",
-						));
+						return Err(Error::bad_request("Field 'acl' provided more than once"));
 					}
 				}
 				_ => {
 					if params.insert(&name, content).is_some() {
 						return Err(Error::bad_request(format!(
-							"Field '{}' provided more than one time",
+							"Field '{}' provided more than once",
 							name
 						)));
 					}
@@ -149,7 +147,7 @@ pub async fn handle_post_object(
 		.ok_or_bad_request("Invalid expiration date")?
 		.into();
 	if Utc::now() - expiration > Duration::zero() {
-		return Err(Error::bad_request("Expiration date is in the paste"));
+		return Err(Error::bad_request("Expiration date is in the past"));
 	}

 	let mut conditions = decoded_policy.into_conditions()?;
@@ -330,7 +328,7 @@ impl Policy {
 					if map.len() != 1 {
 						return Err(Error::bad_request("Invalid policy item"));
 					}
-					let (mut k, v) = map.into_iter().next().expect("size was verified");
+					let (mut k, v) = map.into_iter().next().expect("Size could not be verified");
 					k.make_ascii_lowercase();
 					params.entry(k).or_default().push(Operation::Equal(v));
 				}
@@ -19,7 +19,7 @@ use crate::signature::error::*;

 pub async fn check_payload_signature(
 	garage: &Garage,
-	service: &str,
+	service: &'static str,
 	request: &Request<Body>,
 ) -> Result<(Option<Key>, Option<Hash>), Error> {
 	let mut headers = HashMap::new();
@@ -51,6 +51,7 @@ pub async fn check_payload_signature(
 	};

 	let canonical_request = canonical_request(
+		service,
 		request.method(),
 		request.uri(),
 		&headers,
@@ -184,7 +185,7 @@ fn parse_query_authorization(

 	if duration > 7 * 24 * 3600 {
 		return Err(Error::bad_request(
-			"X-Amz-Exprires may not exceed a week".to_string(),
+			"X-Amz-Expires may not exceed a week".to_string(),
 		));
 	}

@@ -210,7 +211,7 @@ fn parse_query_authorization(
 fn parse_credential(cred: &str) -> Result<(String, String), Error> {
 	let first_slash = cred
 		.find('/')
-		.ok_or_bad_request("Credentials does not contain / in authorization field")?;
+		.ok_or_bad_request("Credentials does not contain '/' in authorization field")?;
 	let (key_id, scope) = cred.split_at(first_slash);
 	Ok((
 		key_id.to_string(),
@@ -231,15 +232,50 @@ pub fn string_to_sign(datetime: &DateTime<Utc>, scope_string: &str, canonical_re
 }

 pub fn canonical_request(
+	service: &'static str,
 	method: &Method,
 	uri: &hyper::Uri,
 	headers: &HashMap<String, String>,
 	signed_headers: &str,
 	content_sha256: &str,
 ) -> String {
+	// There seems to be evidence that in AWSv4 signatures, the path component is url-encoded
+	// a second time when building the canonical request, as specified in this documentation page:
+	// -> https://docs.aws.amazon.com/rolesanywhere/latest/userguide/authentication-sign-process.html
+	// However this documentation page is for a specific service ("roles anywhere"), and
+	// in the S3 service we know for a fact that there is no double-urlencoding, because all of
+	// the tests we made with external software work without it.
+	//
+	// The theory is that double-urlencoding occurs for all services except S3,
+	// which is what is implemented in rusoto_signature:
+	// -> https://docs.rs/rusoto_signature/latest/src/rusoto_signature/signature.rs.html#464
+	//
+	// Digging into the code of the official AWS Rust SDK, we learn that double-URI-encoding can
+	// be set or unset on a per-request basis (the signature crates, aws-sigv4 and aws-sig-auth,
+	// are agnostic to this). Grepping the codebase confirms that S3 is the only API for which
+	// double_uri_encode is set to false, meaning it is true (its default value) for all other
+	// AWS services. We will therefore implement this behavior in Garage as well.
+	//
+	// Note that this documentation page, which is touted as the "authoritative reference" on
+	// AWSv4 signatures, makes no mention of either single- or double-urlencoding:
+	// -> https://docs.aws.amazon.com/IAM/latest/UserGuide/create-signed-request.html
+	// This page of the S3 documentation does also not mention anything specific:
+	// -> https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.html
+	//
+	// Note that there is also the issue of path normalization, which I hope is unrelated to the
+	// one of URI-encoding. At least in aws-sigv4 both parameters can be set independently,
+	// and rusoto_signature does not seem to do any effective path normalization, even though
+	// it mentions it in the comments (same link to the souce code as above).
+	// We make the explicit choice of NOT normalizing paths in the K2V API because doing so
+	// would make non-normalized paths invalid K2V partition keys, and we don't want that.
+	let path: std::borrow::Cow<str> = if service != "s3" {
+		uri_encode(uri.path(), false).into()
+	} else {
+		uri.path().into()
+	};
 	[
 		method.as_str(),
-		uri.path(),
+		&path,
 		&canonical_query_string(uri),
 		&canonical_header_string(headers, signed_headers),
 		"",
@@ -1,6 +1,6 @@
 [package]
 name = "garage_block"
-version = "0.8.2"
+version = "0.8.3"
 authors = ["Alex Auvolat <alex@adnab.me>"]
 edition = "2018"
 license = "AGPL-3.0"
@@ -14,10 +14,10 @@ path = "lib.rs"
 # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

 [dependencies]
-garage_db = { version = "0.8.2", path = "../db" }
-garage_rpc = { version = "0.8.2", path = "../rpc" }
-garage_util = { version = "0.8.2", path = "../util" }
-garage_table = { version = "0.8.2", path = "../table" }
+garage_db.workspace = true
+garage_rpc.workspace = true
+garage_util.workspace = true
+garage_table.workspace = true

 opentelemetry = "0.17"

@@ -28,7 +28,7 @@ hex = "0.4"
 tracing = "0.1"
 rand = "0.8"

-async-compression = { version = "0.3", features = ["tokio", "zstd"] }
+async-compression = { version = "0.4", features = ["tokio", "zstd"] }
 zstd = { version = "0.12", default-features = false }

 serde = { version = "1.0", default-features = false, features = ["derive", "rc"] }
@@ -37,7 +37,7 @@ serde_bytes = "0.11"
 futures = "0.3"
 futures-util = "0.3"
 tokio = { version = "1.0", default-features = false, features = ["rt", "rt-multi-thread", "io-util", "net", "time", "macros", "sync", "signal", "fs"] }
-tokio-util = { version = "0.6", features = ["io"] }
+tokio-util = { version = "0.7", features = ["io"] }

 [features]
 system-libs = [ "zstd/pkg-config" ]
@@ -600,12 +600,32 @@ impl BlockManager {
 	/// Utility: check if block is stored compressed. Error if block is not stored
 	async fn is_block_compressed(&self, hash: &Hash) -> Result<bool, Error> {
 		let mut path = self.block_path(hash);
-		path.set_extension("zst");
-		if fs::metadata(&path).await.is_ok() {
-			return Ok(true);
+
+		// If compression is disabled on node - check for the raw block
+		// first and then a compressed one (as compression may have been
+		// previously enabled).
+		match self.compression_level {
+			None => {
+				if fs::metadata(&path).await.is_ok() {
+					return Ok(false);
+				}
+
+				path.set_extension("zst");
+
+				fs::metadata(&path).await.map(|_| true).map_err(Into::into)
+			}
+			_ => {
+				path.set_extension("zst");
+
+				if fs::metadata(&path).await.is_ok() {
+					return Ok(true);
+				}
+
+				path.set_extension("");
+
+				fs::metadata(&path).await.map(|_| false).map_err(Into::into)
+			}
 		}
-		path.set_extension("");
-		fs::metadata(&path).await.map(|_| false).map_err(Into::into)
 	}

 	async fn lock_mutate(&self, hash: &Hash) -> MutexGuard<'_, BlockManagerLocked> {
@@ -220,14 +220,12 @@ fn randomize_next_scrub_run_time(timestamp: u64) -> u64 {
 	// Take SCRUB_INTERVAL and mix in a random interval of 10 days to attempt to
 	// balance scrub load across different cluster nodes.

-	let next_run_timestamp = timestamp
+	timestamp
 		+ SCRUB_INTERVAL
 			.saturating_add(Duration::from_secs(
 				rand::thread_rng().gen_range(0..3600 * 24 * 10),
 			))
-			.as_millis() as u64;
-
-	next_run_timestamp
+			.as_millis() as u64
 }

 impl Default for ScrubWorkerPersisted {
@@ -241,18 +239,14 @@ impl Default for ScrubWorkerPersisted {
 	}
 }

+#[derive(Default)]
 enum ScrubWorkerState {
 	Running(BlockStoreIterator),
 	Paused(BlockStoreIterator, u64), // u64 = time when to resume scrub
+	#[default]
 	Finished,
 }

-impl Default for ScrubWorkerState {
-	fn default() -> Self {
-		ScrubWorkerState::Finished
-	}
-}
-
 #[derive(Debug)]
 pub enum ScrubWorkerCommand {
 	Start,
@@ -281,6 +275,7 @@ impl ScrubWorker {
 			ScrubWorkerCommand::Start => {
 				self.work = match std::mem::take(&mut self.work) {
 					ScrubWorkerState::Finished => {
+						info!("Scrub worker initializing, now performing datastore scrub");
 						let iterator = BlockStoreIterator::new(&self.manager);
 						ScrubWorkerState::Running(iterator)
 					}
@@ -394,12 +389,21 @@ impl Worker for ScrubWorker {
 						.tranquilizer
 						.tranquilize_worker(self.persister.get_with(|p| p.tranquility)))
 				} else {
+					let now = now_msec();
+					let next_scrub_timestamp = randomize_next_scrub_run_time(now);
+
 					self.persister.set_with(|p| {
-						p.time_last_complete_scrub = now_msec();
-						p.time_next_run_scrub = randomize_next_scrub_run_time(now_msec());
+						p.time_last_complete_scrub = now;
+						p.time_next_run_scrub = next_scrub_timestamp;
 					})?;
 					self.work = ScrubWorkerState::Finished;
 					self.tranquilizer.clear();
+
+					info!(
+						"Datastore scrub completed, next scrub scheduled for {}",
+						msec_to_rfc3339(next_scrub_timestamp)
+					);
+
 					Ok(WorkerState::Idle)
 				}
 			}
@@ -1,6 +1,6 @@
 [package]
 name = "garage_db"
-version = "0.8.2"
+version = "0.8.3"
 authors = ["Alex Auvolat <alex@adnab.me>"]
 edition = "2018"
 license = "AGPL-3.0"
@@ -22,19 +22,19 @@ hexdump = "0.1"
 tracing = "0.1"

 heed = { version = "0.11", default-features = false, features = ["lmdb"], optional = true }
-rusqlite = { version = "0.28", optional = true }
+rusqlite = { version = "0.29", optional = true }
 sled = { version = "0.34", optional = true }

 # cli deps
 clap = { version = "4.1", optional = true, features = ["derive", "env"] }
-pretty_env_logger = { version = "0.4", optional = true }
+pretty_env_logger = { version = "0.5", optional = true }

 [dev-dependencies]
 mktemp = "0.5"

 [features]
 default = [ "sled" ]
-bundled-libs = [ "rusqlite/bundled" ]
+bundled-libs = [ "rusqlite?/bundled" ]
 cli = ["clap", "pretty_env_logger"]
 lmdb = [ "heed" ]
 sqlite = [ "rusqlite" ]
@@ -349,6 +349,6 @@ pub fn recommended_map_size() -> usize {

 #[cfg(target_pointer_width = "32")]
 pub fn recommended_map_size() -> usize {
-	warn!("LMDB is not recommended on 32-bit systems, database size will be limited");
+	tracing::warn!("LMDB is not recommended on 32-bit systems, database size will be limited");
 	1usize << 30
 }
@@ -0,0 +1,12 @@
+[package]
+name = "format_table"
+version = "0.1.1"
+authors = ["Alex Auvolat <alex@adnab.me>"]
+edition = "2018"
+license = "AGPL-3.0"
+description = "Format tables with a stupid API"
+repository = "https://git.deuxfleurs.fr/Deuxfleurs/garage"
+readme = "README.md"
+
+[lib]
+path = "lib.rs"
@@ -0,0 +1,13 @@
+# `format_table`
+
+Format tables with a stupid API. [Documentation](https://docs.rs/format_table).
+
+Example:
+
+```rust
+let mut table = vec!["product\tquantity\tprice".to_string()];
+for (p, q, r) in [("tomato", 12, 15), ("potato", 10, 20), ("rice", 5, 12)] {
+	table.push(format!("{}\t{}\t{}", p, q, r));
+}
+format_table::format_table(table);
+```
@@ -1,3 +1,19 @@
+//! Format tables with a stupid API.
+//!
+//! Example:
+//!
+//! ```rust
+//! let mut table = vec!["product\tquantity\tprice".to_string()];
+//! for (p, q, r) in [("tomato", 12, 15), ("potato", 10, 20), ("rice", 5, 12)] {
+//!     table.push(format!("{}\t{}\t{}", p, q, r));
+//! }
+//! format_table::format_table(table);
+//! ```
+//!
+//! A table to be formatted is a `Vec<String>`, containing one string per line.
+//! Table columns in each line are separated by a `\t` character.
+
+/// Format a table and return the result as a string.
 pub fn format_table_to_string(data: Vec<String>) -> String {
 	let data = data
 		.iter()
@@ -27,6 +43,7 @@ pub fn format_table_to_string(data: Vec<String>) -> String {
 	out
 }

+/// Format a table and print the result to stdout.
 pub fn format_table(data: Vec<String>) {
 	print!("{}", format_table_to_string(data));
 }
@@ -1,6 +1,6 @@
 [package]
 name = "garage"
-version = "0.8.2"
+version = "0.8.3"
 authors = ["Alex Auvolat <alex@adnab.me>"]
 edition = "2018"
 license = "AGPL-3.0"
@@ -21,19 +21,20 @@ path = "tests/lib.rs"
 # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

 [dependencies]
-garage_db = { version = "0.8.2", path = "../db" }
-garage_api = { version = "0.8.2", path = "../api" }
-garage_block = { version = "0.8.2", path = "../block" }
-garage_model = { version = "0.8.2", path = "../model" }
-garage_rpc = { version = "0.8.2", path = "../rpc" }
-garage_table = { version = "0.8.2", path = "../table" }
-garage_util = { version = "0.8.2", path = "../util" }
-garage_web = { version = "0.8.2", path = "../web" }
+format_table.workspace = true
+garage_db.workspace = true
+garage_api.workspace = true
+garage_block.workspace = true
+garage_model.workspace = true
+garage_rpc.workspace = true
+garage_table.workspace = true
+garage_util.workspace = true
+garage_web.workspace = true

 backtrace = "0.3"
 bytes = "1.0"
-bytesize = "1.1"
-timeago = "0.4"
+bytesize = "1.2"
+timeago = { version = "0.4", default-features = false }
 parse_duration = "2.1"
 hex = "0.4"
 tracing = { version = "0.1" }
@@ -41,6 +42,7 @@ tracing-subscriber = { version = "0.3", features = ["env-filter"] }
 rand = "0.8"
 async-trait = "0.1.7"
 sodiumoxide = { version = "0.2.5-0", package = "kuska-sodiumoxide" }
+git-version = "0.3.4"

 serde = { version = "1.0", default-features = false, features = ["derive", "rc"] }
 serde_bytes = "0.11"
@@ -59,7 +61,8 @@ opentelemetry-otlp = { version = "0.10", optional = true }
 prometheus = { version = "0.13", optional = true }

 [dev-dependencies]
-aws-sdk-s3 = "0.19"
+aws-config = "0.55.2"
+aws-sdk-s3 = "0.28"
 chrono = "0.4"
 http = "0.2"
 hmac = "0.12"
@@ -71,6 +74,8 @@ assert-json-diff = "2.0"
 serde_json = "1.0"
 base64 = "0.21"

+k2v-client.workspace = true
+

 [features]
 default = [ "bundled-libs", "metrics", "sled", "k2v" ]
@@ -0,0 +1,160 @@
+use garage_util::data::*;
+
+use garage_table::*;
+
+use garage_model::helper::error::{Error, OkOrBadRequest};
+use garage_model::s3::object_table::*;
+use garage_model::s3::version_table::*;
+
+use crate::cli::*;
+
+use super::*;
+
+impl AdminRpcHandler {
+	pub(super) async fn handle_block_cmd(&self, cmd: &BlockOperation) -> Result<AdminRpc, Error> {
+		match cmd {
+			BlockOperation::ListErrors => Ok(AdminRpc::BlockErrorList(
+				self.garage.block_manager.list_resync_errors()?,
+			)),
+			BlockOperation::Info { hash } => self.handle_block_info(hash).await,
+			BlockOperation::RetryNow { all, blocks } => {
+				self.handle_block_retry_now(*all, blocks).await
+			}
+			BlockOperation::Purge { yes, blocks } => self.handle_block_purge(*yes, blocks).await,
+		}
+	}
+
+	async fn handle_block_info(&self, hash: &String) -> Result<AdminRpc, Error> {
+		let hash = hex::decode(hash).ok_or_bad_request("invalid hash")?;
+		let hash = Hash::try_from(&hash).ok_or_bad_request("invalid hash")?;
+		let refcount = self.garage.block_manager.get_block_rc(&hash)?;
+		let block_refs = self
+			.garage
+			.block_ref_table
+			.get_range(&hash, None, None, 10000, Default::default())
+			.await?;
+		let mut versions = vec![];
+		for br in block_refs {
+			if let Some(v) = self
+				.garage
+				.version_table
+				.get(&br.version, &EmptyKey)
+				.await?
+			{
+				versions.push(Ok(v));
+			} else {
+				versions.push(Err(br.version));
+			}
+		}
+		Ok(AdminRpc::BlockInfo {
+			hash,
+			refcount,
+			versions,
+		})
+	}
+
+	async fn handle_block_retry_now(
+		&self,
+		all: bool,
+		blocks: &[String],
+	) -> Result<AdminRpc, Error> {
+		if all {
+			if !blocks.is_empty() {
+				return Err(Error::BadRequest(
+					"--all was specified, cannot also specify blocks".into(),
+				));
+			}
+			let blocks = self.garage.block_manager.list_resync_errors()?;
+			for b in blocks.iter() {
+				self.garage.block_manager.resync.clear_backoff(&b.hash)?;
+			}
+			Ok(AdminRpc::Ok(format!(
+				"{} blocks returned in queue for a retry now (check logs to see results)",
+				blocks.len()
+			)))
+		} else {
+			for hash in blocks {
+				let hash = hex::decode(hash).ok_or_bad_request("invalid hash")?;
+				let hash = Hash::try_from(&hash).ok_or_bad_request("invalid hash")?;
+				self.garage.block_manager.resync.clear_backoff(&hash)?;
+			}
+			Ok(AdminRpc::Ok(format!(
+				"{} blocks returned in queue for a retry now (check logs to see results)",
+				blocks.len()
+			)))
+		}
+	}
+
+	async fn handle_block_purge(&self, yes: bool, blocks: &[String]) -> Result<AdminRpc, Error> {
+		if !yes {
+			return Err(Error::BadRequest(
+				"Pass the --yes flag to confirm block purge operation.".into(),
+			));
+		}
+
+		let mut obj_dels = 0;
+		let mut ver_dels = 0;
+
+		for hash in blocks {
+			let hash = hex::decode(hash).ok_or_bad_request("invalid hash")?;
+			let hash = Hash::try_from(&hash).ok_or_bad_request("invalid hash")?;
+			let block_refs = self
+				.garage
+				.block_ref_table
+				.get_range(&hash, None, None, 10000, Default::default())
+				.await?;
+
+			for br in block_refs {
+				let version = match self
+					.garage
+					.version_table
+					.get(&br.version, &EmptyKey)
+					.await?
+				{
+					Some(v) => v,
+					None => continue,
+				};
+
+				if let Some(object) = self
+					.garage
+					.object_table
+					.get(&version.bucket_id, &version.key)
+					.await?
+				{
+					let ov = object.versions().iter().rev().find(|v| v.is_complete());
+					if let Some(ov) = ov {
+						if ov.uuid == br.version {
+							let del_uuid = gen_uuid();
+							let deleted_object = Object::new(
+								version.bucket_id,
+								version.key.clone(),
+								vec![ObjectVersion {
+									uuid: del_uuid,
+									timestamp: ov.timestamp + 1,
+									state: ObjectVersionState::Complete(
+										ObjectVersionData::DeleteMarker,
+									),
+								}],
+							);
+							self.garage.object_table.insert(&deleted_object).await?;
+							obj_dels += 1;
+						}
+					}
+				}
+
+				if !version.deleted.get() {
+					let deleted_version =
+						Version::new(version.uuid, version.bucket_id, version.key.clone(), true);
+					self.garage.version_table.insert(&deleted_version).await?;
+					ver_dels += 1;
+				}
+			}
+		}
+		Ok(AdminRpc::Ok(format!(
+			"{} blocks were purged: {} object deletion markers added, {} versions marked deleted",
+			blocks.len(),
+			obj_dels,
+			ver_dels
+		)))
+	}
+}
@@ -0,0 +1,496 @@
+use std::collections::HashMap;
+use std::fmt::Write;
+
+use garage_util::crdt::*;
+use garage_util::time::*;
+
+use garage_table::*;
+
+use garage_model::bucket_alias_table::*;
+use garage_model::bucket_table::*;
+use garage_model::helper::error::{Error, OkOrBadRequest};
+use garage_model::permission::*;
+
+use crate::cli::*;
+
+use super::*;
+
+impl AdminRpcHandler {
+	pub(super) async fn handle_bucket_cmd(&self, cmd: &BucketOperation) -> Result<AdminRpc, Error> {
+		match cmd {
+			BucketOperation::List => self.handle_list_buckets().await,
+			BucketOperation::Info(query) => self.handle_bucket_info(query).await,
+			BucketOperation::Create(query) => self.handle_create_bucket(&query.name).await,
+			BucketOperation::Delete(query) => self.handle_delete_bucket(query).await,
+			BucketOperation::Alias(query) => self.handle_alias_bucket(query).await,
+			BucketOperation::Unalias(query) => self.handle_unalias_bucket(query).await,
+			BucketOperation::Allow(query) => self.handle_bucket_allow(query).await,
+			BucketOperation::Deny(query) => self.handle_bucket_deny(query).await,
+			BucketOperation::Website(query) => self.handle_bucket_website(query).await,
+			BucketOperation::SetQuotas(query) => self.handle_bucket_set_quotas(query).await,
+			BucketOperation::CleanupIncompleteUploads(query) => {
+				self.handle_bucket_cleanup_incomplete_uploads(query).await
+			}
+		}
+	}
+
+	async fn handle_list_buckets(&self) -> Result<AdminRpc, Error> {
+		let buckets = self
+			.garage
+			.bucket_table
+			.get_range(
+				&EmptyKey,
+				None,
+				Some(DeletedFilter::NotDeleted),
+				10000,
+				EnumerationOrder::Forward,
+			)
+			.await?;
+
+		Ok(AdminRpc::BucketList(buckets))
+	}
+
+	async fn handle_bucket_info(&self, query: &BucketOpt) -> Result<AdminRpc, Error> {
+		let bucket_id = self
+			.garage
+			.bucket_helper()
+			.resolve_global_bucket_name(&query.name)
+			.await?
+			.ok_or_bad_request("Bucket not found")?;
+
+		let bucket = self
+			.garage
+			.bucket_helper()
+			.get_existing_bucket(bucket_id)
+			.await?;
+
+		let counters = self
+			.garage
+			.object_counter_table
+			.table
+			.get(&bucket_id, &EmptyKey)
+			.await?
+			.map(|x| x.filtered_values(&self.garage.system.ring.borrow()))
+			.unwrap_or_default();
+
+		let mut relevant_keys = HashMap::new();
+		for (k, _) in bucket
+			.state
+			.as_option()
+			.unwrap()
+			.authorized_keys
+			.items()
+			.iter()
+		{
+			if let Some(key) = self
+				.garage
+				.key_table
+				.get(&EmptyKey, k)
+				.await?
+				.filter(|k| !k.is_deleted())
+			{
+				relevant_keys.insert(k.clone(), key);
+			}
+		}
+		for ((k, _), _, _) in bucket
+			.state
+			.as_option()
+			.unwrap()
+			.local_aliases
+			.items()
+			.iter()
+		{
+			if relevant_keys.contains_key(k) {
+				continue;
+			}
+			if let Some(key) = self.garage.key_table.get(&EmptyKey, k).await? {
+				relevant_keys.insert(k.clone(), key);
+			}
+		}
+
+		Ok(AdminRpc::BucketInfo {
+			bucket,
+			relevant_keys,
+			counters,
+		})
+	}
+
+	#[allow(clippy::ptr_arg)]
+	async fn handle_create_bucket(&self, name: &String) -> Result<AdminRpc, Error> {
+		if !is_valid_bucket_name(name) {
+			return Err(Error::BadRequest(format!(
+				"{}: {}",
+				name, INVALID_BUCKET_NAME_MESSAGE
+			)));
+		}
+
+		if let Some(alias) = self.garage.bucket_alias_table.get(&EmptyKey, name).await? {
+			if alias.state.get().is_some() {
+				return Err(Error::BadRequest(format!("Bucket {} already exists", name)));
+			}
+		}
+
+		// ---- done checking, now commit ----
+
+		let bucket = Bucket::new();
+		self.garage.bucket_table.insert(&bucket).await?;
+
+		self.garage
+			.bucket_helper()
+			.set_global_bucket_alias(bucket.id, name)
+			.await?;
+
+		Ok(AdminRpc::Ok(format!("Bucket {} was created.", name)))
+	}
+
+	async fn handle_delete_bucket(&self, query: &DeleteBucketOpt) -> Result<AdminRpc, Error> {
+		let helper = self.garage.bucket_helper();
+
+		let bucket_id = helper
+			.resolve_global_bucket_name(&query.name)
+			.await?
+			.ok_or_bad_request("Bucket not found")?;
+
+		// Get the alias, but keep in minde here the bucket name
+		// given in parameter can also be directly the bucket's ID.
+		// In that case bucket_alias will be None, and
+		// we can still delete the bucket if it has zero aliases
+		// (a condition which we try to prevent but that could still happen somehow).
+		// We just won't try to delete an alias entry because there isn't one.
+		let bucket_alias = self
+			.garage
+			.bucket_alias_table
+			.get(&EmptyKey, &query.name)
+			.await?;
+
+		// Check bucket doesn't have other aliases
+		let mut bucket = helper.get_existing_bucket(bucket_id).await?;
+		let bucket_state = bucket.state.as_option().unwrap();
+		if bucket_state
+			.aliases
+			.items()
+			.iter()
+			.filter(|(_, _, active)| *active)
+			.any(|(name, _, _)| name != &query.name)
+		{
+			return Err(Error::BadRequest(format!("Bucket {} still has other global aliases. Use `bucket unalias` to delete them one by one.", query.name)));
+		}
+		if bucket_state
+			.local_aliases
+			.items()
+			.iter()
+			.any(|(_, _, active)| *active)
+		{
+			return Err(Error::BadRequest(format!("Bucket {} still has other local aliases. Use `bucket unalias` to delete them one by one.", query.name)));
+		}
+
+		// Check bucket is empty
+		if !helper.is_bucket_empty(bucket_id).await? {
+			return Err(Error::BadRequest(format!(
+				"Bucket {} is not empty",
+				query.name
+			)));
+		}
+
+		if !query.yes {
+			return Err(Error::BadRequest(
+				"Add --yes flag to really perform this operation".to_string(),
+			));
+		}
+
+		// --- done checking, now commit ---
+		// 1. delete authorization from keys that had access
+		for (key_id, _) in bucket.authorized_keys() {
+			helper
+				.set_bucket_key_permissions(bucket.id, key_id, BucketKeyPerm::NO_PERMISSIONS)
+				.await?;
+		}
+
+		// 2. delete bucket alias
+		if bucket_alias.is_some() {
+			helper
+				.purge_global_bucket_alias(bucket_id, &query.name)
+				.await?;
+		}
+
+		// 3. delete bucket
+		bucket.state = Deletable::delete();
+		self.garage.bucket_table.insert(&bucket).await?;
+
+		Ok(AdminRpc::Ok(format!("Bucket {} was deleted.", query.name)))
+	}
+
+	async fn handle_alias_bucket(&self, query: &AliasBucketOpt) -> Result<AdminRpc, Error> {
+		let helper = self.garage.bucket_helper();
+		let key_helper = self.garage.key_helper();
+
+		let bucket_id = helper
+			.resolve_global_bucket_name(&query.existing_bucket)
+			.await?
+			.ok_or_bad_request("Bucket not found")?;
+
+		if let Some(key_pattern) = &query.local {
+			let key = key_helper.get_existing_matching_key(key_pattern).await?;
+
+			helper
+				.set_local_bucket_alias(bucket_id, &key.key_id, &query.new_name)
+				.await?;
+			Ok(AdminRpc::Ok(format!(
+				"Alias {} now points to bucket {:?} in namespace of key {}",
+				query.new_name, bucket_id, key.key_id
+			)))
+		} else {
+			helper
+				.set_global_bucket_alias(bucket_id, &query.new_name)
+				.await?;
+			Ok(AdminRpc::Ok(format!(
+				"Alias {} now points to bucket {:?}",
+				query.new_name, bucket_id
+			)))
+		}
+	}
+
+	async fn handle_unalias_bucket(&self, query: &UnaliasBucketOpt) -> Result<AdminRpc, Error> {
+		let helper = self.garage.bucket_helper();
+		let key_helper = self.garage.key_helper();
+
+		if let Some(key_pattern) = &query.local {
+			let key = key_helper.get_existing_matching_key(key_pattern).await?;
+
+			let bucket_id = key
+				.state
+				.as_option()
+				.unwrap()
+				.local_aliases
+				.get(&query.name)
+				.cloned()
+				.flatten()
+				.ok_or_bad_request("Bucket not found")?;
+
+			helper
+				.unset_local_bucket_alias(bucket_id, &key.key_id, &query.name)
+				.await?;
+
+			Ok(AdminRpc::Ok(format!(
+				"Alias {} no longer points to bucket {:?} in namespace of key {}",
+				&query.name, bucket_id, key.key_id
+			)))
+		} else {
+			let bucket_id = helper
+				.resolve_global_bucket_name(&query.name)
+				.await?
+				.ok_or_bad_request("Bucket not found")?;
+
+			helper
+				.unset_global_bucket_alias(bucket_id, &query.name)
+				.await?;
+
+			Ok(AdminRpc::Ok(format!(
+				"Alias {} no longer points to bucket {:?}",
+				&query.name, bucket_id
+			)))
+		}
+	}
+
+	async fn handle_bucket_allow(&self, query: &PermBucketOpt) -> Result<AdminRpc, Error> {
+		let helper = self.garage.bucket_helper();
+		let key_helper = self.garage.key_helper();
+
+		let bucket_id = helper
+			.resolve_global_bucket_name(&query.bucket)
+			.await?
+			.ok_or_bad_request("Bucket not found")?;
+		let key = key_helper
+			.get_existing_matching_key(&query.key_pattern)
+			.await?;
+
+		let allow_read = query.read || key.allow_read(&bucket_id);
+		let allow_write = query.write || key.allow_write(&bucket_id);
+		let allow_owner = query.owner || key.allow_owner(&bucket_id);
+
+		helper
+			.set_bucket_key_permissions(
+				bucket_id,
+				&key.key_id,
+				BucketKeyPerm {
+					timestamp: now_msec(),
+					allow_read,
+					allow_write,
+					allow_owner,
+				},
+			)
+			.await?;
+
+		Ok(AdminRpc::Ok(format!(
+			"New permissions for {} on {}: read {}, write {}, owner {}.",
+			&key.key_id, &query.bucket, allow_read, allow_write, allow_owner
+		)))
+	}
+
+	async fn handle_bucket_deny(&self, query: &PermBucketOpt) -> Result<AdminRpc, Error> {
+		let helper = self.garage.bucket_helper();
+		let key_helper = self.garage.key_helper();
+
+		let bucket_id = helper
+			.resolve_global_bucket_name(&query.bucket)
+			.await?
+			.ok_or_bad_request("Bucket not found")?;
+		let key = key_helper
+			.get_existing_matching_key(&query.key_pattern)
+			.await?;
+
+		let allow_read = !query.read && key.allow_read(&bucket_id);
+		let allow_write = !query.write && key.allow_write(&bucket_id);
+		let allow_owner = !query.owner && key.allow_owner(&bucket_id);
+
+		helper
+			.set_bucket_key_permissions(
+				bucket_id,
+				&key.key_id,
+				BucketKeyPerm {
+					timestamp: now_msec(),
+					allow_read,
+					allow_write,
+					allow_owner,
+				},
+			)
+			.await?;
+
+		Ok(AdminRpc::Ok(format!(
+			"New permissions for {} on {}: read {}, write {}, owner {}.",
+			&key.key_id, &query.bucket, allow_read, allow_write, allow_owner
+		)))
+	}
+
+	async fn handle_bucket_website(&self, query: &WebsiteOpt) -> Result<AdminRpc, Error> {
+		let bucket_id = self
+			.garage
+			.bucket_helper()
+			.resolve_global_bucket_name(&query.bucket)
+			.await?
+			.ok_or_bad_request("Bucket not found")?;
+
+		let mut bucket = self
+			.garage
+			.bucket_helper()
+			.get_existing_bucket(bucket_id)
+			.await?;
+		let bucket_state = bucket.state.as_option_mut().unwrap();
+
+		if !(query.allow ^ query.deny) {
+			return Err(Error::BadRequest(
+				"You must specify exactly one flag, either --allow or --deny".to_string(),
+			));
+		}
+
+		let website = if query.allow {
+			Some(WebsiteConfig {
+				index_document: query.index_document.clone(),
+				error_document: query.error_document.clone(),
+			})
+		} else {
+			None
+		};
+
+		bucket_state.website_config.update(website);
+		self.garage.bucket_table.insert(&bucket).await?;
+
+		let msg = if query.allow {
+			format!("Website access allowed for {}", &query.bucket)
+		} else {
+			format!("Website access denied for {}", &query.bucket)
+		};
+
+		Ok(AdminRpc::Ok(msg))
+	}
+
+	async fn handle_bucket_set_quotas(&self, query: &SetQuotasOpt) -> Result<AdminRpc, Error> {
+		let bucket_id = self
+			.garage
+			.bucket_helper()
+			.resolve_global_bucket_name(&query.bucket)
+			.await?
+			.ok_or_bad_request("Bucket not found")?;
+
+		let mut bucket = self
+			.garage
+			.bucket_helper()
+			.get_existing_bucket(bucket_id)
+			.await?;
+		let bucket_state = bucket.state.as_option_mut().unwrap();
+
+		if query.max_size.is_none() && query.max_objects.is_none() {
+			return Err(Error::BadRequest(
+				"You must specify either --max-size or --max-objects (or both) for this command to do something.".to_string(),
+			));
+		}
+
+		let mut quotas = bucket_state.quotas.get().clone();
+
+		match query.max_size.as_ref().map(String::as_ref) {
+			Some("none") => quotas.max_size = None,
+			Some(v) => {
+				let bs = v
+					.parse::<bytesize::ByteSize>()
+					.ok_or_bad_request(format!("Invalid size specified: {}", v))?;
+				quotas.max_size = Some(bs.as_u64());
+			}
+			_ => (),
+		}
+
+		match query.max_objects.as_ref().map(String::as_ref) {
+			Some("none") => quotas.max_objects = None,
+			Some(v) => {
+				let mo = v
+					.parse::<u64>()
+					.ok_or_bad_request(format!("Invalid number specified: {}", v))?;
+				quotas.max_objects = Some(mo);
+			}
+			_ => (),
+		}
+
+		bucket_state.quotas.update(quotas);
+		self.garage.bucket_table.insert(&bucket).await?;
+
+		Ok(AdminRpc::Ok(format!(
+			"Quotas updated for {}",
+			&query.bucket
+		)))
+	}
+
+	async fn handle_bucket_cleanup_incomplete_uploads(
+		&self,
+		query: &CleanupIncompleteUploadsOpt,
+	) -> Result<AdminRpc, Error> {
+		let mut bucket_ids = vec![];
+		for b in query.buckets.iter() {
+			bucket_ids.push(
+				self.garage
+					.bucket_helper()
+					.resolve_global_bucket_name(b)
+					.await?
+					.ok_or_bad_request(format!("Bucket not found: {}", b))?,
+			);
+		}
+
+		let duration = parse_duration::parse::parse(&query.older_than)
+			.ok_or_bad_request("Invalid duration passed for --older-than parameter")?;
+
+		let mut ret = String::new();
+		for bucket in bucket_ids {
+			let count = self
+				.garage
+				.bucket_helper()
+				.cleanup_incomplete_uploads(&bucket, duration)
+				.await?;
+			writeln!(
+				&mut ret,
+				"Bucket {:?}: {} incomplete uploads aborted",
+				bucket, count
+			)
+			.unwrap();
+		}
+
+		Ok(AdminRpc::Ok(ret))
+	}
+}
@@ -0,0 +1,149 @@
+use std::collections::HashMap;
+
+use garage_table::*;
+
+use garage_model::helper::error::Error;
+use garage_model::key_table::*;
+
+use crate::cli::*;
+
+use super::*;
+
+impl AdminRpcHandler {
+	pub(super) async fn handle_key_cmd(&self, cmd: &KeyOperation) -> Result<AdminRpc, Error> {
+		match cmd {
+			KeyOperation::List => self.handle_list_keys().await,
+			KeyOperation::Info(query) => self.handle_key_info(query).await,
+			KeyOperation::New(query) => self.handle_create_key(query).await,
+			KeyOperation::Rename(query) => self.handle_rename_key(query).await,
+			KeyOperation::Delete(query) => self.handle_delete_key(query).await,
+			KeyOperation::Allow(query) => self.handle_allow_key(query).await,
+			KeyOperation::Deny(query) => self.handle_deny_key(query).await,
+			KeyOperation::Import(query) => self.handle_import_key(query).await,
+		}
+	}
+
+	async fn handle_list_keys(&self) -> Result<AdminRpc, Error> {
+		let key_ids = self
+			.garage
+			.key_table
+			.get_range(
+				&EmptyKey,
+				None,
+				Some(KeyFilter::Deleted(DeletedFilter::NotDeleted)),
+				10000,
+				EnumerationOrder::Forward,
+			)
+			.await?
+			.iter()
+			.map(|k| (k.key_id.to_string(), k.params().unwrap().name.get().clone()))
+			.collect::<Vec<_>>();
+		Ok(AdminRpc::KeyList(key_ids))
+	}
+
+	async fn handle_key_info(&self, query: &KeyOpt) -> Result<AdminRpc, Error> {
+		let key = self
+			.garage
+			.key_helper()
+			.get_existing_matching_key(&query.key_pattern)
+			.await?;
+		self.key_info_result(key).await
+	}
+
+	async fn handle_create_key(&self, query: &KeyNewOpt) -> Result<AdminRpc, Error> {
+		let key = Key::new(&query.name);
+		self.garage.key_table.insert(&key).await?;
+		self.key_info_result(key).await
+	}
+
+	async fn handle_rename_key(&self, query: &KeyRenameOpt) -> Result<AdminRpc, Error> {
+		let mut key = self
+			.garage
+			.key_helper()
+			.get_existing_matching_key(&query.key_pattern)
+			.await?;
+		key.params_mut()
+			.unwrap()
+			.name
+			.update(query.new_name.clone());
+		self.garage.key_table.insert(&key).await?;
+		self.key_info_result(key).await
+	}
+
+	async fn handle_delete_key(&self, query: &KeyDeleteOpt) -> Result<AdminRpc, Error> {
+		let key_helper = self.garage.key_helper();
+
+		let mut key = key_helper
+			.get_existing_matching_key(&query.key_pattern)
+			.await?;
+
+		if !query.yes {
+			return Err(Error::BadRequest(
+				"Add --yes flag to really perform this operation".to_string(),
+			));
+		}
+
+		key_helper.delete_key(&mut key).await?;
+
+		Ok(AdminRpc::Ok(format!(
+			"Key {} was deleted successfully.",
+			key.key_id
+		)))
+	}
+
+	async fn handle_allow_key(&self, query: &KeyPermOpt) -> Result<AdminRpc, Error> {
+		let mut key = self
+			.garage
+			.key_helper()
+			.get_existing_matching_key(&query.key_pattern)
+			.await?;
+		if query.create_bucket {
+			key.params_mut().unwrap().allow_create_bucket.update(true);
+		}
+		self.garage.key_table.insert(&key).await?;
+		self.key_info_result(key).await
+	}
+
+	async fn handle_deny_key(&self, query: &KeyPermOpt) -> Result<AdminRpc, Error> {
+		let mut key = self
+			.garage
+			.key_helper()
+			.get_existing_matching_key(&query.key_pattern)
+			.await?;
+		if query.create_bucket {
+			key.params_mut().unwrap().allow_create_bucket.update(false);
+		}
+		self.garage.key_table.insert(&key).await?;
+		self.key_info_result(key).await
+	}
+
+	async fn handle_import_key(&self, query: &KeyImportOpt) -> Result<AdminRpc, Error> {
+		let prev_key = self.garage.key_table.get(&EmptyKey, &query.key_id).await?;
+		if prev_key.is_some() {
+			return Err(Error::BadRequest(format!("Key {} already exists in data store. Even if it is deleted, we can't let you create a new key with the same ID. Sorry.", query.key_id)));
+		}
+		let imported_key = Key::import(&query.key_id, &query.secret_key, &query.name);
+		self.garage.key_table.insert(&imported_key).await?;
+
+		self.key_info_result(imported_key).await
+	}
+
+	async fn key_info_result(&self, key: Key) -> Result<AdminRpc, Error> {
+		let mut relevant_buckets = HashMap::new();
+
+		for (id, _) in key
+			.state
+			.as_option()
+			.unwrap()
+			.authorized_buckets
+			.items()
+			.iter()
+		{
+			if let Some(b) = self.garage.bucket_table.get(&EmptyKey, id).await? {
+				relevant_buckets.insert(*id, b);
+			}
+		}
+
+		Ok(AdminRpc::KeyInfo(key, relevant_buckets))
+	}
+}
@@ -0,0 +1,538 @@
+mod block;
+mod bucket;
+mod key;
+
+use std::collections::HashMap;
+use std::fmt::Write;
+use std::sync::Arc;
+
+use async_trait::async_trait;
+use serde::{Deserialize, Serialize};
+
+use format_table::format_table_to_string;
+
+use garage_util::background::BackgroundRunner;
+use garage_util::data::*;
+use garage_util::error::Error as GarageError;
+
+use garage_table::replication::*;
+use garage_table::*;
+
+use garage_rpc::ring::PARTITION_BITS;
+use garage_rpc::*;
+
+use garage_block::manager::BlockResyncErrorInfo;
+
+use garage_model::bucket_table::*;
+use garage_model::garage::Garage;
+use garage_model::helper::error::{Error, OkOrBadRequest};
+use garage_model::key_table::*;
+use garage_model::migrate::Migrate;
+use garage_model::s3::version_table::Version;
+
+use crate::cli::*;
+use crate::repair::online::launch_online_repair;
+
+pub const ADMIN_RPC_PATH: &str = "garage/admin_rpc.rs/Rpc";
+
+#[derive(Debug, Serialize, Deserialize)]
+#[allow(clippy::large_enum_variant)]
+pub enum AdminRpc {
+	BucketOperation(BucketOperation),
+	KeyOperation(KeyOperation),
+	LaunchRepair(RepairOpt),
+	Migrate(MigrateOpt),
+	Stats(StatsOpt),
+	Worker(WorkerOperation),
+	BlockOperation(BlockOperation),
+
+	// Replies
+	Ok(String),
+	BucketList(Vec<Bucket>),
+	BucketInfo {
+		bucket: Bucket,
+		relevant_keys: HashMap<String, Key>,
+		counters: HashMap<String, i64>,
+	},
+	KeyList(Vec<(String, String)>),
+	KeyInfo(Key, HashMap<Uuid, Bucket>),
+	WorkerList(
+		HashMap<usize, garage_util::background::WorkerInfo>,
+		WorkerListOpt,
+	),
+	WorkerVars(Vec<(Uuid, String, String)>),
+	WorkerInfo(usize, garage_util::background::WorkerInfo),
+	BlockErrorList(Vec<BlockResyncErrorInfo>),
+	BlockInfo {
+		hash: Hash,
+		refcount: u64,
+		versions: Vec<Result<Version, Uuid>>,
+	},
+}
+
+impl Rpc for AdminRpc {
+	type Response = Result<AdminRpc, Error>;
+}
+
+pub struct AdminRpcHandler {
+	garage: Arc<Garage>,
+	background: Arc<BackgroundRunner>,
+	endpoint: Arc<Endpoint<AdminRpc, Self>>,
+}
+
+impl AdminRpcHandler {
+	pub fn new(garage: Arc<Garage>, background: Arc<BackgroundRunner>) -> Arc<Self> {
+		let endpoint = garage.system.netapp.endpoint(ADMIN_RPC_PATH.into());
+		let admin = Arc::new(Self {
+			garage,
+			background,
+			endpoint,
+		});
+		admin.endpoint.set_handler(admin.clone());
+		admin
+	}
+
+	// ================ MIGRATION COMMANDS ====================
+
+	async fn handle_migrate(self: &Arc<Self>, opt: MigrateOpt) -> Result<AdminRpc, Error> {
+		if !opt.yes {
+			return Err(Error::BadRequest(
+				"Please provide the --yes flag to initiate migration operation.".to_string(),
+			));
+		}
+
+		let m = Migrate {
+			garage: self.garage.clone(),
+		};
+		match opt.what {
+			MigrateWhat::Buckets050 => m.migrate_buckets050().await,
+		}?;
+		Ok(AdminRpc::Ok("Migration successfull.".into()))
+	}
+
+	// ================ REPAIR COMMANDS ====================
+
+	async fn handle_launch_repair(self: &Arc<Self>, opt: RepairOpt) -> Result<AdminRpc, Error> {
+		if !opt.yes {
+			return Err(Error::BadRequest(
+				"Please provide the --yes flag to initiate repair operations.".to_string(),
+			));
+		}
+		if opt.all_nodes {
+			let mut opt_to_send = opt.clone();
+			opt_to_send.all_nodes = false;
+
+			let mut failures = vec![];
+			let ring = self.garage.system.ring.borrow().clone();
+			for node in ring.layout.node_ids().iter() {
+				let node = (*node).into();
+				let resp = self
+					.endpoint
+					.call(
+						&node,
+						AdminRpc::LaunchRepair(opt_to_send.clone()),
+						PRIO_NORMAL,
+					)
+					.await;
+				if !matches!(resp, Ok(Ok(_))) {
+					failures.push(node);
+				}
+			}
+			if failures.is_empty() {
+				Ok(AdminRpc::Ok("Repair launched on all nodes".to_string()))
+			} else {
+				Err(Error::BadRequest(format!(
+					"Could not launch repair on nodes: {:?} (launched successfully on other nodes)",
+					failures
+				)))
+			}
+		} else {
+			launch_online_repair(&self.garage, &self.background, opt).await?;
+			Ok(AdminRpc::Ok(format!(
+				"Repair launched on {:?}",
+				self.garage.system.id
+			)))
+		}
+	}
+
+	// ================ STATS COMMANDS ====================
+
+	async fn handle_stats(&self, opt: StatsOpt) -> Result<AdminRpc, Error> {
+		if opt.all_nodes {
+			let mut ret = String::new();
+			let ring = self.garage.system.ring.borrow().clone();
+
+			for node in ring.layout.node_ids().iter() {
+				let mut opt = opt.clone();
+				opt.all_nodes = false;
+				opt.skip_global = true;
+
+				writeln!(&mut ret, "\n======================").unwrap();
+				writeln!(&mut ret, "Stats for node {:?}:", node).unwrap();
+
+				let node_id = (*node).into();
+				match self
+					.endpoint
+					.call(&node_id, AdminRpc::Stats(opt), PRIO_NORMAL)
+					.await
+				{
+					Ok(Ok(AdminRpc::Ok(s))) => writeln!(&mut ret, "{}", s).unwrap(),
+					Ok(Ok(x)) => writeln!(&mut ret, "Bad answer: {:?}", x).unwrap(),
+					Ok(Err(e)) => writeln!(&mut ret, "Remote error: {}", e).unwrap(),
+					Err(e) => writeln!(&mut ret, "Network error: {}", e).unwrap(),
+				}
+			}
+
+			writeln!(&mut ret, "\n======================").unwrap();
+			write!(
+				&mut ret,
+				"Cluster statistics:\n\n{}",
+				self.gather_cluster_stats()
+			)
+			.unwrap();
+
+			Ok(AdminRpc::Ok(ret))
+		} else {
+			Ok(AdminRpc::Ok(self.gather_stats_local(opt)?))
+		}
+	}
+
+	fn gather_stats_local(&self, opt: StatsOpt) -> Result<String, Error> {
+		let mut ret = String::new();
+		writeln!(
+			&mut ret,
+			"\nGarage version: {} [features: {}]\nRust compiler version: {}",
+			garage_util::version::garage_version(),
+			garage_util::version::garage_features()
+				.map(|list| list.join(", "))
+				.unwrap_or_else(|| "(unknown)".into()),
+			garage_util::version::rust_version(),
+		)
+		.unwrap();
+
+		writeln!(&mut ret, "\nDatabase engine: {}", self.garage.db.engine()).unwrap();
+
+		// Gather table statistics
+		let mut table = vec!["  Table\tItems\tMklItems\tMklTodo\tGcTodo".into()];
+		table.push(self.gather_table_stats(&self.garage.bucket_table, opt.detailed)?);
+		table.push(self.gather_table_stats(&self.garage.key_table, opt.detailed)?);
+		table.push(self.gather_table_stats(&self.garage.object_table, opt.detailed)?);
+		table.push(self.gather_table_stats(&self.garage.version_table, opt.detailed)?);
+		table.push(self.gather_table_stats(&self.garage.block_ref_table, opt.detailed)?);
+		write!(
+			&mut ret,
+			"\nTable stats:\n{}",
+			format_table_to_string(table)
+		)
+		.unwrap();
+
+		// Gather block manager statistics
+		writeln!(&mut ret, "\nBlock manager stats:").unwrap();
+		let rc_len = if opt.detailed {
+			self.garage.block_manager.rc_len()?.to_string()
+		} else {
+			self.garage
+				.block_manager
+				.rc_fast_len()?
+				.map(|x| x.to_string())
+				.unwrap_or_else(|| "NC".into())
+		};
+
+		writeln!(
+			&mut ret,
+			"  number of RC entries (~= number of blocks): {}",
+			rc_len
+		)
+		.unwrap();
+		writeln!(
+			&mut ret,
+			"  resync queue length: {}",
+			self.garage.block_manager.resync.queue_len()?
+		)
+		.unwrap();
+		writeln!(
+			&mut ret,
+			"  blocks with resync errors: {}",
+			self.garage.block_manager.resync.errors_len()?
+		)
+		.unwrap();
+
+		if !opt.detailed {
+			writeln!(&mut ret, "\nIf values are missing above (marked as NC), consider adding the --detailed flag (this will be slow).").unwrap();
+		}
+
+		if !opt.skip_global {
+			write!(&mut ret, "\n{}", self.gather_cluster_stats()).unwrap();
+		}
+
+		Ok(ret)
+	}
+
+	fn gather_cluster_stats(&self) -> String {
+		let mut ret = String::new();
+
+		// Gather storage node and free space statistics
+		let layout = &self.garage.system.ring.borrow().layout;
+		let mut node_partition_count = HashMap::<Uuid, u64>::new();
+		for short_id in layout.ring_assignation_data.iter() {
+			let id = layout.node_id_vec[*short_id as usize];
+			*node_partition_count.entry(id).or_default() += 1;
+		}
+		let node_info = self
+			.garage
+			.system
+			.get_known_nodes()
+			.into_iter()
+			.map(|n| (n.id, n))
+			.collect::<HashMap<_, _>>();
+
+		let mut table = vec!["  ID\tHostname\tZone\tCapacity\tPart.\tDataAvail\tMetaAvail".into()];
+		for (id, parts) in node_partition_count.iter() {
+			let info = node_info.get(id);
+			let status = info.map(|x| &x.status);
+			let role = layout.roles.get(id).and_then(|x| x.0.as_ref());
+			let hostname = status.map(|x| x.hostname.as_str()).unwrap_or("?");
+			let zone = role.map(|x| x.zone.as_str()).unwrap_or("?");
+			let capacity = role
+				.map(|x| x.capacity_string())
+				.unwrap_or_else(|| "?".into());
+			let avail_str = |x| match x {
+				Some((avail, total)) => {
+					let pct = (avail as f64) / (total as f64) * 100.;
+					let avail = bytesize::ByteSize::b(avail);
+					let total = bytesize::ByteSize::b(total);
+					format!("{}/{} ({:.1}%)", avail, total, pct)
+				}
+				None => "?".into(),
+			};
+			let data_avail = avail_str(status.and_then(|x| x.data_disk_avail));
+			let meta_avail = avail_str(status.and_then(|x| x.meta_disk_avail));
+			table.push(format!(
+				"  {:?}\t{}\t{}\t{}\t{}\t{}\t{}",
+				id, hostname, zone, capacity, parts, data_avail, meta_avail
+			));
+		}
+		write!(
+			&mut ret,
+			"Storage nodes:\n{}",
+			format_table_to_string(table)
+		)
+		.unwrap();
+
+		let meta_part_avail = node_partition_count
+			.iter()
+			.filter_map(|(id, parts)| {
+				node_info
+					.get(id)
+					.and_then(|x| x.status.meta_disk_avail)
+					.map(|c| c.0 / *parts)
+			})
+			.collect::<Vec<_>>();
+		let data_part_avail = node_partition_count
+			.iter()
+			.filter_map(|(id, parts)| {
+				node_info
+					.get(id)
+					.and_then(|x| x.status.data_disk_avail)
+					.map(|c| c.0 / *parts)
+			})
+			.collect::<Vec<_>>();
+		if !meta_part_avail.is_empty() && !data_part_avail.is_empty() {
+			let meta_avail =
+				bytesize::ByteSize(meta_part_avail.iter().min().unwrap() * (1 << PARTITION_BITS));
+			let data_avail =
+				bytesize::ByteSize(data_part_avail.iter().min().unwrap() * (1 << PARTITION_BITS));
+			writeln!(
+				&mut ret,
+				"\nEstimated available storage space cluster-wide (might be lower in practice):"
+			)
+			.unwrap();
+			if meta_part_avail.len() < node_partition_count.len()
+				|| data_part_avail.len() < node_partition_count.len()
+			{
+				writeln!(&mut ret, "  data: < {}", data_avail).unwrap();
+				writeln!(&mut ret, "  metadata: < {}", meta_avail).unwrap();
+				writeln!(&mut ret, "A precise estimate could not be given as information is missing for some storage nodes.").unwrap();
+			} else {
+				writeln!(&mut ret, "  data: {}", data_avail).unwrap();
+				writeln!(&mut ret, "  metadata: {}", meta_avail).unwrap();
+			}
+		}
+
+		ret
+	}
+
+	fn gather_table_stats<F, R>(
+		&self,
+		t: &Arc<Table<F, R>>,
+		detailed: bool,
+	) -> Result<String, Error>
+	where
+		F: TableSchema + 'static,
+		R: TableReplication + 'static,
+	{
+		let (data_len, mkl_len) = if detailed {
+			(
+				t.data.store.len().map_err(GarageError::from)?.to_string(),
+				t.merkle_updater.merkle_tree_len()?.to_string(),
+			)
+		} else {
+			(
+				t.data
+					.store
+					.fast_len()
+					.map_err(GarageError::from)?
+					.map(|x| x.to_string())
+					.unwrap_or_else(|| "NC".into()),
+				t.merkle_updater
+					.merkle_tree_fast_len()?
+					.map(|x| x.to_string())
+					.unwrap_or_else(|| "NC".into()),
+			)
+		};
+
+		Ok(format!(
+			"  {}\t{}\t{}\t{}\t{}",
+			F::TABLE_NAME,
+			data_len,
+			mkl_len,
+			t.merkle_updater.todo_len()?,
+			t.data.gc_todo_len()?
+		))
+	}
+
+	// ================ WORKER COMMANDS ====================
+
+	async fn handle_worker_cmd(&self, cmd: &WorkerOperation) -> Result<AdminRpc, Error> {
+		match cmd {
+			WorkerOperation::List { opt } => {
+				let workers = self.background.get_worker_info();
+				Ok(AdminRpc::WorkerList(workers, *opt))
+			}
+			WorkerOperation::Info { tid } => {
+				let info = self
+					.background
+					.get_worker_info()
+					.get(tid)
+					.ok_or_bad_request(format!("No worker with TID {}", tid))?
+					.clone();
+				Ok(AdminRpc::WorkerInfo(*tid, info))
+			}
+			WorkerOperation::Get {
+				all_nodes,
+				variable,
+			} => self.handle_get_var(*all_nodes, variable).await,
+			WorkerOperation::Set {
+				all_nodes,
+				variable,
+				value,
+			} => self.handle_set_var(*all_nodes, variable, value).await,
+		}
+	}
+
+	async fn handle_get_var(
+		&self,
+		all_nodes: bool,
+		variable: &Option<String>,
+	) -> Result<AdminRpc, Error> {
+		if all_nodes {
+			let mut ret = vec![];
+			let ring = self.garage.system.ring.borrow().clone();
+			for node in ring.layout.node_ids().iter() {
+				let node = (*node).into();
+				match self
+					.endpoint
+					.call(
+						&node,
+						AdminRpc::Worker(WorkerOperation::Get {
+							all_nodes: false,
+							variable: variable.clone(),
+						}),
+						PRIO_NORMAL,
+					)
+					.await??
+				{
+					AdminRpc::WorkerVars(v) => ret.extend(v),
+					m => return Err(GarageError::unexpected_rpc_message(m).into()),
+				}
+			}
+			Ok(AdminRpc::WorkerVars(ret))
+		} else {
+			#[allow(clippy::collapsible_else_if)]
+			if let Some(v) = variable {
+				Ok(AdminRpc::WorkerVars(vec![(
+					self.garage.system.id,
+					v.clone(),
+					self.garage.bg_vars.get(v)?,
+				)]))
+			} else {
+				let mut vars = self.garage.bg_vars.get_all();
+				vars.sort();
+				Ok(AdminRpc::WorkerVars(
+					vars.into_iter()
+						.map(|(k, v)| (self.garage.system.id, k.to_string(), v))
+						.collect(),
+				))
+			}
+		}
+	}
+
+	async fn handle_set_var(
+		&self,
+		all_nodes: bool,
+		variable: &str,
+		value: &str,
+	) -> Result<AdminRpc, Error> {
+		if all_nodes {
+			let mut ret = vec![];
+			let ring = self.garage.system.ring.borrow().clone();
+			for node in ring.layout.node_ids().iter() {
+				let node = (*node).into();
+				match self
+					.endpoint
+					.call(
+						&node,
+						AdminRpc::Worker(WorkerOperation::Set {
+							all_nodes: false,
+							variable: variable.to_string(),
+							value: value.to_string(),
+						}),
+						PRIO_NORMAL,
+					)
+					.await??
+				{
+					AdminRpc::WorkerVars(v) => ret.extend(v),
+					m => return Err(GarageError::unexpected_rpc_message(m).into()),
+				}
+			}
+			Ok(AdminRpc::WorkerVars(ret))
+		} else {
+			self.garage.bg_vars.set(variable, value)?;
+			Ok(AdminRpc::WorkerVars(vec![(
+				self.garage.system.id,
+				variable.to_string(),
+				value.to_string(),
+			)]))
+		}
+	}
+}
+
+#[async_trait]
+impl EndpointHandler<AdminRpc> for AdminRpcHandler {
+	async fn handle(
+		self: &Arc<Self>,
+		message: &AdminRpc,
+		_from: NodeID,
+	) -> Result<AdminRpc, Error> {
+		match message {
+			AdminRpc::BucketOperation(bo) => self.handle_bucket_cmd(bo).await,
+			AdminRpc::KeyOperation(ko) => self.handle_key_cmd(ko).await,
+			AdminRpc::Migrate(opt) => self.handle_migrate(opt.clone()).await,
+			AdminRpc::LaunchRepair(opt) => self.handle_launch_repair(opt.clone()).await,
+			AdminRpc::Stats(opt) => self.handle_stats(opt.clone()).await,
+			AdminRpc::Worker(wo) => self.handle_worker_cmd(wo).await,
+			AdminRpc::BlockOperation(bo) => self.handle_block_cmd(bo).await,
+			m => Err(GarageError::unexpected_rpc_message(m).into()),
+		}
+	}
+}
@@ -1,8 +1,8 @@
 use std::collections::HashSet;
 use std::time::Duration;

+use format_table::format_table;
 use garage_util::error::*;
-use garage_util::formater::format_table;

 use garage_rpc::layout::*;
 use garage_rpc::system::*;
@@ -1,6 +1,6 @@
+use format_table::format_table;
 use garage_util::crdt::Crdt;
 use garage_util::error::*;
-use garage_util::formater::format_table;

 use garage_rpc::layout::*;
 use garage_rpc::system::*;
@@ -216,7 +216,7 @@ pub struct WebsiteOpt {
 	#[structopt(short = "i", long = "index-document", default_value = "index.html")]
 	pub index_document: String,

-	/// Error document: the optionnal document returned when an error occurs
+	/// Error document: the optional document returned when an error occurs
 	#[structopt(short = "e", long = "error-document")]
 	pub error_document: Option<String>,
 }
@@ -1,11 +1,11 @@
 use std::collections::HashMap;
 use std::time::Duration;

+use format_table::format_table;
 use garage_util::background::*;
 use garage_util::crdt::*;
 use garage_util::data::*;
 use garage_util::error::*;
-use garage_util::formater::format_table;
 use garage_util::time::*;

 use garage_block::manager::BlockResyncErrorInfo;
@@ -373,13 +373,18 @@ pub fn print_block_error_list(el: Vec<BlockResyncErrorInfo>) {

 	let mut table = vec!["Hash\tRC\tErrors\tLast error\tNext try".into()];
 	for e in el {
+		let next_try = if e.next_try > now {
+			tf2.convert(Duration::from_millis(e.next_try - now))
+		} else {
+			"asap".to_string()
+		};
 		table.push(format!(
 			"{}\t{}\t{}\t{}\tin {}",
 			hex::encode(e.hash.as_slice()),
 			e.refcount,
 			e.error_count,
 			tf.convert(Duration::from_millis(now - e.last_try)),
-			tf2.convert(Duration::from_millis(e.next_try - now))
+			next_try
 		));
 	}
 	format_table(table);
@@ -108,6 +108,12 @@ async fn main() {
 	][..];
 	if let Some(git_version) = option_env!("GIT_VERSION") {
 		garage_util::version::init_version(git_version);
+	} else {
+		garage_util::version::init_version(git_version::git_version!(
+			prefix = "git:",
+			cargo_prefix = "cargo:",
+			fallback = "unknown"
+		));
 	}
 	garage_util::version::init_features(features);

@@ -1,7 +1,6 @@
 use crate::common;
 use crate::common::ext::CommandExt;
-use aws_sdk_s3::model::BucketLocationConstraint;
-use aws_sdk_s3::output::DeleteBucketOutput;
+use aws_sdk_s3::operation::delete_bucket::DeleteBucketOutput;

 #[tokio::test]
 async fn test_bucket_all() {
@@ -63,10 +62,7 @@ async fn test_bucket_all() {
 			.await
 			.unwrap();

-		match r.location_constraint.unwrap() {
-			BucketLocationConstraint::Unknown(v) if v.as_str() == "garage-integ-test" => (),
-			_ => unreachable!("wrong region"),
-		}
+		assert_eq!(r.location_constraint.unwrap().as_str(), "garage-integ-test");
 	}
 	{
 		// (Stub) check GetVersioning
@@ -1,15 +1,16 @@
-use aws_sdk_s3::{Client, Config, Credentials, Endpoint};
+use aws_sdk_s3::config::Credentials;
+use aws_sdk_s3::{Client, Config};

-use super::garage::{Instance, Key};
+use super::garage::Key;
+use crate::common::garage::DEFAULT_PORT;

-pub fn build_client(instance: &Instance, key: &Key) -> Client {
+pub fn build_client(key: &Key) -> Client {
 	let credentials = Credentials::new(&key.id, &key.secret, None, None, "garage-integ-test");
-	let endpoint = Endpoint::immutable(instance.s3_uri());

 	let config = Config::builder()
+		.endpoint_url(format!("http://127.0.0.1:{}", DEFAULT_PORT))
 		.region(super::REGION)
 		.credentials_provider(credentials)
-		.endpoint_resolver(endpoint)
 		.build();

 	Client::from_conf(config)
@@ -209,6 +209,7 @@ impl<'a> RequestBuilder<'a> {
 		all_headers.extend(self.unsigned_headers.clone());

 		let canonical_request = signature::payload::canonical_request(
+			self.service,
 			&self.method,
 			&Uri::try_from(&uri).unwrap(),
 			&all_headers,
@@ -1,5 +1,8 @@
-use aws_sdk_s3::{Client, Region};
+use aws_sdk_s3::config::Region;
+use aws_sdk_s3::Client;
 use ext::*;
+#[cfg(feature = "k2v")]
+use k2v_client::K2vClient;

 #[macro_use]
 pub mod macros;
@@ -19,6 +22,7 @@ pub struct Context {
 	pub key: garage::Key,
 	pub client: Client,
 	pub custom_request: CustomRequester,
+	#[cfg(feature = "k2v")]
 	pub k2v: K2VContext,
 }

@@ -31,8 +35,9 @@ impl Context {
 	fn new() -> Self {
 		let garage = garage::instance();
 		let key = garage.key(None);
-		let client = client::build_client(garage, &key);
+		let client = client::build_client(&key);
 		let custom_request = CustomRequester::new_s3(garage, &key);
+		#[cfg(feature = "k2v")]
 		let k2v_request = CustomRequester::new_k2v(garage, &key);

 		Context {
@@ -40,6 +45,7 @@ impl Context {
 			client,
 			key,
 			custom_request,
+			#[cfg(feature = "k2v")]
 			k2v: K2VContext {
 				request: k2v_request,
 			},
@@ -68,6 +74,20 @@ impl Context {

 		bucket_name
 	}
+
+	/// Build a K2vClient for a given bucket
+	#[cfg(feature = "k2v")]
+	pub fn k2v_client(&self, bucket: &str) -> K2vClient {
+		let config = k2v_client::K2vClientConfig {
+			region: REGION.to_string(),
+			endpoint: self.garage.k2v_uri().to_string(),
+			aws_access_key_id: self.key.id.clone(),
+			aws_secret_access_key: self.key.secret.clone(),
+			bucket: bucket.to_string(),
+			user_agent: None,
+		};
+		K2vClient::new(config).expect("Could not create K2V client")
+	}
 }

 pub fn context() -> Context {
@@ -6,7 +6,7 @@ use assert_json_diff::assert_json_eq;
 use base64::prelude::*;
 use serde_json::json;

-use super::json_body;
+use crate::json_body;
 use hyper::{Method, StatusCode};

 #[tokio::test]
@@ -6,7 +6,7 @@ use assert_json_diff::assert_json_eq;
 use base64::prelude::*;
 use serde_json::json;

-use super::json_body;
+use crate::json_body;
 use hyper::{Method, StatusCode};

 #[tokio::test]
@@ -3,16 +3,3 @@ pub mod errorcodes;
 pub mod item;
 pub mod poll;
 pub mod simple;
-
-use hyper::{Body, Response};
-
-pub async fn json_body(res: Response<Body>) -> serde_json::Value {
-	let res_body: serde_json::Value = serde_json::from_slice(
-		&hyper::body::to_bytes(res.into_body())
-			.await
-			.unwrap()
-			.to_vec()[..],
-	)
-	.unwrap();
-	res_body
-}
@@ -5,8 +5,8 @@ use std::time::Duration;
 use assert_json_diff::assert_json_eq;
 use serde_json::json;

-use super::json_body;
 use crate::common;
+use crate::json_body;

 #[tokio::test]
 async fn test_poll_item() {
@@ -0,0 +1 @@
+pub mod simple;
@@ -0,0 +1,60 @@
+use std::time::Duration;
+
+use k2v_client::*;
+
+use crate::common;
+
+#[tokio::test]
+async fn test_simple() {
+	let ctx = common::context();
+	let bucket = ctx.create_bucket("test-k2v-client-simple");
+	let k2v_client = ctx.k2v_client(&bucket);
+
+	k2v_client
+		.insert_item("root", "test1", b"Hello, world!".to_vec(), None)
+		.await
+		.unwrap();
+
+	let res = k2v_client.read_item("root", "test1").await.unwrap();
+
+	assert_eq!(res.value.len(), 1);
+	assert_eq!(res.value[0], K2vValue::Value(b"Hello, world!".to_vec()));
+}
+
+#[tokio::test]
+async fn test_special_chars() {
+	let ctx = common::context();
+	let bucket = ctx.create_bucket("test-k2v-client-simple-special-chars");
+	let k2v_client = ctx.k2v_client(&bucket);
+
+	let (pk, sk) = ("root@plépp", "≤≤««");
+	k2v_client
+		.insert_item(pk, sk, b"Hello, world!".to_vec(), None)
+		.await
+		.unwrap();
+
+	let res = k2v_client.read_item(pk, sk).await.unwrap();
+	assert_eq!(res.value.len(), 1);
+	assert_eq!(res.value[0], K2vValue::Value(b"Hello, world!".to_vec()));
+
+	// sleep a bit before read_index
+	tokio::time::sleep(Duration::from_secs(1)).await;
+	let res = k2v_client.read_index(Default::default()).await.unwrap();
+	assert_eq!(res.items.len(), 1);
+	assert_eq!(res.items.keys().next().unwrap(), pk);
+
+	let res = k2v_client
+		.read_batch(&[BatchReadOp {
+			partition_key: pk,
+			filter: Default::default(),
+			single_item: false,
+			conflicts_only: false,
+			tombstones: false,
+		}])
+		.await
+		.unwrap();
+	assert_eq!(res.len(), 1);
+	let res = &res[0];
+	assert_eq!(res.items.len(), 1);
+	assert_eq!(res.items.keys().next().unwrap(), sk);
+}
@@ -8,3 +8,18 @@ mod s3;

 #[cfg(feature = "k2v")]
 mod k2v;
+#[cfg(feature = "k2v")]
+mod k2v_client;
+
+use hyper::{Body, Response};
+
+pub async fn json_body(res: Response<Body>) -> serde_json::Value {
+	let res_body: serde_json::Value = serde_json::from_slice(
+		&hyper::body::to_bytes(res.into_body())
+			.await
+			.unwrap()
+			.to_vec()[..],
+	)
+	.unwrap();
+	res_body
+}
@@ -1,6 +1,6 @@
 use crate::common;
-use aws_sdk_s3::model::{CompletedMultipartUpload, CompletedPart};
-use aws_sdk_s3::types::ByteStream;
+use aws_sdk_s3::primitives::ByteStream;
+use aws_sdk_s3::types::{CompletedMultipartUpload, CompletedPart};

 const SZ_5MB: usize = 5 * 1024 * 1024;
 const SZ_10MB: usize = 10 * 1024 * 1024;
@@ -1,6 +1,6 @@
 use crate::common;
-use aws_sdk_s3::model::{Delete, ObjectIdentifier};
-use aws_sdk_s3::types::ByteStream;
+use aws_sdk_s3::primitives::ByteStream;
+use aws_sdk_s3::types::{Delete, ObjectIdentifier};

 const STD_KEY: &str = "hello world";
 const CTRL_KEY: &str = "\x00\x01\x02\x00";
@@ -2,7 +2,7 @@ use crate::common;

 #[tokio::test]
 async fn test_simple() {
-	use aws_sdk_s3::types::ByteStream;
+	use aws_sdk_s3::primitives::ByteStream;

 	let ctx = common::context();
 	let bucket = ctx.create_bucket("test-simple");
@@ -1,11 +1,11 @@
 use crate::common;
 use crate::common::ext::*;
-use crate::k2v::json_body;
+use crate::json_body;

 use assert_json_diff::assert_json_eq;
 use aws_sdk_s3::{
-	model::{CorsConfiguration, CorsRule, ErrorDocument, IndexDocument, WebsiteConfiguration},
-	types::ByteStream,
+	primitives::ByteStream,
+	types::{CorsConfiguration, CorsRule, ErrorDocument, IndexDocument, WebsiteConfiguration},
 };
 use http::{Request, StatusCode};
 use hyper::{
@@ -72,7 +72,7 @@ async fn test_website() {
 		res_body,
 		json!({
 			"code": "InvalidRequest",
-			"message": "Bad request: Bucket 'my-website' is not authorized for website hosting",
+			"message": "Bad request: Domain 'my-website' is not managed by Garage",
 			"region": "garage-integ-test",
 			"path": "/check",
 		})
@@ -91,24 +91,29 @@ async fn test_website() {
 		BODY.as_ref()
 	);

-	let admin_req = || {
-		Request::builder()
-			.method("GET")
-			.uri(format!(
-				"http://127.0.0.1:{0}/check?domain={1}",
-				ctx.garage.admin_port,
-				BCKT_NAME.to_string()
-			))
-			.body(Body::empty())
-			.unwrap()
-	};
+	for bname in [
+		BCKT_NAME.to_string(),
+		format!("{BCKT_NAME}.web.garage"),
+		format!("{BCKT_NAME}.s3.garage"),
+	] {
+		let admin_req = || {
+			Request::builder()
+				.method("GET")
+				.uri(format!(
+					"http://127.0.0.1:{0}/check?domain={1}",
+					ctx.garage.admin_port, bname
+				))
+				.body(Body::empty())
+				.unwrap()
+		};

-	let mut admin_resp = client.request(admin_req()).await.unwrap();
-	assert_eq!(admin_resp.status(), StatusCode::OK);
-	assert_eq!(
-		to_bytes(admin_resp.body_mut()).await.unwrap().as_ref(),
-		format!("Bucket '{BCKT_NAME}' is authorized for website hosting").as_bytes()
-	);
+		let mut admin_resp = client.request(admin_req()).await.unwrap();
+		assert_eq!(admin_resp.status(), StatusCode::OK);
+		assert_eq!(
+			to_bytes(admin_resp.body_mut()).await.unwrap().as_ref(),
+			format!("Domain '{bname}' is managed by Garage").as_bytes()
+		);
+	}

 	ctx.garage
 		.command()
@@ -142,7 +147,7 @@ async fn test_website() {
 		res_body,
 		json!({
 			"code": "InvalidRequest",
-			"message": "Bad request: Bucket 'my-website' is not authorized for website hosting",
+			"message": "Bad request: Domain 'my-website' is not managed by Garage",
 			"region": "garage-integ-test",
 			"path": "/check",
 		})
@@ -397,7 +402,7 @@ async fn test_website_s3_api() {
 }

 #[tokio::test]
-async fn test_website_check_website_enabled() {
+async fn test_website_check_domain() {
 	let ctx = common::context();

 	let client = Client::new();
@@ -435,13 +440,13 @@ async fn test_website_check_website_enabled() {
 	};

 	let admin_resp = client.request(admin_req()).await.unwrap();
-	assert_eq!(admin_resp.status(), StatusCode::NOT_FOUND);
+	assert_eq!(admin_resp.status(), StatusCode::BAD_REQUEST);
 	let res_body = json_body(admin_resp).await;
 	assert_json_eq!(
 		res_body,
 		json!({
-			"code": "NoSuchBucket",
-			"message": "Bucket not found: ",
+			"code": "InvalidRequest",
+			"message": "Bad request: Domain '' is not managed by Garage",
 			"region": "garage-integ-test",
 			"path": "/check",
 		})
@@ -459,13 +464,13 @@ async fn test_website_check_website_enabled() {
 	};

 	let admin_resp = client.request(admin_req()).await.unwrap();
-	assert_eq!(admin_resp.status(), StatusCode::NOT_FOUND);
+	assert_eq!(admin_resp.status(), StatusCode::BAD_REQUEST);
 	let res_body = json_body(admin_resp).await;
 	assert_json_eq!(
 		res_body,
 		json!({
-			"code": "NoSuchBucket",
-			"message": "Bucket not found: foobar",
+			"code": "InvalidRequest",
+			"message": "Bad request: Domain 'foobar' is not managed by Garage",
 			"region": "garage-integ-test",
 			"path": "/check",
 		})
@@ -483,13 +488,13 @@ async fn test_website_check_website_enabled() {
 	};

 	let admin_resp = client.request(admin_req()).await.unwrap();
-	assert_eq!(admin_resp.status(), StatusCode::NOT_FOUND);
+	assert_eq!(admin_resp.status(), StatusCode::BAD_REQUEST);
 	let res_body = json_body(admin_resp).await;
 	assert_json_eq!(
 		res_body,
 		json!({
-			"code": "NoSuchBucket",
-			"message": "Bucket not found: ☹",
+			"code": "InvalidRequest",
+			"message": "Bad request: Domain '☹' is not managed by Garage",
 			"region": "garage-integ-test",
 			"path": "/check",
 		})
@@ -1,6 +1,6 @@
 [package]
 name = "k2v-client"
-version = "0.1.1"
+version = "0.0.4"
 authors = ["Trinity Pointard <trinity.pointard@gmail.com>", "Alex Auvolat <alex@adnab.me>"]
 edition = "2018"
 license = "AGPL-3.0"
@@ -10,24 +10,28 @@ readme = "../../README.md"

 [dependencies]
 base64 = "0.21"
+sha2 = "0.10"
+hex = "0.4"
 http = "0.2"
 log = "0.4"
-rusoto_core = { version = "0.48.0", default-features = false, features = ["rustls"] }
-rusoto_credential = "0.48.0"
-rusoto_signature = "0.48.0"
-hyper-rustls = { version = "0.23", default-features = false, features = [ "http1", "http2", "tls12" ] }
-serde = "1.0"
+aws-sigv4 = "0.55"
+percent-encoding = "2.2"
+hyper = { version = "0.14", default-features = false, features = ["client", "http1", "http2"] }
+hyper-rustls = { version = "0.24", features = ["http2"] }
+serde = { version = "1.0", features = [ "derive" ] }
 serde_json = "1.0"
 thiserror = "1.0"
-tokio = "1.24"
+tokio = { version = "1.0", default-features = false, features = ["rt", "rt-multi-thread", "io-util", "net", "time", "macros", "sync", "signal", "fs"] }

 # cli deps
 clap = { version = "4.1", optional = true, features = ["derive", "env"] }
-garage_util = { version = "0.8.2", path = "../util", optional = true }
+format_table = { workspace = true, optional = true }
+tracing = { version = "0.1", optional = true }
+tracing-subscriber = { version = "0.3", optional = true, features = ["env-filter"] }


 [features]
-cli = ["clap", "tokio/fs", "tokio/io-std", "garage_util"]
+cli = ["clap", "tokio/fs", "tokio/io-std", "tracing", "tracing-subscriber", "format_table"]

 [lib]
 path = "lib.rs"
@@ -2,12 +2,11 @@ use std::collections::BTreeMap;
 use std::process::exit;
 use std::time::Duration;

+use base64::prelude::*;
+
 use k2v_client::*;

-use garage_util::formater::format_table;
-
-use rusoto_core::credential::AwsCredentials;
-use rusoto_core::Region;
+use format_table::format_table;

 use clap::{Parser, Subcommand};

@@ -155,7 +154,9 @@ impl Value {
 		if let Some(ref text) = self.text {
 			Ok(text.as_bytes().to_vec())
 		} else if let Some(ref b64) = self.b64 {
-			base64::decode(b64).map_err(|_| Error::Message("invalid base64 input".into()))
+			BASE64_STANDARD
+				.decode(b64)
+				.map_err(|_| Error::Message("invalid base64 input".into()))
 		} else if let Some(ref path) = self.file {
 			use tokio::io::AsyncReadExt;
 			if path == "-" {
@@ -230,7 +231,7 @@ impl ReadOutputKind {
 			for val in val.value {
 				match val {
 					K2vValue::Value(v) => {
-						println!("{}", base64::encode(&v))
+						println!("{}", BASE64_STANDARD.encode(&v))
 					}
 					K2vValue::Tombstone => {
 						println!();
@@ -249,7 +250,7 @@ impl ReadOutputKind {
 					if let Ok(string) = std::str::from_utf8(&v) {
 						println!("  utf-8: {}", string);
 					} else {
-						println!("  base64: {}", base64::encode(&v));
+						println!("  base64: {}", BASE64_STANDARD.encode(&v));
 					}
 				}
 				K2vValue::Tombstone => {
@@ -275,7 +276,7 @@ struct BatchOutputKind {
 impl BatchOutputKind {
 	fn display_human_output(&self, values: BTreeMap<String, CausalValue>) -> ! {
 		for (key, values) in values {
-			println!("key: {}", key);
+			println!("sort_key: {}", key);
 			let causality: String = values.causality.into();
 			println!("causality: {}", causality);
 			for value in values.value {
@@ -284,7 +285,7 @@ impl BatchOutputKind {
 						if let Ok(string) = std::str::from_utf8(&v) {
 							println!("  value(utf-8): {}", string);
 						} else {
-							println!("  value(base64): {}", base64::encode(&v));
+							println!("  value(base64): {}", BASE64_STANDARD.encode(&v));
 						}
 					}
 					K2vValue::Tombstone => {
@@ -310,23 +311,19 @@ impl BatchOutputKind {
 			.collect::<Vec<_>>()
 	}

-	fn display_poll_range_output(
-		&self,
-		seen_marker: String,
-		values: BTreeMap<String, CausalValue>,
-	) -> ! {
+	fn display_poll_range_output(&self, poll_range: PollRangeResult) -> ! {
 		if self.json {
 			let json = serde_json::json!({
-				"values": self.values_json(values),
-				"seen_marker": seen_marker,
+				"values": self.values_json(poll_range.items),
+				"seen_marker": poll_range.seen_marker,
 			});

 			let stdout = std::io::stdout();
 			serde_json::to_writer_pretty(stdout, &json).unwrap();
 			exit(0)
 		} else {
-			println!("seen marker: {}", seen_marker);
-			self.display_human_output(values)
+			println!("seen marker: {}", poll_range.seen_marker);
+			self.display_human_output(poll_range.items)
 		}
 	}

@@ -393,16 +390,27 @@ impl Filter {

 #[tokio::main]
 async fn main() -> Result<(), Error> {
+	if std::env::var("RUST_LOG").is_err() {
+		std::env::set_var("RUST_LOG", "warn")
+	}
+
+	tracing_subscriber::fmt()
+		.with_writer(std::io::stderr)
+		.with_env_filter(tracing_subscriber::filter::EnvFilter::from_default_env())
+		.init();
+
 	let args = Args::parse();

-	let region = Region::Custom {
-		name: args.region,
+	let config = K2vClientConfig {
 		endpoint: args.endpoint,
+		region: args.region,
+		aws_access_key_id: args.key_id,
+		aws_secret_access_key: args.secret,
+		bucket: args.bucket,
+		user_agent: None,
 	};

-	let creds = AwsCredentials::new(args.key_id, args.secret, None, None);
-
-	let client = K2vClient::new(region, args.bucket, creds, None)?;
+	let client = K2vClient::new(config)?;

 	match args.command {
 		Command::Insert {
@@ -489,8 +497,8 @@ async fn main() -> Result<(), Error> {
 				)
 				.await?;
 			match res {
-				Some((items, seen_marker)) => {
-					output_kind.display_poll_range_output(seen_marker, items);
+				Some(poll_range_output) => {
+					output_kind.display_poll_range_output(poll_range_output);
 				}
 				None => {
 					if output_kind.json {
@@ -520,7 +528,7 @@ async fn main() -> Result<(), Error> {
 						value
 							.as_object_mut()
 							.unwrap()
-							.insert("sort_key".to_owned(), k.into());
+							.insert("partition_key".to_owned(), k.into());
 						value
 					})
 					.collect::<Vec<_>>();
@@ -537,7 +545,7 @@ async fn main() -> Result<(), Error> {
 				}

 				let mut to_print = Vec::new();
-				to_print.push(format!("key:\tentries\tconflicts\tvalues\tbytes"));
+				to_print.push(format!("partition_key\tentries\tconflicts\tvalues\tbytes"));
 				for (k, v) in res.items {
 					to_print.push(format!(
 						"{}\t{}\t{}\t{}\t{}",
@@ -18,12 +18,20 @@ pub enum Error {
 	NotFound,
 	#[error("io error: {0}")]
 	IoError(#[from] std::io::Error),
-	#[error("rusoto tls error: {0}")]
-	RusotoTls(#[from] rusoto_core::request::TlsError),
-	#[error("rusoto http error: {0}")]
-	RusotoHttp(#[from] rusoto_core::HttpDispatchError),
+	#[error("http error: {0}")]
+	Http(#[from] http::Error),
+	#[error("hyper error: {0}")]
+	Hyper(#[from] hyper::Error),
+	#[error("invalid header: {0}")]
+	Header(#[from] hyper::header::ToStrError),
 	#[error("deserialization error: {0}")]
 	Deserialization(#[from] serde_json::Error),
+	#[error("invalid signature parameters: {0}")]
+	SignParameters(#[from] aws_sigv4::signing_params::BuildError),
+	#[error("could not sign request: {0}")]
+	SignRequest(#[from] aws_sigv4::http_request::SigningError),
+	#[error("request timed out")]
+	Timeout,
 	#[error("{0}")]
 	Message(Cow<'static, str>),
 }
@@ -1,20 +1,23 @@
 use std::collections::BTreeMap;
-use std::time::Duration;
+use std::convert::TryInto;
+use std::time::{Duration, SystemTime};

-use http::header::{ACCEPT, CONTENT_LENGTH, CONTENT_TYPE};
-use http::status::StatusCode;
-use http::HeaderMap;
+use base64::prelude::*;
 use log::{debug, error};
+use percent_encoding::{utf8_percent_encode, AsciiSet, NON_ALPHANUMERIC};
+
+use http::header::{ACCEPT, CONTENT_TYPE};
+use http::status::StatusCode;
+use http::{HeaderName, HeaderValue, Request};
+use hyper::{body::Bytes, Body};
+use hyper::{client::connect::HttpConnector, Client as HttpClient};
+use hyper_rustls::HttpsConnector;
+
+use aws_sigv4::http_request::{sign, SignableRequest, SigningParams, SigningSettings};

-use rusoto_core::{ByteStream, DispatchSignedRequest, HttpClient};
-use rusoto_credential::AwsCredentials;
-use rusoto_signature::region::Region;
-use rusoto_signature::signature::SignedRequest;
 use serde::de::Error as DeError;
 use serde::{Deserialize, Deserializer, Serialize, Serializer};

-use tokio::io::AsyncReadExt;
-
 mod error;

 pub use error::Error;
@@ -22,41 +25,57 @@ pub use error::Error;
 const DEFAULT_TIMEOUT: Duration = Duration::from_secs(5);
 const DEFAULT_POLL_TIMEOUT: Duration = Duration::from_secs(300);
 const SERVICE: &str = "k2v";
-const GARAGE_CAUSALITY_TOKEN: &str = "X-Garage-Causality-Token";
+const AMZ_CONTENT_SHA256: HeaderName = HeaderName::from_static("x-amz-content-sha256");
+const GARAGE_CAUSALITY_TOKEN: HeaderName = HeaderName::from_static("x-garage-causality-token");
+
+const STRICT_ENCODE_SET: AsciiSet = NON_ALPHANUMERIC
+	.remove(b'_')
+	.remove(b'-')
+	.remove(b'.')
+	.remove(b'~');
+const PATH_ENCODE_SET: AsciiSet = NON_ALPHANUMERIC
+	.remove(b'/')
+	.remove(b'_')
+	.remove(b'-')
+	.remove(b'.')
+	.remove(b'~');
+
+pub struct K2vClientConfig {
+	pub endpoint: String,
+	pub region: String,
+	pub aws_access_key_id: String,
+	pub aws_secret_access_key: String,
+	pub bucket: String,
+	pub user_agent: Option<String>,
+}

 /// Client used to query a K2V server.
 pub struct K2vClient {
-	region: Region,
-	bucket: String,
-	creds: AwsCredentials,
-	client: HttpClient,
+	config: K2vClientConfig,
+	user_agent: HeaderValue,
+	client: HttpClient<HttpsConnector<HttpConnector>>,
 }

 impl K2vClient {
 	/// Create a new K2V client.
-	pub fn new(
-		region: Region,
-		bucket: String,
-		creds: AwsCredentials,
-		user_agent: Option<String>,
-	) -> Result<Self, Error> {
+	pub fn new(config: K2vClientConfig) -> Result<Self, Error> {
 		let connector = hyper_rustls::HttpsConnectorBuilder::new()
 			.with_native_roots()
 			.https_or_http()
 			.enable_http1()
 			.enable_http2()
 			.build();
-		let mut client = HttpClient::from_connector(connector);
-		if let Some(ua) = user_agent {
-			client.local_agent_prepend(ua);
-		} else {
-			client.local_agent_prepend(format!("k2v/{}", env!("CARGO_PKG_VERSION")));
-		}
+		let client = HttpClient::builder().build(connector);
+		let user_agent: std::borrow::Cow<str> = match &config.user_agent {
+			Some(ua) => ua.into(),
+			None => format!("k2v/{}", env!("CARGO_PKG_VERSION")).into(),
+		};
+		let user_agent = HeaderValue::from_str(&user_agent)
+			.map_err(|_| Error::Message("invalid user agent".into()))?;
 		Ok(K2vClient {
-			region,
-			bucket,
-			creds,
+			config,
 			client,
+			user_agent,
 		})
 	}

@@ -66,15 +85,10 @@ impl K2vClient {
 		partition_key: &str,
 		sort_key: &str,
 	) -> Result<CausalValue, Error> {
-		let mut req = SignedRequest::new(
-			"GET",
-			SERVICE,
-			&self.region,
-			&format!("/{}/{}", self.bucket, partition_key),
-		);
-		req.add_param("sort_key", sort_key);
-		req.add_header(ACCEPT, "application/octet-stream, application/json");
-
+		let url = self.build_url(Some(partition_key), &[("sort_key", sort_key)]);
+		let req = Request::get(url)
+			.header(ACCEPT, "application/octet-stream, application/json")
+			.body(Bytes::new())?;
 		let res = self.dispatch(req, None).await?;

 		let causality = res
@@ -91,7 +105,7 @@ impl K2vClient {
 		match res.content_type.as_deref() {
 			Some("application/octet-stream") => Ok(CausalValue {
 				causality,
-				value: vec![K2vValue::Value(res.body)],
+				value: vec![K2vValue::Value(res.body.to_vec())],
 			}),
 			Some("application/json") => {
 				let value = serde_json::from_slice(&res.body)?;
@@ -115,16 +129,17 @@ impl K2vClient {
 	) -> Result<Option<CausalValue>, Error> {
 		let timeout = timeout.unwrap_or(DEFAULT_POLL_TIMEOUT);

-		let mut req = SignedRequest::new(
-			"GET",
-			SERVICE,
-			&self.region,
-			&format!("/{}/{}", self.bucket, partition_key),
+		let url = self.build_url(
+			Some(partition_key),
+			&[
+				("sort_key", sort_key),
+				("causality_token", &causality.0),
+				("timeout", &timeout.as_secs().to_string()),
+			],
 		);
-		req.add_param("sort_key", sort_key);
-		req.add_param("causality_token", &causality.0);
-		req.add_param("timeout", &timeout.as_secs().to_string());
-		req.add_header(ACCEPT, "application/octet-stream, application/json");
+		let req = Request::get(url)
+			.header(ACCEPT, "application/octet-stream, application/json")
+			.body(Bytes::new())?;

 		let res = self.dispatch(req, Some(timeout + DEFAULT_TIMEOUT)).await?;

@@ -146,7 +161,7 @@ impl K2vClient {
 		match res.content_type.as_deref() {
 			Some("application/octet-stream") => Ok(Some(CausalValue {
 				causality,
-				value: vec![K2vValue::Value(res.body)],
+				value: vec![K2vValue::Value(res.body.to_vec())],
 			})),
 			Some("application/json") => {
 				let value = serde_json::from_slice(&res.body)?;
@@ -167,7 +182,7 @@ impl K2vClient {
 		filter: Option<PollRangeFilter<'_>>,
 		seen_marker: Option<&str>,
 		timeout: Option<Duration>,
-	) -> Result<Option<(BTreeMap<String, CausalValue>, String)>, Error> {
+	) -> Result<Option<PollRangeResult>, Error> {
 		let timeout = timeout.unwrap_or(DEFAULT_POLL_TIMEOUT);

 		let request = PollRangeRequest {
@@ -176,16 +191,10 @@ impl K2vClient {
 			timeout: timeout.as_secs(),
 		};

-		let mut req = SignedRequest::new(
-			"POST",
-			SERVICE,
-			&self.region,
-			&format!("/{}/{}", self.bucket, partition_key),
-		);
-		req.add_param("poll_range", "");
-
+		let url = self.build_url(Some(partition_key), &[("poll_range", "")]);
 		let payload = serde_json::to_vec(&request)?;
-		req.set_payload(Some(payload));
+		let req = Request::post(url).body(Bytes::from(payload))?;
+
 		let res = self.dispatch(req, Some(timeout + DEFAULT_TIMEOUT)).await?;

 		if res.status == StatusCode::NOT_MODIFIED {
@@ -208,7 +217,10 @@ impl K2vClient {
 			})
 			.collect::<BTreeMap<_, _>>();

-		Ok(Some((items, resp.seen_marker)))
+		Ok(Some(PollRangeResult {
+			items,
+			seen_marker: resp.seen_marker,
+		}))
 	}

 	/// Perform an InsertItem request, inserting a value for a single pk+sk.
@@ -219,18 +231,12 @@ impl K2vClient {
 		value: Vec<u8>,
 		causality: Option<CausalityToken>,
 	) -> Result<(), Error> {
-		let mut req = SignedRequest::new(
-			"PUT",
-			SERVICE,
-			&self.region,
-			&format!("/{}/{}", self.bucket, partition_key),
-		);
-		req.add_param("sort_key", sort_key);
-		req.set_payload(Some(value));
-
+		let url = self.build_url(Some(partition_key), &[("sort_key", sort_key)]);
+		let mut req = Request::put(url);
 		if let Some(causality) = causality {
-			req.add_header(GARAGE_CAUSALITY_TOKEN, &causality.0);
+			req = req.header(GARAGE_CAUSALITY_TOKEN, &causality.0);
 		}
+		let req = req.body(Bytes::from(value))?;

 		self.dispatch(req, None).await?;
 		Ok(())
@@ -243,14 +249,10 @@ impl K2vClient {
 		sort_key: &str,
 		causality: CausalityToken,
 	) -> Result<(), Error> {
-		let mut req = SignedRequest::new(
-			"DELETE",
-			SERVICE,
-			&self.region,
-			&format!("/{}/{}", self.bucket, partition_key),
-		);
-		req.add_param("sort_key", sort_key);
-		req.add_header(GARAGE_CAUSALITY_TOKEN, &causality.0);
+		let url = self.build_url(Some(partition_key), &[("sort_key", sort_key)]);
+		let req = Request::delete(url)
+			.header(GARAGE_CAUSALITY_TOKEN, &causality.0)
+			.body(Bytes::new())?;

 		self.dispatch(req, None).await?;
 		Ok(())
@@ -262,9 +264,9 @@ impl K2vClient {
 		&self,
 		filter: Filter<'_>,
 	) -> Result<PaginatedRange<PartitionInfo>, Error> {
-		let mut req =
-			SignedRequest::new("GET", SERVICE, &self.region, &format!("/{}", self.bucket));
-		filter.insert_params(&mut req);
+		let params = filter.query_params();
+		let url = self.build_url(None, &params);
+		let req = Request::get(url).body(Bytes::new())?;

 		let res = self.dispatch(req, None).await?;

@@ -286,11 +288,10 @@ impl K2vClient {
 	/// *not* atomic: it is possible for some sub-operations to fails and others to success. In
 	/// that case, failure is reported.
 	pub async fn insert_batch(&self, operations: &[BatchInsertOp<'_>]) -> Result<(), Error> {
-		let mut req =
-			SignedRequest::new("POST", SERVICE, &self.region, &format!("/{}", self.bucket));
-
+		let url = self.build_url::<&str>(None, &[]);
 		let payload = serde_json::to_vec(operations)?;
-		req.set_payload(Some(payload));
+		let req = Request::post(url).body(payload.into())?;
+
 		self.dispatch(req, None).await?;
 		Ok(())
 	}
@@ -300,12 +301,10 @@ impl K2vClient {
 		&self,
 		operations: &[BatchReadOp<'_>],
 	) -> Result<Vec<PaginatedRange<CausalValue>>, Error> {
-		let mut req =
-			SignedRequest::new("POST", SERVICE, &self.region, &format!("/{}", self.bucket));
-		req.add_param("search", "");
-
+		let url = self.build_url(None, &[("search", "")]);
 		let payload = serde_json::to_vec(operations)?;
-		req.set_payload(Some(payload));
+		let req = Request::post(url).body(payload.into())?;
+
 		let res = self.dispatch(req, None).await?;

 		let resp: Vec<BatchReadResponse> = serde_json::from_slice(&res.body)?;
@@ -334,12 +333,10 @@ impl K2vClient {
 	/// Perform a DeleteBatch request, deleting mutiple values or range of values at once, without
 	/// providing causality information.
 	pub async fn delete_batch(&self, operations: &[BatchDeleteOp<'_>]) -> Result<Vec<u64>, Error> {
-		let mut req =
-			SignedRequest::new("POST", SERVICE, &self.region, &format!("/{}", self.bucket));
-		req.add_param("delete", "");
-
+		let url = self.build_url(None, &[("delete", "")]);
 		let payload = serde_json::to_vec(operations)?;
-		req.set_payload(Some(payload));
+		let req = Request::post(url).body(payload.into())?;
+
 		let res = self.dispatch(req, None).await?;

 		let resp: Vec<BatchDeleteResponse> = serde_json::from_slice(&res.body)?;
@@ -349,33 +346,67 @@ impl K2vClient {

 	async fn dispatch(
 		&self,
-		mut req: SignedRequest,
+		mut req: Request<Bytes>,
 		timeout: Option<Duration>,
 	) -> Result<Response, Error> {
-		req.sign(&self.creds);
-		let mut res = self
-			.client
-			.dispatch(req, Some(timeout.unwrap_or(DEFAULT_TIMEOUT)))
-			.await?;
+		req.headers_mut()
+			.insert(http::header::USER_AGENT, self.user_agent.clone());

-		let causality_token = res
-			.headers
-			.remove(GARAGE_CAUSALITY_TOKEN)
-			.map(CausalityToken);
-		let content_type = res.headers.remove(CONTENT_TYPE);
+		use sha2::{Digest, Sha256};
+		let mut hasher = Sha256::new();
+		hasher.update(req.body());
+		let hash = hex::encode(&hasher.finalize());
+		req.headers_mut()
+			.insert(AMZ_CONTENT_SHA256, hash.try_into().unwrap());
+
+		debug!("request uri: {:?}", req.uri());
+
+		// Sign request
+		let signing_settings = SigningSettings::default();
+		let signing_params = SigningParams::builder()
+			.access_key(&self.config.aws_access_key_id)
+			.secret_key(&self.config.aws_secret_access_key)
+			.region(&self.config.region)
+			.service_name(SERVICE)
+			.time(SystemTime::now())
+			.settings(signing_settings)
+			.build()?;
+		// Convert the HTTP request into a signable request
+		let signable_request = SignableRequest::from(&req);
+
+		// Sign and then apply the signature to the request
+		let (signing_instructions, _signature) =
+			sign(signable_request, &signing_params)?.into_parts();
+		signing_instructions.apply_to_request(&mut req);
+
+		// Send and wait for timeout
+		let res = tokio::select! {
+			res = self.client.request(req.map(Body::from)) => res?,
+			_ = tokio::time::sleep(timeout.unwrap_or(DEFAULT_TIMEOUT)) => {
+				return Err(Error::Timeout);
+			}
+		};
+
+		let (mut res, body) = res.into_parts();
+		let causality_token = match res.headers.remove(GARAGE_CAUSALITY_TOKEN) {
+			Some(v) => Some(CausalityToken(v.to_str()?.to_string())),
+			None => None,
+		};
+		let content_type = match res.headers.remove(CONTENT_TYPE) {
+			Some(v) => Some(v.to_str()?.to_string()),
+			None => None,
+		};

 		let body = match res.status {
-			StatusCode::OK => read_body(&mut res.headers, res.body).await?,
-			StatusCode::NO_CONTENT => Vec::new(),
+			StatusCode::OK => hyper::body::to_bytes(body).await?,
+			StatusCode::NO_CONTENT => Bytes::new(),
 			StatusCode::NOT_FOUND => return Err(Error::NotFound),
-			StatusCode::NOT_MODIFIED => Vec::new(),
+			StatusCode::NOT_MODIFIED => Bytes::new(),
 			s => {
-				let err_body = read_body(&mut res.headers, res.body)
-					.await
-					.unwrap_or_default();
+				let err_body = hyper::body::to_bytes(body).await.unwrap_or_default();
 				let err_body_str = std::str::from_utf8(&err_body)
 					.map(String::from)
-					.unwrap_or_else(|_| base64::encode(&err_body));
+					.unwrap_or_else(|_| BASE64_STANDARD.encode(&err_body));

 				if s.is_client_error() || s.is_server_error() {
 					error!("Error response {}: {}", res.status, err_body_str);
@@ -408,7 +439,7 @@ impl K2vClient {
 			"Response body: {}",
 			std::str::from_utf8(&body)
 				.map(String::from)
-				.unwrap_or_else(|_| base64::encode(&body))
+				.unwrap_or_else(|_| BASE64_STANDARD.encode(&body))
 		);

 		Ok(Response {
@@ -418,16 +449,26 @@ impl K2vClient {
 			content_type,
 		})
 	}
-}

-async fn read_body(headers: &mut HeaderMap<String>, body: ByteStream) -> Result<Vec<u8>, Error> {
-	let body_len = headers
-		.get(CONTENT_LENGTH)
-		.and_then(|h| h.parse().ok())
-		.unwrap_or(0);
-	let mut res = Vec::with_capacity(body_len);
-	body.into_async_read().read_to_end(&mut res).await?;
-	Ok(res)
+	fn build_url<V: AsRef<str>>(&self, partition_key: Option<&str>, query: &[(&str, V)]) -> String {
+		let mut url = format!("{}/{}", self.config.endpoint, self.config.bucket);
+		if let Some(pk) = partition_key {
+			url.push('/');
+			url.extend(utf8_percent_encode(pk, &PATH_ENCODE_SET));
+		}
+		if !query.is_empty() {
+			url.push('?');
+			for (i, (k, v)) in query.iter().enumerate() {
+				if i > 0 {
+					url.push('&');
+				}
+				url.extend(utf8_percent_encode(k, &STRICT_ENCODE_SET));
+				url.push('=');
+				url.extend(utf8_percent_encode(v.as_ref(), &STRICT_ENCODE_SET));
+			}
+		}
+		url
+	}
 }

 /// An opaque token used to convey causality between operations.
@@ -482,9 +523,11 @@ impl<'de> Deserialize<'de> for K2vValue {
 	{
 		let val: Option<&str> = Option::deserialize(d)?;
 		Ok(match val {
-			Some(s) => {
-				K2vValue::Value(base64::decode(s).map_err(|_| DeError::custom("invalid base64"))?)
-			}
+			Some(s) => K2vValue::Value(
+				BASE64_STANDARD
+					.decode(s)
+					.map_err(|_| DeError::custom("invalid base64"))?,
+			),
 			None => K2vValue::Tombstone,
 		})
 	}
@@ -498,7 +541,7 @@ impl Serialize for K2vValue {
 		match self {
 			K2vValue::Tombstone => serializer.serialize_none(),
 			K2vValue::Value(v) => {
-				let b64 = base64::encode(v);
+				let b64 = BASE64_STANDARD.encode(v);
 				serializer.serialize_str(&b64)
 			}
 		}
@@ -530,6 +573,7 @@ pub struct Filter<'a> {
 	pub reverse: bool,
 }

+/// Filter for a poll range operations.
 #[derive(Debug, Default, Clone, Serialize)]
 pub struct PollRangeFilter<'a> {
 	pub start: Option<&'a str>,
@@ -537,6 +581,15 @@ pub struct PollRangeFilter<'a> {
 	pub prefix: Option<&'a str>,
 }

+/// Response to a poll_range query
+#[derive(Debug, Default, Clone, Serialize)]
+pub struct PollRangeResult {
+	/// List of items that have changed since last PollRange call.
+	pub items: BTreeMap<String, CausalValue>,
+	/// opaque string representing items already seen for future PollRange calls.
+	pub seen_marker: String,
+}
+
 #[derive(Debug, Clone, Serialize)]
 #[serde(rename_all = "camelCase")]
 struct PollRangeRequest<'a> {
@@ -554,22 +607,24 @@ struct PollRangeResponse {
 }

 impl<'a> Filter<'a> {
-	fn insert_params(&self, req: &mut SignedRequest) {
-		if let Some(start) = &self.start {
-			req.add_param("start", start);
+	fn query_params(&self) -> Vec<(&'static str, std::borrow::Cow<str>)> {
+		let mut res = Vec::<(&'static str, std::borrow::Cow<str>)>::with_capacity(8);
+		if let Some(start) = self.start.as_deref() {
+			res.push(("start", start.into()));
 		}
-		if let Some(end) = &self.end {
-			req.add_param("end", end);
+		if let Some(end) = self.end.as_deref() {
+			res.push(("end", end.into()));
 		}
-		if let Some(prefix) = &self.prefix {
-			req.add_param("prefix", prefix);
+		if let Some(prefix) = self.prefix.as_deref() {
+			res.push(("prefix", prefix.into()));
 		}
 		if let Some(limit) = &self.limit {
-			req.add_param("limit", &limit.to_string());
+			res.push(("limit", limit.to_string().into()));
 		}
 		if self.reverse {
-			req.add_param("reverse", "true");
+			res.push(("reverse", "true".into()));
 		}
+		res
 	}
 }

@@ -691,7 +746,7 @@ struct ErrorResponse {
 }

 struct Response {
-	body: Vec<u8>,
+	body: Bytes,
 	status: StatusCode,
 	causality_token: Option<CausalityToken>,
 	content_type: Option<String>,
@@ -1,6 +1,6 @@
 [package]
 name = "garage_model"
-version = "0.8.2"
+version = "0.8.3"
 authors = ["Alex Auvolat <alex@adnab.me>"]
 edition = "2018"
 license = "AGPL-3.0"
@@ -14,11 +14,11 @@ path = "lib.rs"
 # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

 [dependencies]
-garage_db = { version = "0.8.2", default-features = false, path = "../db" }
-garage_rpc = { version = "0.8.2", path = "../rpc" }
-garage_table = { version = "0.8.2", path = "../table" }
-garage_block = { version = "0.8.2", path = "../block" }
-garage_util = { version = "0.8.2", path = "../util" }
+garage_db.workspace = true
+garage_rpc.workspace = true
+garage_table.workspace = true
+garage_block.workspace = true
+garage_util.workspace = true

 async-trait = "0.1.7"
 arc-swap = "1.0"
@@ -79,7 +79,7 @@ impl RangeSeenMarker {

 		let bytes = nonversioned_encode(&self)?;
 		let bytes = zstd::stream::encode_all(&mut &bytes[..], zstd::DEFAULT_COMPRESSION_LEVEL)?;
-		Ok(BASE64_STANDARD.encode(&bytes))
+		Ok(BASE64_STANDARD.encode(bytes))
 	}

 	/// Decode from msgpack+zstd+b64 representation, returns None on error.
@@ -1,6 +1,6 @@
 [package]
 name = "garage_rpc"
-version = "0.8.2"
+version = "0.8.3"
 authors = ["Alex Auvolat <alex@adnab.me>"]
 edition = "2018"
 license = "AGPL-3.0"
@@ -14,11 +14,11 @@ path = "lib.rs"
 # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

 [dependencies]
-garage_util = { version = "0.8.2", path = "../util" }
+garage_util.workspace = true

 arc-swap = "1.0"
 bytes = "1.0"
-gethostname = "0.2"
+gethostname = "0.4"
 hex = "0.4"
 tracing = "0.1"
 rand = "0.8"
@@ -8,16 +8,26 @@ use serde::{Deserialize, Serialize};

 use netapp::NodeID;

+use garage_util::config::ConsulDiscoveryAPI;
 use garage_util::config::ConsulDiscoveryConfig;

+const META_PREFIX: &str = "fr-deuxfleurs-garagehq";
+
 #[derive(Deserialize, Clone, Debug)]
 struct ConsulQueryEntry {
 	#[serde(rename = "Address")]
 	address: String,
 	#[serde(rename = "ServicePort")]
 	service_port: u16,
-	#[serde(rename = "NodeMeta")]
-	node_meta: HashMap<String, String>,
+	#[serde(rename = "ServiceMeta")]
+	meta: HashMap<String, String>,
+}
+
+#[derive(Serialize, Clone, Debug)]
+#[serde(untagged)]
+enum PublishRequest {
+	Catalog(ConsulPublishEntry),
+	Service(ConsulPublishService),
 }

 #[derive(Serialize, Clone, Debug)]
@@ -26,17 +36,31 @@ struct ConsulPublishEntry {
 	node: String,
 	#[serde(rename = "Address")]
 	address: IpAddr,
-	#[serde(rename = "NodeMeta")]
-	node_meta: HashMap<String, String>,
 	#[serde(rename = "Service")]
-	service: ConsulPublishService,
+	service: ConsulPublishCatalogService,
+}
+
+#[derive(Serialize, Clone, Debug)]
+struct ConsulPublishCatalogService {
+	#[serde(rename = "ID")]
+	service_id: String,
+	#[serde(rename = "Service")]
+	service_name: String,
+	#[serde(rename = "Tags")]
+	tags: Vec<String>,
+	#[serde(rename = "Meta")]
+	meta: HashMap<String, String>,
+	#[serde(rename = "Address")]
+	address: IpAddr,
+	#[serde(rename = "Port")]
+	port: u16,
 }

 #[derive(Serialize, Clone, Debug)]
 struct ConsulPublishService {
 	#[serde(rename = "ID")]
 	service_id: String,
-	#[serde(rename = "Service")]
+	#[serde(rename = "Name")]
 	service_name: String,
 	#[serde(rename = "Tags")]
 	tags: Vec<String>,
@@ -44,10 +68,11 @@ struct ConsulPublishService {
 	address: IpAddr,
 	#[serde(rename = "Port")]
 	port: u16,
+	#[serde(rename = "Meta")]
+	meta: HashMap<String, String>,
 }

 // ----
-
 pub struct ConsulDiscovery {
 	config: ConsulDiscoveryConfig,
 	client: reqwest::Client,
@@ -55,44 +80,48 @@ pub struct ConsulDiscovery {

 impl ConsulDiscovery {
 	pub fn new(config: ConsulDiscoveryConfig) -> Result<Self, ConsulError> {
-		let client = match (&config.client_cert, &config.client_key) {
-			(Some(client_cert), Some(client_key)) => {
-				let mut client_cert_buf = vec![];
-				File::open(client_cert)?.read_to_end(&mut client_cert_buf)?;
+		let mut builder: reqwest::ClientBuilder = reqwest::Client::builder().use_rustls_tls();
+		if config.tls_skip_verify {
+			builder = builder.danger_accept_invalid_certs(true);
+		} else if let Some(ca_cert) = &config.ca_cert {
+			let mut ca_cert_buf = vec![];
+			File::open(ca_cert)?.read_to_end(&mut ca_cert_buf)?;
+			builder =
+				builder.add_root_certificate(reqwest::Certificate::from_pem(&ca_cert_buf[..])?);
+		}

-				let mut client_key_buf = vec![];
-				File::open(client_key)?.read_to_end(&mut client_key_buf)?;
+		match &config.api {
+			ConsulDiscoveryAPI::Catalog => match (&config.client_cert, &config.client_key) {
+				(Some(client_cert), Some(client_key)) => {
+					let mut client_cert_buf = vec![];
+					File::open(client_cert)?.read_to_end(&mut client_cert_buf)?;

-				let identity = reqwest::Identity::from_pem(
-					&[&client_cert_buf[..], &client_key_buf[..]].concat()[..],
-				)?;
+					let mut client_key_buf = vec![];
+					File::open(client_key)?.read_to_end(&mut client_key_buf)?;

-				if config.tls_skip_verify {
-					reqwest::Client::builder()
-						.use_rustls_tls()
-						.danger_accept_invalid_certs(true)
-						.identity(identity)
-						.build()?
-				} else if let Some(ca_cert) = &config.ca_cert {
-					let mut ca_cert_buf = vec![];
-					File::open(ca_cert)?.read_to_end(&mut ca_cert_buf)?;
+					let identity = reqwest::Identity::from_pem(
+						&[&client_cert_buf[..], &client_key_buf[..]].concat()[..],
+					)?;

-					reqwest::Client::builder()
-						.use_rustls_tls()
-						.add_root_certificate(reqwest::Certificate::from_pem(&ca_cert_buf[..])?)
-						.identity(identity)
-						.build()?
-				} else {
-					reqwest::Client::builder()
-						.use_rustls_tls()
-						.identity(identity)
-						.build()?
+					builder = builder.identity(identity);
+				}
+				(None, None) => {}
+				_ => return Err(ConsulError::InvalidTLSConfig),
+			},
+			ConsulDiscoveryAPI::Agent => {
+				if let Some(token) = &config.token {
+					let mut headers = reqwest::header::HeaderMap::new();
+					headers.insert(
+						"x-consul-token",
+						reqwest::header::HeaderValue::from_str(&token)?,
+					);
+					builder = builder.default_headers(headers);
 				}
 			}
-			(None, None) => reqwest::Client::new(),
-			_ => return Err(ConsulError::InvalidTLSConfig),
 		};

+		let client: reqwest::Client = builder.build()?;
+
 		Ok(Self { client, config })
 	}

@@ -111,8 +140,8 @@ impl ConsulDiscovery {
 		for ent in entries {
 			let ip = ent.address.parse::<IpAddr>().ok();
 			let pubkey = ent
-				.node_meta
-				.get("pubkey")
+				.meta
+				.get(&format!("{}-pubkey", META_PREFIX))
 				.and_then(|k| hex::decode(k).ok())
 				.and_then(|k| NodeID::from_slice(&k[..]));
 			if let (Some(ip), Some(pubkey)) = (ip, pubkey) {
@@ -138,29 +167,49 @@ impl ConsulDiscovery {
 		rpc_public_addr: SocketAddr,
 	) -> Result<(), ConsulError> {
 		let node = format!("garage:{}", hex::encode(&node_id[..8]));
+		let tags = [
+			vec!["advertised-by-garage".into(), hostname.into()],
+			self.config.tags.clone(),
+		]
+		.concat();

-		let advertisement = ConsulPublishEntry {
-			node: node.clone(),
-			address: rpc_public_addr.ip(),
-			node_meta: [
-				("pubkey".to_string(), hex::encode(node_id)),
-				("hostname".to_string(), hostname.to_string()),
-			]
-			.iter()
-			.cloned()
-			.collect(),
-			service: ConsulPublishService {
+		let mut meta = self.config.meta.clone().unwrap_or_default();
+		meta.insert(format!("{}-pubkey", META_PREFIX), hex::encode(node_id));
+		meta.insert(format!("{}-hostname", META_PREFIX), hostname.to_string());
+
+		let url = format!(
+			"{}/v1/{}",
+			self.config.consul_http_addr,
+			(match &self.config.api {
+				ConsulDiscoveryAPI::Catalog => "catalog/register",
+				ConsulDiscoveryAPI::Agent => "agent/service/register?replace-existing-checks",
+			})
+		);
+
+		let req = self.client.put(&url);
+		let advertisement: PublishRequest = match &self.config.api {
+			ConsulDiscoveryAPI::Catalog => PublishRequest::Catalog(ConsulPublishEntry {
+				node: node.clone(),
+				address: rpc_public_addr.ip(),
+				service: ConsulPublishCatalogService {
+					service_id: node.clone(),
+					service_name: self.config.service_name.clone(),
+					tags,
+					meta: meta.clone(),
+					address: rpc_public_addr.ip(),
+					port: rpc_public_addr.port(),
+				},
+			}),
+			ConsulDiscoveryAPI::Agent => PublishRequest::Service(ConsulPublishService {
 				service_id: node.clone(),
 				service_name: self.config.service_name.clone(),
-				tags: vec!["advertised-by-garage".into(), hostname.into()],
+				tags,
+				meta,
 				address: rpc_public_addr.ip(),
 				port: rpc_public_addr.port(),
-			},
+			}),
 		};
-
-		let url = format!("{}/v1/catalog/register", self.config.consul_http_addr);
-
-		let http = self.client.put(&url).json(&advertisement).send().await?;
+		let http = req.json(&advertisement).send().await?;
 		http.error_for_status()?;

 		Ok(())
@@ -176,4 +225,6 @@ pub enum ConsulError {
 	Reqwest(#[error(source)] reqwest::Error),
 	#[error(display = "Invalid Consul TLS configuration")]
 	InvalidTLSConfig,
+	#[error(display = "Token error: {}", _0)]
+	Token(#[error(source)] reqwest::header::InvalidHeaderValue),
 }
@@ -1,6 +1,6 @@
 [package]
 name = "garage_table"
-version = "0.8.2"
+version = "0.8.3"
 authors = ["Alex Auvolat <alex@adnab.me>"]
 edition = "2018"
 license = "AGPL-3.0"
@@ -14,9 +14,9 @@ path = "lib.rs"
 # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

 [dependencies]
-garage_db = { version = "0.8.2", path = "../db" }
-garage_rpc = { version = "0.8.2", path = "../rpc" }
-garage_util = { version = "0.8.2", path = "../util" }
+garage_db.workspace = true
+garage_rpc.workspace = true
+garage_util.workspace = true

 opentelemetry = "0.17"

@@ -44,22 +44,22 @@ pub struct TableData<F: TableSchema, R: TableReplication> {
 impl<F: TableSchema, R: TableReplication> TableData<F, R> {
 	pub fn new(system: Arc<System>, instance: F, replication: R, db: &db::Db) -> Arc<Self> {
 		let store = db
-			.open_tree(&format!("{}:table", F::TABLE_NAME))
+			.open_tree(format!("{}:table", F::TABLE_NAME))
 			.expect("Unable to open DB tree");

 		let merkle_tree = db
-			.open_tree(&format!("{}:merkle_tree", F::TABLE_NAME))
+			.open_tree(format!("{}:merkle_tree", F::TABLE_NAME))
 			.expect("Unable to open DB Merkle tree tree");
 		let merkle_todo = db
-			.open_tree(&format!("{}:merkle_todo", F::TABLE_NAME))
+			.open_tree(format!("{}:merkle_todo", F::TABLE_NAME))
 			.expect("Unable to open DB Merkle TODO tree");

 		let insert_queue = db
-			.open_tree(&format!("{}:insert_queue", F::TABLE_NAME))
+			.open_tree(format!("{}:insert_queue", F::TABLE_NAME))
 			.expect("Unable to open insert queue DB tree");

 		let gc_todo = db
-			.open_tree(&format!("{}:gc_todo_v2", F::TABLE_NAME))
+			.open_tree(format!("{}:gc_todo_v2", F::TABLE_NAME))
 			.expect("Unable to open GC DB tree");
 		let gc_todo = CountedTree::new(gc_todo).expect("Cannot count gc_todo_v2");

@@ -90,7 +90,7 @@ impl<F: TableSchema, R: TableReplication> TableData<F, R> {

 	pub fn read_entry(&self, p: &F::P, s: &F::S) -> Result<Option<ByteBuf>, Error> {
 		let tree_key = self.tree_key(p, s);
-		if let Some(bytes) = self.store.get(&tree_key)? {
+		if let Some(bytes) = self.store.get(tree_key)? {
 			Ok(Some(ByteBuf::from(bytes.to_vec())))
 		} else {
 			Ok(None)
@@ -132,10 +132,10 @@ impl<F: TableSchema, R: TableReplication> TableData<F, R> {
 		}
 	}

-	fn read_range_aux<'a>(
+	fn read_range_aux(
 		&self,
 		partition_hash: Hash,
-		range: db::ValueIter<'a>,
+		range: db::ValueIter,
 		filter: &Option<F::Filter>,
 		limit: usize,
 	) -> Result<Vec<Arc<ByteBuf>>, Error> {
@@ -34,8 +34,9 @@ impl DeletedFilter {
 	}
 }

-#[derive(Clone, Copy, Debug, Serialize, Deserialize, PartialEq, Eq)]
+#[derive(Clone, Copy, Debug, Default, Serialize, Deserialize, PartialEq, Eq)]
 pub enum EnumerationOrder {
+	#[default]
 	Forward,
 	Reverse,
 }
@@ -49,9 +50,3 @@ impl EnumerationOrder {
 		}
 	}
 }
-
-impl Default for EnumerationOrder {
-	fn default() -> Self {
-		EnumerationOrder::Forward
-	}
-}
@@ -1,6 +1,6 @@
 [package]
 name = "garage_util"
-version = "0.8.2"
+version = "0.8.3"
 authors = ["Alex Auvolat <alex@adnab.me>"]
 edition = "2018"
 license = "AGPL-3.0"
@@ -14,7 +14,7 @@ path = "lib.rs"
 # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

 [dependencies]
-garage_db = { version = "0.8.2", path = "../db" }
+garage_db.workspace = true

 arc-swap = "1.0"
 async-trait = "0.1"
@@ -22,7 +22,6 @@ blake2 = "0.10"
 bytes = "1.0"
 digest = "0.10"
 err-derive = "0.3"
-git-version = "0.3.4"
 hexdump = "0.1"
 xxhash-rust = { version = "0.8", default-features = false, features = ["xxh3"] }
 hex = "0.4"
@@ -135,8 +135,19 @@ pub struct AdminConfig {
 	pub trace_sink: Option<String>,
 }

+#[derive(Deserialize, Debug, Clone, Default)]
+#[serde(rename_all = "lowercase")]
+pub enum ConsulDiscoveryAPI {
+	#[default]
+	Catalog,
+	Agent,
+}
+
 #[derive(Deserialize, Debug, Clone)]
 pub struct ConsulDiscoveryConfig {
+	/// The consul api to use when registering: either `catalog` (the default) or `agent`
+	#[serde(default)]
+	pub api: ConsulDiscoveryAPI,
 	/// Consul http or https address to connect to to discover more peers
 	pub consul_http_addr: String,
 	/// Consul service name to use
@@ -147,9 +158,17 @@ pub struct ConsulDiscoveryConfig {
 	pub client_cert: Option<String>,
 	/// Client TLS key to use when connecting to Consul
 	pub client_key: Option<String>,
+	/// /// Token to use for connecting to consul
+	pub token: Option<String>,
 	/// Skip TLS hostname verification
 	#[serde(default)]
 	pub tls_skip_verify: bool,
+	/// Additional tags to add to the service
+	#[serde(default)]
+	pub tags: Vec<String>,
+	/// Additional service metadata to add
+	#[serde(default)]
+	pub meta: Option<std::collections::HashMap<String, String>>,
 }

 #[derive(Deserialize, Debug, Clone)]
@@ -223,7 +242,7 @@ fn secret_from_file(
 			#[cfg(unix)]
 			if std::env::var("GARAGE_ALLOW_WORLD_READABLE_SECRETS").as_deref() != Ok("true") {
 				use std::os::unix::fs::MetadataExt;
-				let metadata = std::fs::metadata(&file_path)?;
+				let metadata = std::fs::metadata(file_path)?;
 				if metadata.mode() & 0o077 != 0 {
 					return Err(format!("File {} is world-readable! (mode: 0{:o}, expected 0600)\nRefusing to start until this is fixed, or environment variable GARAGE_ALLOW_WORLD_READABLE_SECRETS is set to true.", file_path, metadata.mode()).into());
 				}
@@ -344,7 +363,7 @@ mod tests {
 			replication_mode = "3"
 			rpc_bind_addr = "[::]:3901"
 			rpc_secret_file = "{}"
-		
+
 			[s3_api]
 			s3_region = "garage"
 			api_bind_addr = "[::]:3900"
@@ -388,7 +407,7 @@ mod tests {
 			rpc_bind_addr = "[::]:3901"
 			rpc_secret= "dummy"
 			rpc_secret_file = "dummy"
-		
+
 			[s3_api]
 			s3_region = "garage"
 			api_bind_addr = "[::]:3900"
@@ -13,7 +13,7 @@ pub fn handle_forwarded_for_headers(headers: &HeaderMap<HeaderValue>) -> Result<
 		.to_str()
 		.ok_or_message("Error parsing X-Forwarded-For header")?;

-	let client_ip = IpAddr::from_str(&forwarded_for_ip_str)
+	let client_ip = IpAddr::from_str(forwarded_for_ip_str)
 		.ok_or_message("Valid IP address not found in X-Forwarded-For header")?;

 	Ok(client_ip.to_string())
@@ -10,7 +10,6 @@ pub mod crdt;
 pub mod data;
 pub mod encode;
 pub mod error;
-pub mod formater;
 pub mod forwarded_headers;
 pub mod metrics;
 pub mod migrate;
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Alex Auvolat	cece1be1bb	bump version to 0.8.3	2023-08-28 13:17:26 +02:00
Alex Auvolat	769b6fe054	fix test_website_check_domain	2023-08-28 12:40:28 +02:00
Alex Auvolat	e66c78d6ea	integration test: move json_body to root of crate	2023-08-28 12:32:57 +02:00
Alex Auvolat	51011e68b1	move alpine linux info to binary package page	2023-08-28 12:20:34 +02:00
Alex	a54a1f5616	Merge pull request 'doc: Add information about Alpine Linux package to Quick Start' (#564 ) from jirutka/garage:alpine into next-0.8 Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/564	2023-08-28 10:18:33 +00:00
Alex Auvolat	9b4ce4a8ad	admin api: refactor caddy check api code	2023-08-28 12:17:10 +02:00
Alex	2bbe2da5ad	Merge pull request 'support index on path missing a trailing slash' (#612 ) from compat/index-without-trailing-slash into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/612	2023-08-28 10:15:01 +00:00
Alex	29353adbe5	Merge pull request 'cargo: Bump dependencies' (#606 ) from jpds/garage:cargo-bumps-230801 into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/606	2023-08-28 10:13:39 +00:00
Alex Auvolat	c5cafa0000	web_server.rs: handle error properly and refactor	2023-08-28 12:05:14 +02:00
Alex Auvolat	74478443ec	update cargo.nix	2023-08-28 11:31:40 +02:00
Jonathan Davies	d66d81ae2d	cargo: Updated gethostname v0.2.3 -> v0.4.3.	2023-08-28 09:30:27 +00:00
Jonathan Davies	7d8296ec59	cargo: Updated pretty_env_logger v0.4.0 -> v0.5.0.	2023-08-28 09:30:27 +00:00
Jonathan Davies	f607ac6792	garage/api: cargo: Updated idna dependency to 0.4.	2023-08-28 09:30:27 +00:00
Jonathan Davies	96d1d81ab7	garage/db: cargo: Updated rusqlite to 0.29.	2023-08-28 09:30:27 +00:00
Jonathan Davies	5185701aa8	cargo: Updated: * addr2line v0.19.0 -> v0.20.0 * async-compression v0.4.0 -> v0.4.1 * clap v4.3.8 -> v4.3.19 * hyper v0.14.26 -> v0.14.27 * ipnet v2.7.2 -> v2.8.0 * rmp v0.8.11 -> v0.8.12 * serde v1.0.164 -> v1.0.188 * tokio v1.29.0 -> v1.31.0 * zstd v0.12.3+zstd.1.5.2 -> v0.12.4 * Others in `cargo update`	2023-08-28 09:30:27 +00:00
Alex	d539a56d3a	Merge pull request 'Support {s3,web}.root_domains for the Caddy on-demand TLS endpoint (<admin>/check?domain=xx)' (#610 ) from bug/support-root-domains-on-demand-tls into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/610	2023-08-28 09:18:13 +00:00
Alex	bd50333ade	Merge pull request 'reverse-proxy.md: Added caching section for Caddy.' (#614 ) from jpds/garage:caddy-cache into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/614	2023-08-28 08:51:33 +00:00
Alex	170c6a2eac	Merge pull request 'backup.md: Added restic-android note.' (#616 ) from jpds/garage:doc-restic-android into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/616	2023-08-28 08:50:57 +00:00
Jonathan Davies	7f7d85654d	backup.md: Added restic-android note.	2023-08-18 18:02:19 +01:00
Jonathan Davies	245a0882e1	reverse-proxy.md: Added caching section for Caddy.	2023-08-16 11:49:52 +01:00
Quentin Dufour	63da1d2443	support index on path missing a trailing slash	2023-08-08 15:28:57 +02:00
Quentin Dufour	24e533f262	support {s3,web}.root_domains in /check endpoint	2023-08-08 11:05:42 +02:00
Alex	67b1457c77	Merge pull request 'post_object.rs: Fixed typos / grammar.' (#607 ) from jpds/garage:post-objects-typos into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/607	2023-08-04 07:09:21 +00:00
Jonathan Davies	59bfc68f2e	post_object.rs: Fixed typos / grammar.	2023-08-01 15:31:39 +01:00
Alex	a98855157b	Merge pull request 'operations/durability-repairs-md: Fix typo' (#604 ) from maxjustus/garage:main into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/604	2023-07-28 14:31:51 +00:00
Max Justus Spransy	4d7bbf7878	operations/durability-repairs-md: Fix typo	2023-07-24 10:01:48 -07:00
Alex	18eb73d52e	Merge pull request 'flake-compat: use nix-community fork' (#599 ) from flokli/garage:flake-compat into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/599	2023-07-18 21:54:51 +00:00
Florian Klink	79ca8e76a4	nix/common.nix: use pattern from nix-community/flake-compat This is still a bit confusing, as normally the flake.defaultNix attrset gets exposed via a top-level default.nix, but at least it brings us closer to that.	2023-07-16 12:52:14 +03:00
Florian Klink	1bbf604224	flake.nix: switch to nix-community/flake-compat edolstra/flake-compat is unmaintained. cargo2nix also still pulls in edolstra/flake-compat, make it follow the nix-community one.	2023-07-16 12:40:47 +03:00
Alex	6ba611361e	Merge pull request 'tree-wide: fix some typos' (#598 ) from flokli/garage:fix-typos into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/598	2023-07-14 15:51:44 +00:00
Florian Klink	c855284760	src/util: fix typo	2023-07-14 14:25:40 +03:00
Florian Klink	b1ca1784a1	src/garage/cli: fix typo	2023-07-14 14:25:33 +03:00
Florian Klink	f0b7a0af3d	doc/drafts: fix typo	2023-07-14 14:25:14 +03:00
Florian Klink	194549ca46	doc/book: fix typo	2023-07-14 14:24:40 +03:00
Florian Klink	202d3f0e3c	doc/api: fix typo	2023-07-14 14:24:27 +03:00
Alex	7605d0cb11	Merge pull request 'cargo: tokio-1.29 and async-compression-0.4' (#593 ) from jpds/garage:tokio-1.29 into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/593	2023-07-03 17:03:09 +00:00
Alex Auvolat	031804171a	Update Cargo.nix	2023-07-03 11:33:36 +02:00
Jonathan Davies	aee0d97f22	cargo: Updated async-compression to 0.4.	2023-06-28 11:17:16 +01:00
Jonathan Davies	098c388f1b	cargo: Updated tokio to 1.29.	2023-06-28 11:16:41 +01:00
Alex	e716320b0a	Merge pull request 'cargo: roxmltree-0.18 and aws-sdk-s3-0.28 bump' (#591 ) from jpds/garage:roxmltree-0.18 into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/591	2023-06-27 17:20:58 +00:00
Alex	e466edbaec	Merge pull request 'introduce dedicated return type for PollRange' (#590 ) from trinity-1686a/garage:k2v-client-poll-range-result into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/590	2023-06-27 08:28:26 +00:00
Alex Auvolat	76355453dd	Update Cargo.nix	2023-06-27 10:23:02 +02:00
Alex	ee494f5aa2	Merge pull request 'don't build sqlite by default' (#592 ) from trinity-1686a/garage:dont-build-sqlite into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/592	2023-06-27 08:14:38 +00:00
Jonathan Davies	f31d98097a	Cargo.lock: Updated.	2023-06-26 18:03:47 +01:00
Jonathan Davies	a6da7e588f	tests/bucket.rs: Adjusted as previously used function is now private.	2023-06-26 18:03:43 +01:00
trinity-1686a	e5835704b7	don't build sqlite by default `bundled-libs` is enabled by default, and causes sqlite to be built too, even if the sqlite backend isn't enabled.	2023-06-26 11:15:11 +02:00
Jonathan Davies	7f8bf2d801	src/garage/tests: Updated types for aws-sdk-s3 bump.	2023-06-25 21:31:35 +01:00
Jonathan Davies	4297233d3e	garage/Cargo.toml: Updated aws-sdk-s3 to 0.28, added aws-config.	2023-06-25 21:17:15 +01:00
Jonathan Davies	b94ba47f29	api/Cargo.toml: Updated roxmltree to 0.18.	2023-06-24 14:15:26 +01:00
trinity-1686a	33b3cf8e22	introduce dedicated return type for PollRange	2023-06-24 10:17:20 +02:00
Alex	736083063f	Merge pull request 'doc: Added ejabberd S3 section' (#588 ) from jpds/garage:doc-ejabberd-s3 into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/588	2023-06-20 09:23:43 +00:00
Jonathan Davies	a5ae566e0b	apps/index.md: Fixed endpoint URL example.	2023-06-19 10:15:30 +01:00
Jonathan Davies	185f9e78f3	operations/durability-repairs.md: Added note about randomized scrub times.	2023-06-19 10:15:30 +01:00
Jonathan Davies	fb971a5f01	cookbook/encryption.md: Added Cyberduck note.	2023-06-19 10:15:30 +01:00
Jonathan Davies	6af2cde23f	cookbook/encryption.md: Added note on XMPP.	2023-06-19 10:15:30 +01:00
Jonathan Davies	97eb389274	docs/apps: Added ejabberd section.	2023-06-19 10:15:30 +01:00
Alex	5e291c64b3	Merge pull request 'Documentation updates' (#587 ) from doc-updates into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/587	2023-06-14 10:57:32 +00:00
Alex Auvolat	9092c71a01	doc: encryption organization	2023-06-14 12:51:47 +02:00
Alex Auvolat	120f8b3bfb	doc: better doc on systemd's DynamicUser (fix #430 )	2023-06-14 12:39:46 +02:00
Alex Auvolat	39c3738a07	Add a page about encryption (fix #416 )	2023-06-14 12:39:46 +02:00
Alex Auvolat	7169ee6ee6	doc: reformulate in monitoring page	2023-06-14 12:39:46 +02:00
Alex Auvolat	dd7533a260	doc: add an operations&maintenance section and move some pages there	2023-06-14 12:39:40 +02:00
Alex Auvolat	9233661967	Add documentation on durability and repair procedures (fix #219 )	2023-06-14 11:54:21 +02:00
Alex Auvolat	3aadba724d	doc: english improvement	2023-06-14 11:21:56 +02:00
Alex Auvolat	5a186be363	Doc: update goals, add docker alias Fix #235	2023-06-14 11:09:31 +02:00
Alex	01346143ca	Merge pull request 'Split src/garage/admin.rs into smaller files' (#586 ) from main-split-admin into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/586	2023-06-13 14:56:34 +00:00
Alex Auvolat	eb9cecf05c	Split garage/admin.rs into smaller files	2023-06-13 16:46:28 +02:00
Alex Auvolat	802ed75721	move admin.rs to admin/mod.rs, before splitting	2023-06-13 16:42:14 +02:00
Alex	fc29548933	Merge pull request 'fix timestamps wrapping around in `garage block list-errors` (fix #584 )' (#585 ) from fix-future-timestamps into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/585	2023-06-13 12:51:16 +00:00
Alex Auvolat	1ea4937c8b	fix timestamps wrapping around in `garage block list-errors` (fix #584 )	2023-06-12 20:07:33 +02:00
Alex	6aec73b641	Merge pull request 'payload.rs: Fixed two typoes' (#581 ) from jpds/garage:payload-typoes into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/581	2023-06-09 08:59:47 +00:00
Jonathan Davies	8a945ee996	payload.rs: Surround / in inverted commas.	2023-06-06 16:26:06 +01:00
Jonathan Davies	180992d0f1	payload.rs: Fixed typo in error message.	2023-06-06 16:25:29 +01:00
Alex	44548a9114	Merge pull request 'feature: Register consul services with agent API' (#567 ) from unrob/garage:roberto/consul-agent-registration into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/567 Reviewed-by: Alex <alex@adnab.me>	2023-06-02 14:35:00 +00:00
Roberto Hidalgo	32ad4538ee	fix references to old config names	2023-05-22 08:57:15 -06:00
Roberto Hidalgo	ef8a7add08	set default for [consul-services] api	2023-05-22 08:57:15 -06:00
Roberto Hidalgo	2d46d24d06	update docs	2023-05-22 08:57:15 -06:00
Roberto Hidalgo	b770504126	simplify code according to feedback	2023-05-22 08:57:15 -06:00
Roberto Hidalgo	6b69404f1a	rename mode to consul_http_api	2023-05-22 08:57:15 -06:00
Roberto Hidalgo	011f473048	revert rpc/Cargo.toml	2023-05-22 08:57:15 -06:00
Roberto Hidalgo	fd7dbea5b8	follow feedback, fold into existing feature	2023-05-22 08:57:15 -06:00
Roberto Hidalgo	bd6485565e	allow additional ServiceMeta, docs	2023-05-22 08:57:15 -06:00
Roberto Hidalgo	4d6e6fc155	cargo fmt	2023-05-22 08:57:15 -06:00
Roberto Hidalgo	02ba9016ab	register consul services against local agent instead of catalog api	2023-05-22 08:57:15 -06:00
Alex	9d833bb7ef	Merge pull request 'K2V-client improvements' (#577 ) from k2v-client-aws-sigv4 into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/577	2023-05-22 09:03:08 +00:00
Alex Auvolat	c3d3b837eb	bump k2v-client to v0.0.4	2023-05-22 10:47:15 +02:00
Alex Auvolat	130e01505b	Fix k2v_client with unicode in partition keys	2023-05-22 10:45:09 +02:00
Alex Auvolat	e2ce5970c6	Add basic k2v_client integration tests	2023-05-22 10:45:06 +02:00
Alex Auvolat	644e872264	Port k2v-client to aws-sigv4 since rusoto_signature is deprecated	2023-05-19 12:08:29 +02:00
Alex	03efc191c1	Merge pull request 'K2V: double urlencoding' (#574 ) from fix-k2v-urlencoding into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/574	2023-05-18 09:33:03 +00:00
Alex Auvolat	4420db7310	add tracing to k2v-client	2023-05-18 11:18:21 +02:00
Alex Auvolat	746b0090e4	k2v signature verification: double urlencoding (see comment in source code)	2023-05-18 11:18:06 +02:00
Alex	c26a4308b4	Merge pull request 'Split format_table into separate crate and reduce k2v-client dependencies' (#572 ) from split-format-table into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/572	2023-05-17 12:33:45 +00:00
Alex Auvolat	217d429937	fix clippy lint in format-table crate	2023-05-17 13:06:37 +02:00
Alex Auvolat	a1cec2cd60	Split format_table into separate crate and reduce k2v-client dependencies	2023-05-17 13:01:37 +02:00
Alex	b66f247580	Merge pull request 'fixes to K2V client' (#571 ) from k2v-client-fixes into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/571	2023-05-16 20:20:31 +00:00
Alex Auvolat	16f2a32bb7	cargo fmt	2023-05-16 19:46:57 +02:00
Alex Auvolat	472444ed8e	k2v-client 0.0.2	2023-05-16 19:46:57 +02:00
Alex Auvolat	bb03805b58	k2v-cli: fix sort_key being partition_key and specify which key	2023-05-16 19:46:57 +02:00
Alex Auvolat	e4f955d672	fix base64 uses	2023-05-16 19:46:56 +02:00
Alex	ea9b15f669	Merge pull request 'cargo: tokio-1.28 and hyper-0.14.26 update' (#569 ) from jpds/garage:tokio-1.28 into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/569	2023-05-11 10:16:33 +00:00
Alex Auvolat	2e6bb3f766	update Cargo.nix	2023-05-11 11:34:18 +02:00
Alex	375270afd1	Merge pull request '*: apply clippy recommendations.' (#570 ) from jpds/garage:clippy-fixes into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/570	2023-05-11 09:33:03 +00:00
Jonathan Davies	c783194e8b	*: apply clippy recommendations.	2023-05-09 20:49:34 +01:00
Jonathan Davies	fdcd7dee5a	Cargo.lock: Updated for: * tokio 1.28 * hyper 0.14.26	2023-05-09 14:43:52 +01:00
Jonathan Davies	0f0795103d	block/Cargo.toml: Bump tokio-util to 0.7.	2023-05-09 14:33:21 +01:00
Jonathan Davies	c9d26e8c50	k2v-client/Cargo.toml: Make tokio dep match other packages.	2023-05-09 14:33:00 +01:00
Alex	b925f53dc3	Merge pull request 'move git-version dependency to main crate to reduce rebuilds' (#568 ) from move-git-version into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/568	2023-05-09 09:53:33 +00:00
Alex	2f495575d8	Merge pull request 'block/manager.rs: Prioritize raw blocks when no compression configured' (#566 ) from jpds/garage:skip-compressed-blocks-scrub-no-compression into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/566	2023-05-09 09:39:48 +00:00
Alex Auvolat	9e0a9c1c15	move git-version dependency to main crate to reduce rebuilds	2023-05-09 11:35:32 +02:00
Jonathan Davies	9c788059e2	block/manager.rs: In is_block_compressed - check which compression_level is configured on a node and check for raw block first if compression is disabled (to help reduce syscalls during a scrub).	2023-05-09 10:28:19 +01:00
Alex	5684e1990c	Merge pull request 'Really allow to disable `sled` feature' (#563 ) from jirutka/garage:workspace-deps into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/563	2023-05-09 09:08:35 +00:00
Alex	14c50f2f84	Merge pull request 'Fix undefined macro warn! on 32-bit' (#562 ) from jirutka/garage:fix-undefined-warn into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/562	2023-05-09 08:52:11 +00:00
Alex	0fab9c3b8c	Merge pull request 'Helm: Include newer config parameters as values' (#565 ) from jonatan/garage:main into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/565	2023-05-09 08:49:00 +00:00
Jakub Jirutka	75759a163c	Allow to really disable sled feature	2023-05-09 08:46:15 +00:00
Jakub Jirutka	d2deee0b8b	Declare garage crates using workspace.dependencies This will allow to really disable "sled" feature without declaring `default-features = false` in every Cargo.toml where garage_db and garage_model is used. See https://doc.rust-lang.org/cargo/reference/workspaces.html#the-dependencies-table	2023-05-09 08:46:15 +00:00
Alex	8499cd5c21	Merge pull request 'Remove unnecessary/unused "timeago" features' (#559 ) from jirutka/garage:timeago-features into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/559	2023-05-09 08:44:22 +00:00
Jonatan Steuernagel	4ea7983093	Helm: Increment patch version	2023-05-08 08:03:21 +02:00
Jonatan Steuernagel	d5e39d11eb	Helm: Include newer config parameters as values Add all missing parameters from the reference manual. Primarily to enable the use of the new lmdb engine	2023-05-08 07:47:31 +02:00
Jakub Jirutka	06caa12d49	doc: Add information about Alpine Linux package to Quick Start	2023-05-07 19:28:43 +02:00
Jakub Jirutka	6d3ace1ea9	Fix undefined macro warn! on 32-bit Compiling garage_db v0.8.2 (garage-0.8.2/src/db) error: cannot find macro `warn` in this scope --> src/db/lmdb_adapter.rs:352:2 \| 352 \| warn!("LMDB is not recommended on 32-bit systems, database size will be limited"); \| ^^^^ \| = help: consider importing this macro: tracing::warn = note: `warn` is in scope, but it is an attribute: `#[warn]` error: could not compile `garage_db` due to previous error	2023-05-07 17:01:44 +02:00
Jakub Jirutka	833cf082da	Remove unnecessary/unused "timeago" features To decrease dependency bloat and binary size.	2023-05-07 01:03:54 +02:00
Alex	1ecd88c01f	Merge pull request 'Update rust toolchain to 1.68 and simplify Nix stuff' (#554 ) from nix-update-simplify into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/554	2023-04-25 14:56:49 +00:00
Alex Auvolat	5efcdc0de3	Update rust toolchain to 1.68 and simplify Nix stuff	2023-04-25 14:46:47 +02:00
Alex	a16eb7e4b8	Merge pull request 'api/Cargo.toml: Bumped quick-xml to version 0.26.' (#552 ) from jpds/garage:quick-xml-0.26 into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/552	2023-04-24 09:00:00 +00:00
Alex	6742070517	Merge pull request 'block/repair.rs: Added log entries for scrub start/finish.' (#551 ) from jpds/garage:scrub-log into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/551	2023-04-24 08:29:36 +00:00
Alex Auvolat	6894878146	update cargo.nix	2023-04-24 10:26:14 +02:00
Alex	02b0ba5f44	Merge pull request 'cookbook/real-world: fix typo' (#549 ) from yuka/garage:main into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/549	2023-04-24 08:24:55 +00:00
Jonathan Davies	fb3bd11dce	block/repair.rs: Added log entries for scrub start/finish.	2023-04-23 22:22:26 +01:00
Jonathan Davies	c168383113	api/Cargo.toml: Bumped quick-xml to version 0.26.	2023-04-23 20:14:28 +01:00
yuka	04a0063df9	cookbook/real-world: fix typo	2023-04-21 16:46:58 +00:00
arthurlutz	a2a35ac7a8	docs(book/quickstart): adapt aws s3 commands to example Signed-off-by: arthurlutz <arthurlutz@noreply.localhost>	2023-04-03 06:18:28 +00:00
Alex	f167310f42	Merge pull request 'Update Helm chart versions (app + chart)' (#535 ) from elwin013/garage:update-helm-chart-appVersion-to-0.8.2 into main Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/535	2023-03-24 16:06:30 +00:00
Kamil Banach	66ed0bdd91	Update Helm chart versions (app + chart) * chart version: 0.4.0 * app version: v0.8.2	2023-03-23 20:20:46 +01:00
Jonathan Davies	11b154b33b	cli.md: Pointed Cyberduck profile at upstream link.	2023-03-20 10:46:02 +00:00