mirror of
https://github.com/deuxfleurs-org/garage.git
synced 2026-08-15 16:43:10 +00:00
Compare commits
8 Commits
v0.5-beta1
...
v0.5.0.1
| Author | SHA1 | Date | |
|---|---|---|---|
 Alex Auvolat
|
4b369347c0 | ||
|
Alex Auvolat
|
224c89ad6e | ||
|
Quentin Dufour
|
7c2037ba87 | ||
|
trinity-1686a
|
c4ac8835d3 | ||
|
Alex Auvolat
|
ccce75bc25 | ||
|
trinity-1686a
|
7f26ed55cd | ||
|
Quentin Dufour
|
8811bb08e6 | ||
|
Quentin Dufour
|
85b2e4ca29 |
@@ -1,27 +0,0 @@
|
||||
Testing
|
||||
-------
|
||||
|
||||
How are we going to test that our replication method works correctly?
|
||||
We will have to introduce lots of dummy data and then add/remove nodes many times.
|
||||
|
||||
|
||||
Attaining S3 compatibility
|
||||
--------------------------
|
||||
|
||||
- test multipart uploads
|
||||
- get ranges
|
||||
|
||||
- fix sync not working in some cases ? (when starting from empty?)
|
||||
|
||||
- api_server following the S3 semantics for head/get/put/list/delete: verify more that it works as intended
|
||||
- PUT requests: verify content-md5 if provided
|
||||
- possibly other necessary endpoints ?
|
||||
|
||||
|
||||
Lower priority
|
||||
--------------
|
||||
|
||||
- less a priority: hinted handoff
|
||||
- repair: re-propagate block ref table to rc
|
||||
- FIXME in rpc_server when garage shuts down and futures can be interrupted
|
||||
(tokio::spawn should be replaced by a new function background::spawn_joinable)
|
||||
@@ -8,7 +8,7 @@
|
||||
- [Multi-node deployment](./cookbook/real_world.md)
|
||||
- [Building from source](./cookbook/from_source.md)
|
||||
- [Integration with systemd](./cookbook/systemd.md)
|
||||
- [Gateways](./cookbook/gateways.md)
|
||||
- [Configuring a gateway node](./cookbook/gateways.md)
|
||||
- [Exposing buckets as websites](./cookbook/exposing_websites.md)
|
||||
- [Configuring a reverse proxy](./cookbook/reverse_proxy.md)
|
||||
- [Recovering from failures](./cookbook/recovering.md)
|
||||
@@ -30,9 +30,9 @@
|
||||
- [S3 compatibility status](./reference_manual/s3_compatibility.md)
|
||||
|
||||
- [Design](./design/index.md)
|
||||
- [Related Work](./design/related_work.md)
|
||||
- [Goals and use Cases](./design/goals.md)
|
||||
- [Related work](./design/related_work.md)
|
||||
- [Internals](./design/internals.md)
|
||||
- [Design draft](./design/design_draft.md)
|
||||
|
||||
- [Development](./development/index.md)
|
||||
- [Setup your environment](./development/devenv.md)
|
||||
@@ -41,5 +41,7 @@
|
||||
- [Miscellaneous notes](./development/miscellaneous_notes.md)
|
||||
|
||||
- [Working Documents](./working_documents/index.md)
|
||||
- [Load Balancing Data](./working_documents/load_balancing.md)
|
||||
- [S3 compatibility target](./working_documents/compatibility_target.md)
|
||||
- [Load balancing data](./working_documents/load_balancing.md)
|
||||
- [Migrating from 0.3 to 0.4](./working_documents/migration_04.md)
|
||||
- [Design draft](./working_documents/design_draft.md)
|
||||
|
||||
@@ -1,7 +1,19 @@
|
||||
# Connect it to
|
||||
# Connect it to...
|
||||
|
||||
To configure an S3 client to interact with Garage, you will need the following
|
||||
parameters:
|
||||
Garage implements the Amazon S3 protocol, which makes it compatible with many existing software programs.
|
||||
|
||||
In particular, you will find here instructions to connect it with:
|
||||
|
||||
- [web applications](./apps.md)
|
||||
- [website hosting](./websites.md)
|
||||
- [software repositories](./repositories.md)
|
||||
- [CLI tools](./cli.md)
|
||||
- [your own code](./code.md)
|
||||
|
||||
### Generic instructions
|
||||
|
||||
To configure S3-compatible software to interact with Garage,
|
||||
you will need the following parameters:
|
||||
|
||||
- An **API endpoint**: this corresponds to the HTTP or HTTPS address
|
||||
used to contact the Garage server. When runing Garage locally this will usually
|
||||
@@ -27,12 +39,3 @@ provided that you follow the following guidelines:
|
||||
If this is not configured explicitly, clients usually try to talk to region `us-east-1`.
|
||||
Garage should normally redirect your client to the correct region,
|
||||
but in case your client does not support this you might have to configure it manually.
|
||||
|
||||
We will now provide example configurations for the most common clients per category:
|
||||
|
||||
- [Apps](./apps.md)
|
||||
- [Websites](./websites.md)
|
||||
- [Repositories](./repositories.md)
|
||||
- [CLI tools](./cli.md)
|
||||
- [Your code](./code.md)
|
||||
|
||||
|
||||
@@ -6,11 +6,11 @@ Gateways allow you to expose Garage endpoints (S3 API and websites) without stor
|
||||
|
||||
You can configure Garage as a gateway on all nodes that will consume your S3 API, it will provide you the following benefits:
|
||||
|
||||
- **It removes 1 or 2 network RTT** Instead of (querying your reverse proxy then) querying a random node of the cluster that will forward your request to the nodes effectively storing the data, your local gateway will directly knows which node to query.
|
||||
- **It removes 1 or 2 network RTT.** Instead of (querying your reverse proxy then) querying a random node of the cluster that will forward your request to the nodes effectively storing the data, your local gateway will directly knows which node to query.
|
||||
|
||||
- **It ease server management** Instead of tracking in your reverse proxy and DNS what are the current Garage nodes, your gateway being part of the cluster keeps this information for you. In your software, you will always specify `http://localhost:3900`.
|
||||
- **It eases server management.** Instead of tracking in your reverse proxy and DNS what are the current Garage nodes, your gateway being part of the cluster keeps this information for you. In your software, you will always specify `http://localhost:3900`.
|
||||
|
||||
- **It simplifies security** Instead of having to maintain and renew a TLS certificate, you leverage the Secret Handshake protocol we use for our cluster. The S3 API protocol will be in plain text but limited to your local machine.
|
||||
- **It simplifies security.** Instead of having to maintain and renew a TLS certificate, you leverage the Secret Handshake protocol we use for our cluster. The S3 API protocol will be in plain text but limited to your local machine.
|
||||
|
||||
## Limitations
|
||||
|
||||
|
||||
@@ -4,22 +4,23 @@ A cookbook, when you cook, is a collection of recipes.
|
||||
Similarly, Garage's cookbook contains a collection of recipes that are known to works well!
|
||||
This chapter could also be referred as "Tutorials" or "Best practices".
|
||||
|
||||
- **[Deploying Garage](real_world.md):** This page will walk you through all of the necessary
|
||||
- **[Multi-node deployment](real_world.md):** This page will walk you through all of the necessary
|
||||
steps to deploy Garage in a real-world setting.
|
||||
|
||||
- **[Configuring S3 clients](clients.md):** This page will explain how to configure
|
||||
popular S3 clients to interact with a Garage server.
|
||||
|
||||
- **[Hosting a website](website.md):** This page explains how to use Garage
|
||||
to host a static website.
|
||||
|
||||
- **[Recovering from failures](recovering.md):** Garage's first selling point is resilience
|
||||
to hardware failures. This section explains how to recover from such a failure in the
|
||||
best possible way.
|
||||
|
||||
- **[Building from source](from_source.md):** This page explains how to build Garage from
|
||||
source in case a binary is not provided for your architecture, or if you want to
|
||||
hack with us!
|
||||
|
||||
- **[Starting with Systemd](from_source.md):** This page explains how to run Garage
|
||||
- **[Integration with Systemd](systemd.md):** This page explains how to run Garage
|
||||
as a Systemd service (instead of as a Docker container).
|
||||
|
||||
- **[Configuring a gateway node](gateways.md):** This page explains how to run a gateway node in a Garage cluster, i.e. a Garage node that doesn't store data but accelerates access to data present on the other nodes.
|
||||
|
||||
- **[Hosting a website](exposing_websites.md):** This page explains how to use Garage
|
||||
to host a static website.
|
||||
|
||||
- **[Configuring a reverse-proxy](reverse_proxy.md):** This page explains how to configure a reverse-proxy to add TLS support to your S3 api endpoint.
|
||||
|
||||
- **[Recovering from failures](recovering.md):** Garage's first selling point is resilience
|
||||
to hardware failures. This section explains how to recover from such a failure in the
|
||||
best possible way.
|
||||
|
||||
@@ -286,5 +286,5 @@ and is covered in the [quick start guide](../quick_start/index.md).
|
||||
Remember also that the CLI is self-documented thanks to the `--help` flag and
|
||||
the `help` subcommand (e.g. `garage help`, `garage key --help`).
|
||||
|
||||
Configuring an S3 client to interact with Garage is covered
|
||||
[in the next section](clients.md).
|
||||
Configuring S3-compatible applicatiosn to interact with Garage
|
||||
is covered in the [Integrations](/connect/index.html) section.
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
# Goals and use cases
|
||||
|
||||
## Goals and non-goals
|
||||
|
||||
Garage is a lightweight geo-distributed data store that implements the
|
||||
[Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html)
|
||||
object storage protocole. It enables applications to store large blobs such
|
||||
as pictures, video, images, documents, etc., in a redundant multi-node
|
||||
setting. S3 is versatile enough to also be used to publish a static
|
||||
website.
|
||||
|
||||
Garage is an opinionated object storage solutoin, we focus on the following **desirable properties**:
|
||||
|
||||
- **Self-contained & lightweight**: works everywhere and integrates well in existing environments to target [hyperconverged infrastructures](https://en.wikipedia.org/wiki/Hyper-converged_infrastructure).
|
||||
- **Highly resilient**: highly resilient to network failures, network latency, disk failures, sysadmin failures.
|
||||
- **Simple**: simple to understand, simple to operate, simple to debug.
|
||||
- **Internet enabled**: made for multi-sites (eg. datacenters, offices, households, etc.) interconnected through regular Internet connections.
|
||||
|
||||
We also noted that the pursuit of some other goals are detrimental to our initial goals.
|
||||
The following has been identified as **non-goals** (if these points matter to you, you should not use Garage):
|
||||
|
||||
- **Extreme performances**: high performances constrain a lot the design and the infrastructure; we seek performances through minimalism only.
|
||||
- **Feature extensiveness**: we do not plan to add additional features compared to the ones provided by the S3 API.
|
||||
- **Storage optimizations**: erasure coding or any other coding technique both increase the difficulty of placing data and synchronizing; we limit ourselves to duplication.
|
||||
- **POSIX/Filesystem compatibility**: we do not aim at being POSIX compatible or to emulate any kind of filesystem. Indeed, in a distributed environment, such synchronizations are translated in network messages that impose severe constraints on the deployment.
|
||||
|
||||
## Use-cases
|
||||
|
||||
*Are you also using Garage in your organization? [Open a PR](https://git.deuxfleurs.fr/Deuxfleurs/garage) to add your use case here!*
|
||||
|
||||
### Deuxfleurs
|
||||
|
||||
[Deuxfleurs](https://deuxfleurs.fr) is an experimental non-profit hosting
|
||||
organization that develops Garage. Deuxfleurs is focused on building highly
|
||||
available infrastructure through redundancy in multiple geographical
|
||||
locations. They use Garage themselves for the following tasks:
|
||||
|
||||
- Hosting of [main website](https://deuxfleurs.fr), [this website](https://garagehq.deuxfleurs.fr), as well as the personal website of many of the members of the organization
|
||||
|
||||
- As a [Matrix media backend](https://github.com/matrix-org/synapse-s3-storage-provider)
|
||||
|
||||
- To store personal data and shared documents through [Bagage](https://git.deuxfleurs.fr/Deuxfleurs/bagage), a homegrown WebDav-to-S3 proxy
|
||||
|
||||
- In the Drone continuous integration platform to store task logs
|
||||
|
||||
- As a Nix binary cache
|
||||
|
||||
- As a backup target using `rclone`
|
||||
|
||||
The Deuxfleurs Garage cluster is a multi-site cluster currently composed of
|
||||
4 nodes in 2 physical locations. In the future it will be expanded to at
|
||||
least 3 physical locations to fully exploit Garage's potential for high
|
||||
availability.
|
||||
@@ -1,5 +1,26 @@
|
||||
# Design
|
||||
|
||||
The design section helps you to see Garage from a "big picture" perspective.
|
||||
It will allow you to understand if Garage is a good fit for you,
|
||||
how to better use it, how to contribute to it, what can Garage could and could not do, etc.
|
||||
The design section helps you to see Garage from a "big picture"
|
||||
perspective. It will allow you to understand if Garage is a good fit for
|
||||
you, how to better use it, how to contribute to it, what can Garage could
|
||||
and could not do, etc.
|
||||
|
||||
- **[Goals and use cases](goals.md):** This page explains why Garage was concieved and what practical use cases it targets.
|
||||
|
||||
- **[Related work](related_work.md):** This pages presents the theoretical background on which Garage is built, and describes other software storage solutions and why they didn't work for us.
|
||||
|
||||
- **[Internals](internals.md):** This page enters into more details on how Garage manages data internally.
|
||||
|
||||
## Talks
|
||||
|
||||
We love to talk and hear about Garage, that's why we keep a log here:
|
||||
|
||||
- [(fr, 2021-11-13, video) Garage : Mille et une façons de stocker vos données](https://video.tedomum.net/w/moYKcv198dyMrT8hCS5jz9) and [slides (html)](https://rfid.deuxfleurs.fr/presentations/2021-11-13/garage/) - during [RFID#1](https://rfid.deuxfleurs.fr/programme/2021-11-13/) event
|
||||
|
||||
- [(en, 2021-04-28) Distributed object storage is centralised](https://git.deuxfleurs.fr/Deuxfleurs/garage/raw/commit/b1f60579a13d3c5eba7f74b1775c84639ea9b51a/doc/talks/2021-04-28_spirals-team/talk.pdf)
|
||||
|
||||
- [(fr, 2020-12-02) Garage : jouer dans la cour des grands quand on est un hébergeur associatif](https://git.deuxfleurs.fr/Deuxfleurs/garage/raw/commit/b1f60579a13d3c5eba7f74b1775c84639ea9b51a/doc/talks/2020-12-02_wide-team/talk.pdf)
|
||||
|
||||
*Did you write or talk about Garage? [Open a pull request](https://git.deuxfleurs.fr/Deuxfleurs/garage/) to add a link here!*
|
||||
|
||||
|
||||
|
||||
@@ -4,14 +4,17 @@
|
||||
|
||||
TODO: write this section
|
||||
|
||||
- The Dynamo ring
|
||||
- The Dynamo ring (see [this paper](https://dl.acm.org/doi/abs/10.1145/1323293.1294281) and [that paper](https://www.usenix.org/conference/nsdi16/technical-sessions/presentation/eisenbud))
|
||||
|
||||
- CRDTs
|
||||
- CRDTs (see [this paper](https://link.springer.com/chapter/10.1007/978-3-642-24550-3_29))
|
||||
|
||||
- Consistency model of Garage tables
|
||||
|
||||
See this presentation (in French) for some first information:
|
||||
<https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/main/doc/talks/2020-12-02_wide-team/talk.pdf>
|
||||
In the meantime, you can find some information at the following links:
|
||||
|
||||
- [this presentation (in French)](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/main/doc/talks/2020-12-02_wide-team/talk.pdf)
|
||||
|
||||
- [an old design draft](/working_documents/design_draft.md)
|
||||
|
||||
|
||||
## Garbage collection
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# Related Work
|
||||
# Related work
|
||||
|
||||
## Context
|
||||
|
||||
@@ -41,14 +41,35 @@ There were many attempts in research too. I am only thinking to [LBFS](https://p
|
||||
|
||||
## Existing software
|
||||
|
||||
**[Pithos](https://github.com/exoscale/pithos) :**
|
||||
**[MinIO](https://min.io/):** MinIO shares our *Self-contained & lightweight* goal but selected two of our non-goals: *Storage optimizations* through erasure coding and *POSIX/Filesystem compatibility* through strong consistency.
|
||||
However, by pursuing these two non-goals, MinIO do not reach our desirable properties.
|
||||
Firstly, it fails on the *Simple* property: due to the erasure coding, MinIO has severe limitations on how drives can be added or deleted from a cluster.
|
||||
Secondly, it fails on the *Internet enabled* property: due to its strong consistency, MinIO is latency sensitive.
|
||||
Furthermore, MinIO has no knowledge of "sites" and thus can not distribute data to minimize the failure of a given site.
|
||||
|
||||
**[Openstack Swift](https://docs.openstack.org/swift/latest/):**
|
||||
OpenStack Swift at least fails on the *Self-contained & lightweight* goal.
|
||||
Starting it requires around 8GB of RAM, which is too much especially in an hyperconverged infrastructure.
|
||||
We also do not classify Swift as *Simple*.
|
||||
|
||||
**[Ceph](https://ceph.io/ceph-storage/object-storage/):**
|
||||
This review holds for the whole Ceph stack, including the RADOS paper, Ceph Object Storage module, the RADOS Gateway, etc.
|
||||
At its core, Ceph has been designed to provide *POSIX/Filesystem compatibility* which requires strong consistency, which in turn
|
||||
makes Ceph latency-sensitive and fails our *Internet enabled* goal.
|
||||
Due to its industry oriented design, Ceph is also far from being *Simple* to operate and from being *Self-contained & lightweight* which makes it hard to integrate it in an hyperconverged infrastructure.
|
||||
In a certain way, Ceph and MinIO are closer together than they are from Garage or OpenStack Swift.
|
||||
|
||||
**[Pithos](https://github.com/exoscale/pithos):**
|
||||
Pithos has been abandonned and should probably not used yet, in the following we explain why we did not pick their design.
|
||||
Pithos was relying as a S3 proxy in front of Cassandra (and was working with Scylla DB too).
|
||||
From its designers' mouth, storing data in Cassandra has shown its limitations justifying the project abandonment.
|
||||
They built a closed-source version 2 that does not store blobs in the database (only metadata) but did not communicate further on it.
|
||||
We considered there v2's design but concluded that it does not fit both our *Self-contained & lightweight* and *Simple* properties. It makes the development, the deployment and the operations more complicated while reducing the flexibility.
|
||||
|
||||
**[IPFS](https://ipfs.io/) :**
|
||||
**[Riak CS](https://docs.riak.com/riak/cs/2.1.1/index.html):**
|
||||
*Not written yet*
|
||||
|
||||
**[IPFS](https://ipfs.io/):**
|
||||
*Not written yet*
|
||||
|
||||
## Specific research papers
|
||||
|
||||
@@ -2,3 +2,13 @@
|
||||
|
||||
Now that you are a Garage expert, you want to enhance it, you are in the right place!
|
||||
We discuss here how to hack on Garage, how we manage its development, etc.
|
||||
|
||||
## Rust API (docs.rs)
|
||||
If you encounter a specific bug in Garage or plan to patch it, you may jump directly to the source code's documentation!
|
||||
|
||||
- [garage\_api](https://docs.rs/garage_api/latest/garage_api/) - contains the S3 standard API endpoint
|
||||
- [garage\_model](https://docs.rs/garage_model/latest/garage_model/) - contains Garage's model built on the table abstraction
|
||||
- [garage\_rpc](https://docs.rs/garage_rpc/latest/garage_rpc/) - contains Garage's federation protocol
|
||||
- [garage\_table](https://docs.rs/garage_table/latest/garage_table/) - contains core Garage's CRDT datatypes
|
||||
- [garage\_util](https://docs.rs/garage_util/latest/garage_util/) - contains garage helpers
|
||||
- [garage\_web](https://docs.rs/garage_web/latest/garage_web/) - contains the S3 website endpoint
|
||||
|
||||
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 1.4 MiB |
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 310 KiB |
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 35 KiB |
+38
-69
@@ -12,91 +12,62 @@
|
||||
]
|
||||
</p>
|
||||
|
||||
```
|
||||
This very website is hosted using Garage. In other words: the doc is the PoC!
|
||||
```
|
||||
|
||||
# The Garage Geo-Distributed Data Store
|
||||
# Data resiliency for everyone
|
||||
|
||||
Garage is a lightweight geo-distributed data store that implements the
|
||||
[Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html)
|
||||
object storage protocole. It enables applications to store large blobs such
|
||||
as pictures, video, images, documents, etc., in a redundant multi-node
|
||||
setting. S3 is versatile enough to also be used to publish a static
|
||||
website.
|
||||
Garage is an **open-source** distributed **storage service** you can **self-host** to fullfill many needs:
|
||||
|
||||
Garage comes from the observation that despite the numerous existing
|
||||
implementation of object stores, many people have broken data management
|
||||
policies (backup/replication on a single site or none at all). To promote
|
||||
better data management policies, we focused on the following **desirable
|
||||
properties**:
|
||||
<p align="center" style="text-align:center; margin-bottom: 5rem;">
|
||||
<img alt="Summary of the possible usages with a related icon: host a website, store media and backup target" src="img/usage.svg" />
|
||||
</p>
|
||||
|
||||
- **Self-contained & lightweight**: works everywhere and integrates well in existing environments to target [hyperconverged infrastructures](https://en.wikipedia.org/wiki/Hyper-converged_infrastructure).
|
||||
- **Highly resilient**: highly resilient to network failures, network latency, disk failures, sysadmin failures.
|
||||
- **Simple**: simple to understand, simple to operate, simple to debug.
|
||||
- **Internet enabled**: made for multi-sites (eg. datacenters, offices, households, etc.) interconnected through regular Internet connections.
|
||||
<p align="center" style="text-align:center; margin-bottom: 5rem;">
|
||||
<a href="/design/use_cases.html">⮞ learn more about use cases ⮜</a>
|
||||
</p>
|
||||
|
||||
We also noted that the pursuit of some other goals are detrimental to our initial goals.
|
||||
The following has been identified as **non-goals** (if these points matter to you, you should not use Garage):
|
||||
Garage implements the **[Amazon S3 API](https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html)** and thus is already **compatible** with many applications:
|
||||
|
||||
- **Extreme performances**: high performances constrain a lot the design and the infrastructure; we seek performances through minimalism only.
|
||||
- **Feature extensiveness**: complete implementation of the S3 API or any other API to make Garage a drop-in replacement is not targeted as it could lead to decisions impacting our desirable properties.
|
||||
- **Storage optimizations**: erasure coding or any other coding technique both increase the difficulty of placing data and synchronizing; we limit ourselves to duplication.
|
||||
- **POSIX/Filesystem compatibility**: we do not aim at being POSIX compatible or to emulate any kind of filesystem. Indeed, in a distributed environment, such synchronizations are translated in network messages that impose severe constraints on the deployment.
|
||||
<p align="center" style="text-align:center; margin-bottom: 8rem;">
|
||||
<img alt="Garage is already compatible with Nextcloud, Mastodon, Matrix Synapse, Cyberduck, RClone and Peertube" src="img/software.svg" />
|
||||
</p>
|
||||
|
||||
## Use Cases
|
||||
<p align="center" style="text-align:center; margin-bottom: 5rem;">
|
||||
<a href="/connect/index.html">⮞ learn more about integrations ⮜</a>
|
||||
</p>
|
||||
|
||||
**[Deuxfleurs](https://deuxfleurs.fr):** Garage is used by Deuxfleurs which
|
||||
is a non-profit hosting organization. Especially, it is used to host their
|
||||
main website, this documentation and some of its members' blogs.
|
||||
Deuxfleurs also uses Garage as their [Matrix's media
|
||||
backend](https://github.com/matrix-org/synapse-s3-storage-provider).
|
||||
Deuxfleurs also uses it in its continuous integration platform to store
|
||||
Drone's job logs and a Nix binary cache.
|
||||
|
||||
*Are you using Garage? [Open a pull request](https://git.deuxfleurs.fr/Deuxfleurs/garage/) to add your organization here!*
|
||||
Garage provides **data resiliency** by **replicating** data 3x over **distant** servers:
|
||||
|
||||
## Comparison to existing software
|
||||
<p align="center" style="text-align:center; margin-bottom: 5rem;">
|
||||
<img alt="An example deployment on a map with servers in 5 zones: UK, France, Belgium, Germany and Switzerland. Each chunk of data is replicated in 3 of these 5 zones." src="img/map.svg" />
|
||||
</p>
|
||||
|
||||
**[MinIO](https://min.io/):** MinIO shares our *Self-contained & lightweight* goal but selected two of our non-goals: *Storage optimizations* through erasure coding and *POSIX/Filesystem compatibility* through strong consistency.
|
||||
However, by pursuing these two non-goals, MinIO do not reach our desirable properties.
|
||||
Firstly, it fails on the *Simple* property: due to the erasure coding, MinIO has severe limitations on how drives can be added or deleted from a cluster.
|
||||
Secondly, it fails on the *Internet enabled* property: due to its strong consistency, MinIO is latency sensitive.
|
||||
Furthermore, MinIO has no knowledge of "sites" and thus can not distribute data to minimize the failure of a given site.
|
||||
<p align="center" style="text-align:center; margin-bottom: 5rem;">
|
||||
<a href="/design/index.html">⮞ learn more about our design ⮜</a>
|
||||
</p>
|
||||
|
||||
**[Openstack Swift](https://docs.openstack.org/swift/latest/):**
|
||||
OpenStack Swift at least fails on the *Self-contained & lightweight* goal.
|
||||
Starting it requires around 8GB of RAM, which is too much especially in an hyperconverged infrastructure.
|
||||
We also do not classify Swift as *Simple*.
|
||||
Did you notice that *this website* is hosted and served by Garage?
|
||||
|
||||
**[Ceph](https://ceph.io/ceph-storage/object-storage/):**
|
||||
This review holds for the whole Ceph stack, including the RADOS paper, Ceph Object Storage module, the RADOS Gateway, etc.
|
||||
At its core, Ceph has been designed to provide *POSIX/Filesystem compatibility* which requires strong consistency, which in turn
|
||||
makes Ceph latency-sensitive and fails our *Internet enabled* goal.
|
||||
Due to its industry oriented design, Ceph is also far from being *Simple* to operate and from being *Self-contained & lightweight* which makes it hard to integrate it in an hyperconverged infrastructure.
|
||||
In a certain way, Ceph and MinIO are closer together than they are from Garage or OpenStack Swift.
|
||||
## Keeping requirements low
|
||||
|
||||
*More comparisons are available in our [Related Work](design/related_work.md) chapter.*
|
||||
We worked hard to keep requirements as low as possible as we target the largest possible public.
|
||||
|
||||
## Other Resources
|
||||
* **CPU:** any x86\_64 CPU from the last 10 years, ARMv7 or ARMv8.
|
||||
* **RAM:** 1GB
|
||||
* **Disk Space:** at least 16GB
|
||||
* **Network:** 200ms or less, 50 Mbps or more
|
||||
* **Heterogeneous hardware:** build a cluster with whatever second-hand machines are available
|
||||
|
||||
This website is not the only source of information about Garage!
|
||||
We reference here other places on the Internet where you can learn more about Garage.
|
||||
*For the network, as we do not use consensus algorithms like Paxos or Raft, Garage is not as latency sensitive.*
|
||||
*Thanks to Rust and its zero-cost abstractions, we keep CPU and memory low.*
|
||||
|
||||
### Rust API (docs.rs)
|
||||
## Built on the shoulder of giants
|
||||
|
||||
If you encounter a specific bug in Garage or plan to patch it, you may jump directly to the source code's documentation!
|
||||
- [Dynamo: Amazon’s Highly Available Key-value Store ](https://dl.acm.org/doi/abs/10.1145/1323293.1294281) by DeCandia et al.
|
||||
- [Conflict-Free Replicated Data Types](https://link.springer.com/chapter/10.1007/978-3-642-24550-3_29) by Shapiro et al.
|
||||
- [Maglev: A Fast and Reliable Software Network Load Balancer](https://www.usenix.org/conference/nsdi16/technical-sessions/presentation/eisenbud) by Eisenbud et al.
|
||||
|
||||
- [garage\_api](https://docs.rs/garage_api/latest/garage_api/) - contains the S3 standard API endpoint
|
||||
- [garage\_model](https://docs.rs/garage_model/latest/garage_model/) - contains Garage's model built on the table abstraction
|
||||
- [garage\_rpc](https://docs.rs/garage_rpc/latest/garage_rpc/) - contains Garage's federation protocol
|
||||
- [garage\_table](https://docs.rs/garage_table/latest/garage_table/) - contains core Garage's CRDT datatypes
|
||||
- [garage\_util](https://docs.rs/garage_util/latest/garage_util/) - contains garage helpers
|
||||
- [garage\_web](https://docs.rs/garage_web/latest/garage_web/) - contains the S3 website endpoint
|
||||
|
||||
### Talks
|
||||
|
||||
We love to talk and hear about Garage, that's why we keep a log here:
|
||||
## Talks
|
||||
|
||||
- [(fr, 2021-11-13, video) Garage : Mille et une façons de stocker vos données](https://video.tedomum.net/w/moYKcv198dyMrT8hCS5jz9) and [slides (html)](https://rfid.deuxfleurs.fr/presentations/2021-11-13/garage/) - during [RFID#1](https://rfid.deuxfleurs.fr/programme/2021-11-13/) event
|
||||
|
||||
@@ -104,8 +75,6 @@ We love to talk and hear about Garage, that's why we keep a log here:
|
||||
|
||||
- [(fr, 2020-12-02, pdf) Garage : jouer dans la cour des grands quand on est un hébergeur associatif](https://git.deuxfleurs.fr/Deuxfleurs/garage/raw/commit/b1f60579a13d3c5eba7f74b1775c84639ea9b51a/doc/talks/2020-12-02_wide-team/talk.pdf)
|
||||
|
||||
*Did you write or talk about Garage? [Open a pull request](https://git.deuxfleurs.fr/Deuxfleurs/garage/) to add a link here!*
|
||||
|
||||
## Community
|
||||
|
||||
If you want to discuss with us, you can join our Matrix channel at [#garage:deuxfleurs.fr](https://matrix.to/#/#garage:deuxfleurs.fr).
|
||||
@@ -116,7 +85,7 @@ Our code repository and issue tracker, which is the place where you should repor
|
||||
Garage's source code, is released under the [AGPL v3 License](https://www.gnu.org/licenses/agpl-3.0.en.html).
|
||||
Please note that if you patch Garage and then use it to provide any service over a network, you must share your code!
|
||||
|
||||
# Funding
|
||||
# Sponsors and funding
|
||||
|
||||
The Deuxfleurs association has received a grant from [NGI POINTER](https://pointer.ngi.eu/), to fund 3 people working on Garage full-time for a year: from October 2021 to September 2022.
|
||||
|
||||
|
||||
@@ -0,0 +1,105 @@
|
||||
# S3 compatibility target
|
||||
|
||||
If there is a specific S3 functionnality you have a need for, feel free to open
|
||||
a PR to put the corresponding endpoints higher in the list. Please explain
|
||||
your motivations for doing so in the PR message.
|
||||
|
||||
| Priority | Endpoints |
|
||||
| -------------------------- | --------- |
|
||||
| **S-tier** (high priority) | |
|
||||
| | HeadBucket |
|
||||
| | GetBucketLocation |
|
||||
| | *CreateBucket* |
|
||||
| | *DeleteBucket* |
|
||||
| | ListBuckets |
|
||||
| | ListObjects |
|
||||
| | ListObjectsV2 |
|
||||
| | HeadObject |
|
||||
| | GetObject |
|
||||
| | PutObject |
|
||||
| | CopyObject |
|
||||
| | DeleteObject |
|
||||
| | DeleteObjects |
|
||||
| | CreateMultipartUpload |
|
||||
| | CompleteMultipartUpload |
|
||||
| | AbortMultipartUpload |
|
||||
| | UploadPart |
|
||||
| | *ListMultipartUploads* |
|
||||
| | *ListParts* |
|
||||
| **A-tier** (will implement) | |
|
||||
| | *GetBucketCors* |
|
||||
| | *PutBucketCors* |
|
||||
| | *DeleteBucketCors* |
|
||||
| | *UploadPartCopy* |
|
||||
| | *GetBucketWebsite* |
|
||||
| | *PutBucketWebsite* |
|
||||
| | *DeleteBucketWebsite* |
|
||||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~ | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
||||
| **B-tier** | |
|
||||
| | GetBucketAcl |
|
||||
| | PutBucketAcl |
|
||||
| | GetObjectLockConfiguration |
|
||||
| | PutObjectLockConfiguration |
|
||||
| | GetObjectRetention |
|
||||
| | PutObjectRetention |
|
||||
| | GetObjectLegalHold |
|
||||
| | PutObjectLegalHold |
|
||||
| **C-tier** | |
|
||||
| | GetBucketVersioning |
|
||||
| | PutBucketVersioning |
|
||||
| | ListObjectVersions |
|
||||
| | GetObjectAcl |
|
||||
| | PutObjectAcl |
|
||||
| **garbage-tier** | |
|
||||
| | DeleteBucketEncryption |
|
||||
| | DeleteBucketAnalyticsConfiguration |
|
||||
| | DeleteBucketIntelligentTieringConfiguration |
|
||||
| | DeleteBucketInventoryConfiguration |
|
||||
| | DeleteBucketLifecycle |
|
||||
| | DeleteBucketMetricsConfiguration |
|
||||
| | DeleteBucketOwnershipControls |
|
||||
| | DeleteBucketPolicy |
|
||||
| | DeleteBucketReplication |
|
||||
| | DeleteBucketTagging |
|
||||
| | DeleteObjectTagging |
|
||||
| | DeletePublicAccessBlock |
|
||||
| | GetBucketAccelerateConfiguration |
|
||||
| | GetBucketAnalyticsConfiguration |
|
||||
| | GetBucketEncryption |
|
||||
| | GetBucketIntelligentTieringConfiguration |
|
||||
| | GetBucketInventoryConfiguration |
|
||||
| | GetBucketLifecycleConfiguration |
|
||||
| | GetBucketLogging |
|
||||
| | GetBucketMetricsConfiguration |
|
||||
| | GetBucketNotificationConfiguration |
|
||||
| | GetBucketOwnershipControls |
|
||||
| | GetBucketPolicy |
|
||||
| | GetBucketPolicyStatus |
|
||||
| | GetBucketReplication |
|
||||
| | GetBucketRequestPayment |
|
||||
| | GetBucketTagging |
|
||||
| | GetObjectTagging |
|
||||
| | GetObjectTorrent |
|
||||
| | GetPublicAccessBlock |
|
||||
| | ListBucketAnalyticsConfigurations |
|
||||
| | ListBucketIntelligentTieringConfigurations |
|
||||
| | ListBucketInventoryConfigurations |
|
||||
| | ListBucketMetricsConfigurations |
|
||||
| | PutBucketAccelerateConfiguration |
|
||||
| | PutBucketAnalyticsConfiguration |
|
||||
| | PutBucketEncryption |
|
||||
| | PutBucketIntelligentTieringConfiguration |
|
||||
| | PutBucketInventoryConfiguration |
|
||||
| | PutBucketLifecycleConfiguration |
|
||||
| | PutBucketLogging |
|
||||
| | PutBucketMetricsConfiguration |
|
||||
| | PutBucketNotificationConfiguration |
|
||||
| | PutBucketOwnershipControls |
|
||||
| | PutBucketPolicy |
|
||||
| | PutBucketReplication |
|
||||
| | PutBucketRequestPayment |
|
||||
| | PutBucketTagging |
|
||||
| | PutObjectTagging |
|
||||
| | PutPublicAccessBlock |
|
||||
| | RestoreObject |
|
||||
| | SelectObjectContent |
|
||||
-83
@@ -1,83 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
set -xe
|
||||
|
||||
cd $(dirname $0)
|
||||
|
||||
mkdir -p pki
|
||||
cd pki
|
||||
|
||||
# Create a certificate authority that both the client side and the server side of
|
||||
# the RPC protocol will use to authenticate the other side.
|
||||
if [ ! -f garage-ca.key ]; then
|
||||
echo "Generating Garage CA keys..."
|
||||
openssl genpkey -algorithm ED25519 -out garage-ca.key
|
||||
openssl req -x509 -new -nodes -key garage-ca.key -sha256 -days 3650 -out garage-ca.crt -subj "/C=FR/O=Garage"
|
||||
fi
|
||||
|
||||
|
||||
# Generate a certificate that can be used either as a server certificate
|
||||
# or a client certificate. This is what the RPC client and server will use
|
||||
# to prove that they are authenticated by the CA.
|
||||
if [ ! -f garage.crt ]; then
|
||||
echo "Generating Garage agent keys..."
|
||||
if [ ! -f garage.key ]; then
|
||||
openssl genpkey -algorithm ED25519 -out garage.key
|
||||
fi
|
||||
openssl req -new -sha256 -key garage.key -subj "/C=FR/O=Garage/CN=garage" \
|
||||
-out garage.csr
|
||||
openssl req -in garage.csr -noout -text
|
||||
openssl x509 -req -in garage.csr \
|
||||
-extensions v3_req \
|
||||
-extfile <(cat <<EOF
|
||||
[req]
|
||||
distinguished_name = req_distinguished_name
|
||||
req_extensions = v3_req
|
||||
prompt = no
|
||||
|
||||
[req_distinguished_name]
|
||||
C = FR
|
||||
O = Garage
|
||||
CN = garage
|
||||
|
||||
[v3_req]
|
||||
keyUsage = keyEncipherment, dataEncipherment
|
||||
extendedKeyUsage = serverAuth, clientAuth
|
||||
subjectAltName = @alt_names
|
||||
[alt_names]
|
||||
DNS.1 = garage
|
||||
EOF
|
||||
) \
|
||||
-CA garage-ca.crt -CAkey garage-ca.key -CAcreateserial \
|
||||
-out garage.crt -days 365
|
||||
fi
|
||||
|
||||
# Client-only certificate used for the CLI
|
||||
if [ ! -f garage-client.crt ]; then
|
||||
echo "Generating Garage client keys..."
|
||||
if [ ! -f garage-client.key ]; then
|
||||
openssl genpkey -algorithm ED25519 -out garage-client.key
|
||||
fi
|
||||
openssl req -new -sha256 -key garage-client.key -subj "/C=FR/O=Garage" \
|
||||
-out garage-client.csr
|
||||
openssl req -in garage-client.csr -noout -text
|
||||
openssl x509 -req -in garage-client.csr \
|
||||
-extensions v3_req \
|
||||
-extfile <(cat <<EOF
|
||||
[req]
|
||||
distinguished_name = req_distinguished_name
|
||||
req_extensions = v3_req
|
||||
prompt = no
|
||||
|
||||
[req_distinguished_name]
|
||||
C = FR
|
||||
O = Garage
|
||||
|
||||
[v3_req]
|
||||
keyUsage = keyEncipherment, dataEncipherment
|
||||
extendedKeyUsage = clientAuth
|
||||
EOF
|
||||
) \
|
||||
-CA garage-ca.crt -CAkey garage-ca.key -CAcreateserial \
|
||||
-out garage-client.crt -days 365
|
||||
fi
|
||||
@@ -47,6 +47,10 @@ EOF
|
||||
|
||||
echo -en "$LABEL configuration written to $CONF_PATH\n"
|
||||
|
||||
(garage -c /tmp/config.$count.toml server 2>&1|while read r; do echo -en "$LABEL $r\n"; done) &
|
||||
done
|
||||
# >>>>>>>>>>>>>>>> END FOR LOOP ON NODES
|
||||
|
||||
if [ -z "$SKIP_HTTPS" ]; then
|
||||
echo -en "$LABEL Starting dummy HTTPS reverse proxy\n"
|
||||
mkdir -p /tmp/garagessl
|
||||
@@ -62,10 +66,6 @@ if [ -z "$SKIP_HTTPS" ]; then
|
||||
socat openssl-listen:4443,reuseaddr,fork,cert=/tmp/garagessl/test.pem,verify=0 tcp4-connect:localhost:3911 &
|
||||
fi
|
||||
|
||||
(garage -c /tmp/config.$count.toml server 2>&1|while read r; do echo -en "$LABEL $r\n"; done) &
|
||||
done
|
||||
# >>>>>>>>>>>>>>>> END FOR LOOP ON NODES
|
||||
|
||||
sleep 3
|
||||
# Establish connections between nodes
|
||||
for count in $(seq 1 3); do
|
||||
|
||||
@@ -125,6 +125,21 @@ if [ -z "$SKIP_AWS" ]; then
|
||||
rm /tmp/garage-index.html
|
||||
fi
|
||||
|
||||
if [ -z "$SKIP_AWS" ]; then
|
||||
echo "🪣 Test bucket logic "
|
||||
AWS_ACCESS_KEY_ID=`cat /tmp/garage.s3 |cut -d' ' -f1`
|
||||
[ $(aws s3 ls | wc -l) == 1 ]
|
||||
garage -c /tmp/config.1.toml bucket create seau
|
||||
garage -c /tmp/config.1.toml bucket allow --read seau --key $AWS_ACCESS_KEY_ID
|
||||
[ $(aws s3 ls | wc -l) == 2 ]
|
||||
garage -c /tmp/config.1.toml bucket deny --read seau --key $AWS_ACCESS_KEY_ID
|
||||
[ $(aws s3 ls | wc -l) == 1 ]
|
||||
garage -c /tmp/config.1.toml bucket allow --read seau --key $AWS_ACCESS_KEY_ID
|
||||
[ $(aws s3 ls | wc -l) == 2 ]
|
||||
garage -c /tmp/config.1.toml bucket delete --yes seau
|
||||
[ $(aws s3 ls | wc -l) == 1 ]
|
||||
fi
|
||||
|
||||
echo "🏁 Teardown"
|
||||
AWS_ACCESS_KEY_ID=`cat /tmp/garage.s3 |cut -d' ' -f1`
|
||||
AWS_SECRET_ACCESS_KEY=`cat /tmp/garage.s3 |cut -d' ' -f2`
|
||||
|
||||
+149
-154
@@ -1,4 +1,3 @@
|
||||
use std::collections::HashMap;
|
||||
use std::net::SocketAddr;
|
||||
use std::sync::Arc;
|
||||
|
||||
@@ -6,7 +5,7 @@ use futures::future::Future;
|
||||
use hyper::header;
|
||||
use hyper::server::conn::AddrStream;
|
||||
use hyper::service::{make_service_fn, service_fn};
|
||||
use hyper::{Body, Method, Request, Response, Server};
|
||||
use hyper::{Body, Request, Response, Server};
|
||||
|
||||
use garage_util::error::Error as GarageError;
|
||||
|
||||
@@ -22,6 +21,7 @@ use crate::s3_delete::*;
|
||||
use crate::s3_get::*;
|
||||
use crate::s3_list::*;
|
||||
use crate::s3_put::*;
|
||||
use crate::s3_router::{Authorization, Endpoint};
|
||||
|
||||
/// Run the S3 API server
|
||||
pub async fn run_api_server(
|
||||
@@ -65,10 +65,15 @@ async fn handler(
|
||||
}
|
||||
Err(e) => {
|
||||
let body: Body = Body::from(e.aws_xml(&garage.config.s3_api.s3_region, uri.path()));
|
||||
let http_error = Response::builder()
|
||||
let mut http_error_builder = Response::builder()
|
||||
.status(e.http_status_code())
|
||||
.header("Content-Type", "application/xml")
|
||||
.body(body)?;
|
||||
.header("Content-Type", "application/xml");
|
||||
|
||||
if let Some(header_map) = http_error_builder.headers_mut() {
|
||||
e.add_headers(header_map)
|
||||
}
|
||||
|
||||
let http_error = http_error_builder.body(body)?;
|
||||
|
||||
if e.http_status_code().is_server_error() {
|
||||
warn!("Response: error {}, {}", e.http_status_code(), e);
|
||||
@@ -81,8 +86,6 @@ async fn handler(
|
||||
}
|
||||
|
||||
async fn handler_inner(garage: Arc<Garage>, req: Request<Body>) -> Result<Response<Body>, Error> {
|
||||
let path = req.uri().path().to_string();
|
||||
let path = percent_encoding::percent_decode_str(&path).decode_utf8()?;
|
||||
let (api_key, content_sha256) = check_signature(&garage, &req).await?;
|
||||
|
||||
let authority = req
|
||||
@@ -100,166 +103,158 @@ async fn handler_inner(garage: Arc<Garage>, req: Request<Body>) -> Result<Respon
|
||||
.as_ref()
|
||||
.and_then(|root_domain| host_to_bucket(&host, root_domain));
|
||||
|
||||
if path == "/" && bucket.is_none() {
|
||||
return handle_list_buckets(&api_key);
|
||||
}
|
||||
|
||||
let (bucket, key) = parse_bucket_key(&path, bucket)?;
|
||||
let allowed = match req.method() {
|
||||
&Method::HEAD | &Method::GET => api_key.allow_read(bucket),
|
||||
_ => api_key.allow_write(bucket),
|
||||
let endpoint = Endpoint::from_request(&req, bucket.map(ToOwned::to_owned))?;
|
||||
let allowed = match endpoint.authorization_type() {
|
||||
Authorization::None => true,
|
||||
Authorization::Read(bucket) => api_key.allow_read(bucket),
|
||||
Authorization::Write(bucket) => api_key.allow_write(bucket),
|
||||
};
|
||||
|
||||
if !allowed {
|
||||
return Err(Error::Forbidden(
|
||||
"Operation is not allowed for this key.".to_string(),
|
||||
));
|
||||
}
|
||||
|
||||
let mut params = HashMap::new();
|
||||
if let Some(query) = req.uri().query() {
|
||||
let query_pairs = url::form_urlencoded::parse(query.as_bytes());
|
||||
for (key, val) in query_pairs {
|
||||
params.insert(key.to_lowercase(), val.to_string());
|
||||
match endpoint {
|
||||
Endpoint::ListBuckets => handle_list_buckets(&api_key),
|
||||
Endpoint::HeadObject { bucket, key, .. } => handle_head(garage, &req, &bucket, &key).await,
|
||||
Endpoint::GetObject { bucket, key, .. } => handle_get(garage, &req, &bucket, &key).await,
|
||||
Endpoint::UploadPart {
|
||||
bucket,
|
||||
key,
|
||||
part_number,
|
||||
upload_id,
|
||||
} => {
|
||||
handle_put_part(
|
||||
garage,
|
||||
req,
|
||||
&bucket,
|
||||
&key,
|
||||
part_number,
|
||||
&upload_id,
|
||||
content_sha256,
|
||||
)
|
||||
.await
|
||||
}
|
||||
}
|
||||
|
||||
if let Some(key) = key {
|
||||
match *req.method() {
|
||||
Method::HEAD => {
|
||||
// HeadObject query
|
||||
Ok(handle_head(garage, &req, bucket, key).await?)
|
||||
Endpoint::CopyObject { bucket, key } => {
|
||||
let copy_source = req.headers().get("x-amz-copy-source").unwrap().to_str()?;
|
||||
let copy_source = percent_encoding::percent_decode_str(copy_source).decode_utf8()?;
|
||||
let (source_bucket, source_key) = parse_bucket_key(©_source, None)?;
|
||||
if !api_key.allow_read(source_bucket) {
|
||||
return Err(Error::Forbidden(format!(
|
||||
"Reading from bucket {} not allowed for this key",
|
||||
source_bucket
|
||||
)));
|
||||
}
|
||||
Method::GET => {
|
||||
// GetObject query
|
||||
Ok(handle_get(garage, &req, bucket, key).await?)
|
||||
}
|
||||
Method::PUT => {
|
||||
if params.contains_key(&"partnumber".to_string())
|
||||
&& params.contains_key(&"uploadid".to_string())
|
||||
{
|
||||
// UploadPart query
|
||||
let part_number = params.get("partnumber").unwrap();
|
||||
let upload_id = params.get("uploadid").unwrap();
|
||||
Ok(handle_put_part(
|
||||
garage,
|
||||
req,
|
||||
let source_key = source_key.ok_or_bad_request("No source key specified")?;
|
||||
handle_copy(garage, &req, &bucket, &key, source_bucket, source_key).await
|
||||
}
|
||||
Endpoint::PutObject { bucket, key } => {
|
||||
handle_put(garage, req, &bucket, &key, content_sha256).await
|
||||
}
|
||||
Endpoint::AbortMultipartUpload {
|
||||
bucket,
|
||||
key,
|
||||
upload_id,
|
||||
} => handle_abort_multipart_upload(garage, &bucket, &key, &upload_id).await,
|
||||
Endpoint::DeleteObject { bucket, key, .. } => handle_delete(garage, &bucket, &key).await,
|
||||
Endpoint::CreateMultipartUpload { bucket, key } => {
|
||||
handle_create_multipart_upload(garage, &req, &bucket, &key).await
|
||||
}
|
||||
Endpoint::CompleteMultipartUpload {
|
||||
bucket,
|
||||
key,
|
||||
upload_id,
|
||||
} => {
|
||||
handle_complete_multipart_upload(garage, req, &bucket, &key, &upload_id, content_sha256)
|
||||
.await
|
||||
}
|
||||
Endpoint::CreateBucket { bucket } => {
|
||||
debug!(
|
||||
"Body: {}",
|
||||
std::str::from_utf8(&hyper::body::to_bytes(req.into_body()).await?)
|
||||
.unwrap_or("<invalid utf8>")
|
||||
);
|
||||
let empty_body: Body = Body::from(vec![]);
|
||||
let response = Response::builder()
|
||||
.header("Location", format!("/{}", bucket))
|
||||
.body(empty_body)
|
||||
.unwrap();
|
||||
Ok(response)
|
||||
}
|
||||
Endpoint::HeadBucket { .. } => {
|
||||
let empty_body: Body = Body::from(vec![]);
|
||||
let response = Response::builder().body(empty_body).unwrap();
|
||||
Ok(response)
|
||||
}
|
||||
Endpoint::DeleteBucket { .. } => Err(Error::Forbidden(
|
||||
"Cannot delete buckets using S3 api, please talk to Garage directly".into(),
|
||||
)),
|
||||
Endpoint::GetBucketLocation { .. } => handle_get_bucket_location(garage),
|
||||
Endpoint::GetBucketVersioning { .. } => handle_get_bucket_versioning(),
|
||||
Endpoint::ListObjects {
|
||||
bucket,
|
||||
delimiter,
|
||||
encoding_type,
|
||||
marker,
|
||||
max_keys,
|
||||
prefix,
|
||||
} => {
|
||||
handle_list(
|
||||
garage,
|
||||
&ListObjectsQuery {
|
||||
is_v2: false,
|
||||
bucket,
|
||||
delimiter: delimiter.map(|d| d.to_string()),
|
||||
max_keys: max_keys.unwrap_or(1000),
|
||||
prefix: prefix.unwrap_or_default(),
|
||||
marker,
|
||||
continuation_token: None,
|
||||
start_after: None,
|
||||
urlencode_resp: encoding_type.map(|e| e == "url").unwrap_or(false),
|
||||
},
|
||||
)
|
||||
.await
|
||||
}
|
||||
Endpoint::ListObjectsV2 {
|
||||
bucket,
|
||||
delimiter,
|
||||
encoding_type,
|
||||
max_keys,
|
||||
prefix,
|
||||
continuation_token,
|
||||
start_after,
|
||||
list_type,
|
||||
..
|
||||
} => {
|
||||
if list_type == "2" {
|
||||
handle_list(
|
||||
garage,
|
||||
&ListObjectsQuery {
|
||||
is_v2: true,
|
||||
bucket,
|
||||
key,
|
||||
part_number,
|
||||
upload_id,
|
||||
content_sha256,
|
||||
)
|
||||
.await?)
|
||||
} else if req.headers().contains_key("x-amz-copy-source") {
|
||||
// CopyObject query
|
||||
let copy_source = req.headers().get("x-amz-copy-source").unwrap().to_str()?;
|
||||
let copy_source =
|
||||
percent_encoding::percent_decode_str(copy_source).decode_utf8()?;
|
||||
let (source_bucket, source_key) = parse_bucket_key(©_source, None)?;
|
||||
if !api_key.allow_read(source_bucket) {
|
||||
return Err(Error::Forbidden(format!(
|
||||
"Reading from bucket {} not allowed for this key",
|
||||
source_bucket
|
||||
)));
|
||||
}
|
||||
let source_key = source_key.ok_or_bad_request("No source key specified")?;
|
||||
Ok(handle_copy(garage, &req, bucket, key, source_bucket, source_key).await?)
|
||||
} else {
|
||||
// PutObject query
|
||||
Ok(handle_put(garage, req, bucket, key, content_sha256).await?)
|
||||
}
|
||||
delimiter: delimiter.map(|d| d.to_string()),
|
||||
max_keys: max_keys.unwrap_or(1000),
|
||||
prefix: prefix.unwrap_or_default(),
|
||||
marker: None,
|
||||
continuation_token,
|
||||
start_after,
|
||||
urlencode_resp: encoding_type.map(|e| e == "url").unwrap_or(false),
|
||||
},
|
||||
)
|
||||
.await
|
||||
} else {
|
||||
Err(Error::BadRequest(format!(
|
||||
"Invalid endpoint: list-type={}",
|
||||
list_type
|
||||
)))
|
||||
}
|
||||
Method::DELETE => {
|
||||
if params.contains_key(&"uploadid".to_string()) {
|
||||
// AbortMultipartUpload query
|
||||
let upload_id = params.get("uploadid").unwrap();
|
||||
Ok(handle_abort_multipart_upload(garage, bucket, key, upload_id).await?)
|
||||
} else {
|
||||
// DeleteObject query
|
||||
Ok(handle_delete(garage, bucket, key).await?)
|
||||
}
|
||||
}
|
||||
Method::POST => {
|
||||
if params.contains_key(&"uploads".to_string()) {
|
||||
// CreateMultipartUpload call
|
||||
Ok(handle_create_multipart_upload(garage, &req, bucket, key).await?)
|
||||
} else if params.contains_key(&"uploadid".to_string()) {
|
||||
// CompleteMultipartUpload call
|
||||
let upload_id = params.get("uploadid").unwrap();
|
||||
Ok(handle_complete_multipart_upload(
|
||||
garage,
|
||||
req,
|
||||
bucket,
|
||||
key,
|
||||
upload_id,
|
||||
content_sha256,
|
||||
)
|
||||
.await?)
|
||||
} else {
|
||||
Err(Error::BadRequest(
|
||||
"Not a CreateMultipartUpload call, what is it?".to_string(),
|
||||
))
|
||||
}
|
||||
}
|
||||
_ => Err(Error::BadRequest("Invalid method".to_string())),
|
||||
}
|
||||
} else {
|
||||
match *req.method() {
|
||||
Method::PUT => {
|
||||
// CreateBucket
|
||||
// If we're here, the bucket already exists, so just answer ok
|
||||
debug!(
|
||||
"Body: {}",
|
||||
std::str::from_utf8(&hyper::body::to_bytes(req.into_body()).await?)
|
||||
.unwrap_or("<invalid utf8>")
|
||||
);
|
||||
let empty_body: Body = Body::from(vec![]);
|
||||
let response = Response::builder()
|
||||
.header("Location", format!("/{}", bucket))
|
||||
.body(empty_body)
|
||||
.unwrap();
|
||||
Ok(response)
|
||||
}
|
||||
Method::HEAD => {
|
||||
// HeadBucket
|
||||
let empty_body: Body = Body::from(vec![]);
|
||||
let response = Response::builder().body(empty_body).unwrap();
|
||||
Ok(response)
|
||||
}
|
||||
Method::DELETE => {
|
||||
// DeleteBucket query
|
||||
Err(Error::Forbidden(
|
||||
"Cannot delete buckets using S3 api, please talk to Garage directly".into(),
|
||||
))
|
||||
}
|
||||
Method::GET => {
|
||||
if params.contains_key("location") {
|
||||
// GetBucketLocation call
|
||||
Ok(handle_get_bucket_location(garage)?)
|
||||
} else if params.contains_key("versioning") {
|
||||
// GetBucketVersioning
|
||||
Ok(handle_get_bucket_versioning()?)
|
||||
} else {
|
||||
// ListObjects or ListObjectsV2 query
|
||||
let q = parse_list_objects_query(bucket, ¶ms)?;
|
||||
Ok(handle_list(garage, &q).await?)
|
||||
}
|
||||
}
|
||||
Method::POST => {
|
||||
if params.contains_key(&"delete".to_string()) {
|
||||
// DeleteObjects
|
||||
Ok(handle_delete_objects(garage, bucket, req, content_sha256).await?)
|
||||
} else {
|
||||
debug!(
|
||||
"Body: {}",
|
||||
std::str::from_utf8(&hyper::body::to_bytes(req.into_body()).await?)
|
||||
.unwrap_or("<invalid utf8>")
|
||||
);
|
||||
Err(Error::BadRequest("Unsupported call".to_string()))
|
||||
}
|
||||
}
|
||||
_ => Err(Error::BadRequest("Invalid method".to_string())),
|
||||
Endpoint::DeleteObjects { bucket } => {
|
||||
handle_delete_objects(garage, &bucket, req, content_sha256).await
|
||||
}
|
||||
endpoint => Err(Error::NotImplemented(endpoint.name().to_owned())),
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+27
-2
@@ -1,5 +1,8 @@
|
||||
use std::convert::TryInto;
|
||||
|
||||
use err_derive::Error;
|
||||
use hyper::StatusCode;
|
||||
use hyper::header::HeaderValue;
|
||||
use hyper::{HeaderMap, StatusCode};
|
||||
|
||||
use garage_util::error::Error as GarageError;
|
||||
|
||||
@@ -57,11 +60,15 @@ pub enum Error {
|
||||
|
||||
/// The client sent a range header with invalid value
|
||||
#[error(display = "Invalid HTTP range: {:?}", _0)]
|
||||
InvalidRange(#[error(from)] http_range::HttpRangeParseError),
|
||||
InvalidRange(#[error(from)] (http_range::HttpRangeParseError, u64)),
|
||||
|
||||
/// The client sent an invalid request
|
||||
#[error(display = "Bad request: {}", _0)]
|
||||
BadRequest(String),
|
||||
|
||||
/// The client sent a request for an action not supported by garage
|
||||
#[error(display = "Unimplemented action: {}", _0)]
|
||||
NotImplemented(String),
|
||||
}
|
||||
|
||||
impl From<roxmltree::Error> for Error {
|
||||
@@ -90,6 +97,8 @@ impl Error {
|
||||
Error::InternalError(_) | Error::Hyper(_) | Error::Http(_) => {
|
||||
StatusCode::INTERNAL_SERVER_ERROR
|
||||
}
|
||||
Error::InvalidRange(_) => StatusCode::RANGE_NOT_SATISFIABLE,
|
||||
Error::NotImplemented(_) => StatusCode::NOT_IMPLEMENTED,
|
||||
_ => StatusCode::BAD_REQUEST,
|
||||
}
|
||||
}
|
||||
@@ -127,6 +136,22 @@ impl Error {
|
||||
.into()
|
||||
})
|
||||
}
|
||||
|
||||
pub fn add_headers(&self, header_map: &mut HeaderMap<HeaderValue>) {
|
||||
use hyper::header;
|
||||
#[allow(clippy::single_match)]
|
||||
match self {
|
||||
Error::InvalidRange((_, len)) => {
|
||||
header_map.append(
|
||||
header::CONTENT_RANGE,
|
||||
format!("bytes */{}", len)
|
||||
.try_into()
|
||||
.expect("header value only contain ascii"),
|
||||
);
|
||||
}
|
||||
_ => (),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Trait to map error to the Bad Request error code
|
||||
|
||||
+1
-1
@@ -26,7 +26,7 @@ pub fn host_to_bucket<'a>(host: &'a str, root: &str) -> Option<&'a str> {
|
||||
/// The HTTP host contains both a host and a port.
|
||||
/// Extracting the port is more complex than just finding the colon (:) symbol due to IPv6
|
||||
/// We do not use the collect pattern as there is no way in std rust to collect over a stack allocated value
|
||||
/// check here: https://docs.rs/collect_slice/1.2.0/collect_slice/
|
||||
/// check here: <https://docs.rs/collect_slice/1.2.0/collect_slice/>
|
||||
pub fn authority_to_host(authority: &str) -> Result<String, Error> {
|
||||
let mut iter = authority.chars().enumerate();
|
||||
let (_, first_char) = iter
|
||||
|
||||
@@ -19,4 +19,5 @@ mod s3_delete;
|
||||
pub mod s3_get;
|
||||
mod s3_list;
|
||||
mod s3_put;
|
||||
mod s3_router;
|
||||
mod s3_xml;
|
||||
|
||||
@@ -45,6 +45,7 @@ pub fn handle_list_buckets(api_key: &Key) -> Result<Response<Body>, Error> {
|
||||
.authorized_buckets
|
||||
.items()
|
||||
.iter()
|
||||
.filter(|(_, _, perms)| perms.allow_read || perms.allow_write)
|
||||
.map(|(name, ts, _)| s3_xml::Bucket {
|
||||
creation_date: s3_xml::Value(msec_to_rfc3339(*ts)),
|
||||
name: s3_xml::Value(name.to_string()),
|
||||
|
||||
+5
-8
@@ -156,11 +156,12 @@ pub async fn handle_get(
|
||||
let range = match req.headers().get("range") {
|
||||
Some(range) => {
|
||||
let range_str = range.to_str()?;
|
||||
let mut ranges = http_range::HttpRange::parse(range_str, last_v_meta.size)?;
|
||||
let mut ranges = http_range::HttpRange::parse(range_str, last_v_meta.size)
|
||||
.map_err(|e| (e, last_v_meta.size))?;
|
||||
if ranges.len() > 1 {
|
||||
return Err(Error::BadRequest(
|
||||
"Multiple ranges not supported".to_string(),
|
||||
));
|
||||
// garage does not support multi-range requests yet, so we respond with the entire
|
||||
// object when multiple ranges are requested
|
||||
None
|
||||
} else {
|
||||
ranges.pop()
|
||||
}
|
||||
@@ -235,10 +236,6 @@ async fn handle_get_range(
|
||||
begin: u64,
|
||||
end: u64,
|
||||
) -> Result<Response<Body>, Error> {
|
||||
if end > version_meta.size {
|
||||
return Err(Error::BadRequest("Range not included in file".to_string()));
|
||||
}
|
||||
|
||||
let resp_builder = object_headers(version, version_meta)
|
||||
.header("Content-Length", format!("{}", end - begin))
|
||||
.header(
|
||||
|
||||
+1
-27
@@ -1,4 +1,4 @@
|
||||
use std::collections::{BTreeMap, BTreeSet, HashMap};
|
||||
use std::collections::{BTreeMap, BTreeSet};
|
||||
use std::sync::Arc;
|
||||
|
||||
use hyper::{Body, Response};
|
||||
@@ -35,32 +35,6 @@ struct ListResultInfo {
|
||||
etag: String,
|
||||
}
|
||||
|
||||
pub fn parse_list_objects_query(
|
||||
bucket: &str,
|
||||
params: &HashMap<String, String>,
|
||||
) -> Result<ListObjectsQuery, Error> {
|
||||
Ok(ListObjectsQuery {
|
||||
is_v2: params.get("list-type").map(|x| x == "2").unwrap_or(false),
|
||||
bucket: bucket.to_string(),
|
||||
delimiter: params.get("delimiter").filter(|x| !x.is_empty()).cloned(),
|
||||
max_keys: params
|
||||
.get("max-keys")
|
||||
.map(|x| {
|
||||
x.parse::<usize>()
|
||||
.ok_or_bad_request("Invalid value for max-keys")
|
||||
})
|
||||
.unwrap_or(Ok(1000))?,
|
||||
prefix: params.get("prefix").cloned().unwrap_or_default(),
|
||||
marker: params.get("marker").cloned(),
|
||||
continuation_token: params.get("continuation-token").cloned(),
|
||||
start_after: params.get("start-after").cloned(),
|
||||
urlencode_resp: params
|
||||
.get("encoding-type")
|
||||
.map(|x| x == "url")
|
||||
.unwrap_or(false),
|
||||
})
|
||||
}
|
||||
|
||||
pub async fn handle_list(
|
||||
garage: Arc<Garage>,
|
||||
query: &ListObjectsQuery,
|
||||
|
||||
+1
-6
@@ -354,15 +354,10 @@ pub async fn handle_put_part(
|
||||
req: Request<Body>,
|
||||
bucket: &str,
|
||||
key: &str,
|
||||
part_number_str: &str,
|
||||
part_number: u64,
|
||||
upload_id: &str,
|
||||
content_sha256: Option<Hash>,
|
||||
) -> Result<Response<Body>, Error> {
|
||||
// Check parameters
|
||||
let part_number = part_number_str
|
||||
.parse::<u64>()
|
||||
.ok_or_bad_request("Invalid part number")?;
|
||||
|
||||
let version_uuid = decode_upload_id(upload_id)?;
|
||||
|
||||
let content_md5 = match req.headers().get("content-md5") {
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
+10
-1
@@ -1,5 +1,6 @@
|
||||
use err_derive::Error;
|
||||
use hyper::StatusCode;
|
||||
use hyper::header::HeaderValue;
|
||||
use hyper::{HeaderMap, StatusCode};
|
||||
|
||||
use garage_util::error::Error as GarageError;
|
||||
|
||||
@@ -47,4 +48,12 @@ impl Error {
|
||||
_ => StatusCode::BAD_REQUEST,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn add_headers(&self, header_map: &mut HeaderMap<HeaderValue>) {
|
||||
#[allow(clippy::single_match)]
|
||||
match self {
|
||||
Error::ApiError(e) => e.add_headers(header_map),
|
||||
_ => (),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -62,6 +62,7 @@ fn error_to_res(e: Error) -> Response<Body> {
|
||||
let body: Body = Body::from(format!("{}\n", e));
|
||||
let mut http_error = Response::new(body);
|
||||
*http_error.status_mut() = e.http_status_code();
|
||||
e.add_headers(http_error.headers_mut());
|
||||
http_error
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user