gsadmin/garage
1
0
Fork 0
mirror of https://github.com/deuxfleurs-org/garage.git synced 2026-09-04 19:25:42 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request 'Make initial setup easier' (#1329) from easy-bootstrap into main-v2

Browse Source 
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/1329
This commit is contained in:
Alex
2026-02-14 18:26:43 +00:00
parent c22c4ff2e5 1d588282bf
commit 3af9e8d3d2
6 changed files with 430 additions and 147 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/book/cookbook/real-world.md
+1 -1
View File
@@ -103,7 +103,7 @@ to check [the most recent versions on the Docker Hub](https://hub.docker.com/r/d
For example: For example:
``` ```
sudo docker pull dxflrs/garage:v2.2.0 docker pull dxflrs/garage:v2.2.0
``` ```
## Deploying and configuring Garage ## Deploying and configuring Garage
doc/book/quick-start/_index.md
+210 -137
View File
@@ -43,12 +43,10 @@ or if you want a build customized for your system,
you can [build Garage from source](@/documentation/cookbook/from-source.md). you can [build Garage from source](@/documentation/cookbook/from-source.md).
If none of these option work for you, you can also run Garage in a Docker If none of these option work for you, you can also run Garage in a Docker
container. When using Docker, the commands used in this guide will not work container. For simplicity, a minimal command to launch Garage using Docker is
anymore. We recommend reading the tutorial on [configuring a provided in this quick start guide. We recommend reading the tutorial on
multi-node cluster](@/documentation/cookbook/real-world.md) to learn about [configuring a multi-node cluster](@/documentation/cookbook/real-world.md) to
using Garage as a Docker container. For simplicity, a minimal command to launch learn about the full Docker workflow for Garage.
Garage using Docker is provided in this quick start guide as well.
## Configuring and starting Garage ## Configuring and starting Garage
@@ -82,9 +80,6 @@ bind_addr = "[::]:3902"
root_domain = ".web.garage.localhost" root_domain = ".web.garage.localhost"
index = "index.html" index = "index.html"
[k2v_api]
api_bind_addr = "[::]:3904"
[admin] [admin]
api_bind_addr = "[::]:3903" api_bind_addr = "[::]:3903"
admin_token = "$(openssl rand -base64 32)" admin_token = "$(openssl rand -base64 32)"
@@ -95,10 +90,13 @@ EOF
See the [Configuration file format](https://garagehq.deuxfleurs.fr/documentation/reference-manual/configuration/) See the [Configuration file format](https://garagehq.deuxfleurs.fr/documentation/reference-manual/configuration/)
for complete options and values. for complete options and values.
Now that your configuration file has been created, you may save it to the directory of your choice. By default, Garage looks for its configuration file in **`/etc/garage.toml`.**
By default, Garage looks for **`/etc/garage.toml`.** Since we have written our configuration file in the working directory, we will have to set
You can also store it somewhere else, but you will have to specify `-c path/to/garage.toml` the following environment variable:
at each invocation of the `garage` binary (for example: `garage -c ./garage.toml server`, `garage -c ./garage.toml status`).
```bash
export GARAGE_CONFIG_FILE=$(pwd)/garage.toml
```
As you can see, the `rpc_secret` is a 32 bytes hexadecimal string. As you can see, the `rpc_secret` is a 32 bytes hexadecimal string.
You can regenerate it with `openssl rand -hex 32`. You can regenerate it with `openssl rand -hex 32`.
@@ -111,15 +109,36 @@ Garage server will not be persistent. Change these to locations on your local di
your data to be persisted properly. your data to be persisted properly.
### Configuring initial access credentials
Since `v2.3.0`, Garage can automatically create a default access key and a default storage bucket,
based on values provided in environment variables.
To use this feature, export the following environment variables:
```bash
export GARAGE_DEFAULT_ACCESS_KEY="GK$(openssl rand -hex 16)"
export GARAGE_DEFAULT_SECRET_KEY="$(openssl rand -hex 32)"
export GARAGE_DEFAULT_BUCKET="default-bucket"
```
The example above creates a random access key ID and associated secret key.
You can also provide an access key ID and secret key of your own.
### Launching the Garage server ### Launching the Garage server
Use the following command to launch the Garage server: Use the following command to launch the Garage server:
``` ```bash
garage -c path/to/garage.toml server garage server --single-node --default-bucket
``` ```
If you have placed the `garage.toml` file in `/etc` (its default location), you can simply run `garage server`. The `--single-node` flag instructs Garage to automatically configure a single-node cluster without data replication.
The `--default-bucket` flag instructs Garage to create a default access key and a default bucket using the environment variables we defined above.
Both flags are optional and can be omitted, in which case you will have to follow manual configuration steps described below.
**For older versions of Garage (before v2.3.0):** automatic configuration using `--single-node` and `--default-bucket` is not available,
you must follow the manual configuration steps.
Alternatively, if you cannot or do not wish to run the Garage binary directly, Alternatively, if you cannot or do not wish to run the Garage binary directly,
you may use Docker to run Garage in a container using the following command: you may use Docker to run Garage in a container using the following command:
@@ -127,21 +146,58 @@ you may use Docker to run Garage in a container using the following command:
```bash ```bash
docker run \ docker run \
-d \ -d \
--name garaged \ --name garage-container \
-p 3900:3900 -p 3901:3901 -p 3902:3902 -p 3903:3903 \ -p 3900:3900 -p 3901:3901 -p 3902:3902 -p 3903:3903 \
-v /path/to/garage.toml:/etc/garage.toml \ -v $(pwd)/garage.toml:/etc/garage.toml \
-v /tmp/meta:/tmp/meta \ -e GARAGE_DEFAULT_ACCESS_KEY \
-v /tmp/data:/tmp/data \ -e GARAGE_DEFAULT_SECRET_KEY \
-e GARAGE_DEFAULT_BUCKET \
dxflrs/garage:v2.2.0 dxflrs/garage:v2.2.0
/garage server --single-node --default-bucket
``` ```
Under Linux, you can substitute `--network host` for `-p 3900:3900 -p 3901:3901 -p 3902:3902 -p 3903:3903` Note that this command will NOT create persistent volumes for Garage's data, so
your cluster will be wiped if the container terminates. To persist Garage's
data, you must manually add volumes for the `data` and `metadata` directories
and configure their correct paths in your `garage.toml` files (see [configuring
a multi-node cluster](@/documentation/cookbook/real-world.md)).
#### Troubleshooting Under Linux, you can substitute `--network host` for `-p 3900:3900 -p 3901:3901 -p 3902:3902 -p 3903:3903`.
### Checking that Garage runs correctly
The `garage` utility is also used as a CLI tool to administrate your Garage
deployment. It needs read access to your configuration file and to the metadata directory
to obtain connection parameters to contact the local Garage node.
Use the following command to show the status of your cluster:
```
garage status
```
If you are running Garage in a Docker container, you can use the following command instead:
```bash
docker exec garage-container /garage status
```
This should show something like this:
```
==== HEALTHY NODES ====
ID Hostname Address Tags Zone Capacity DataAvail Version
563e1ac825ee3323 linuxbox 127.0.0.1:3901 [default] dc1 19.9 GiB 19.5 GiB (97.6%) v2.3.0
```
### Troubleshooting
Ensure your configuration file, `metadata_dir` and `data_dir` are readable by the user running the `garage` server or Docker. Ensure your configuration file, `metadata_dir` and `data_dir` are readable by the user running the `garage` server or Docker.
You can tune Garage's verbosity by setting the `RUST_LOG=` environment variable. \ When running the `garage` CLI, ensure that the path to your configuration file is correctly specified (see below),
and that it can read it and read from your metadata directory.
You can tune Garage's verbosity by setting the `RUST_LOG=` environment variable.
Available log levels are (from less verbose to more verbose): `error`, `warn`, `info` *(default)*, `debug` and `trace`. Available log levels are (from less verbose to more verbose): `error`, `warn`, `info` *(default)*, `debug` and `trace`.
```bash ```bash
@@ -154,36 +210,135 @@ Log level `info` is the default value and is recommended for most use cases.
Log level `debug` can help you check why your S3 API calls are not working. Log level `debug` can help you check why your S3 API calls are not working.
### Checking that Garage runs correctly
The `garage` utility is also used as a CLI tool to configure your Garage deployment. ## Uploading and downloading from Garage
It uses values from the TOML configuration file to find the Garage daemon running on the
local node, therefore if your configuration file is not at `/etc/garage.toml` you will
again have to specify `-c path/to/garage.toml` at each invocation.
If you are running Garage in a Docker container, you can set `alias garage="docker exec -ti <container name> /garage"` This section will show how to download and upload files on Garage using a third-party tool named `awscli`.
to use the Garage binary inside your container.
If the `garage` CLI is able to correctly detect the parameters of your local Garage node,
the following command should be enough to show the status of your cluster:
``` ### Install and configure `awscli`
garage status
If you have python on your system, you can install it with:
```bash
python -m pip install --user awscli
``` ```
This should show something like this: Now that `awscli` is installed, you must configure it to talk to your Garage
instance using the credentials defined above. Here is a simple way to create
a configuration file in `~/.awsrc` using a single command that will save the
secrets from your environment:
```bash
cat > ~/.awsrc <<EOF
export AWS_ENDPOINT_URL='http://localhost:3900'
export AWS_DEFAULT_REGION='garage'
export AWS_ACCESS_KEY_ID='$GARAGE_DEFAULT_ACCESS_KEY'
export AWS_SECRET_ACCESS_KEY='$GARAGE_DEFAULT_SECRET_KEY'
aws --version
EOF
```
Note that you need to have at least `awscli` `>=1.29.0` or `>=2.13.0`, otherwise you
need to specify `--endpoint-url` explicitly on each `awscli` invocation.
Now, each time you want to use `awscli` on this target, run:
```bash
source ~/.awsrc
```
*You can create multiple files with different names if you
have multiple Garage clusters or different keys.
Switching from one cluster to another is as simple as
sourcing the right file.*
### Example usage of `awscli`
```bash
# list buckets
aws s3 ls
# list objects of a bucket
aws s3 ls s3://default-bucket
# copy from your filesystem to garage
aws s3 cp /proc/cpuinfo s3://default-bucket/cpuinfo.txt
# copy from garage to your filesystem
aws s3 cp s3://default-bucket/cpuinfo.txt /tmp/cpuinfo.txt
```
Note that you can use `awscli` for more advanced operations like
creating a bucket, pre-signing a request or managing your website.
[Read the full documentation to know more](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3/index.html).
Some features are however not implemented like ACL or policy.
Check [our S3 compatibility list](@/documentation/reference-manual/s3-compatibility.md).
### Other tools for interacting with Garage
The following tools can also be used to send and receive files from/to Garage:
- [minio-client](@/documentation/connect/cli.md#minio-client)
- [s3cmd](@/documentation/connect/cli.md#s3cmd)
- [rclone](@/documentation/connect/cli.md#rclone)
- [Cyberduck](@/documentation/connect/cli.md#cyberduck)
- [WinSCP](@/documentation/connect/cli.md#winscp)
An exhaustive list is maintained in the ["Integrations" > "Browsing tools" section](@/documentation/connect/_index.md).
## Manual configuration
This section provides instructions that are equivalent to using the
`--single-node` and `--default-bucket` flags for automatic configuration. If
you are using an older version of Garage (before v2.3.0), you must follow
these instructions as automatic configuration is not available.
We will have to run quite a few `garage` administration commands to get started.
If you ever get lost, don't forget that the `help` command and the `--help` flags can help you anywhere,
the CLI tool is self-documented! Two examples:
```
garage help
garage bucket allow --help
```
### Configuring the `garage` CLI
Remember that the `garage` CLI needs to know the path of your `garage.toml` configuration file.
If it is not in the default location of `/etc/garage.toml`, you can specify it either:
- by setting the `GARAGE_CONFIG_FILE` environment variable;
- by adding the `-c` flag to each `garage` command, for example: `garage -c ./garage.toml status`.
If you are running Garage in a Docker container, you can set the following alias
to provide a fake `garage`command that uses the Garage binary inside your container:
```bash
alias garage="docker exec -ti <container name> /garage"
```
You can test that your `garage` CLI is configured correctly by running a basic command such as `garage status`.
### Creating a cluster layout
When you first start a cluster without automatic configuration, the output of `garage status` will look as follows:
``` ```
==== HEALTHY NODES ==== ==== HEALTHY NODES ====
ID Hostname Address Tag Zone Capacity ID Hostname Address Tags Zone Capacity DataAvail Version
563e1ac825ee3323 linuxbox 127.0.0.1:3901 NO ROLE ASSIGNED 563e1ac825ee3323 linuxbox 127.0.0.1:3901 NO ROLE ASSIGNED v2.2.0
``` ```
## Creating a cluster layout Creating a cluster layout for a Garage deployment means informing Garage of the
disk space available on each node of the cluster using the `-c` flag, as well
Creating a cluster layout for a Garage deployment means informing Garage as the name of the zone (e.g. datacenter) each machine is located in using the
of the disk space available on each node of the cluster, `-c`, `-z` flag.
as well as the name of the zone (e.g. datacenter), `-z`, each machine is located in.
For our test deployment, we are have only one node with zone named `dc1` and a For our test deployment, we are have only one node with zone named `dc1` and a
capacity of `1G`, though the capacity is ignored for a single node deployment capacity of `1G`, though the capacity is ignored for a single node deployment
@@ -204,38 +359,29 @@ garage layout apply --version 1
``` ```
## Creating buckets and keys ### Creating buckets and keys
In this section, we will suppose that we want to create a bucket named `nextcloud-bucket`
that will be accessed through a key named `nextcloud-app-key`.
Don't forget that `help` command and `--help` subcommands can help you anywhere,
the CLI tool is self-documented! Two examples:
```
garage help
garage bucket allow --help
```
### Create a bucket
Let's take an example where we want to deploy NextCloud using Garage as the Let's take an example where we want to deploy NextCloud using Garage as the
main data storage. main data storage. We will suppose that we want to create a bucket named
`nextcloud-bucket` that will be accessed through a key named
`nextcloud-app-key`.
First, create a bucket with the following command: #### Create a bucket
First, create the bucket with the following command:
``` ```
garage bucket create nextcloud-bucket garage bucket create nextcloud-bucket
``` ```
Check that everything went well: Check that the bucket was created properly:
``` ```
garage bucket list garage bucket list
garage bucket info nextcloud-bucket garage bucket info nextcloud-bucket
``` ```
### Create an API key #### Create an API key
The `nextcloud-bucket` bucket now exists on the Garage server, The `nextcloud-bucket` bucket now exists on the Garage server,
however it cannot be accessed until we add an API key with the proper access rights. however it cannot be accessed until we add an API key with the proper access rights.
@@ -258,14 +404,14 @@ Secret key: 7d37d093435a41f2aab8f13c19ba067d9776c90215f56614adad6ece597dbb34
Authorized buckets: Authorized buckets:
``` ```
Check that everything works as intended: Check that the key was created properly:
``` ```
garage key list garage key list
garage key info nextcloud-app-key garage key info nextcloud-app-key
``` ```
### Allow a key to access a bucket #### Allow a key to access a bucket
Now that we have a bucket and a key, we need to give permissions to the key on the bucket: Now that we have a bucket and a key, we need to give permissions to the key on the bucket:
@@ -284,78 +430,5 @@ You can check at any time the allowed keys on your bucket with:
garage bucket info nextcloud-bucket garage bucket info nextcloud-bucket
``` ```
You should now be able to read and write objects to the bucket using the
## Uploading and downloading from Garage credentials created above.
To download and upload files on garage, we can use a third-party tool named `awscli`.
### Install and configure `awscli`
If you have python on your system, you can install it with:
```bash
python -m pip install --user awscli
```
Now that `awscli` is installed, you must configure it to talk to your Garage instance,
with your key. There are multiple ways to do that, the simplest one is to create a file
named `~/.awsrc` with this content:
```bash
export AWS_ACCESS_KEY_ID=xxxx # put your Key ID here
export AWS_SECRET_ACCESS_KEY=xxxx # put your Secret key here
export AWS_DEFAULT_REGION='garage'
export AWS_ENDPOINT_URL='http://localhost:3900'
aws --version
```
Note you need to have at least `awscli` `>=1.29.0` or `>=2.13.0`, otherwise you
need to specify `--endpoint-url` explicitly on each `awscli` invocation.
Now, each time you want to use `awscli` on this target, run:
```bash
source ~/.awsrc
```
*You can create multiple files with different names if you
have multiple Garage clusters or different keys.
Switching from one cluster to another is as simple as
sourcing the right file.*
### Example usage of `awscli`
```bash
# list buckets
aws s3 ls
# list objects of a bucket
aws s3 ls s3://nextcloud-bucket
# copy from your filesystem to garage
aws s3 cp /proc/cpuinfo s3://nextcloud-bucket/cpuinfo.txt
# copy from garage to your filesystem
aws s3 cp s3://nextcloud-bucket/cpuinfo.txt /tmp/cpuinfo.txt
```
Note that you can use `awscli` for more advanced operations like
creating a bucket, pre-signing a request or managing your website.
[Read the full documentation to know more](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3/index.html).
Some features are however not implemented like ACL or policy.
Check [our s3 compatibility list](@/documentation/reference-manual/s3-compatibility.md).
### Other tools for interacting with Garage
The following tools can also be used to send and receive files from/to Garage:
- [minio-client](@/documentation/connect/cli.md#minio-client)
- [s3cmd](@/documentation/connect/cli.md#s3cmd)
- [rclone](@/documentation/connect/cli.md#rclone)
- [Cyberduck](@/documentation/connect/cli.md#cyberduck)
- [WinSCP](@/documentation/connect/cli.md#winscp)
An exhaustive list is maintained in the ["Integrations" > "Browsing tools" section](@/documentation/connect/_index.md).
src/garage/cli/structs.rs
+21 -1
View File
@@ -8,7 +8,7 @@ use crate::cli::local::convert_db;
pub enum Command { pub enum Command {
/// Run Garage server /// Run Garage server
#[structopt(name = "server", version = garage_version())] #[structopt(name = "server", version = garage_version())]
Server, Server(ServerOpt),
/// Get network status /// Get network status
#[structopt(name = "status", version = garage_version())] #[structopt(name = "status", version = garage_version())]
@@ -79,6 +79,26 @@ pub enum Command {
}, },
} }
// ---------------------------
// ---- garage server ... ----
// ---------------------------
#[derive(StructOpt, Debug)]
pub struct ServerOpt {
/// Automatically configure a single-node layout in the cluster.
/// Garage will refuse to run if the cluster already has other nodes.
#[structopt(long = "single-node")]
pub(crate) single_node: bool,
/// Configure a default S3 API key using environment variables `GARAGE_DEFAULT_ACCESS_KEY` and
/// `GARAGE_DEFAULT_SECRET_KEY`. Requires `--single-node`.
#[structopt(long = "default-access-key")]
pub(crate) default_access_key: bool,
/// Configure a default bucket using environment variable `GARAGE_DEFAULT_BUCKET`.
/// Implies `--default-access-key`. Requires `--single-node`.
#[structopt(long = "default-bucket")]
pub(crate) default_bucket: bool,
}
// ------------------------- // -------------------------
// ---- garage node ... ---- // ---- garage node ... ----
// ------------------------- // -------------------------
src/garage/main.rs
+2 -2
View File
@@ -158,7 +158,7 @@ fn main() {
async fn run(opt: Opt) -> Result<(), Error> { async fn run(opt: Opt) -> Result<(), Error> {
match opt.cmd { match opt.cmd {
Command::Server => server::run_server(opt.config_file, opt.secrets).await, Command::Server(sopt) => server::run_server(opt.config_file, opt.secrets, sopt).await,
Command::OfflineRepair(repair_opt) => { Command::OfflineRepair(repair_opt) => {
cli::local::repair::offline_repair(opt.config_file, opt.secrets, repair_opt).await cli::local::repair::offline_repair(opt.config_file, opt.secrets, repair_opt).await
} }
@@ -188,7 +188,7 @@ async fn run(opt: Opt) -> Result<(), Error> {
fn init_logging(opt: &Opt) { fn init_logging(opt: &Opt) {
if std::env::var("RUST_LOG").is_err() { if std::env::var("RUST_LOG").is_err() {
let default_log = match &opt.cmd { let default_log = match &opt.cmd {
Command::Server => "netapp=info,garage=info", Command::Server(_) => "netapp=info,garage=info",
_ => "netapp=warn,garage=warn", _ => "netapp=warn,garage=warn",
}; };
src/garage/server.rs
+181 -2
View File
@@ -1,4 +1,5 @@
use std::path::PathBuf; use std::path::PathBuf;
use std::sync::Arc;
use tokio::sync::watch; use tokio::sync::watch;
@@ -16,6 +17,7 @@ use garage_api_k2v::api_server::K2VApiServer;
use crate::secrets::{fill_secrets, Secrets}; use crate::secrets::{fill_secrets, Secrets};
use crate::tracing_setup::init_tracing; use crate::tracing_setup::init_tracing;
use crate::ServerOpt;
async fn wait_from(mut chan: watch::Receiver<bool>) { async fn wait_from(mut chan: watch::Receiver<bool>) {
while !*chan.borrow() { while !*chan.borrow() {
@@ -25,8 +27,12 @@ async fn wait_from(mut chan: watch::Receiver<bool>) {
} }
} }
pub async fn run_server(config_file: PathBuf, secrets: Secrets) -> Result<(), Error> { pub async fn run_server(
info!("Loading configuration..."); config_file: PathBuf,
secrets: Secrets,
opt: ServerOpt,
) -> Result<(), Error> {
info!("Loading configuration from {}...", config_file.display());
let config = fill_secrets(read_config(config_file)?, secrets)?; let config = fill_secrets(read_config(config_file)?, secrets)?;
// ---- Initialize Garage internals ---- // ---- Initialize Garage internals ----
@@ -44,6 +50,9 @@ pub async fn run_server(config_file: PathBuf, secrets: Secrets) -> Result<(), Er
info!("Initializing Garage main data store..."); info!("Initializing Garage main data store...");
let garage = Garage::new(config.clone())?; let garage = Garage::new(config.clone())?;
// Handle --single-node, --default-bucket and --default-access-key
initial_config(&garage, opt).await?;
info!("Initializing background runner..."); info!("Initializing background runner...");
let watch_cancel = watch_shutdown_signal(); let watch_cancel = watch_shutdown_signal();
let (background, await_background_done) = BackgroundRunner::new(watch_cancel.clone()); let (background, await_background_done) = BackgroundRunner::new(watch_cancel.clone());
@@ -164,6 +173,176 @@ pub async fn run_server(config_file: PathBuf, secrets: Secrets) -> Result<(), Er
Ok(()) Ok(())
} }
async fn initial_config(garage: &Arc<Garage>, opt: ServerOpt) -> Result<(), Error> {
use garage_model::bucket_alias_table::is_valid_bucket_name;
use garage_model::bucket_table::Bucket;
use garage_model::key_table::*;
use garage_model::permission::BucketKeyPerm;
use garage_rpc::layout::*;
use garage_rpc::replication_mode::ReplicationFactor;
use garage_table::*;
use garage_util::time::now_msec;
if opt.single_node {
if garage.replication_factor != ReplicationFactor::new(1).unwrap() {
return Err(Error::Message(
"Single-node mode requires replication_factor = 1 in the configuration file."
.into(),
));
}
let layout_version = garage.system.cluster_layout().inner().current().version;
if layout_version > 1 {
return Err(Error::Message("Refusing to run in single-node mode: layout version is already superior to 1. Remove the --single-node flag to run the server in full mode.".into()));
}
if layout_version == 0 {
// Setup initial layout
let mut layout = garage.system.cluster_layout().inner().clone();
let our_id = garage.system.id;
// Check no other nodes are present in the system
let nodes = garage.system.get_known_nodes();
if nodes.iter().any(|x| x.id != our_id) {
return Err(Error::Message("Refusing to run in single-node mode: more nodes are already present in the cluster.".into()));
}
// Automatically determine this node's capacity
let capacity = garage
.system
.local_status()
.data_disk_avail
.map(|(_avail, total)| total)
.unwrap_or(1024 * 1024 * 1024); // Default to 1GB
assert!(layout.current().roles.is_empty());
layout.staging.get_mut().roles.clear();
layout.staging.get_mut().roles.update_in_place(
our_id,
NodeRoleV(Some(NodeRole {
zone: "dc1".to_string(),
capacity: Some(capacity),
tags: vec!["default".to_string()],
})),
);
let (layout, msg) = layout.apply_staged_changes(1)?;
info!(
"Created initial layout for single-node configuration:\n{}",
msg.join("\n")
);
garage
.system
.layout_manager
.update_cluster_layout(&layout)
.await?;
}
}
if (opt.default_bucket || opt.default_access_key) && !opt.single_node {
return Err(Error::Message(
"Flags --default-access-key and --default-bucket can only be used in single-node mode."
.into(),
));
}
if opt.default_access_key || opt.default_bucket {
let rdenv = |name: &str| {
std::env::var(name)
.map_err(|_| Error::Message(format!("Environment variable `{}` is not set", name)))
};
// Create default access key if it does not exist
let key_id = rdenv("GARAGE_DEFAULT_ACCESS_KEY")?;
let secret_key = rdenv("GARAGE_DEFAULT_SECRET_KEY")?;
let existing_key = garage.key_table.get(&EmptyKey, &key_id).await?;
let key = match existing_key {
Some(key) => {
match key.state.as_option() {
None => return Err(Error::Message(format!("Access key {} was deleted in the cluster, cannot add it back", key_id))),
Some(st) if st.secret_key != secret_key => return Err(Error::Message(format!("Access key {} is associated with a secret key different than the one given in GARAGE_DEFAULT_SECRET_KEY", key_id))),
_ => (),
}
key
}
None => {
info!("Creating default access key `{}`", key_id);
let mut key = Key::import(&key_id, &secret_key, "default access key")
.map_err(|e| Error::Message(format!("Invalid default access key: {}", e)))?;
key.state
.as_option_mut()
.unwrap()
.allow_create_bucket
.update(true);
garage.key_table.insert(&key).await?;
key
}
};
if opt.default_bucket {
// Create default bucket if it does not exist
let bucket_name = rdenv("GARAGE_DEFAULT_BUCKET")?;
if !is_valid_bucket_name(&bucket_name, garage.config.allow_punycode) {
return Err(Error::Message(
"Invalid default bucket name, see S3 specification for allowed bucket names."
.into(),
));
}
let helper = garage.locked_helper().await;
let bucket = match helper.bucket().resolve_global_bucket_fast(&bucket_name)? {
Some(bucket) => bucket,
None => {
info!("Creating default bucket `{}`", bucket_name);
let bucket = Bucket::new();
garage.bucket_table.insert(&bucket).await?;
helper
.set_global_bucket_alias(bucket.id, &bucket_name)
.await
.map_err(|e| {
Error::Message(format!("Cannot create default bucket: {}", e))
})?;
bucket
}
};
helper
.set_bucket_key_permissions(
bucket.id,
&key.key_id,
BucketKeyPerm {
timestamp: now_msec(),
allow_read: true,
allow_write: true,
allow_owner: true,
},
)
.await
.map_err(|e| {
Error::Message(format!(
"Cannot configure permissions on default bucket: {}",
e
))
})?;
}
}
Ok(())
}
#[cfg(unix)] #[cfg(unix)]
fn watch_shutdown_signal() -> watch::Receiver<bool> { fn watch_shutdown_signal() -> watch::Receiver<bool> {
use tokio::signal::unix::*; use tokio::signal::unix::*;
src/model/key_table.rs
+15 -4
View File
@@ -155,12 +155,23 @@ impl Key {
/// Import a key from it's parts /// Import a key from it's parts
pub fn import(key_id: &str, secret_key: &str, name: &str) -> Result<Self, &'static str> { pub fn import(key_id: &str, secret_key: &str, name: &str) -> Result<Self, &'static str> {
if key_id.len() != 26 || &key_id[..2] != "GK" || hex::decode(&key_id[2..]).is_err() { if key_id.len() < 8 {
return Err("The specified key ID is not a valid Garage key ID (starts with `GK`, followed by 12 hex-encoded bytes)"); return Err("Key identifiers should be at least 8 characters long");
} }
if secret_key.len() != 64 || hex::decode(secret_key).is_err() { if !key_id
return Err("The specified secret key is not a valid Garage secret key (composed of 32 hex-encoded bytes)"); .chars()
.all(|c| c.is_ascii_alphanumeric() || "-_.".contains(c))
{
return Err("Key identifiers should be composed only of ASCII alphanumeric characters and characters '-', '_' and '.'");
}
if secret_key.len() < 16 {
return Err("Secret keys should be at least 16 characters long");
}
if !secret_key.chars().all(|c| c.is_ascii_graphic()) {
return Err("Secret keys should be composed only of graphic ASCII characters (U+0021 to U+007E)");
} }
Ok(Self { Ok(Self {