Commit Graph

68 Commits

Author SHA1 Message Date
Alex Auvolat f914db057a cli_v2: implement LaunchRepairOperation and remove old stuff 2025-02-05 15:36:47 +01:00
Alex Auvolat 89ff9f5576 admin api: base infrastructure for local endpoints
admin api: rename EndpointHandler into RequestHandler to avoid confusion with RPC

wip: infrastructure for local api calls

admin api: fix things

admin api: first local endpoint to work with new scheme

admin api: implement SetWorkerVariable
2025-02-03 18:54:51 +01:00
Alex Auvolat 69ddaafc60 wip: migrate garage status and garage layout assign 2025-01-30 12:07:12 +01:00
Alex Auvolat 145130481e wip: proxy admin api requests through admin rpc, prepare new cli 2025-01-30 10:44:08 +01:00
Stefan Majer 2eb9fcae20 Fix all typos 2025-01-16 13:22:00 +01:00
Alex Auvolat f7cd4eb600 Merge branch 'main' into next-0.10 2024-03-26 16:34:40 +01:00
Alex Auvolat 84018be862 [syslog] warning when syslog support is not enabled 2024-03-20 14:39:04 +01:00
Alex Auvolat fe8a7819fa [syslog] Add support to logging to syslog
Original patch by Jakub Jirutka for Alpine Linux port.
2024-03-20 14:22:18 +01:00
Alex Auvolat 44454aac01 [rm-sled] Remove the Sled database engine 2024-03-08 14:11:02 +01:00
Alex Auvolat b96f84b894 [networking-fixes] add option to bind outgoing RPC sockets (fix #638)
Thanks to yuka for the original patch.
2024-02-19 11:45:44 +01:00
Alex Auvolat 5ea24254a9 [import-netapp] import Netapp code into Garage codebase 2024-02-15 12:15:07 +01:00
Alex Auvolat 25e5738568 [fix-secrets-695] take into account rpc secret from file for cli commands (fix #695) 2024-02-12 10:42:17 +01:00
Alex Auvolat 82a29bf6e5 help, comments: make clear that full-length node ID = public key
Generally, avoid using the "public key" terminology
2024-01-16 14:04:11 +01:00
Alex Auvolat 4c5be79b80 Merge tag 'v0.8.5' into sync-08-09
Garage v0.8.5

This minor release includes the following improvements and fixes:

New features:

- Configuration: make LMDB's `map_size` configurable and make `block_size` and `sled_cache_capacity` expressable as strings (such as `10M`) (#628, #630)

- Add support for binding to Unix sockets for the S3, K2V, Admin and Web API servers (#640)

- Move the `convert_db` command into the main Garage binary (#645)

- Add support for specifying RPC secret and admin tokens as environment variables (#643)

- Add `allow_world_readable_secrets` option to config file (#663, #685)

Bug fixes:

- Use `statvfs` instead of mount list to determine free space in metadata/data directories (#611, #631)

- Add missing casts to fix 32-bit build (#632)

- Fix error when none of the HTTP servers (S3/K2V/Admin/Web) is started and fix shutdown hang (#613, #633)

- Add missing CORS headers to PostObject response (#609, #656)

- Monitoring: finer histogram boundaries in Prometheus exported metrics (#531, #686)

Other:

- Documentation improvements (#641)
2024-01-16 12:12:27 +01:00
Alex Auvolat 7228695ee2 config: refactor secret sourcing 2024-01-15 17:18:46 +01:00
Alex 0215b11402 Merge pull request 'Add support for specifying rpc_secret_file, metrics_token_file and admin_token_file using environment variables' (#643) from networkException/garage:token-file-env into main-0.8.x
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/643
2023-10-19 09:33:12 +00:00
networkException 8599051c49 garage: support specifying token / secret as environment variables
this patch adds support for specifying the `rpc_secret_file`,
`metrics_token_file` and `admin_token_file` as environment variables.
2023-10-19 03:39:02 +02:00
networkException 4a19ee94bb garage: fix admin-token description 2023-10-19 03:31:50 +02:00
Alex Auvolat d4932c31ea Merge branch 'main' into next 2023-10-10 13:57:21 +02:00
Alex Auvolat 2d5d7a7031 Move convert_db command into main garage binary 2023-10-10 12:13:15 +02:00
Alex Auvolat 351d734e6c Merge branch 'main' into next 2023-05-09 12:40:08 +02:00
Alex Auvolat 9e0a9c1c15 move git-version dependency to main crate to reduce rebuilds 2023-05-09 11:35:32 +02:00
Alex Auvolat fa78d806e3 Merge branch 'main' into next 2023-04-25 12:34:26 +02:00
Alex Auvolat 80e2326998 fixes for pr 499 2023-02-06 12:23:55 +01:00
Alex Auvolat 656b8d42de secrets can be passed directly in config, as file, or as env 2023-02-03 15:27:39 +01:00
Felix Scheinost f2106c2733 Implement rpc_secret_file 2023-01-04 18:35:10 +01:00
Alex Auvolat 570e5e5bbb Merge branch 'main' into next 2023-01-04 11:34:43 +01:00
Alex Auvolat de9d6cddf7 Prettier worker list table; remove useless CLI log messages 2022-12-12 17:17:05 +01:00
Alex Auvolat d75b37b018 Return more info when layout's .check() fails, fix compilation, fix test 2022-11-08 14:58:39 +01:00
Alex Auvolat e03d9062f7 Show a nice message and a backtrace when Garage panics 2022-11-04 16:39:02 +01:00
Alex Auvolat 002b9fc50c Add TLS support for Consul discovery + refactoring 2022-10-18 18:38:20 +02:00
Alex Auvolat e46dc2a8ef Allow for hostnames in bootstrap_peers and rpc_public_addr (fix #353) 2022-09-14 16:09:38 +02:00
Alex Auvolat d9d199a6c9 Merge branch 'main' into lx-perf-improvements 2022-09-08 15:49:17 +02:00
Alex Auvolat ceb1f0229a Move version back into util 2022-09-07 18:36:46 +02:00
Alex Auvolat f310fce34b Inject GIT_VERSION even later 2022-09-07 18:30:15 +02:00
Alex Auvolat 2e00809af5 Error messages when system-libs XOR bundled-libs != 1 2022-09-07 17:57:12 +02:00
Alex Auvolat 28d86e7602 Report build features in garage --help 2022-09-07 17:05:21 +02:00
Alex Auvolat db61f41030 Move GIT_VERSION injection later in build chain to reduce build times 2022-09-07 11:59:56 +02:00
Jakub Jirutka e7af006c1c Make OTLP exporter optional via feature "telemetry-otlp"
opentelemetry-otlp add 48 (!) extra dependencies and increases the
size of the garage binary by ~11 % (with fat LTO).
2022-09-06 01:14:47 +02:00
Alex Auvolat efbca67ce4 Add env filter to tracing subscriber 2022-08-31 14:39:12 +02:00
Alex Auvolat 44cd98d2e4 Tracing-subscriber: write to stderr 2022-08-31 14:28:17 +02:00
Alex Auvolat dd5304f6fc Replace logging crate pretty_env_logger by tracing_subscriber::fmt 2022-08-31 14:24:41 +02:00
Quentin Dufour 2c7bae935a Configure structopt to report the right version
By default, structopt reports the value provided by
the env var CARGO_PKG_VERSION, feeded by Cargo when reading
Cargo.toml. However for Garage we use a versioning based on git,
so we often report a version that is behind the real version.
In this commit, we create garage_util::version::garage() that
reports the right version and configure all structopt subcommands
to call this function instead of using the env var.
2022-08-11 10:21:45 +02:00
Alex 77e3fd6db2 improve internal item counter mechanisms and implement bucket quotas (#326)
- [x] Refactoring of internal counting API
- [x] Repair procedure for counters (it's an offline procedure!!!)
- [x] New counter for objects in buckets
- [x] Add quotas to buckets struct
- [x] Add CLI to manage bucket quotas
- [x] Add admin API to manage bucket quotas
- [x] Apply quotas by adding checks on put operations
- [x] Proof-read

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/326
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
2022-06-15 20:20:28 +02:00
Alex 382e74c798 First version of admin API (#298)
**Spec:**

- [x] Start writing
- [x] Specify all layout endpoints
- [x] Specify all endpoints for operations on keys
- [x] Specify all endpoints for operations on key/bucket permissions
- [x] Specify all endpoints for operations on buckets
- [x] Specify all endpoints for operations on bucket aliases

View rendered spec at <https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/admin-api/doc/drafts/admin-api.md>

**Code:**

- [x] Refactor code for admin api to use common api code that was created for K2V

**General endpoints:**

- [x] Metrics
- [x] GetClusterStatus
- [x] ConnectClusterNodes
- [x] GetClusterLayout
- [x] UpdateClusterLayout
- [x] ApplyClusterLayout
- [x] RevertClusterLayout

**Key-related endpoints:**

- [x] ListKeys
- [x] CreateKey
- [x] ImportKey
- [x] GetKeyInfo
- [x] UpdateKey
- [x] DeleteKey

**Bucket-related endpoints:**

- [x] ListBuckets
- [x] CreateBucket
- [x] GetBucketInfo
- [x] DeleteBucket
- [x] PutBucketWebsite
- [x] DeleteBucketWebsite

**Operations on key/bucket permissions:**

- [x] BucketAllowKey
- [x] BucketDenyKey

**Operations on bucket aliases:**

- [x] GlobalAliasBucket
- [x] GlobalUnaliasBucket
- [x] LocalAliasBucket
- [x] LocalUnaliasBucket

**And also:**

- [x] Separate error type for the admin API (this PR includes a quite big refactoring of error handling)
- [x] Add management of website access
- [ ] Check that nothing is missing wrt what can be done using the CLI
- [ ] Improve formatting of the spec
- [x] Make sure everyone is cool with the API design

Fix #231
Fix #295

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/298
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
2022-05-24 12:16:39 +02:00
Alex Auvolat ba6b56ae68 Fix some new clippy lints 2022-03-14 12:27:49 +01:00
Alex Auvolat bb04d94fa9 Update to Netapp 0.4 which supports distributed tracing 2022-03-14 10:52:30 +01:00
Alex Auvolat 8c2fb0c066 Add tracing integration with opentelemetry 2022-03-14 10:52:13 +01:00
Alex Auvolat df35feba18 New buckets for 0.6.0: make bucket id a SK and not a HK, CLI updates 2022-01-04 12:53:14 +01:00
Alex Auvolat beeef4758e Some movement of helper code and refactoring of error handling 2022-01-04 12:52:46 +01:00