Commit Graph

670 Commits

Author SHA1 Message Date
Alex Auvolat bfb4353df5 Update Grafana dashboard 2022-03-14 10:55:30 +01:00
Alex Auvolat 9b2b531f4d Make admin server optional 2022-03-14 10:54:25 +01:00
Alex Auvolat a19341b188 Add Grafana dashboard for Garage 2022-03-14 10:54:25 +01:00
Alex Auvolat 2377a92f6b Add wrapper over sled tree to count items (used for big queues) 2022-03-14 10:54:25 +01:00
Alex Auvolat 203e8d2c34 Bump version to 0.7 because of incompatible Netapp 2022-03-14 10:54:24 +01:00
Alex Auvolat f869ca625d Add spans to table calls, change span names in RPC 2022-03-14 10:54:12 +01:00
Alex Auvolat 0cc31ee169 add missing netapp telemetry feature 2022-03-14 10:54:11 +01:00
Alex Auvolat dc8d0496cc Refactoring: rename config files, make modifications less invasive 2022-03-14 10:53:51 +01:00
Alex Auvolat d9a35359bf Add metrics to web endpoint 2022-03-14 10:53:50 +01:00
Alex Auvolat 2a5609b292 Add metrics to API endpoint 2022-03-14 10:53:36 +01:00
Alex Auvolat 818daa5c78 Refactor how durations are measured 2022-03-14 10:53:35 +01:00
Alex Auvolat f0d0cd9a20 Remove strum crate dependency; add protobuf nix dependency 2022-03-14 10:53:00 +01:00
Alex Auvolat 55d4471599 Remove ... at end of hex IDs 2022-03-14 10:52:31 +01:00
Alex Auvolat bb04d94fa9 Update to Netapp 0.4 which supports distributed tracing 2022-03-14 10:52:30 +01:00
Alex Auvolat 8c2fb0c066 Add tracing integration with opentelemetry 2022-03-14 10:52:13 +01:00
Maximilien b6561f6e1b Add docker-compose for traces & metrics 2022-03-14 10:51:52 +01:00
Alex Auvolat 2cab84b1fe Add many metrics in table/ and rpc/ 2022-03-14 10:51:50 +01:00
Maximilien R 1e2cf26373 Implement basic metrics in table 2022-03-14 10:51:17 +01:00
mricher e349af13a7 Update dependencies and add admin module with metrics
- Global dependencies updated in Cargo.lock
- New module created in src/admin to host:
  - the (future) admin REST API
  - the metric collection
- add configuration block

No metrics implemented yet
2022-03-14 10:51:12 +01:00
Max Audron 9d44127245 add support for kubernetes service discovery
This commit adds support to discover garage instances running in
kubernetes.

Once enabled by setting `kubernetes_namespace` and
`kubernetes_service_name` garage will create a Custom Resources
`garagenodes.deuxfleurs.fr` with nodes public key as the resource name.
and IP and Port information as spec in the namespace configured by
`kubernetes_namespace`.

For discovering nodes the resources are filtered with the optionally set
`kubernetes_service_name` which sets a label
`garage.deuxfleurs.fr/service` on the resources.

This allows to separate multiple garage deployments in a single
namespace.

the `kubernetes_skip_crd` variable allows to disable the creation of the
CRD by garage itself. The user must deploy this manually.
2022-03-12 13:05:52 +01:00
Quentin Dufour c00b2c9948 Functional tests for admin commands 2022-03-07 17:32:07 +01:00
Quentin Dufour 8df1e186de Functional tests for website endpoints 2022-03-07 17:32:07 +01:00
Quentin Dufour 2ef60b8417 Functional test for multipart endpoints 2022-03-07 17:32:07 +01:00
Quentin Dufour 1e639ec67c Functional test for ListMultipartUploads 2022-03-07 17:32:07 +01:00
Quentin Dufour cfea1e0315 Functional tests for bucket endpoints 2022-03-07 17:32:02 +01:00
Quentin Dufour 05eb79929e Functional tests for object operations 2022-03-07 17:05:10 +01:00
Quentin Dufour 0f4e0e8bb9 Move ListObjects tests to Rust 2022-03-07 17:05:10 +01:00
Quentin Dufour 2a3afcaf65 Test WinSCP 2022-03-03 14:29:10 +01:00
Alex Auvolat 8a5bbc3b0b More permissive OPTIONS on S3 API v0.6.1 2022-03-01 11:15:16 +01:00
Alex Auvolat 97f245f218 Add tracing output to signature calculation 2022-02-28 12:22:39 +01:00
Alex Auvolat 8129a98291 Process CORS earlier in pipeline 2022-02-28 12:22:39 +01:00
Quentin Dufour 54e02b4c3b Force static builds for all platforms 2022-02-24 16:12:37 +01:00
trinity-1686a f6f8b7f1ad Support for PostObject (#222)
Add support for [PostObject](https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectPOST.html)

- [x] routing PostObject properly
- [x] parsing multipart body
- [x] validating signature
- [x] validating policy
- [x] validating content length
- [x] actually saving data

Co-authored-by: trinity-1686a <trinity@deuxfleurs.fr>
Co-authored-by: Trinity Pointard <trinity.pointard@gmail.com>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/222
Reviewed-by: Alex <alex@adnab.me>
Co-authored-by: trinity-1686a <trinity.pointard@gmail.com>
Co-committed-by: trinity-1686a <trinity.pointard@gmail.com>
2022-02-21 23:02:30 +01:00
Alex Auvolat e312ba977e Add FOSDEM talk and move all .pdf files to Git LFS 2022-02-16 20:01:36 +01:00
Alex Auvolat 2465163e39 documentation: add mention to install build-essential 2022-02-16 12:18:24 +01:00
Jill 84613e66a2 garage(tests): Remove RNG stuff 2022-02-11 10:50:55 +01:00
Jill c8b30ebc79 garage(tests): Remove superfluous test 2022-02-10 17:55:51 +01:00
Jill d7decda3f4 garage(tests): Add random suffix to created buckets. 2022-02-10 17:55:50 +01:00
Jill cd13ea461b garage(tests): Add some unsafe-usage doc and tweaks 2022-02-10 17:55:50 +01:00
Quentin Dufour 5d19f3d2d7 Add integration tests to Drone 2022-02-10 17:55:50 +01:00
Quentin Dufour 084dcdbd3a Upgrade cargo2nix 2022-02-10 17:55:50 +01:00
Jill 3baa841d6f tests: Fix garage integration test 2022-02-10 17:55:49 +01:00
Jill dd407e7041 tests: Add garage integration tests (base) 2022-02-10 17:55:49 +01:00
Alex Auvolat af261e1789 Fix a bug when a migration is followed by a rebalance
Nodes would stabilize on different encoding formats for the values,
some having the pre-migration format and some having the post-migration
format. This would be reflected in the Merkle trees never converging
and thus having an infinite resync loop.
2022-02-10 17:38:27 +01:00
Alex 4ae03aa774 Small documentation updates (#237)
Fixes #234, among other things

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/237
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
2022-02-10 15:58:09 +01:00
Quentin 3e1373fafc Add a new S3 comparison Matrix to documentation (#220)
Co-authored-by: Quentin Dufour <quentin@deuxfleurs.fr>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/220
Co-authored-by: Quentin <quentin@dufour.io>
Co-committed-by: Quentin <quentin@dufour.io>
2022-02-07 16:04:52 +01:00
Alex Auvolat 7d68b7060e Fix anchors in links 2022-02-07 16:01:48 +01:00
Quentin Dufour 99ed67503c Update quickstart to set endpoints ending in localhost 2022-02-07 16:01:48 +01:00
Quentin Dufour 5a1fb7cce7 Improve integration part of the doc 2022-02-07 16:01:45 +01:00
Alex 1c0ba930b8 Reorganize documentation for new website (#213)
This PR should be merged after the new website is deployed.

- [x] Rename files
- [x] Add front matter section to all `.md` files in the book (necessary for Zola)
- [x] Change all internal links to use Zola's linking system that checks broken links
- [x] Some updates to documentation contents and organization

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/213
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
2022-02-07 11:51:12 +01:00