gsadmin/garage
1
0
Fork 0
mirror of https://github.com/deuxfleurs-org/garage.git synced 2026-07-29 01:08:57 +00:00
Code Issues Packages Projects Releases Wiki Activity
736 Commits 31 Branches 89 Tags
2b93a01d2bead391e15f529dc5b4db80dcbeeba8
Commit Graph

736 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Alex Auvolat 2b93a01d2b ListBucket and GetBucketInfo 2022-05-12 10:20:34 +02:00
Alex Auvolat aeb978552a Short doc on UpdateKey 2022-05-11 11:51:11 +02:00
Alex Auvolat 393b76ecba Implement CreateKey, DeleteKey and rudimentary UpdateKey 2022-05-11 11:40:26 +02:00
Alex Auvolat 5c00c9fb46 First key endpoints: ListKeys and GetKeyInfo 2022-05-11 11:10:28 +02:00
Alex Auvolat f97a7845e9 Add API access key admin endpoints 2022-05-11 10:27:40 +02:00
Alex Auvolat bb6ec9ebd9 Update Cargo.nix and improve log message 2022-05-10 13:36:35 +02:00
Alex Auvolat dd54d0b2b1 Refactor code for apply/revert, implement Update/Apply/RevertLayout 2022-05-10 13:25:10 +02:00
Alex Auvolat 01c4876fb4 Specify remaining cluster-related endpoints 2022-05-10 13:25:10 +02:00
Alex Auvolat e4c61124d8 Add first draft of admin api 2022-05-10 13:25:10 +02:00
Alex Auvolat ec03e3d16c Fmt & cleanup 2022-05-10 13:25:10 +02:00
Alex Auvolat 7a19daafbd Implement /status Admin endpoint 2022-05-10 13:25:10 +02:00
Alex Auvolat 99fcfa3844 Make background runner terminate correctly 2022-05-10 13:25:10 +02:00
Alex Auvolat 633958c7b1 Refactor admin API to be in api/admin and use common code 2022-05-10 13:25:06 +02:00
Alex 5768bf3622 First implementation of K2V (#293) 
**Specification:**

View spec at [this URL](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/k2v/doc/drafts/k2v-spec.md)

- [x] Specify the structure of K2V triples
- [x] Specify the DVVS format used for causality detection
- [x] Specify the K2V index (just a counter of number of values per partition key)
- [x] Specify single-item endpoints: ReadItem, InsertItem, DeleteItem
- [x] Specify index endpoint: ReadIndex
- [x] Specify multi-item endpoints: InsertBatch, ReadBatch, DeleteBatch
- [x] Move to JSON objects instead of tuples
- [x] Specify endpoints for polling for updates on single values (PollItem)

**Implementation:**

- [x] Table for K2V items, causal contexts
- [x] Indexing mechanism and table for K2V index
- [x] Make API handlers a bit more generic
- [x] K2V API endpoint
- [x] K2V API router
- [x] ReadItem
- [x] InsertItem
- [x] DeleteItem
- [x] PollItem
- [x] ReadIndex
- [x] InsertBatch
- [x] ReadBatch
- [x] DeleteBatch

**Testing:**

- [x] Just a simple Python script that does some requests to check visually that things are going right (does not contain parsing of results or assertions on returned values)
- [x] Actual tests:
  - [x] Adapt testing framework
  - [x] Simple test with InsertItem + ReadItem
  - [x] Test with several Insert/Read/DeleteItem + ReadIndex
  - [x] Test all combinations of return formats for ReadItem
  - [x] Test with ReadBatch, InsertBatch, DeleteBatch
  - [x] Test with PollItem
  - [x] Test error codes
- [ ] Fix most broken stuff
  - [x] test PollItem broken randomly
  - [x] when invalid causality tokens are given, errors should be 4xx not 5xx

**Improvements:**

- [x] Descending range queries
  - [x] Specify
  - [x] Implement
  - [x] Add test
- [x] Batch updates to index counter
- [x] Put K2V behind `k2v` feature flag

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/293
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
 2022-05-10 13:16:57 +02:00
Alex Auvolat def78c5e6f Update netapp to 0.4.4, fix #300 v0.7.1 2022-05-09 12:08:47 +02:00
Alex 277a20ec44 Fix layout show to not show changes when there are no changes (#297) 
fixes #295, partially

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/297
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
 2022-05-09 11:14:55 +02:00
Alex Auvolat c9ef3e461b fix clippy 2022-04-19 12:50:40 +02:00
Alex Auvolat c93008d333 Prettier code for defragmentation 2022-04-19 12:50:40 +02:00
Alex Auvolat e5341ca47b Defragmentation in UploadPartCopy: first pass (not pretty but it compiles) 2022-04-19 12:50:36 +02:00
withinboredom a4f9f19ac3 remove size limitation in UploadPartCopy (#280) 
This removes the >1mb s3_copy restriction.

This restriction doesn't seem to be documented anywhere (I could be wrong). It also causes some software to fail (such as #248).

Co-authored-by: Rob Landers <landers.robert@gmail.com>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/280
Co-authored-by: withinboredom <landers.robert@gmail.com>
Co-committed-by: withinboredom <landers.robert@gmail.com>
 2022-04-19 12:49:43 +02:00
Baptiste Jonglez 47e57518ec Add documentation on running Kopia with Garage 2022-04-10 13:04:07 +02:00
Alex Auvolat dffcd9f4b1 update Cargo.nix v0.7.0 2022-04-08 14:35:09 +02:00
Alex Auvolat 5d404dcd54 Add missing opentelemetry features 2022-04-08 14:21:04 +02:00
Quentin Dufour 62f0715abe Add/Fix OpenTelemetry 2022-04-07 16:12:35 +02:00
Quentin Dufour 7e1ac51b58 Add files to quickly test k8s 2022-04-07 16:12:35 +02:00
Alex Auvolat 94f1e48fff Update to netapp 0.4.2 (a tiny fix) 2022-04-07 11:50:03 +02:00
Alex Auvolat cb5836d53c Bring maximum exponential backoff time down from 16h to 1h 2022-04-07 11:49:29 +02:00
Quentin Dufour 8e3ee82c3e Be clearer on what upgrades are (not) supported 2022-04-06 21:45:59 +02:00
Quentin Dufour a122a8cb46 Add an "upgrading" section, add a guide for 0.7 2022-04-05 10:08:31 +02:00
Quentin Dufour 9fd8ec1dee Add documentation for winscp+sftpgo 2022-03-31 10:25:56 +02:00
Alex Auvolat 0091002ef2 New replication modes and their documentation 2022-03-28 16:26:04 +02:00
Alex Auvolat 8f9cf3a5d1 fix a clippy lint 2022-03-28 15:48:55 +02:00
Alex Auvolat 913f7754bb Add blocks in errored state to garage stats 2022-03-28 15:47:23 +02:00
Alex Auvolat 42dde54126 Log admin GET requests at debug level instead of info 
to reduce noise in logs
 2022-03-28 15:46:52 +02:00
Alex Auvolat dca2ffdf91 document administrative options 2022-03-28 12:26:08 +02:00
Quentin Dufour 0cf4efac89 Compile kuberetes-discovery only when release=true v0.7.0-rc1 2022-03-24 16:57:43 +01:00
Alex Auvolat 9d0ed78887 Add feature flag for Kubernetes discovery 2022-03-24 16:57:43 +01:00
Alex Auvolat 509d256c58 Make layout optimization work in relative terms 2022-03-24 15:27:14 +01:00
Alex Auvolat 2814d41842 Allow garage layout assign to assign to several nodes at once 2022-03-24 15:27:13 +01:00
Alex Auvolat 7e0e2ffda2 Slight change and add comment to layout assignation algo 2022-03-24 15:27:13 +01:00
Alex Auvolat 413ab0eaed Small change to partition assignation algorithm 
This change helps ensure that nodes for each partition are spread
over all datacenters, a property that wasn't ensured previously
when going from a 2 DC deployment to a 3 DC deployment
 2022-03-24 15:27:10 +01:00
Alex Auvolat 43945234ae Add missing src/block to toplevel cargo.toml 2022-03-23 10:26:10 +01:00
Alex Auvolat 3dc9214172 Add lots of comments on how the resync queue works 
(I don't really want to change/refactor that code though)
 2022-03-23 10:25:39 +01:00
Alex Auvolat 077dd1cde9 Clippy 2022-03-23 10:25:39 +01:00
Alex Auvolat 2d13f0aa13 run cargo2nix 2022-03-23 10:25:37 +01:00
Alex Auvolat e480aaf338 Make background tranquility a configurable parameter 2022-03-23 10:25:19 +01:00
Alex Auvolat 8fd6745745 Move block RC code to separate rc.rs 2022-03-23 10:25:19 +01:00
Alex Auvolat c3982a90b6 Move DataBlock out of manager.rs 2022-03-23 10:25:19 +01:00
Alex Auvolat c1d9854d2c Move block manager to separate module 2022-03-23 10:25:15 +01:00
trinity-1686a 8565f7dc31 cleanup 2022-03-23 10:22:37 +01:00