c1d1646c4d
Change the way new layout assignations are computed.
...
The function now computes an optimal assignation (with respect to partition size) that minimizes the distance to the former assignation, using flow algorithms.
This commit was written by Mendes Oulamara <mendes.oulamara@pm.me >
2022-05-01 09:54:19 +02:00
c9ef3e461b
fix clippy
2022-04-19 12:50:40 +02:00
c93008d333
Prettier code for defragmentation
2022-04-19 12:50:40 +02:00
e5341ca47b
Defragmentation in UploadPartCopy: first pass (not pretty but it compiles)
2022-04-19 12:50:36 +02:00
a4f9f19ac3
remove size limitation in UploadPartCopy ( #280 )
...
This removes the >1mb s3_copy restriction.
This restriction doesn't seem to be documented anywhere (I could be wrong). It also causes some software to fail (such as #248 ).
Co-authored-by: Rob Landers <landers.robert@gmail.com >
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/280
Co-authored-by: withinboredom <landers.robert@gmail.com >
Co-committed-by: withinboredom <landers.robert@gmail.com >
2022-04-19 12:49:43 +02:00
5d404dcd54
Add missing opentelemetry features
2022-04-08 14:21:04 +02:00
94f1e48fff
Update to netapp 0.4.2 (a tiny fix)
2022-04-07 11:50:03 +02:00
cb5836d53c
Bring maximum exponential backoff time down from 16h to 1h
2022-04-07 11:49:29 +02:00
0091002ef2
New replication modes and their documentation
2022-03-28 16:26:04 +02:00
8f9cf3a5d1
fix a clippy lint
2022-03-28 15:48:55 +02:00
913f7754bb
Add blocks in errored state to garage stats
2022-03-28 15:47:23 +02:00
42dde54126
Log admin GET requests at debug level instead of info
...
to reduce noise in logs
2022-03-28 15:46:52 +02:00
9d0ed78887
Add feature flag for Kubernetes discovery
2022-03-24 16:57:43 +01:00
509d256c58
Make layout optimization work in relative terms
2022-03-24 15:27:14 +01:00
2814d41842
Allow garage layout assign to assign to several nodes at once
2022-03-24 15:27:13 +01:00
7e0e2ffda2
Slight change and add comment to layout assignation algo
2022-03-24 15:27:13 +01:00
413ab0eaed
Small change to partition assignation algorithm
...
This change helps ensure that nodes for each partition are spread
over all datacenters, a property that wasn't ensured previously
when going from a 2 DC deployment to a 3 DC deployment
2022-03-24 15:27:10 +01:00
3dc9214172
Add lots of comments on how the resync queue works
...
(I don't really want to change/refactor that code though)
2022-03-23 10:25:39 +01:00
077dd1cde9
Clippy
2022-03-23 10:25:39 +01:00
e480aaf338
Make background tranquility a configurable parameter
2022-03-23 10:25:19 +01:00
8fd6745745
Move block RC code to separate rc.rs
2022-03-23 10:25:19 +01:00
c3982a90b6
Move DataBlock out of manager.rs
2022-03-23 10:25:19 +01:00
c1d9854d2c
Move block manager to separate module
2022-03-23 10:25:15 +01:00
8565f7dc31
cleanup
2022-03-23 10:22:37 +01:00
8db6b84559
add test for create bucket and put website with streaming signature
2022-03-23 10:22:37 +01:00
1eb7fdb08f
add test framework for arbitraty S3 requests
...
and implement some basic test with it
2022-03-23 10:22:36 +01:00
e934934f14
garage_api: Update streaming payload stream unit tests
2022-03-23 10:22:36 +01:00
98545a16dd
garage_api: Handle streaming payload early in request handling
2022-03-23 10:22:36 +01:00
db46cdef79
Update netapp to v0.4.1
2022-03-15 17:09:57 +01:00
ba6b56ae68
Fix some new clippy lints
2022-03-14 12:27:49 +01:00
0af314b295
Add comment for fsync
2022-03-14 11:54:00 +01:00
d78bf379fb
Fix resync queue to not drop items
2022-03-14 11:51:37 +01:00
f7e6f4616f
Spawn a single resync worker
2022-03-14 11:51:37 +01:00
dc5ec4ecf9
Add appropriate fsync() calls in write_block
...
to ensure that data is persisted properly
2022-03-14 11:51:32 +01:00
fe62d01b7e
Implement exponential backoff for resync retries
2022-03-14 11:41:20 +01:00
9b2b531f4d
Make admin server optional
2022-03-14 10:54:25 +01:00
2377a92f6b
Add wrapper over sled tree to count items (used for big queues)
2022-03-14 10:54:25 +01:00
203e8d2c34
Bump version to 0.7 because of incompatible Netapp
2022-03-14 10:54:24 +01:00
f869ca625d
Add spans to table calls, change span names in RPC
2022-03-14 10:54:12 +01:00
0cc31ee169
add missing netapp telemetry feature
2022-03-14 10:54:11 +01:00
dc8d0496cc
Refactoring: rename config files, make modifications less invasive
2022-03-14 10:53:51 +01:00
d9a35359bf
Add metrics to web endpoint
2022-03-14 10:53:50 +01:00
2a5609b292
Add metrics to API endpoint
2022-03-14 10:53:36 +01:00
818daa5c78
Refactor how durations are measured
2022-03-14 10:53:35 +01:00
f0d0cd9a20
Remove strum crate dependency; add protobuf nix dependency
2022-03-14 10:53:00 +01:00
55d4471599
Remove ... at end of hex IDs
2022-03-14 10:52:31 +01:00
bb04d94fa9
Update to Netapp 0.4 which supports distributed tracing
2022-03-14 10:52:30 +01:00
8c2fb0c066
Add tracing integration with opentelemetry
2022-03-14 10:52:13 +01:00
2cab84b1fe
Add many metrics in table/ and rpc/
2022-03-14 10:51:50 +01:00
1e2cf26373
Implement basic metrics in table
2022-03-14 10:51:17 +01:00
Alex Auvolat