5dd200c015
layout: move block_read_nodes_of to rpc_helper to avoid double-locking
...
(in theory, this could have caused a deadlock)
2023-12-08 12:02:24 +01:00
d6d239fc79
block manager: read_block using old layout versions if necessary
2023-11-27 11:52:57 +01:00
33c8a489b0
layou: implement ack locking
2023-11-15 15:40:44 +01:00
90e1619b1e
table: take into account multiple write sets in inserts
2023-11-14 15:40:46 +01:00
3b361d2959
layout: prepare for write sets
2023-11-14 14:28:16 +01:00
8a2b1dd422
wip: split out layout management from System into separate LayoutManager
2023-11-09 12:55:36 +01:00
7f9ba49c71
block manager: remove data_dir field
2023-09-11 11:57:36 +02:00
be91ef6294
block manager: fix bug where rebalance didn't delete old copies
2023-09-07 16:04:03 +02:00
6b008b5bd3
block manager: add rebalance operation to rebalance multi-hdd setups
2023-09-07 13:44:11 +02:00
99ed18350f
block manager: refactor and fix monitoring/statistics
2023-09-07 12:41:36 +02:00
f38a31b330
block manager: avoid incorrect data_dir configs and avoid losing files
2023-09-06 17:49:30 +02:00
a44f486931
block manager: refactoring & increase max worker count to 8
2023-09-06 16:35:28 +02:00
93114a9747
block manager: refactoring
2023-09-06 16:35:28 +02:00
1b8c265c14
block manager: get rid of check_block_status
2023-09-06 16:35:28 +02:00
a09f86729c
block manager: move blocks in write_block if necessary
2023-09-06 16:35:28 +02:00
887b3233f4
block manager: use data paths from layout
2023-09-06 16:35:28 +02:00
6c420c0880
block manager: multi-directory layout computation
2023-09-06 16:35:28 +02:00
71c0188055
block manager: skeleton for multi-hdd support
2023-09-06 16:35:28 +02:00
e7e164a280
Make fsync an option for meta and data
2023-06-09 16:23:21 +02:00
9c788059e2
block/manager.rs: In is_block_compressed - check which compression_level
...
is configured on a node and check for raw block first if compression is
disabled (to help reduce syscalls during a scrub).
2023-05-09 10:28:19 +01:00
d218f475cb
block/manager.rs: Set defaults for scrub_persister.
2023-03-09 17:08:47 +00:00
148b66b843
block/manager.rs: Display scrub-next-run.
2023-03-06 13:43:09 +00:00
4cfb469d2b
block/metrics.rs: Added compression_level metric.
2023-01-10 10:40:03 +00:00
02e8eb167e
Merge pull request 'PutObject: better cleanup when request is interrupted in the middle' ( #462 ) from interrupted-cleanup into main
...
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/462
2023-01-04 14:43:45 +00:00
f3f27293df
Uniform framework for bg variable management
2023-01-04 13:07:13 +01:00
936b6cb563
When saving block, delete .tmp file if we could not complete
2023-01-03 17:34:26 +01:00
d56c472712
Refactor background runner and get rid of job worker
2022-12-14 12:51:42 +01:00
2183518edc
Spawn all background workers in a separate step
2022-12-14 12:28:07 +01:00
041b60ed1d
Add block.rc_size, table.size and table.merkle_tree_size metrics
2022-12-13 15:54:03 +01:00
d6040e32a6
cli: prettier table in garage stats
2022-12-13 15:43:22 +01:00
687660b27f
Implement block list-errors and block info
2022-12-13 14:23:45 +01:00
56592e1853
RPC performance changes
...
- configurable ping timeout
- single, much higher, configurable RPC timeout
- no more concurrency semaphore
2022-09-19 20:31:00 +02:00
b823151a0b
improvements in block manager
2022-09-12 16:57:38 +02:00
6b958979bd
Merge branch 'lx-perf-improvements' into netapp-stream-body
2022-09-06 22:13:01 +02:00
c2cc08852b
Reenable node ordering
2022-09-06 19:31:42 +02:00
07e6bcde85
Merge branch 'main' into lx-perf-improvements
2022-09-05 12:40:17 +02:00
5d4b937a00
Ability to have up to 4 concurrently working resync workers
2022-09-02 17:18:13 +02:00
5e8baa433d
Make BlockManagerLocked fully private again
2022-09-02 16:52:22 +02:00
47be652a1f
block manager: refactor: split resync into separate file
2022-09-02 16:47:15 +02:00
943d76c583
Ability to dynamically set resync tranquility
2022-09-02 15:34:21 +02:00
99b532b85b
Apply PRIO_SECONDARY to block data transfers
2022-09-01 16:35:43 +02:00
df094bd807
Less strict timeouts
2022-09-01 16:30:44 +02:00
bc977f9a7a
Update to Netapp with OrderTag support and exploit OrderTags
2022-09-01 12:58:20 +02:00
70231d68b2
Fix bytes_read counter
2022-08-31 19:44:27 +02:00
e935861854
Factor out node request order selection logic & use in manager
2022-07-29 12:25:03 +02:00
605a630333
Use streaming in block manager
2022-07-29 12:25:02 +02:00
8e7e680afe
First adaptation to WIP netapp with streaming body
2022-07-29 12:25:02 +02:00
2f111e6b3d
Performance improvements:
...
- reduce contention on mutation_lock by having 256 of them
- better lmdb defaults
2022-07-29 12:24:48 +02:00
1b2e1296eb
Compute hashes on dedicated threads
2022-07-29 12:24:44 +02:00
4f38cadf6e
Background task manager ( #332 )
...
- [x] New background worker trait
- [x] Adapt all current workers to use new API
- [x] Command to list currently running workers, and whether they are active, idle, or dead
- [x] Error reporting
- Optimizations
- [x] Merkle updater: several items per iteration
- [ ] Use `tokio::task::spawn_blocking` where appropriate so that CPU-intensive tasks don't block other things going on
- scrub:
- [x] have only one worker with a channel to start/pause/cancel
- [x] automatic scrub
- [x] ability to view and change tranquility from CLI
- [x] persistence of a few info
- [ ] Testing
Co-authored-by: Alex Auvolat <alex@adnab.me >
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/332
Co-authored-by: Alex <alex@adnab.me >
Co-committed-by: Alex <alex@adnab.me >
2022-07-08 13:30:26 +02:00
Alex Auvolat