Commit Graph

2770 Commits

Author SHA1 Message Date
Alex Auvolat fd8074ad9b Update .drone.yml signature 2022-09-05 16:09:01 +02:00
Alex Auvolat 07e6bcde85 Merge branch 'main' into lx-perf-improvements 2022-09-05 12:40:17 +02:00
Alex 0009fd136c Merge pull request 'Make block resync speed dynamically configurable' (#369) from resync-ajustable-speed into main
Included in this PR:

- [x] Small refactor, resync code is moved to a separate `block/resync.rs` file
- [x] Block resync tranquility is no longer in config file, it is set dynamically using `garage worker set resync-tranquility` (this parameter is persisted over Garage restarts)
- [x] Up to 4 block resync workers can be activated to run simultaneously to speed up big resyncs, this parameter is set dynamically using `garage worker set resync-n-workers`

Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/369
2022-09-05 12:35:08 +02:00
Jakub Jirutka 7511ba5530 Allow linking against system-provided libsqlite
Unfortunately, rusqlite uses the opposite logic for enabling/disabling
bundled libraries to others (libsodium-sys, zstd-sys). Cargo features
are very limited and doesn't allow to enable feature A in a dependency
iff feature B is disabled.

Note, lmdb-rkv-sys doesn't need any special treatment because it
automatically links against system liblmdb if found via pkgconf.

Linux distros should build garage with
`--no-default-features --features system-libs` to disable bundled-libs
and enable system-libs.
2022-09-03 19:15:57 +02:00
Jakub Jirutka a6e40b75ea Add feature "system-libs" to enable linking against system libraries
If this feature is enabled, libsodium-sys and zstd-sys will link
dynamically against system-provided libraries instead of building
and linking statically the bundled (possibly outdated and vulnerable)
copies of them. This feature is intended mainly for linux package
maintainers.
2022-09-03 18:44:34 +02:00
Alex Auvolat e1751c8a9c fix clippy 2022-09-02 17:24:26 +02:00
Alex Auvolat 5d4b937a00 Ability to have up to 4 concurrently working resync workers 2022-09-02 17:18:13 +02:00
Alex Auvolat 5e8baa433d Make BlockManagerLocked fully private again 2022-09-02 16:52:22 +02:00
Alex Auvolat 47be652a1f block manager: refactor: split resync into separate file 2022-09-02 16:47:15 +02:00
Alex Auvolat 943d76c583 Ability to dynamically set resync tranquility 2022-09-02 15:34:21 +02:00
Alex Auvolat 6226f5ceca Update to netapp 0.4.5 - fixed ping 2022-09-02 14:33:12 +02:00
Alex Auvolat 13b5f28c7e Make use of BytesBuf from new Netapp 2022-09-02 13:46:42 +02:00
Alex Auvolat 1ef87ac4cb cargo fmt 2022-09-02 13:38:29 +02:00
Alex Auvolat 99b532b85b Apply PRIO_SECONDARY to block data transfers 2022-09-01 16:35:43 +02:00
Alex Auvolat e648bf7b69 update cargo.nix 2022-09-01 16:31:04 +02:00
Alex Auvolat df094bd807 Less strict timeouts 2022-09-01 16:30:44 +02:00
Alex Auvolat f3bf34b6a1 update netapp: straming + fix-ping 2022-09-01 14:23:54 +02:00
Alex Auvolat bc977f9a7a Update to Netapp with OrderTag support and exploit OrderTags 2022-09-01 12:58:20 +02:00
Alex Auvolat 4b726b0941 netapp recv with unbounded channel removes deadlock 2022-09-01 09:47:28 +02:00
Alex Auvolat 70231d68b2 Fix bytes_read counter 2022-08-31 19:44:27 +02:00
Alex Auvolat e598231ca4 update netapp git commit 2022-08-31 19:27:25 +02:00
Alex Auvolat c9bc9d89de Merge branch 'lx-perf-improvements' into netapp-stream-body 2022-08-31 17:42:31 +02:00
Alex Auvolat eb97e13a6a update cargo.nix 2022-08-31 17:42:00 +02:00
Alex Auvolat efbca67ce4 Add env filter to tracing subscriber 2022-08-31 14:39:12 +02:00
Alex Auvolat 44cd98d2e4 Tracing-subscriber: write to stderr 2022-08-31 14:28:17 +02:00
Alex Auvolat dd5304f6fc Replace logging crate pretty_env_logger by tracing_subscriber::fmt 2022-08-31 14:24:41 +02:00
Alex Auvolat 322dafc761 Try to fix clippy 2022-08-29 17:32:45 +02:00
Alex Auvolat 5d065b8a0f cargo2nix fix to fetchCrateGit 2022-08-29 17:24:53 +02:00
Alex Auvolat 52749e28f7 Merge branch 'lx-perf-improvements' into netapp-stream-body 2022-08-29 16:48:43 +02:00
Alex Auvolat 4da67b0035 Update drone signature 2022-08-29 16:48:31 +02:00
Alex Auvolat 1921f4f7e6 Merge branch 'lx-perf-improvements' into netapp-stream-body 2022-08-29 16:45:05 +02:00
Alex Auvolat ebc20a8798 Merge branch 'main' into lx-perf-improvements 2022-08-29 16:44:13 +02:00
Mendes 81083dd415 Added a first draft version of the algorithm and analysis for the non-strict mode. 2022-08-19 21:21:41 +02:00
Quentin Dufour 532eca7ff9 Add some documentation for Caddy 2022-08-12 10:33:41 +02:00
Quentin Dufour 2c7bae935a Configure structopt to report the right version
By default, structopt reports the value provided by
the env var CARGO_PKG_VERSION, feeded by Cargo when reading
Cargo.toml. However for Garage we use a versioning based on git,
so we often report a version that is behind the real version.
In this commit, we create garage_util::version::garage() that
reports the right version and configure all structopt subcommands
to call this function instead of using the env var.
v0.7.2_ci-test-2
2022-08-11 10:21:45 +02:00
Quentin Dufour 8cd02639dc drone: set TARGET env as needed by "to_s3" func 2022-08-03 11:19:26 +02:00
Alex Auvolat e935861854 Factor out node request order selection logic & use in manager 2022-07-29 12:25:03 +02:00
Alex Auvolat f0ee3056d3 Update cargo.nix 2022-07-29 12:25:03 +02:00
Alex Auvolat 126b037307 update netapp 2022-07-29 12:25:03 +02:00
Alex Auvolat 33750c04ed Update cargo.nix 2022-07-29 12:25:03 +02:00
Alex Auvolat 68087ee13d Fix clippy 2022-07-29 12:25:03 +02:00
Alex Auvolat 605a630333 Use streaming in block manager 2022-07-29 12:25:02 +02:00
Alex Auvolat a35d4da721 update netapp to 0.5 2022-07-29 12:25:02 +02:00
Alex Auvolat 8e7e680afe First adaptation to WIP netapp with streaming body 2022-07-29 12:25:02 +02:00
Alex Auvolat 16f6a1a65d fix clippy 2022-07-29 12:24:49 +02:00
Alex Auvolat ad35b18bb1 Faster chunker 2022-07-29 12:24:49 +02:00
Alex Auvolat 49154a78d8 Update cargo.nix 2022-07-29 12:24:48 +02:00
Alex Auvolat ff4771c36a cargo fmt 2022-07-29 12:24:48 +02:00
Alex Auvolat 381eb9a5a1 Fix tests 2022-07-29 12:24:48 +02:00
Alex Auvolat 2cad656a03 More make clippy happy 2022-07-29 12:24:48 +02:00