Commit Graph

25 Commits

Author SHA1 Message Date
Alex Auvolat 4c139bcbca add error case for layout not ready, and fail earlier in many places 2025-09-13 20:52:51 +02:00
Alex Auvolat 1e13a66b42 rework bucket helper functions to use local access where relevant 2025-03-25 13:00:48 +01:00
Alex Auvolat 620dc58560 remove async_trait for traits declared in garage_net 2025-02-05 20:22:16 +01:00
Stefan Majer 2eb9fcae20 Fix all typos 2025-01-16 13:22:00 +01:00
Alex Auvolat 8bfc16ba7d Merge branch 'main' into next-0.10 2024-03-28 15:01:05 +01:00
Alex Auvolat 0d3e285d13 [fix-buffering] implement block_ram_buffer_max to avoid excessive RAM usage 2024-03-27 16:22:40 +01:00
Alex Auvolat c0eeb0b0f3 [next-0.10] fixes to k2v rpc + comment fixes 2024-03-27 10:44:03 +01:00
Alex Auvolat eb4a6ce106 Merge branch 'main' into next-0.10 2024-02-15 14:06:34 +01:00
Alex Auvolat 5ea24254a9 [import-netapp] import Netapp code into Garage codebase 2024-02-15 12:15:07 +01:00
Alex Auvolat acd49de9f9 rpc: fix write set quorums 2023-11-15 13:07:42 +01:00
Alex Auvolat 90e1619b1e table: take into account multiple write sets in inserts 2023-11-14 15:40:46 +01:00
Alex Auvolat 3b361d2959 layout: prepare for write sets 2023-11-14 14:28:16 +01:00
Alex Auvolat 8a2b1dd422 wip: split out layout management from System into separate LayoutManager 2023-11-09 12:55:36 +01:00
Alex Auvolat 8e93d69974 More clippy fixes 2023-01-26 17:26:32 +01:00
Alex Auvolat 638c5a3ce0 PollRange: add extra RPC delay after quorum is achieved,
to give a chance to the 3rd node to respond
2023-01-11 16:12:07 +01:00
Alex Auvolat 5b5ca63cf6 Poll cleanup 2023-01-11 15:17:27 +01:00
Alex Auvolat bba13f40fc Correctly return bad requests when seeh marker is invalid 2023-01-11 12:27:19 +01:00
Alex Auvolat ba384e61c0 PollRange: return immediately if no seen marker is provided 2023-01-11 12:03:17 +01:00
Alex Auvolat 09a3dad0f2 Lock once for insert_many 2023-01-11 11:35:36 +01:00
Alex Auvolat 43fd6c1526 PollRange RPC 2023-01-10 12:54:24 +01:00
Alex Auvolat 9f5419f465 Make K2V item timestamps globally increasing on each node 2023-01-10 11:03:52 +01:00
Alex Auvolat a48e2e0cb2 K2V: Subscription to ranges of items 2023-01-10 10:30:59 +01:00
Alex Auvolat f8e528c15d Small refactor of tables internals 2022-12-14 10:48:49 +01:00
Alex Auvolat 56592e1853 RPC performance changes
- configurable ping timeout
- single, much higher, configurable RPC timeout
- no more concurrency semaphore
2022-09-19 20:31:00 +02:00
Alex 5768bf3622 First implementation of K2V (#293)
**Specification:**

View spec at [this URL](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/k2v/doc/drafts/k2v-spec.md)

- [x] Specify the structure of K2V triples
- [x] Specify the DVVS format used for causality detection
- [x] Specify the K2V index (just a counter of number of values per partition key)
- [x] Specify single-item endpoints: ReadItem, InsertItem, DeleteItem
- [x] Specify index endpoint: ReadIndex
- [x] Specify multi-item endpoints: InsertBatch, ReadBatch, DeleteBatch
- [x] Move to JSON objects instead of tuples
- [x] Specify endpoints for polling for updates on single values (PollItem)

**Implementation:**

- [x] Table for K2V items, causal contexts
- [x] Indexing mechanism and table for K2V index
- [x] Make API handlers a bit more generic
- [x] K2V API endpoint
- [x] K2V API router
- [x] ReadItem
- [x] InsertItem
- [x] DeleteItem
- [x] PollItem
- [x] ReadIndex
- [x] InsertBatch
- [x] ReadBatch
- [x] DeleteBatch

**Testing:**

- [x] Just a simple Python script that does some requests to check visually that things are going right (does not contain parsing of results or assertions on returned values)
- [x] Actual tests:
  - [x] Adapt testing framework
  - [x] Simple test with InsertItem + ReadItem
  - [x] Test with several Insert/Read/DeleteItem + ReadIndex
  - [x] Test all combinations of return formats for ReadItem
  - [x] Test with ReadBatch, InsertBatch, DeleteBatch
  - [x] Test with PollItem
  - [x] Test error codes
- [ ] Fix most broken stuff
  - [x] test PollItem broken randomly
  - [x] when invalid causality tokens are given, errors should be 4xx not 5xx

**Improvements:**

- [x] Descending range queries
  - [x] Specify
  - [x] Implement
  - [x] Add test
- [x] Batch updates to index counter
- [x] Put K2V behind `k2v` feature flag

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/293
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
2022-05-10 13:16:57 +02:00