gsadmin/garage
1
0
Fork 0
mirror of https://github.com/deuxfleurs-org/garage.git synced 2026-08-17 09:07:48 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,656 Commits 31 Branches 89 Tags
f59a8b7f621b0abf8c8e31b7cdd8994a7154b16c
Commit Graph

83 Commits

Author SHA1 Message Date
Gwen Lg f59a8b7f62 style: corrects the use of ';' to improve readability 
- remove unnecessary semicolon
and enable lint warning: unnecessary semicolon
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.93.0/index.html#unnecessary_semicolon
- add `;` to the last statement for consitent formatting
and enable lint `clippy::semicolon_if_nothing_returned`
warning: consider adding a `;` to the last statement for consistent formatting
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.93.0/index.html#semicolon_if_nothing_returned
 2026-02-17 18:38:44 +00:00
Gwen Lg 4a0be692b3 docs: various fixes in Rust code documentation 
- add backticks on struct doc comments
lint message: unclosed HTML tag `M`
- add a blanck line for proper doc formating
lint message: doc list item without indentation
help: if this is supposed to be its own paragraph, add a blank line
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#doc_lazy_continuation
- fix hyperlink in doc + one url invalid
lint message: this URL is not a hyperlink
note: bare URLs are not automatically turned into clickable links
- ajust space in doc
lint message: doc list item overindented
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#doc_overindented_list_items
 2026-01-29 20:49:28 +01:00
Gwen Lg 21db7a4d4b chore: remove various useless code 
- call expect directly on Result
lint message: called `ok().expect()` on a `Result` value
help: you can call `expect()` directly on the `Result`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#ok_expect
- use assign operation instead of manual implementation
lint message: manual implementation of an assign operation
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#assign_op_pattern
- remove useless call to format
lint message: useless use of `format!`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#useless_format
- remove useless `?Sized`
lint message: `?Sized` bound is ignored because of a `Sized` requirement
note: ...because `Deserialize` has the bound `Sized`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#needless_maybe_sized
- use is_some instead of pattern maching with Some(_)
lint message: redundant pattern matching, consider using `is_some()`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#redundant_pattern_matching
- remove unneeded unit return type
lint message: unneeded unit return type
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#unused_unit
- remove redundant closure
lint message: redundant closure
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#redundant_closure
- use derive Default instead of manual implementation
lint message: this `impl` can be derived
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#derivable_impls
- remove unneeded `return` statement
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#needless_return
- remove empty string from println call
lint message: empty string literal in `println!`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#println_empty_string
- remove clone() on type than implement Copy
lint message: using `clone` on type `Option<ChecksumValue>` which implements the `Copy` trait
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#clone_on_copy
- remove useless let binding
lint message: this let-binding has unit value
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#let_unit_value
- remove useless len comparison to zero, already test of empty
lint message: length comparison to zero
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#len_zero
- remove useless `as_deref` call
lint message: derefed type is same as origin
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#needless_option_as_deref
- remove useless conversion of the same type
lint message: useless conversion to the same type: `replication_mode::ReplicationFactor`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#useless_conversion
- remove useless bool_comparison
lint message: equality checks against false can be replaced by a negation
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#bool_comparison
- remove useless to_string
lint message: unnecessary use of `to_string`
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#unnecessary_to_owned

Signed-off-by: Gwen Lg <me@gwenlg.fr>
 2026-01-29 20:49:28 +01:00
Gwen Lg 141b3f24f1 chore: clean relative to references and borrows 
- lint message: the borrowed expression implements the required traits
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#needless_borrows_for_generic_args
- lint message: this expression creates a reference which is immediately dereferenced by the compiler
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#needless_borrow
- lint message: you don't need to add `&` to all patterns
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#match_ref_pat
- remove useless taken reference
lint message: needlessly taken reference of left operand
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#op_ref
- use &Path instead of &PathBuf as fn parameters
lint message: writing `&PathBuf` instead of `&Path` involves a new object where a slice will do
help: for further information visit https://rust-lang.github.io/rust-clippy/rust-1.92.0/index.html#ptr_arg
 2026-01-29 20:49:28 +01:00
Gwen Lg e331f88c85 chore: fix typos in rust code comment or error message 2026-01-29 14:53:27 +01:00
trinity-1686a 5333285b50 don't append index document to redirect urls 2025-11-06 18:03:58 +01:00
Alex Auvolat dc1a4ffd76 Merge branch 'main' into next-v2 2025-06-13 14:01:39 +02:00
trinity-1686a 26bc807905 put web error in a basic webpage 
before, it was a plain string, with an xml content type

this caused browsers to show very ugly and meaningless pages
 2025-06-10 22:23:06 +02:00
Alex Auvolat 29ce490dd6 Merge branch 'main' into next-v2 2025-03-05 14:50:22 +01:00
Alex Auvolat bf27a3ec98 web: implement x-amz-website-redirect-location 2025-02-19 17:04:10 +01:00
Alex Auvolat c5df820e2c api: start refactor of signature to calculate checksums earlier 2025-02-17 18:47:06 +01:00
Alex Auvolat 2f55889835 add configuration option to enable/disable monitoring bucket in web metrics 2025-02-14 14:59:00 +01:00
Jonathan Davies 8b9cc5ca3f web_server.rs: Added bucket domain to observability. 2025-02-14 14:36:20 +01:00
trinity-1686a f034e834fa Merge branch 'main' into 1686a/s3-redirects 2025-02-08 16:01:11 +01:00
Alex Auvolat afa28706e5 split s3/cors.rs into also common/cors.rs 2025-01-31 18:42:14 +01:00
Alex Auvolat 84f1db91c4 fix things up 2025-01-31 18:34:57 +01:00
Alex Auvolat 2aaba39ddc refactor web_server.rs 2025-01-04 20:11:54 +01:00
Alex Auvolat 47467df83e avoid handling status_code-related logic in api/s3/get.rs 2025-01-04 19:52:14 +01:00
Alex Auvolat 22487ceddf move Redirect::compute_target to standalone function in web_server.rs 2025-01-04 18:22:42 +01:00
trinity-1686a c939d2a936 clippy 2024-12-22 15:26:06 +01:00
trinity-1686a 65e9dde8c9 add tests 2024-12-22 15:20:09 +01:00
trinity-1686a c9b733a4a6 support redirection on s3 endpoint 2024-12-14 17:46:27 +01:00
Yureka fb55682c66 add request context helper 2024-03-04 13:26:39 +01:00
Alex Auvolat 02e98e2d10 [header-override-650] implement header overriding in GetObject (fix #650) 2024-02-09 15:58:46 +01:00
Alex Auvolat 5c63193d1d [dep-upgrade-202402] fix shutdown issue introduced when upgrading hyper 2024-02-08 23:43:59 +01:00
Alex Auvolat fe48d60d2b [dep-upgrade-202402] refactor http listener code 2024-02-07 14:34:40 +01:00
Alex Auvolat a22bd31920 [dep-upgrade-202402] migration to http/hyper 1.0 for k2v api 2024-02-05 19:27:12 +01:00
Alex Auvolat 0bb5b77530 [dep-upgrade-202402] wip: port to http/hyper crates v1 2024-02-05 18:49:54 +01:00
networkException 8ec6a53b35 everywhere: support unix sockets when binding in various places 
this patch implements binding to paths as a unix socket for generic
server and web server.
 2023-09-29 18:57:44 +02:00
Alex Auvolat c5cafa0000 web_server.rs: handle error properly and refactor 2023-08-28 12:05:14 +02:00
Quentin Dufour 63da1d2443 support index on path missing a trailing slash 2023-08-08 15:28:57 +02:00
Jonathan Davies c783194e8b *: apply clippy recommendations. 2023-05-09 20:49:34 +01:00
Jonathan Davies 4e0fc3d6c9 web/web_server.rs: Handle X-Forwarded-For here too. 2023-03-06 11:43:54 +00:00
Baptiste Jonglez e85a200189 Fix duplicated content-type in error document 
Fixes #492
 2023-01-29 22:51:23 +01:00
Alex Auvolat 8d04ae7014 cargo2nix unstable (patched), rust 1.63.0, nixpkgs 22.05 (32-bit builds are broken) 2022-10-14 14:30:48 +02:00
Alex Auvolat 2559f63e9b Make all HTTP services optionnal 2022-09-07 17:54:16 +02:00
Alex 382e74c798 First version of admin API (#298) 
**Spec:**

- [x] Start writing
- [x] Specify all layout endpoints
- [x] Specify all endpoints for operations on keys
- [x] Specify all endpoints for operations on key/bucket permissions
- [x] Specify all endpoints for operations on buckets
- [x] Specify all endpoints for operations on bucket aliases

View rendered spec at <https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/admin-api/doc/drafts/admin-api.md>

**Code:**

- [x] Refactor code for admin api to use common api code that was created for K2V

**General endpoints:**

- [x] Metrics
- [x] GetClusterStatus
- [x] ConnectClusterNodes
- [x] GetClusterLayout
- [x] UpdateClusterLayout
- [x] ApplyClusterLayout
- [x] RevertClusterLayout

**Key-related endpoints:**

- [x] ListKeys
- [x] CreateKey
- [x] ImportKey
- [x] GetKeyInfo
- [x] UpdateKey
- [x] DeleteKey

**Bucket-related endpoints:**

- [x] ListBuckets
- [x] CreateBucket
- [x] GetBucketInfo
- [x] DeleteBucket
- [x] PutBucketWebsite
- [x] DeleteBucketWebsite

**Operations on key/bucket permissions:**

- [x] BucketAllowKey
- [x] BucketDenyKey

**Operations on bucket aliases:**

- [x] GlobalAliasBucket
- [x] GlobalUnaliasBucket
- [x] LocalAliasBucket
- [x] LocalUnaliasBucket

**And also:**

- [x] Separate error type for the admin API (this PR includes a quite big refactoring of error handling)
- [x] Add management of website access
- [ ] Check that nothing is missing wrt what can be done using the CLI
- [ ] Improve formatting of the spec
- [x] Make sure everyone is cool with the API design

Fix #231
Fix #295

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/298
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
 2022-05-24 12:16:39 +02:00
Alex 5768bf3622 First implementation of K2V (#293) 
**Specification:**

View spec at [this URL](https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/k2v/doc/drafts/k2v-spec.md)

- [x] Specify the structure of K2V triples
- [x] Specify the DVVS format used for causality detection
- [x] Specify the K2V index (just a counter of number of values per partition key)
- [x] Specify single-item endpoints: ReadItem, InsertItem, DeleteItem
- [x] Specify index endpoint: ReadIndex
- [x] Specify multi-item endpoints: InsertBatch, ReadBatch, DeleteBatch
- [x] Move to JSON objects instead of tuples
- [x] Specify endpoints for polling for updates on single values (PollItem)

**Implementation:**

- [x] Table for K2V items, causal contexts
- [x] Indexing mechanism and table for K2V index
- [x] Make API handlers a bit more generic
- [x] K2V API endpoint
- [x] K2V API router
- [x] ReadItem
- [x] InsertItem
- [x] DeleteItem
- [x] PollItem
- [x] ReadIndex
- [x] InsertBatch
- [x] ReadBatch
- [x] DeleteBatch

**Testing:**

- [x] Just a simple Python script that does some requests to check visually that things are going right (does not contain parsing of results or assertions on returned values)
- [x] Actual tests:
  - [x] Adapt testing framework
  - [x] Simple test with InsertItem + ReadItem
  - [x] Test with several Insert/Read/DeleteItem + ReadIndex
  - [x] Test all combinations of return formats for ReadItem
  - [x] Test with ReadBatch, InsertBatch, DeleteBatch
  - [x] Test with PollItem
  - [x] Test error codes
- [ ] Fix most broken stuff
  - [x] test PollItem broken randomly
  - [x] when invalid causality tokens are given, errors should be 4xx not 5xx

**Improvements:**

- [x] Descending range queries
  - [x] Specify
  - [x] Implement
  - [x] Add test
- [x] Batch updates to index counter
- [x] Put K2V behind `k2v` feature flag

Co-authored-by: Alex Auvolat <alex@adnab.me>
Reviewed-on: https://git.deuxfleurs.fr/Deuxfleurs/garage/pulls/293
Co-authored-by: Alex <alex@adnab.me>
Co-committed-by: Alex <alex@adnab.me>
 2022-05-10 13:16:57 +02:00
Alex Auvolat ba6b56ae68 Fix some new clippy lints 2022-03-14 12:27:49 +01:00
Alex Auvolat d9a35359bf Add metrics to web endpoint 2022-03-14 10:53:50 +01:00
Alex Auvolat 8a5bbc3b0b More permissive OPTIONS on S3 API 2022-03-01 11:15:16 +01:00
Alex Auvolat 8129a98291 Process CORS earlier in pipeline 2022-02-28 12:22:39 +01:00
Alex Auvolat 6dab836f3a Multipart improvements 
- support part_number for HeadObject
- add checks in complete_multipart_upload
 2022-01-24 21:04:40 +01:00
Alex Auvolat 513a6b15f9 Handle OPTIONS on website endpoint 2022-01-24 12:32:28 +01:00
Alex Auvolat ea7fb901eb Implement {Put,Get,Delete}BucketCors and CORS in general 
- OPTIONS request against API endpoint
- Returning corresponding CORS headers on API calls
- Returning corresponding CORS headers on website GET's
 2022-01-24 11:58:00 +01:00
Alex Auvolat d4dd2e2640 Make use of website config, return error document on error 2022-01-13 14:25:19 +01:00
Alex Auvolat df35feba18 New buckets for 0.6.0: make bucket id a SK and not a HK, CLI updates 2022-01-04 12:53:14 +01:00
Alex Auvolat 1bcd6fabbd New buckets for 0.6.0: small changes 
- Fix bucket delete

- fix merge of bucket creation date

- Replace deletable with option in aliases
    Rationale: if two aliases point to conflicting bucket, resolving
    by making an arbitrary choice risks making data accessible when it
    shouldn't be. We'd rather resolve to deleting the alias until
    someone puts it back.
 2022-01-04 12:52:47 +01:00
Alex Auvolat b1cfd16913 New buckets for 0.6.0: small fixes, including: 
- ensure bucket names are correct aws s3 names
- when making aliases, ensure timestamps of links in both ways are the
  same
- fix small remarks by trinity
- don't have a separate website_access field
 2022-01-04 12:46:41 +01:00
Alex Auvolat 0bbb6673e7 Model changes 2022-01-04 12:45:52 +01:00