fix(ecstore): stop treating DiskNotFound as object not-found in listing (#4536)

is_all_not_found and is_all_volume_not_found delegated to
is_err_bucket_not_found, which matches StorageError::DiskNotFound. When
every erasure set in every pool was unreachable, the per-set aggregate
DiskNotFound slice was classified as "all not found", so list_merged
returned Ok(empty) and walk_result_from_set_errors was fed an
availability failure disguised as an empty listing. Clients saw a
successful empty ListObjects and mistook a full outage for an empty
bucket.

Introduce is_err_strict_not_found (FileNotFound / VolumeNotFound /
FileVersionNotFound / ObjectNotFound / VersionNotFound, excluding
DiskNotFound) for is_all_not_found, mirroring MinIO's isAllNotFound and
DiskError::is_all_not_found. Give is_all_volume_not_found its own
is_err_strict_volume_not_found (VolumeNotFound / BucketNotFound, minus
DiskNotFound) so genuine volume/bucket absence still surfaces while a
missing object under an existing volume is not escalated. Add
is_all_disk_not_found and a pre-check in walk_result_from_set_errors so
an all-offline slice surfaces DiskNotFound instead of being swallowed as
an empty listing, while partial outages (a healthy set present) stay
tolerated as before.

Regression tests cover all-DiskNotFound rejection, genuine not-found
acceptance, volume-not-found semantics, partial/empty short-circuit, and
the walk full-offline vs partial-offline paths.

Co-authored-by: heihutu <heihutu@gmail.com>
This commit is contained in:
houseme
2026-07-09 01:38:36 +08:00
committed by GitHub
parent 15808254d3
commit e0619e355f
2 changed files with 140 additions and 3 deletions
+28 -1
View File
@@ -20,7 +20,8 @@ use crate::core::sets::Sets;
use crate::disk::error::DiskError;
use crate::disk::{DiskAPI, DiskInfo, DiskStore, RUSTFS_META_BUCKET, WalkDirOptions};
use crate::error::{
Error, Result, StorageError, is_all_not_found, is_all_volume_not_found, is_err_bucket_not_found, to_object_err,
Error, Result, StorageError, is_all_disk_not_found, is_all_not_found, is_all_volume_not_found, is_err_bucket_not_found,
to_object_err,
};
use crate::object_api::{ObjectInfo, ObjectOptions};
use crate::set_disk::SetDisks;
@@ -106,6 +107,14 @@ fn walk_result_from_set_errors(errs: &[Option<Error>]) -> Result<()> {
return Ok(());
}
// Every set is unreachable (all drives offline / drive-id mismatch, with no
// healthy set contributing entries): this is an availability failure, not an
// empty listing. Surface it instead of silently swallowing DiskNotFound in
// the not-found loop below.
if is_all_disk_not_found(errs) {
return Err(StorageError::DiskNotFound);
}
for err in errs.iter().flatten() {
if err == &Error::Unexpected || err.is_not_found() {
continue;
@@ -8842,6 +8851,24 @@ mod test {
.expect("successful sets and unexpected EOF-style markers should not fail the walk");
}
// Regression for #952 (ECA-11): when every set is unreachable the walk must
// surface an availability error, not silently succeed with an empty listing.
#[test]
fn walk_result_from_set_errors_surfaces_full_offline() {
let err = walk_result_from_set_errors(&[Some(StorageError::DiskNotFound), Some(StorageError::DiskNotFound)])
.expect_err("all sets unreachable must surface an availability error, not an empty listing");
assert_eq!(err, StorageError::DiskNotFound);
}
// A partial outage (some set healthy) must still be tolerated as before: the
// healthy set's success short-circuits the all-offline detection.
#[test]
fn walk_result_from_set_errors_tolerates_partial_offline() {
walk_result_from_set_errors(&[None, Some(StorageError::DiskNotFound)])
.expect("a partial outage with a healthy set must not fail the walk");
}
// use std::sync::Arc;
// use crate::cache_value::metacache_set::list_path_raw;