gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-09-08 04:58:12 +00:00
Code Issues Packages Projects Releases Wiki Activity

chore(scanner): stage Scanner/Heal follow-up slices (#7374)

Browse Source 
* fix(scanner): remove unused digest import

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* feat(scanner): add raw page owner index (#7375)

* feat(scanner): add raw page owner index

Add a serializable raw enumeration page owner index for scanner resume work.

The index exposes unsupported, building, and ready states, validates committed page identity by recomputing digests, and uses generation checks for CAS-style page commits.

Focused tests cover small-budget restart progress, page digest/source drift rejection, corrupt deserialized state, CAS failure, precommit crash, empty sources, and invalid entry boundaries.

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* feat(scanner): persist raw page owner resume state (#7379)

Wire the scanner raw enumeration partial-cache writer to the raw page owner index so interrupted bucket walks can retain validated page-builder state across scanner restarts.

Keep complete owner sources terminal-only, add partial-source ingestion for in-progress raw directory reads, and validate the persisted page index through bucket checkpoint preparation.

Co-authored-by: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: zhi22915 <qiuzgang@gmail.com>

* test(scanner): fence segment producer observations (#7381)

Require the segment observation fixture to carry source, incarnation, key-format, baseline, process epoch, generation-window, gap, overflow, and producer-coverage proof before accepting a narrowed proposal. Keep the diagnostic path fixture-only and remove its ordinary stderr output.

Co-authored-by: zhi22915 <qiuzgang@gmail.com>

* fix(ecstore): isolate pool metadata read probes (#7367)

Co-authored-by: zhi22915 <qiuzgang@gmail.com>

* test(heal): cover MRF crash successor matrix (#7369)

* test(heal): cover MRF crash successor matrix

Add process-boundary MRF replay coverage for the successor snapshot window after a retained startup journal is flushed but before cleanup deletes it. Extend the mixed authoritative/legacy reader fixture with a scoped v2 journal epoch to pin the no-merge contract.

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(heal): cover service-kill MRF replay (#7380)

Add a Unix process fixture that waits after publishing the pending MRF successor snapshot, then is terminated by the parent before restart replay.

Co-authored-by: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: zhi22915 <qiuzgang@gmail.com>

* test(heal): cover transport-lost start receipts (#7371)

Add gRPC transport fault fixtures for heal-control start admission. The tests distinguish pre-admission transport loss from post-admission response loss, then verify exact envelope retries reuse the canonical receipt while fresh forceStart requests create distinct tasks.

Co-authored-by: zhi22915 <qiuzgang@gmail.com>

* test(scanner): add crash-restart heal evidence case (#7370)

* test(scanner): add crash-restart heal evidence case

Add a distinct W21 background target crash case to the scanner/heal evidence registry and oracle path.

Keep the existing restart lane on graceful process restart, keep the crash lane on hard kill, and make the wiring checker reject evidence/oracle mismatches.

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

* test(scanner): support older Python wiring checks

Let the scanner/heal evidence wiring checker run under Python 3.9/3.10 by falling back to tomli and chunked SHA-256 hashing when the Python 3.11 standard APIs are unavailable.

Co-Authored-By: heihutu <heihutu@gmail.com>

Co-Authored-By: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: zhi22915 <qiuzgang@gmail.com>

* fix(scanner): reject stale raw page source seeds (#7382)

Do not prefill a resumed raw page owner with previously indexed entries when starting a new raw directory observation pass. The next pass must observe the same prefix again before the page index can advance; otherwise the index is discarded fail-closed.

Co-authored-by: zhi22915 <qiuzgang@gmail.com>

* fix(scanner): defer raw page revalidation until observed (#7384)

A resumed raw page owner index must not prefill entries from older cache state, but it also must not discard a valid multi-entry index before the current raw directory pass has observed enough entries to prove identity. Track the persisted index floor and only run the strict owner identity check once the current pass reaches that floor.

Co-authored-by: zhi22915 <qiuzgang@gmail.com>

---------

Co-authored-by: zhi22915 <qiuzgang@gmail.com>
This commit is contained in:
houseme
2026-09-07 18:13:39 +08:00
committed by GitHub
parent b370e746a0
commit 4c2a0cdf9a
14 changed files with 2108 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/scanner/src/scanner_folder.rs
+93 -14
View File
@@ -25,6 +25,7 @@ use crate::data_usage_define::{
PendingScannerHealKind, ScannerSizeSummaryExt, SizeReconciliationEntry, SizeSummary, hash_path,
};
use crate::error::ScannerError;
use crate::raw_page_index::{RawEnumerationPageIndex, RawEnumerationPageIndexError};
use crate::runtime_config::{
scanner_alert_excess_folders, scanner_alert_excess_version_size, scanner_alert_excess_versions, scanner_yield_every_n_objects,
};
@@ -90,6 +91,8 @@ const DATA_SCANNER_FORCE_COMPACT_AT_FOLDERS: usize = 250_000;
const SCANNER_LIST_PATH_RAW_STALL_TIMEOUT: Duration = Duration::from_secs(60);
const SCANNER_ENTRY_PROGRESS_BATCH: u64 = 32;
const SCANNER_ENTRY_PROGRESS_INTERVAL: Duration = Duration::from_secs(30);
const SCANNER_RAW_ENUMERATION_PAGE_ENTRY_LIMIT: usize = 128;
const SCANNER_RAW_ENUMERATION_PAGE_BUILD_BUDGET: usize = 1;
// Erasure data directories contain direct part.N files; keep namespace probes bounded.
const ERASURE_DATA_DIR_PROBE_ENTRY_LIMIT: usize = 64;
const DEFAULT_HEAL_OBJECT_SELECT_PROB: u32 = 1024;
@@ -751,17 +754,34 @@ struct RawEnumerationProgress {
last_entry: Option<String>,
entries_seen: u64,
digest: Sha256,
observed_entries: Vec<String>,
revalidate_after_entries: usize,
page_index: Option<RawEnumerationPageIndex>,
}
impl RawEnumerationProgress {
fn new(parent: &str) -> Self {
fn new(parent: &str, page_index: Option<RawEnumerationPageIndex>) -> Self {
let mut digest = Sha256::new();
update_raw_enumeration_digest(&mut digest, b"parent", parent.as_bytes());
let mut revalidate_after_entries = 0;
let page_index = match page_index {
Some(index) => match index.indexed_entries() {
Ok(entries) => {
revalidate_after_entries = entries.len();
Some(index)
}
Err(_) => None,
},
None => RawEnumerationPageIndex::new(parent, SCANNER_RAW_ENUMERATION_PAGE_ENTRY_LIMIT).ok(),
};
Self {
parent: parent.to_string(),
last_entry: None,
entries_seen: 0,
digest,
observed_entries: Vec::new(),
revalidate_after_entries,
page_index,
}
}
@@ -769,19 +789,55 @@ impl RawEnumerationProgress {
update_raw_enumeration_digest(&mut self.digest, b"entry", entry.as_bytes());
self.last_entry = Some(entry.to_string());
self.entries_seen = self.entries_seen.saturating_add(1);
self.observed_entries.push(entry.to_string());
if let Some(index) = &mut self.page_index {
if self.observed_entries.len() < self.revalidate_after_entries {
return;
}
let result = index
.generation()
.ok_or(RawEnumerationPageIndexError::Unsupported)
.and_then(|generation| {
index.ingest_partial_owner_entries(
self.observed_entries.clone(),
SCANNER_RAW_ENUMERATION_PAGE_BUILD_BUDGET,
generation,
)
});
match result {
Ok(outcome) if outcome.ready_to_commit => {
if let Some(generation) = index.generation()
&& index.commit_building_page(generation).is_err()
{
self.page_index = None;
}
}
Ok(_) => {}
Err(_) => {
self.page_index = None;
}
}
}
}
fn into_cursor(self) -> Option<DataUsageRawEnumerationCursor> {
fn cursor(&self) -> Option<DataUsageRawEnumerationCursor> {
if self.entries_seen == 0 {
return None;
}
Some(DataUsageRawEnumerationCursor::new(
self.parent,
self.last_entry,
self.parent.clone(),
self.last_entry.clone(),
self.entries_seen,
self.digest.finalize().into(),
self.digest.clone().finalize().into(),
))
}
fn page_index(&self) -> Option<RawEnumerationPageIndex> {
self.page_index.clone().and_then(|index| match index.indexed_entries() {
Ok(entries) if !entries.is_empty() => Some(index),
_ => None,
})
}
}
fn update_raw_enumeration_digest(digest: &mut Sha256, label: &[u8], value: &[u8]) {
@@ -1049,6 +1105,19 @@ impl FolderScanner {
if self.old_cache.info.scan_progress.is_none() {
return;
}
let page_index = self
.old_cache
.validated_raw_enumeration_page_index()
.filter(|index| match index.status() {
crate::raw_page_index::RawEnumerationPageOwnerStatus::Building {
parent: index_parent, ..
}
| crate::raw_page_index::RawEnumerationPageOwnerStatus::Ready {
parent: index_parent, ..
} => index_parent == parent,
crate::raw_page_index::RawEnumerationPageOwnerStatus::Unsupported => false,
})
.cloned();
if let Some(position) = self
.raw_enumeration_progress
.iter()
@@ -1056,7 +1125,8 @@ impl FolderScanner {
{
self.raw_enumeration_progress.truncate(position + 1);
} else {
self.raw_enumeration_progress.push(RawEnumerationProgress::new(parent));
self.raw_enumeration_progress
.push(RawEnumerationProgress::new(parent, page_index));
}
if let Some(progress) = self.raw_enumeration_progress.last_mut() {
progress.record_entry(entry);
@@ -1073,11 +1143,11 @@ impl FolderScanner {
});
}
fn take_raw_enumeration_cursor(&mut self) -> Option<DataUsageRawEnumerationCursor> {
self.raw_enumeration_progress
.drain(..)
.next()
.and_then(RawEnumerationProgress::into_cursor)
fn take_raw_enumeration_resume_state(&mut self) -> (Option<DataUsageRawEnumerationCursor>, Option<RawEnumerationPageIndex>) {
match self.raw_enumeration_progress.drain(..).next() {
Some(progress) => (progress.cursor(), progress.page_index()),
None => (None, None),
}
}
fn carry_forward_old_children(&mut self, parent_hash: &DataUsageHash, entry: &mut DataUsageEntry) {
@@ -2686,6 +2756,7 @@ pub(crate) async fn scan_data_folder_scoped(
new_cache.info.scan_resume_after = None;
new_cache.info.scan_checkpoint = None;
new_cache.info.scan_raw_enumeration_cursor = None;
new_cache.info.scan_raw_enumeration_page_index = None;
new_cache.info.scan_coverage_receipt = None;
if had_scan_checkpoint {
global_metrics().record_scanner_checkpoint_cleared();
@@ -2703,9 +2774,10 @@ pub(crate) async fn scan_data_folder_scoped(
let root_hash = hash_path(&cache.info.name);
let root_has_progress = data_usage_root_has_progress(&root);
let pending_heals_changed = scanner.pending_heals_changed;
let raw_enumeration_cursor = scanner.take_raw_enumeration_cursor();
let carry_forward_cache =
(raw_enumeration_cursor.is_some() && !root_has_progress).then(|| scanner.old_cache.cache.clone());
let (raw_enumeration_cursor, raw_enumeration_page_index) = scanner.take_raw_enumeration_resume_state();
let carry_forward_cache = ((raw_enumeration_cursor.is_some() || raw_enumeration_page_index.is_some())
&& !root_has_progress)
.then(|| scanner.old_cache.cache.clone());
if root_has_progress {
scanner.carry_forward_old_children(&root_hash, &mut root);
}
@@ -2722,8 +2794,15 @@ pub(crate) async fn scan_data_folder_scoped(
new_cache.info.scan_resume_after = None;
new_cache.info.scan_coverage_receipt = None;
}
if raw_enumeration_page_index.is_some() {
new_cache.info.scan_raw_enumeration_page_index = raw_enumeration_page_index;
new_cache.info.scan_checkpoint = None;
new_cache.info.scan_resume_after = None;
new_cache.info.scan_coverage_receipt = None;
}
if partial_cache_is_useful(&root, pending_heals_changed)
|| new_cache.info.scan_raw_enumeration_cursor.is_some()
|| new_cache.info.scan_raw_enumeration_page_index.is_some()
|| !new_cache.info.size_reconciliation.is_empty()
{
if new_cache.root().is_some() {