gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-08-21 20:06:37 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(ecstore): pin bitrot algorithms with a startup self-test (HS-11) (#6165)

Browse Source 
feat(ecstore): pin bitrot algorithms with a startup self-test

A drifted HighwayHash implementation fails silently: every shard reads
back corrupt, heal rewrites healthy data, and cross-platform clusters
disagree about which copy is good. Mirror MinIO's bitrotSelfTest by
verifying, once at process start:

- known-answer digests for HighwayHash256S / HighwayHash256SLegacy over
  a deterministic 4096-byte xorshift64* payload, plus the externally
  verifiable FIPS SHA-256 "abc" vector guarding the HashAlgorithm
  plumbing itself;
- an end-to-end roundtrip per streaming variant (encode -> size formula
  -> bitrot_verify -> BitrotReader read-back), over full blocks and a
  partial tail;
- tamper detection: one flipped byte in the final data block and one in
  the leading hash must both be rejected as a hash mismatch, not by an
  incidental read error.

The check costs microseconds and runs inline in
init_background_service_runtime before any shard can be written or
verified. Outcome surfaces as one structured bitrot_selftest log event,
the rustfs_bitrot_selftest_status gauge (1=passed / 0=failed / 2=skipped),
a bitrotSelftest field on the admin server-info response, and
RUSTFS_BITROT_SELFTEST_STRICT=on turns a failure into a startup error
(MinIO Fatal parity; the default only degrades the status so a bad build
cannot brick an existing fleet on upgrade).

Closes rustfs/backlog#1873 (HS-11).

Co-authored-by: heihutu <heihutu@gmail.com>
This commit is contained in:
houseme
2026-08-17 15:04:23 +08:00
committed by GitHub
parent d091554ffe
commit 89e2513205
9 changed files with 535 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
rustfs/src/admin/handlers/system.rs
+29
View File
@@ -417,6 +417,13 @@ struct SystemAdminDiscovery {
struct ServerInfoResponse {
info: InfoMessage,
admin_discovery: SystemAdminDiscovery,
/// Startup bitrot algorithm self-test outcome (rustfs/backlog#1873):
/// `passed` (algorithms verified at boot), `failed` (a drifted hash
/// implementation — the process is serving with degraded integrity
/// checking unless `RUSTFS_BITROT_SELFTEST_STRICT` aborted it), or
/// `unknown` (not yet run or disabled).
#[serde(rename = "bitrotSelftest")]
bitrot_selftest: &'static str,
}
#[derive(Serialize)]
@@ -433,6 +440,14 @@ fn system_admin_discovery(usecase: &DefaultAdminUsecase) -> SystemAdminDiscovery
}
}
fn bitrot_selftest_status_str() -> &'static str {
match crate::bitrot_selftest::bitrot_selftest_passed() {
Some(true) => "passed",
Some(false) => "failed",
None => "unknown",
}
}
#[async_trait::async_trait]
impl Operation for ServerInfoHandler {
async fn call(&self, req: S3Request<Body>, _params: Params<'_, '_>) -> S3Result<S3Response<(StatusCode, Body)>> {
@@ -464,6 +479,7 @@ impl Operation for ServerInfoHandler {
let response = ServerInfoResponse {
info,
admin_discovery: system_admin_discovery(&usecase),
bitrot_selftest: bitrot_selftest_status_str(),
};
let data = serde_json::to_vec(&response).map_err(|e| {
@@ -1535,6 +1551,18 @@ mod tests {
);
}
/// The startup bitrot self-test outcome must surface in server info as one
/// of three closed-set strings, never an internal enum or a null
/// (rustfs/backlog#1873). This test pins the string mapping; whether the
/// process-global cell holds Some(true)/Some(false)/None is owned by
/// `crate::bitrot_selftest`'s own tests.
#[test]
fn bitrot_selftest_status_str_is_a_closed_set_of_operators_strings() {
let rendered = super::bitrot_selftest_status_str();
assert!(matches!(rendered, "passed" | "failed" | "unknown"));
assert_eq!(super::bitrot_selftest_status_str(), rendered);
}
#[test]
fn server_info_response_exposes_admin_discovery_paths() {
let usecase = DefaultAdminUsecase::without_context();
@@ -1556,6 +1584,7 @@ mod tests {
pools: None,
},
admin_discovery: system_admin_discovery(&usecase),
bitrot_selftest: super::bitrot_selftest_status_str(),
};
let value = serde_json::to_value(response).expect("server info response should serialize");