mirror of
https://github.com/rustfs/rustfs.git
synced 2026-09-06 03:59:14 +00:00
Sergey Shlukov
1747ed0292
* fix(ecstore): stop scan_dir emitting entries past a limit hit inside a subdirectory scan_dir's flush loop recurses into a pending subdirectory when the current sibling entry's page limit is reached mid-recursion, but kept writing the current (later-sorting) entry regardless. gather_results then builds the next page's continuation marker from that later entry, which permanently skips the still-unscanned tail of the subdirectory on resume instead of just deferring it to the next page. Add a limit re-check right after the flush loop, before the current entry is written, so scan_dir stops cleanly at the true last-written key. Reproduces and fixes the rc.5 recursive ListObjectsV2 data-loss report (7826/7881 keys, contiguous 55-key block silently dropped). Adds scan_dir_does_not_emit_entries_past_a_limit_hit_inside_a_subdirectory. * fix(ecstore): re-check the page limit on every dir_stack flush iteration The flush loop that drains dir_stack can pop and recurse into more than one pending subdirectory per outer iteration (whenever more than one stack entry sorts below the current sibling entry). The limit re-check added in the previous commit only ran once, after the whole flush loop exited - so if the first recursive scan_dir call already exhausted the page limit, the loop's next pop+recurse still went ahead and scanned (and emitted entries for) another subdirectory beyond where the page was supposed to stop. Confirmed against production data: a bucket with ~1.17M objects under one prefix still cut a recursive ListObjectsV2 listing short (825 of an expected much larger next page, IsTruncated=false) even with the first fix deployed, at a two-level-nested subdirectory. Move the check inside the while loop so it runs before every pop, not just once after. --------- Co-authored-by: Claude Agent <agent@local>