gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-08-19 19:16:17 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(replication): allow loopback replication targets under an explicit test opt-in (#4725)

Browse Source 
* fix(replication): allow loopback replication targets under an explicit test opt-in

Commit 5c7c757a3 (#4712) activated the previously-dormant replication e2e
suite (they had never run anywhere). All 9 fast tests then failed on main
because the SSRF egress guard rejects the 127.0.0.1 targets the e2e harness
configures: `target endpoint is not allowed: outbound URL host '127.0.0.1'
is not allowed: loopback address`. The whole harness runs on loopback, so
every replication test hit this before reaching its actual assertion.

Loopback is a genuine SSRF vector and must stay rejected in production, so
this does not relax the guard. Instead `validate_replication_target_endpoint`
gains an off-by-default opt-in (`RUSTFS_REPLICATION_ALLOW_LOOPBACK_TARGET`)
that re-enables loopback targets (127.0.0.1 / ::1 / localhost) for single-host
multi-instance dev and the e2e harness. Private addresses stay unconditionally
allowed as before; the opt-in does not widen into link-local or the cloud
metadata endpoint. The e2e harness sets the env for every server it spawns
(single-node and cluster paths), overridable via extra_env.

Verified end-to-end: all 9 previously-failing replication_extension_test
smoke tests pass against a locally built binary. New unit tests in
bucket_target_sys pin the matrix — public/private always allowed, loopback
gated on the opt-in in both IP and hostname forms, and metadata/link-local
still rejected even with the opt-in on.

Refs: backlog#1147

Co-Authored-By: heihutu <heihutu@gmail.com>

* test(replication): rename optin -> opt_in to satisfy typos check

Pure rename of three unit-test function names; no behaviour change.

Co-Authored-By: heihutu <heihutu@gmail.com>

---------

Co-authored-by: heihutu <heihutu@gmail.com>
This commit is contained in:
houseme
2026-07-11 15:49:39 +08:00
committed by GitHub
parent f63af3df63
commit 2ebe8e561b
4 changed files with 300 additions and 95 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.config/nextest.toml
+17 -15
View File
@@ -138,16 +138,19 @@ test-group = 'e2e-reliability'
# silently unrun) until it is explicitly blessed as fast here. Keep the two
# regexes byte-identical. Count invariant: 20 here + 16 nightly = 36 total
# (authority: `cargo nextest list`; docs/testing/e2e-suite-inventory.md).
# NOTE (2026-07-11): the 20 fast bucket-replication admin-path tests were
# pulled OUT of this PR smoke lane — they set a remote replication target at a
# loopback endpoint (127.0.0.1, a second local server), which RustFS's target
# SSRF guard rejects by default ("outbound URL host '127.0.0.1' is not
# allowed: loopback address"), so they failed on EVERY PR after repl-1 (#4712)
# merged. All replication e2e tests now run only in the e2e-repl-nightly lane
# below until the nightly job is configured to allow loopback targets (repl-1
# follow-up, backlog#1147). Restore an allowlist clause here once that lands.
# HISTORY (2026-07-11): the 20 fast tests were briefly pulled out of this lane
# (#4724) because they set a loopback (127.0.0.1) replication target that the
# SSRF egress guard rejected on every PR after repl-1 (#4712). That is fixed —
# the guard now honours an off-by-default opt-in and this suite's source servers
# set it (RUSTFS_REPLICATION_ALLOW_LOOPBACK_TARGET) — so the allowlist below is
# restored.
[profile.e2e-smoke]
default-filter = 'package(e2e_test) & test(/^(delete_marker_migration_semantics|version_id_regression|list_objects_v2_pagination|list_object_versions_regression|list_objects_duplicates|list_buckets_double_slash|leading_slash_key|special_chars|create_bucket_region|delete_objects_versioning|head_object_consistency|head_object_range|copy_object_metadata|copy_source_invalid_date|content_encoding|anonymous_access|bucket_policy_check)_test::/)'
default-filter = """
package(e2e_test) & (
test(/^(delete_marker_migration_semantics|version_id_regression|list_objects_v2_pagination|list_object_versions_regression|list_objects_duplicates|list_buckets_double_slash|leading_slash_key|special_chars|create_bucket_region|delete_objects_versioning|head_object_consistency|head_object_range|copy_object_metadata|copy_source_invalid_date|content_encoding|anonymous_access|bucket_policy_check)_test::/)
| test(/^replication_extension_test::(test_replication_check_succeeds_with_remote_target|test_replication_check_rejects_target_without_object_lock|test_set_remote_target_rejects_unversioned_source_bucket|test_replication_check_rejects_unversioned_source_bucket|test_replication_check_rejects_missing_replication_config|test_replication_check_rejects_invalid_bucket|test_set_remote_target_rejects_same_bucket_on_same_deployment|test_set_remote_target_rejects_unversioned_target_bucket|test_set_remote_target_update_requires_arn|test_set_remote_target_update_rejects_missing_target|test_set_remote_target_rejects_invalid_target_url|test_set_remote_target_rejects_self_signed_https_target_without_skip_tls_verify|test_set_remote_target_rejects_private_ca_https_target_without_ca_cert_pem|test_list_remote_targets_rejects_empty_bucket|test_list_remote_targets_rejects_invalid_bucket|test_remove_remote_target_rejects_missing_target|test_remove_remote_target_rejects_missing_arn|test_remove_remote_target_rejects_invalid_bucket|test_remove_remote_target_rejects_target_used_by_replication|test_delete_bucket_replication_removes_remote_target)$/)
)
"""
fail-fast = false
# ---------------------------------------------------------------------------
@@ -185,13 +188,12 @@ fail-fast = false
# labor with ci-5's future e2e-full merge gate: these tests run ONLY here, not
# double-run there. TODO(ci-7): fold this interim repl-owned lane into the ci
# domain's consolidated scheduled e2e workflow once it exists.
# Runs ALL replication_extension_test tests (the 20 fast admin-path tests were
# moved back here from e2e-smoke on 2026-07-11 — see the note on e2e-smoke
# above). The nightly job must be configured to allow loopback replication
# targets (repl-1 follow-up) for these to pass; keeping them out of the per-PR
# gate un-blocks all PRs in the meantime.
[profile.e2e-repl-nightly]
default-filter = 'package(e2e_test) & test(/^replication_extension_test::/)'
default-filter = """
package(e2e_test)
& test(/^replication_extension_test::/)
& !test(/^replication_extension_test::(test_replication_check_succeeds_with_remote_target|test_replication_check_rejects_target_without_object_lock|test_set_remote_target_rejects_unversioned_source_bucket|test_replication_check_rejects_unversioned_source_bucket|test_replication_check_rejects_missing_replication_config|test_replication_check_rejects_invalid_bucket|test_set_remote_target_rejects_same_bucket_on_same_deployment|test_set_remote_target_rejects_unversioned_target_bucket|test_set_remote_target_update_requires_arn|test_set_remote_target_update_rejects_missing_target|test_set_remote_target_rejects_invalid_target_url|test_set_remote_target_rejects_self_signed_https_target_without_skip_tls_verify|test_set_remote_target_rejects_private_ca_https_target_without_ca_cert_pem|test_list_remote_targets_rejects_empty_bucket|test_list_remote_targets_rejects_invalid_bucket|test_remove_remote_target_rejects_missing_target|test_remove_remote_target_rejects_missing_arn|test_remove_remote_target_rejects_invalid_bucket|test_remove_remote_target_rejects_target_used_by_replication|test_delete_bucket_replication_removes_remote_target)$/)
"""
fail-fast = false
[profile.e2e-repl-nightly.junit]