rustfs/INTERNODE_TRANSPORT_CAPABILITIES.md

Internode Transport Capabilities

Status: design note for backend-neutral capability reporting. This document does not add an RDMA backend and does not require RDMA hardware or crates.

Purpose

InternodeDataTransportCapabilities describes what a backend can honestly do for RustFS internode data-plane transfers. The fields are intentionally neutral: they can describe the current TCP/HTTP backend and a future high-speed backend without naming a vendor stack or transport implementation.

The capability report is descriptive. It does not select a backend, negotiate with peers, or weaken object correctness semantics.

Capability Fields

Field	Meaning
streaming_read	The backend can open a remote disk reader for read_file_stream.
streaming_write	The backend can open a remote disk writer for create_file or append_file.
streaming_walk_dir	The backend can stream walk_dir responses.
zero_copy_candidate	The backend has an API shape that could avoid an extra user-space payload copy. This is not a promise that every transfer is zero-copy.
registered_memory_required	The backend requires pinned or registered buffers before payload transfer.
ordered_delivery	Bytes for each opened transfer are delivered in order.
max_transfer_size	Optional RustFS-level cap for a single transfer. None means no additional cap beyond the backend/protocol/runtime limits.
fallback_supported	The backend can participate in the behavior-preserving TCP fallback path.

TCP/HTTP Backend

The default TCP/HTTP backend reports only capabilities it actually provides:

Field	TCP/HTTP value	Reason
streaming_read	true	HttpReader streams /rustfs/rpc/read_file_stream responses.
streaming_write	true	HttpWriter streams /rustfs/rpc/put_file_stream request bodies.
streaming_walk_dir	true	HttpReader streams /rustfs/rpc/walk_dir responses.
zero_copy_candidate	false	The current path exposes AsyncRead/AsyncWrite and HTTP body chunks; it copies through normal user-space buffers.
registered_memory_required	false	TCP/HTTP does not require RustFS-managed registered memory.
ordered_delivery	true	Each HTTP request body or response body is consumed as an ordered byte stream.
max_transfer_size	None	RustFS does not impose an extra per-transfer cap at the capability layer.
fallback_supported	true	TCP/HTTP is the behavior-preserving default and fallback path.

Future Backend Fit

A future high-speed backend can be described without changing the meaning of the existing TCP report:

Capability shape	Interpretation
zero_copy_candidate=true, registered_memory_required=true	The backend can only use its fast path with buffers that satisfy registration or pinning rules.
zero_copy_candidate=true, registered_memory_required=false	The backend may expose owned chunks or another lower-copy path without requiring caller-registered memory.
max_transfer_size=Some(n)	The backend has a RustFS-visible transfer size ceiling and callers must split larger transfers or use fallback behavior.
ordered_delivery=false	The backend cannot be used behind the current stream API without an ordering or reassembly layer.

Unsupported or mismatched capabilities must not silently change quorum, integrity verification, retry, or timeout semantics.