pub enum ConnectionError {
+ConnectionError in iroh_quinn - Rust ConnectionError
pub enum ConnectionError {
VersionMismatch,
TransportError(Error),
ConnectionClosed(ConnectionClose),
@@ -20,8 +20,8 @@ and §LocallyClosed
The local application closed the connection
§CidsExhausted
The connection could not be created because not enough of the CID space is available
Try using longer connection IDs.
-Trait Implementations§
Source§impl Clone for ConnectionError
Source§fn clone(&self) -> ConnectionError
Returns a duplicate of the value. Read more1.0.0 · Source§fn clone_from(&mut self, source: &Self)
Performs copy-assignment from source. Read moreSource§impl Debug for ConnectionError
Source§impl Display for ConnectionError
Source§impl Error for ConnectionError
Source§fn source(&self) -> Option<&(dyn Error + 'static)>
Returns the lower-level source of this error, if any. Read more1.0.0 · Source§fn description(&self) -> &str
👎Deprecated since 1.42.0: use the Display impl or to_string()Source§impl From<Close> for ConnectionError
Source§fn from(x: Close) -> ConnectionError
Converts to this type from the input type.Source§impl From<CloseReason> for ConnectionError
Source§fn from(value: CloseReason) -> ConnectionError
Converts to this type from the input type.Source§impl From<ConnectionError> for ReadError
Source§fn from(source: ConnectionError) -> Self
Converts to this type from the input type.Source§impl From<ConnectionError> for ResetError
Source§fn from(source: ConnectionError) -> Self
Converts to this type from the input type.Source§impl From<ConnectionError> for SendDatagramError
Source§fn from(source: ConnectionError) -> Self
Converts to this type from the input type.Source§impl From<ConnectionError> for StoppedError
Source§fn from(source: ConnectionError) -> Self
Converts to this type from the input type.Source§impl From<ConnectionError> for WriteError
Source§fn from(source: ConnectionError) -> Self
Converts to this type from the input type.Source§impl From<Error> for ConnectionError
Source§fn from(source: Error) -> ConnectionError
Converts to this type from the input type.Source§impl PartialEq for ConnectionError
Source§impl Eq for ConnectionError
Source§impl StructuralPartialEq for ConnectionError
Auto Trait Implementations§
§impl !Freeze for ConnectionError
§impl !RefUnwindSafe for ConnectionError
§impl Send for ConnectionError
§impl Sync for ConnectionError
§impl Unpin for ConnectionError
§impl !UnwindSafe for ConnectionError
Blanket Implementations§
Trait Implementations§
Source§impl Clone for ConnectionError
Source§fn clone(&self) -> ConnectionError
Returns a duplicate of the value. Read more1.0.0 · Source§fn clone_from(&mut self, source: &Self)
Performs copy-assignment from source. Read moreSource§impl Debug for ConnectionError
Source§impl Display for ConnectionError
Source§impl Error for ConnectionError
Source§fn source(&self) -> Option<&(dyn Error + 'static)>
Returns the lower-level source of this error, if any. Read more1.0.0 · Source§fn description(&self) -> &str
👎Deprecated since 1.42.0: use the Display impl or to_string()Source§impl From<Close> for ConnectionError
Source§fn from(x: Close) -> ConnectionError
Converts to this type from the input type.Source§impl From<CloseReason> for ConnectionError
Source§fn from(value: CloseReason) -> ConnectionError
Converts to this type from the input type.Source§impl From<ConnectionError> for ReadError
Source§fn from(source: ConnectionError) -> Self
Converts to this type from the input type.Source§impl From<ConnectionError> for ResetError
Source§fn from(source: ConnectionError) -> Self
Converts to this type from the input type.Source§impl From<ConnectionError> for SendDatagramError
Source§fn from(source: ConnectionError) -> Self
Converts to this type from the input type.Source§impl From<ConnectionError> for StoppedError
Source§fn from(source: ConnectionError) -> Self
Converts to this type from the input type.Source§impl From<ConnectionError> for WriteError
Source§fn from(source: ConnectionError) -> Self
Converts to this type from the input type.Source§impl From<Error> for ConnectionError
Source§fn from(source: Error) -> ConnectionError
Converts to this type from the input type.Source§impl PartialEq for ConnectionError
Source§impl Eq for ConnectionError
Source§impl StructuralPartialEq for ConnectionError
Auto Trait Implementations§
§impl !Freeze for ConnectionError
§impl !RefUnwindSafe for ConnectionError
§impl Send for ConnectionError
§impl Sync for ConnectionError
§impl Unpin for ConnectionError
§impl !UnwindSafe for ConnectionError
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read moreSource§impl<T> CloneToUninit for Twhere
diff --git a/pr/255/docs/iroh_quinn_proto/enum.ClosePathError.html b/pr/255/docs/iroh_quinn_proto/enum.ClosePathError.html
index 33de795f5..968bc51b4 100644
--- a/pr/255/docs/iroh_quinn_proto/enum.ClosePathError.html
+++ b/pr/255/docs/iroh_quinn_proto/enum.ClosePathError.html
@@ -1,11 +1,11 @@
-ClosePathError in iroh_quinn_proto - Rust ClosePathError
pub enum ClosePathError {
+ClosePathError in iroh_quinn_proto - Rust ClosePathError
pub enum ClosePathError {
ClosedPath,
LastOpenPath,
}
Expand description
Errors triggered when abandoning a path
Variants§
§ClosedPath
The path is already closed or was never opened
§LastOpenPath
This is the last path, which can not be abandoned
-Trait Implementations§
Source§impl Clone for ClosePathError
Source§fn clone(&self) -> ClosePathError
Returns a duplicate of the value. Read more1.0.0 · Source§fn clone_from(&mut self, source: &Self)
Performs copy-assignment from source. Read moreSource§impl Debug for ClosePathError
Source§impl Display for ClosePathError
Source§impl Error for ClosePathError
1.30.0 · Source§fn source(&self) -> Option<&(dyn Error + 'static)>
Returns the lower-level source of this error, if any. Read more1.0.0 · Source§fn description(&self) -> &str
👎Deprecated since 1.42.0: use the Display impl or to_string()Source§impl PartialEq for ClosePathError
Source§impl Eq for ClosePathError
Source§impl StructuralPartialEq for ClosePathError
Auto Trait Implementations§
§impl Freeze for ClosePathError
§impl RefUnwindSafe for ClosePathError
§impl Send for ClosePathError
§impl Sync for ClosePathError
§impl Unpin for ClosePathError
§impl UnwindSafe for ClosePathError
Blanket Implementations§
Trait Implementations§
Source§impl Clone for ClosePathError
Source§fn clone(&self) -> ClosePathError
Returns a duplicate of the value. Read more1.0.0 · Source§fn clone_from(&mut self, source: &Self)
Performs copy-assignment from source. Read moreSource§impl Debug for ClosePathError
Source§impl Display for ClosePathError
Source§impl Error for ClosePathError
1.30.0 · Source§fn source(&self) -> Option<&(dyn Error + 'static)>
Returns the lower-level source of this error, if any. Read more1.0.0 · Source§fn description(&self) -> &str
👎Deprecated since 1.42.0: use the Display impl or to_string()Source§impl PartialEq for ClosePathError
Source§impl Eq for ClosePathError
Source§impl StructuralPartialEq for ClosePathError
Auto Trait Implementations§
§impl Freeze for ClosePathError
§impl RefUnwindSafe for ClosePathError
§impl Send for ClosePathError
§impl Sync for ClosePathError
§impl Unpin for ClosePathError
§impl UnwindSafe for ClosePathError
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read moreSource§impl<T> CloneToUninit for Twhere
diff --git a/pr/255/docs/iroh_quinn_proto/enum.ConnectionError.html b/pr/255/docs/iroh_quinn_proto/enum.ConnectionError.html
index 85a5829ed..b20802d9d 100644
--- a/pr/255/docs/iroh_quinn_proto/enum.ConnectionError.html
+++ b/pr/255/docs/iroh_quinn_proto/enum.ConnectionError.html
@@ -1,4 +1,4 @@
-ConnectionError in iroh_quinn_proto - Rust ConnectionError
pub enum ConnectionError {
+ConnectionError in iroh_quinn_proto - Rust ConnectionError
pub enum ConnectionError {
VersionMismatch,
TransportError(TransportError),
ConnectionClosed(ConnectionClose),
@@ -20,8 +20,8 @@ and §LocallyClosed
The local application closed the connection
§CidsExhausted
The connection could not be created because not enough of the CID space is available
Try using longer connection IDs.
-Trait Implementations§
Source§impl Clone for ConnectionError
Source§fn clone(&self) -> ConnectionError
Returns a duplicate of the value. Read more1.0.0 · Source§fn clone_from(&mut self, source: &Self)
Performs copy-assignment from source. Read moreSource§impl Debug for ConnectionError
Source§impl Display for ConnectionError
Source§impl Error for ConnectionError
Source§fn source(&self) -> Option<&(dyn Error + 'static)>
Returns the lower-level source of this error, if any. Read more1.0.0 · Source§fn description(&self) -> &str
👎Deprecated since 1.42.0: use the Display impl or to_string()Source§impl From<ConnectionError> for Error
Source§fn from(x: ConnectionError) -> Self
Converts to this type from the input type.Source§impl From<Error> for ConnectionError
Source§fn from(source: TransportError) -> Self
Converts to this type from the input type.Source§impl PartialEq for ConnectionError
Source§impl Eq for ConnectionError
Source§impl StructuralPartialEq for ConnectionError
Auto Trait Implementations§
§impl !Freeze for ConnectionError
§impl !RefUnwindSafe for ConnectionError
§impl Send for ConnectionError
§impl Sync for ConnectionError
§impl Unpin for ConnectionError
§impl !UnwindSafe for ConnectionError
Blanket Implementations§
Trait Implementations§
Source§impl Clone for ConnectionError
Source§fn clone(&self) -> ConnectionError
Returns a duplicate of the value. Read more1.0.0 · Source§fn clone_from(&mut self, source: &Self)
Performs copy-assignment from source. Read moreSource§impl Debug for ConnectionError
Source§impl Display for ConnectionError
Source§impl Error for ConnectionError
Source§fn source(&self) -> Option<&(dyn Error + 'static)>
Returns the lower-level source of this error, if any. Read more1.0.0 · Source§fn description(&self) -> &str
👎Deprecated since 1.42.0: use the Display impl or to_string()Source§impl From<ConnectionError> for Error
Source§fn from(x: ConnectionError) -> Self
Converts to this type from the input type.Source§impl From<Error> for ConnectionError
Source§fn from(source: TransportError) -> Self
Converts to this type from the input type.Source§impl PartialEq for ConnectionError
Source§impl Eq for ConnectionError
Source§impl StructuralPartialEq for ConnectionError
Auto Trait Implementations§
§impl !Freeze for ConnectionError
§impl !RefUnwindSafe for ConnectionError
§impl Send for ConnectionError
§impl Sync for ConnectionError
§impl Unpin for ConnectionError
§impl !UnwindSafe for ConnectionError
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read moreSource§impl<T> CloneToUninit for Twhere
diff --git a/pr/255/docs/iroh_quinn_proto/enum.Event.html b/pr/255/docs/iroh_quinn_proto/enum.Event.html
index c11cee02e..4ee4a9e01 100644
--- a/pr/255/docs/iroh_quinn_proto/enum.Event.html
+++ b/pr/255/docs/iroh_quinn_proto/enum.Event.html
@@ -1,4 +1,4 @@
-Event in iroh_quinn_proto - Rust Event
pub enum Event {
+Event in iroh_quinn_proto - Rust Event
pub enum Event {
HandshakeDataReady,
Connected,
HandshakeConfirmed,
@@ -22,7 +22,7 @@
§DatagramsUnblocked
One or more application datagrams have been sent after blocking
§Path(PathEvent)
(Multi)Path events
§NatTraversal(Event)
Iroh’s nat traversal events
-
Trait Implementations§
Auto Trait Implementations§
§impl !Freeze for Event
§impl !RefUnwindSafe for Event
§impl Send for Event
§impl Sync for Event
§impl Unpin for Event
§impl !UnwindSafe for Event
Blanket Implementations§
Trait Implementations§
Auto Trait Implementations§
§impl !Freeze for Event
§impl !RefUnwindSafe for Event
§impl Send for Event
§impl Sync for Event
§impl Unpin for Event
§impl !UnwindSafe for Event
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read moreSource§impl<T> From<T> for T
Source§fn from(t: T) -> T
Returns the argument unchanged.
diff --git a/pr/255/docs/iroh_quinn_proto/enum.PathError.html b/pr/255/docs/iroh_quinn_proto/enum.PathError.html
index 40b9777ed..c6c1acbf1 100644
--- a/pr/255/docs/iroh_quinn_proto/enum.PathError.html
+++ b/pr/255/docs/iroh_quinn_proto/enum.PathError.html
@@ -1,4 +1,4 @@
-PathError in iroh_quinn_proto - Rust PathError
pub enum PathError {
+PathError in iroh_quinn_proto - Rust PathError
pub enum PathError {
MultipathNotNegotiated,
ServerSideNotAllowed,
MaxPathIdReached,
@@ -12,8 +12,8 @@
§RemoteCidsExhausted
No remote CIDs available to open a new path
§ValidationFailed
Path could not be validated and will be abandoned
§InvalidRemoteAddress(SocketAddr)
The remote address for the path is not supported by the endpoint
-
Trait Implementations§
Source§impl Error for PathError
1.30.0 · Source§fn source(&self) -> Option<&(dyn Error + 'static)>
Returns the lower-level source of this error, if any. Read more1.0.0 · Source§fn description(&self) -> &str
👎Deprecated since 1.42.0: use the Display impl or to_string()Source§impl Copy for PathError
Source§impl Eq for PathError
Source§impl StructuralPartialEq for PathError
Auto Trait Implementations§
§impl Freeze for PathError
§impl RefUnwindSafe for PathError
§impl Send for PathError
§impl Sync for PathError
§impl Unpin for PathError
§impl UnwindSafe for PathError
Blanket Implementations§
Trait Implementations§
Source§impl Error for PathError
1.30.0 · Source§fn source(&self) -> Option<&(dyn Error + 'static)>
Returns the lower-level source of this error, if any. Read more1.0.0 · Source§fn description(&self) -> &str
👎Deprecated since 1.42.0: use the Display impl or to_string()Source§impl Copy for PathError
Source§impl Eq for PathError
Source§impl StructuralPartialEq for PathError
Auto Trait Implementations§
§impl Freeze for PathError
§impl RefUnwindSafe for PathError
§impl Send for PathError
§impl Sync for PathError
§impl Unpin for PathError
§impl UnwindSafe for PathError
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read moreSource§impl<T> CloneToUninit for Twhere
diff --git a/pr/255/docs/iroh_quinn_proto/enum.PathEvent.html b/pr/255/docs/iroh_quinn_proto/enum.PathEvent.html
index a49993ecc..7ce42b7f7 100644
--- a/pr/255/docs/iroh_quinn_proto/enum.PathEvent.html
+++ b/pr/255/docs/iroh_quinn_proto/enum.PathEvent.html
@@ -47,7 +47,7 @@ changes the status.
Fields
§id: PathIdPath over which the observed address was reported, PathId::ZERO when multipath is
not negotiated
§addr: SocketAddrThe address observed by the remote over this path
-
Trait Implementations§
Trait Implementations§
Source§impl Eq for PathEvent
Source§impl StructuralPartialEq for PathEvent
Auto Trait Implementations§
§impl Freeze for PathEvent
§impl RefUnwindSafe for PathEvent
§impl Send for PathEvent
§impl Sync for PathEvent
§impl Unpin for PathEvent
§impl UnwindSafe for PathEvent
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
diff --git a/pr/255/docs/iroh_quinn_proto/struct.Connection.html b/pr/255/docs/iroh_quinn_proto/struct.Connection.html
index 616cf1f9f..d6c8d9ac4 100644
--- a/pr/255/docs/iroh_quinn_proto/struct.Connection.html
+++ b/pr/255/docs/iroh_quinn_proto/struct.Connection.html
@@ -33,7 +33,7 @@ increasing time. Specifically, calling Instant may be interleaved in any order with a
call to handle_event at that same instant; however
events or timeouts with different instants must not be interleaved.
-
Implementations§
Source§impl Connection
Sourcepub fn poll_timeout(&mut self) -> Option<Instant>
Returns the next time at which handle_timeout should be called
+Implementations§
Source§impl Connection
Sourcepub fn poll_timeout(&mut self) -> Option<Instant>
Returns the next time at which handle_timeout should be called
The value returned may change after:
- the application performed some I/O on the connection
@@ -41,17 +41,17 @@ events or timeouts with different instants must not be interleaved.
- a call to
poll_transmit returned Some
- a call was made to
handle_timeout
-Sourcepub fn poll(&mut self) -> Option<Event>
Returns application-facing events
Connections should be polled for events after:
- a call was made to
handle_event
- a call was made to
handle_timeout
-Sourcepub fn poll_endpoint_events(&mut self) -> Option<EndpointEvent>
Return endpoint-facing events
-Sourcepub fn recv_stream(&mut self, id: StreamId) -> RecvStream<'_>
Provide control over streams
-Sourcepub fn send_stream(&mut self, id: StreamId) -> SendStream<'_>
Provide control over streams
-Sourcepub fn open_path_ensure(
+
Sourcepub fn poll_endpoint_events(&mut self) -> Option<EndpointEvent>
Return endpoint-facing events
+Sourcepub fn recv_stream(&mut self, id: StreamId) -> RecvStream<'_>
Provide control over streams
+Sourcepub fn send_stream(&mut self, id: StreamId) -> SendStream<'_>
Provide control over streams
+Sourcepub fn open_path_ensure(
&mut self,
network_path: FourTuple,
initial_status: PathStatus,
@@ -65,7 +65,7 @@ remote addresses.
remote, should it not be known yet. We assume that if we already have a path to the remote,
the OS is likely to use the same interface to talk to said remote.
See also open_path. Returns (path_id, true) if the path already existed. (path_id, false) if was opened.
-
Sourcepub fn open_path(
&mut self,
network_path: FourTuple,
initial_status: PathStatus,
@@ -73,7 +73,7 @@ the OS is likely to use the same interface to talk to said remote.
) -> Result<PathId, PathError>
Opens a new path
Further errors might occur and they will be emitted in PathEvent::LocallyClosed events.
When the path is opened it will be reported as an PathEvent::Opened.
-Sourcepub fn close_path(
+
Sourcepub fn close_path(
&mut self,
now: Instant,
path_id: PathId,
@@ -82,32 +82,32 @@ When the path is opened it will be reported as an Sourcepub fn paths(&self) -> Vec<PathId>
Returns all known paths.
+
Sourcepub fn paths(&self) -> Vec<PathId>
Returns all known paths.
There is no guarantee any of these paths are open or usable.
-Sourcepub fn path_status(&self, path_id: PathId) -> Result<PathStatus, ClosedPath>
Gets the local PathStatus for a known PathId
-Sourcepub fn network_path(&self, path_id: PathId) -> Result<FourTuple, ClosedPath>
Returns the path’s network path represented as a 4-tuple.
-Sourcepub fn set_path_status(
+
Sourcepub fn path_status(&self, path_id: PathId) -> Result<PathStatus, ClosedPath>
Gets the local PathStatus for a known PathId
+Sourcepub fn network_path(&self, path_id: PathId) -> Result<FourTuple, ClosedPath>
Returns the path’s network path represented as a 4-tuple.
+Sourcepub fn set_path_status(
&mut self,
path_id: PathId,
status: PathStatus,
) -> Result<PathStatus, SetPathStatusError>
Sets the PathStatus for a known PathId
Returns the previous path status on success.
-Sourcepub fn remote_path_status(&self, path_id: PathId) -> Option<PathStatus>
Returns the remote path status
-Sourcepub fn set_path_max_idle_timeout(
+
Sourcepub fn remote_path_status(&self, path_id: PathId) -> Option<PathStatus>
Returns the remote path status
+Sourcepub fn set_path_max_idle_timeout(
&mut self,
path_id: PathId,
timeout: Option<Duration>,
) -> Result<Option<Duration>, ClosedPath>
Sets the max_idle_timeout for a specific path
See TransportConfig::default_path_max_idle_timeout for details.
Returns the previous value of the setting.
-Sourcepub fn set_path_keep_alive_interval(
+
Sourcepub fn set_path_keep_alive_interval(
&mut self,
path_id: PathId,
interval: Option<Duration>,
) -> Result<Option<Duration>, ClosedPath>
Sets the keep_alive_interval for a specific path
See TransportConfig::default_path_keep_alive_interval for details.
Returns the previous value of the setting.
-Sourcepub fn poll_transmit(
+
Sourcepub fn poll_transmit(
&mut self,
now: Instant,
max_datagrams: NonZeroUsize,
@@ -121,60 +121,60 @@ for a path that was never opened locally.
max_datagrams specifies how many datagrams can be returned inside a
single Transmit using GSO. This must be at least 1.
-
Sourcepub fn handle_event(&mut self, event: ConnectionEvent)
Process ConnectionEvents generated by the associated Endpoint
+Sourcepub fn handle_event(&mut self, event: ConnectionEvent)
Process ConnectionEvents generated by the associated Endpoint
Will execute protocol logic upon receipt of a connection event, in turn preparing signals
(including application Events, EndpointEvents and outgoing datagrams) that should be
extracted through the relevant methods.
-Sourcepub fn handle_timeout(&mut self, now: Instant)
Process timer expirations
+Sourcepub fn handle_timeout(&mut self, now: Instant)
Process timer expirations
Executes protocol logic, potentially preparing signals (including application Events,
EndpointEvents and outgoing datagrams) that should be extracted through the relevant
methods.
It is most efficient to call this immediately after the system clock reaches the latest
Instant that was output by poll_timeout; however spurious extra calls will simply
no-op and therefore are safe.
-Sourcepub fn close(&mut self, now: Instant, error_code: VarInt, reason: Bytes)
Close a connection immediately
+Sourcepub fn close(&mut self, now: Instant, error_code: VarInt, reason: Bytes)
Close a connection immediately
This does not ensure delivery of outstanding data. It is the application’s responsibility to
call this only when all important communications have been completed, e.g. by calling
SendStream::finish on outstanding streams and waiting for the corresponding
StreamEvent::Finished event.
If Streams::send_streams returns 0, all outstanding stream data has been
delivered. There may still be data from the peer that has not been received.
-Sourcepub fn stats(&mut self) -> ConnectionStats
Returns connection statistics
-Sourcepub fn path_stats(&mut self, path_id: PathId) -> Option<PathStats>
Returns path statistics
-Sourcepub fn stats(&mut self) -> ConnectionStats
Returns connection statistics
+Sourcepub fn path_stats(&mut self, path_id: PathId) -> Option<PathStats>
Returns path statistics
+Sourcepub fn ping(&mut self)
Ping the remote endpoint
Causes an ACK-eliciting packet to be transmitted on the connection.
-Sourcepub fn ping_path(&mut self, path: PathId) -> Result<(), ClosedPath>
Ping the remote endpoint over a specific path
+Sourcepub fn ping_path(&mut self, path: PathId) -> Result<(), ClosedPath>
Ping the remote endpoint over a specific path
Causes an ACK-eliciting packet to be transmitted on the path.
-Sourcepub fn force_key_update(&mut self)
Update traffic keys spontaneously
+Sourcepub fn force_key_update(&mut self)
Update traffic keys spontaneously
This can be useful for testing key updates, as they otherwise only happen infrequently.
-Sourcepub fn crypto_session(&self) -> &dyn Session
Get a session reference
-Sourcepub fn is_handshaking(&self) -> bool
Whether the connection is in the process of being established
+Sourcepub fn crypto_session(&self) -> &dyn Session
Get a session reference
+Sourcepub fn is_handshaking(&self) -> bool
Whether the connection is in the process of being established
If this returns false, the connection may be either established or closed, signaled by the
emission of a Connected or ConnectionLost message respectively.
-Sourcepub fn is_closed(&self) -> bool
Whether the connection is closed
Closed connections cannot transport any further data. A connection becomes closed when
either peer application intentionally closes it, or when either transport layer detects an
error such as a time-out or certificate validation failure.
A ConnectionLost event is emitted with details when the connection becomes closed.
-Sourcepub fn is_drained(&self) -> bool
Whether there is no longer any need to keep the connection around
+Sourcepub fn is_drained(&self) -> bool
Whether there is no longer any need to keep the connection around
Closed connections become drained after a brief timeout to absorb any remaining in-flight
packets from the peer. All drained connections have been closed.
-Sourcepub fn accepted_0rtt(&self) -> bool
For clients, if the peer accepted the 0-RTT data packets
+Sourcepub fn accepted_0rtt(&self) -> bool
For clients, if the peer accepted the 0-RTT data packets
The value is meaningless until after the handshake completes.
-Sourcepub fn has_pending_retransmits(&self) -> bool
Whether there are any pending retransmits
-Sourcepub fn path_observed_address(
+
Sourcepub fn has_pending_retransmits(&self) -> bool
Whether there are any pending retransmits
+Sourcepub fn path_observed_address(
&self,
path_id: PathId,
) -> Result<Option<SocketAddr>, ClosedPath>
Get the address observed by the remote over the given path
-Sourcepub fn rtt(&self, path_id: PathId) -> Option<Duration>
Current best estimate of this connection’s latency (round-trip-time)
-Sourcepub fn congestion_state(&self, path_id: PathId) -> Option<&dyn Controller>
Current state of this connection’s congestion controller, for debugging purposes
-Sourcepub fn set_max_concurrent_streams(&mut self, dir: Dir, count: VarInt)
Modify the number of remotely initiated streams that may be concurrently open
+Sourcepub fn rtt(&self, path_id: PathId) -> Option<Duration>
Current best estimate of this connection’s latency (round-trip-time)
+Sourcepub fn congestion_state(&self, path_id: PathId) -> Option<&dyn Controller>
Current state of this connection’s congestion controller, for debugging purposes
+Sourcepub fn set_max_concurrent_streams(&mut self, dir: Dir, count: VarInt)
Modify the number of remotely initiated streams that may be concurrently open
No streams may be opened by the peer unless fewer than count are already open. Large
counts increase both minimum and worst-case memory consumption.
-Sourcepub fn set_max_concurrent_paths(
+
Sourcepub fn set_max_concurrent_paths(
&mut self,
now: Instant,
count: NonZeroU32,
@@ -185,36 +185,36 @@ actively reduce paths they must be closed using TransportConfig) this can not enable
multipath and will fail.
-
Sourcepub fn max_concurrent_streams(&self, dir: Dir) -> u64
Current number of remotely initiated streams that may be concurrently open
+Sourcepub fn max_concurrent_streams(&self, dir: Dir) -> u64
Current number of remotely initiated streams that may be concurrently open
If the target for this limit is reduced using set_max_concurrent_streams,
it will not change immediately, even if fewer streams are open. Instead, it will
decrement by one for each time a remotely initiated stream of matching directionality is closed.
-Sourcepub fn set_send_window(&mut self, send_window: u64)
Sourcepub fn set_receive_window(&mut self, receive_window: VarInt)
Sourcepub fn is_multipath_negotiated(&self) -> bool
Whether the Multipath for QUIC extension is enabled.
+Sourcepub fn set_send_window(&mut self, send_window: u64)
Sourcepub fn set_receive_window(&mut self, receive_window: VarInt)
Sourcepub fn is_multipath_negotiated(&self) -> bool
Whether the Multipath for QUIC extension is enabled.
Multipath is only enabled after the handshake is completed and if it was enabled by both
peers.
-Sourcepub fn local_address_changed(&mut self)
Handle a change in the local address, i.e. an active migration
-Sourcepub fn current_mtu(&self) -> u16
Storage size required for the largest packet that can be transmitted on all currently
+
Sourcepub fn local_address_changed(&mut self)
Handle a change in the local address, i.e. an active migration
+Sourcepub fn current_mtu(&self) -> u16
Storage size required for the largest packet that can be transmitted on all currently
available paths
Buffers passed to Connection::poll_transmit should be at least this large.
When multipath is enabled, this value is the minimum MTU across all available paths.
-Sourcepub fn add_nat_traversal_address(
+
Sourcepub fn add_nat_traversal_address(
&mut self,
address: SocketAddr,
) -> Result<(), Error>
Add addresses the local endpoint considers are reachable for nat traversal
-Sourcepub fn remove_nat_traversal_address(
+
Sourcepub fn remove_nat_traversal_address(
&mut self,
address: SocketAddr,
) -> Result<(), Error>
Removes an address the endpoing no longer considers reachable for nat traversal
Addresses not present in the set will be silently ignored.
-Sourcepub fn get_local_nat_traversal_addresses(
+
Sourcepub fn get_local_nat_traversal_addresses(
&self,
) -> Result<Vec<SocketAddr>, Error>
Get the current local nat traversal addresses
-Sourcepub fn get_remote_nat_traversal_addresses(
+
Sourcepub fn get_remote_nat_traversal_addresses(
&self,
) -> Result<Vec<SocketAddr>, Error>
Get the currently advertised nat traversal addresses by the server
-Sourcepub fn initiate_nat_traversal_round(
+
Sourcepub fn initiate_nat_traversal_round(
&mut self,
now: Instant,
) -> Result<Vec<SocketAddr>, Error>
pub fn initiate_nat_traversal_round( &mut self, now: Instant, ) -> Result<Vec<SocketAddr>, Error>
Initiates a new nat traversal round
@@ -224,7 +224,7 @@ initiated, the previous one is cancelled, and paths that have not been opened arReturns the server addresses that are now being probed. If addresses fail due to spurious errors, these might succeed later and not be returned in this set.
-Trait Implementations§
Auto Trait Implementations§
impl !Freeze for Connection
impl !RefUnwindSafe for Connection
impl Send for Connection
impl Sync for Connection
impl Unpin for Connection
impl !UnwindSafe for Connection
Blanket Implementations§
Trait Implementations§
Auto Trait Implementations§
impl !Freeze for Connection
impl !RefUnwindSafe for Connection
impl Send for Connection
impl Sync for Connection
impl Unpin for Connection
impl !UnwindSafe for Connection
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> From<T> for T
impl<T> From<T> for T
Source§fn from(t: T) -> T
fn from(t: T) -> T
Returns the argument unchanged.
diff --git a/pr/255/docs/iroh_quinn_proto/struct.MultipathNotNegotiated.html b/pr/255/docs/iroh_quinn_proto/struct.MultipathNotNegotiated.html index 2d0745307..70dc9443e 100644 --- a/pr/255/docs/iroh_quinn_proto/struct.MultipathNotNegotiated.html +++ b/pr/255/docs/iroh_quinn_proto/struct.MultipathNotNegotiated.html @@ -1,5 +1,5 @@ -MultipathNotNegotiated
pub struct MultipathNotNegotiated { /* private fields */ }Expand description
Error when the multipath extension was not negotiated, but attempted to be used.
-Trait Implementations§
Source§impl Clone for MultipathNotNegotiated
impl Clone for MultipathNotNegotiated
Source§fn clone(&self) -> MultipathNotNegotiated
fn clone(&self) -> MultipathNotNegotiated
Returns a duplicate of the value. Read more
1.0.0 · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source. Read moreSource§impl Debug for MultipathNotNegotiated
impl Debug for MultipathNotNegotiated
Source§impl Display for MultipathNotNegotiated
impl Display for MultipathNotNegotiated
Source§impl Error for MultipathNotNegotiated
impl Error for MultipathNotNegotiated
1.30.0 · Source§fn source(&self) -> Option<&(dyn Error + 'static)>
fn source(&self) -> Option<&(dyn Error + 'static)>
Returns the lower-level source of this error, if any. Read more
1.0.0 · Source§fn description(&self) -> &str
fn description(&self) -> &str
👎Deprecated since 1.42.0: use the Display impl or to_string()
impl Copy for MultipathNotNegotiated
Auto Trait Implementations§
impl Freeze for MultipathNotNegotiated
impl RefUnwindSafe for MultipathNotNegotiated
impl Send for MultipathNotNegotiated
impl Sync for MultipathNotNegotiated
impl Unpin for MultipathNotNegotiated
impl UnwindSafe for MultipathNotNegotiated
Blanket Implementations§
Source§impl<T> Any for Twhere
+MultipathNotNegotiated in iroh_quinn_proto - Rust MultipathNotNegotiated
pub struct MultipathNotNegotiated { /* private fields */ }
Expand description
Error when the multipath extension was not negotiated, but attempted to be used.
+Trait Implementations§
Source§impl Clone for MultipathNotNegotiated
Source§fn clone(&self) -> MultipathNotNegotiated
Returns a duplicate of the value. Read more1.0.0 · Source§fn clone_from(&mut self, source: &Self)
Performs copy-assignment from source. Read moreSource§impl Debug for MultipathNotNegotiated
Source§impl Display for MultipathNotNegotiated
Source§impl Error for MultipathNotNegotiated
1.30.0 · Source§fn source(&self) -> Option<&(dyn Error + 'static)>
Returns the lower-level source of this error, if any. Read more1.0.0 · Source§fn description(&self) -> &str
👎Deprecated since 1.42.0: use the Display impl or to_string()Source§impl Copy for MultipathNotNegotiated
Auto Trait Implementations§
§impl Freeze for MultipathNotNegotiated
§impl RefUnwindSafe for MultipathNotNegotiated
§impl Send for MultipathNotNegotiated
§impl Sync for MultipathNotNegotiated
§impl Unpin for MultipathNotNegotiated
§impl UnwindSafe for MultipathNotNegotiated
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read moreSource§impl<T> CloneToUninit for Twhere
diff --git a/pr/255/docs/iroh_quinn_proto/struct.TransportError.html b/pr/255/docs/iroh_quinn_proto/struct.TransportError.html
index c4be43306..b6dc15d42 100644
--- a/pr/255/docs/iroh_quinn_proto/struct.TransportError.html
+++ b/pr/255/docs/iroh_quinn_proto/struct.TransportError.html
@@ -11,7 +11,7 @@
§reason: StringHuman-readable explanation of the reason
§crypto: Option<Arc<dyn Error + Send + Sync>>An underlying crypto (e.g. TLS) layer error
Implementations§
Trait Implementations§
Source§impl Error for Error
1.30.0 · Source§fn source(&self) -> Option<&(dyn Error + 'static)>
Returns the lower-level source of this error, if any. Read more1.0.0 · Source§fn description(&self) -> &str
👎Deprecated since 1.42.0: use the Display impl or to_string()Source§impl From<Error> for ConnectionClose
Source§fn from(x: TransportError) -> Self
Converts to this type from the input type.Source§impl From<Error> for ConnectionError
Source§fn from(source: TransportError) -> Self
Converts to this type from the input type.Source§impl From<Error> for TransportError
Trait Implementations§
Source§impl Error for Error
1.30.0 · Source§fn source(&self) -> Option<&(dyn Error + 'static)>
Returns the lower-level source of this error, if any. Read more1.0.0 · Source§fn description(&self) -> &str
👎Deprecated since 1.42.0: use the Display impl or to_string()Source§impl From<Error> for ConnectionClose
Source§fn from(x: TransportError) -> Self
Converts to this type from the input type.Source§impl From<Error> for ConnectionError
Source§fn from(source: TransportError) -> Self
Converts to this type from the input type.Source§impl From<Error> for TransportError
Source§impl Eq for Error
Auto Trait Implementations§
§impl Freeze for Error
§impl !RefUnwindSafe for Error
§impl Send for Error
§impl Sync for Error
§impl Unpin for Error
§impl !UnwindSafe for Error
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
diff --git a/pr/255/docs/src/iroh_quinn_proto/connection/mod.rs.html b/pr/255/docs/src/iroh_quinn_proto/connection/mod.rs.html
index 81ec7afd0..e1b5a58ea 100644
--- a/pr/255/docs/src/iroh_quinn_proto/connection/mod.rs.html
+++ b/pr/255/docs/src/iroh_quinn_proto/connection/mod.rs.html
@@ -343,826 +343,826 @@
343/// Return value for `poll_transmit_path_space`.
344#[derive(Debug)]
345enum PollPathSpaceStatus {
-346 /// Nothing to send on this space, continue to the next one.
-347 NextSpace,
-348 /// We have something to send, and data has been accumulated
-349 /// on the passed in `transmit`.
-350 Send { last_packet_number: Option<u64> },
-351 /// A transmit is ready to be sent out.
-352 SendTransmit {
-353 /// The transmit to send.
-354 transmit: Transmit,
-355 },
-356 /// Nothing to send currently.
-357 NothingToSend {
-358 /// Set to `true` if we consider the current inability to send something because of congestion control
-359 congestion_blocked: bool,
-360 },
-361}
-362
-363impl Connection {
-364 pub(crate) fn new(
-365 endpoint_config: Arc<EndpointConfig>,
-366 config: Arc<TransportConfig>,
-367 init_cid: ConnectionId,
-368 loc_cid: ConnectionId,
-369 rem_cid: ConnectionId,
-370 network_path: FourTuple,
-371 crypto: Box<dyn crypto::Session>,
-372 cid_gen: &dyn ConnectionIdGenerator,
-373 now: Instant,
-374 version: u32,
-375 allow_mtud: bool,
-376 rng_seed: [u8; 32],
-377 side_args: SideArgs,
-378 qlog: QlogSink,
-379 ) -> Self {
-380 let pref_addr_cid = side_args.pref_addr_cid();
-381 let path_validated = side_args.path_validated();
-382 let connection_side = ConnectionSide::from(side_args);
-383 let side = connection_side.side();
-384 let mut rng = StdRng::from_seed(rng_seed);
-385 let initial_space = {
-386 let mut space = PacketSpace::new(now, SpaceId::Initial, &mut rng);
-387 space.crypto = Some(crypto.initial_keys(init_cid, side));
-388 space
-389 };
-390 let handshake_space = PacketSpace::new(now, SpaceId::Handshake, &mut rng);
-391 #[cfg(test)]
-392 let data_space = match config.deterministic_packet_numbers {
-393 true => PacketSpace::new_deterministic(now, SpaceId::Data),
-394 false => PacketSpace::new(now, SpaceId::Data, &mut rng),
-395 };
-396 #[cfg(not(test))]
-397 let data_space = PacketSpace::new(now, SpaceId::Data, &mut rng);
-398 let state = State::handshake(state::Handshake {
-399 rem_cid_set: side.is_server(),
-400 expected_token: Bytes::new(),
-401 client_hello: None,
-402 allow_server_migration: side.is_client(),
-403 });
-404 let local_cid_state = FxHashMap::from_iter([(
-405 PathId::ZERO,
-406 CidState::new(
-407 cid_gen.cid_len(),
-408 cid_gen.cid_lifetime(),
-409 now,
-410 if pref_addr_cid.is_some() { 2 } else { 1 },
-411 ),
-412 )]);
-413
-414 let mut path = PathData::new(network_path, allow_mtud, None, 0, now, &config);
-415 // TODO(@divma): consider if we want to delay this until the path is validated
-416 path.open = true;
-417 let mut this = Self {
-418 endpoint_config,
-419 crypto,
-420 handshake_cid: loc_cid,
-421 rem_handshake_cid: rem_cid,
-422 local_cid_state,
-423 paths: BTreeMap::from_iter([(
-424 PathId::ZERO,
-425 PathState {
-426 data: path,
-427 prev: None,
-428 },
-429 )]),
-430 path_generation_counter: 0,
-431 allow_mtud,
-432 state,
-433 side: connection_side,
-434 zero_rtt_enabled: false,
-435 zero_rtt_crypto: None,
-436 key_phase: false,
-437 // A small initial key phase size ensures peers that don't handle key updates correctly
-438 // fail sooner rather than later. It's okay for both peers to do this, as the first one
-439 // to perform an update will reset the other's key phase size in `update_keys`, and a
-440 // simultaneous key update by both is just like a regular key update with a really fast
-441 // response. Inspired by quic-go's similar behavior of performing the first key update
-442 // at the 100th short-header packet.
-443 key_phase_size: rng.random_range(10..1000),
-444 peer_params: TransportParameters::default(),
-445 orig_rem_cid: rem_cid,
-446 initial_dst_cid: init_cid,
-447 retry_src_cid: None,
-448 events: VecDeque::new(),
-449 endpoint_events: VecDeque::new(),
-450 spin_enabled: config.allow_spin && rng.random_ratio(7, 8),
-451 spin: false,
-452 spaces: [initial_space, handshake_space, data_space],
-453 highest_space: SpaceId::Initial,
-454 prev_crypto: None,
-455 next_crypto: None,
-456 accepted_0rtt: false,
-457 permit_idle_reset: true,
-458 idle_timeout: match config.max_idle_timeout {
-459 None | Some(VarInt(0)) => None,
-460 Some(dur) => Some(Duration::from_millis(dur.0)),
-461 },
-462 timers: TimerTable::default(),
-463 authentication_failures: 0,
-464 close: false,
-465
-466 ack_frequency: AckFrequencyState::new(get_max_ack_delay(
-467 &TransportParameters::default(),
-468 )),
-469
-470 app_limited: false,
-471 receiving_ecn: false,
-472 total_authed_packets: 0,
+346 /// We have something to send, and data has been accumulated
+347 /// on the passed in `transmit`.
+348 Send { last_packet_number: Option<u64> },
+349 /// A transmit is ready to be sent out.
+350 SendTransmit {
+351 /// The transmit to send.
+352 transmit: Transmit,
+353 },
+354 /// Nothing to send currently on this space/path.
+355 NothingToSend {
+356 /// Set to `true` if we consider the current inability to send something because of congestion control
+357 congestion_blocked: bool,
+358 },
+359}
+360
+361impl Connection {
+362 pub(crate) fn new(
+363 endpoint_config: Arc<EndpointConfig>,
+364 config: Arc<TransportConfig>,
+365 init_cid: ConnectionId,
+366 loc_cid: ConnectionId,
+367 rem_cid: ConnectionId,
+368 network_path: FourTuple,
+369 crypto: Box<dyn crypto::Session>,
+370 cid_gen: &dyn ConnectionIdGenerator,
+371 now: Instant,
+372 version: u32,
+373 allow_mtud: bool,
+374 rng_seed: [u8; 32],
+375 side_args: SideArgs,
+376 qlog: QlogSink,
+377 ) -> Self {
+378 let pref_addr_cid = side_args.pref_addr_cid();
+379 let path_validated = side_args.path_validated();
+380 let connection_side = ConnectionSide::from(side_args);
+381 let side = connection_side.side();
+382 let mut rng = StdRng::from_seed(rng_seed);
+383 let initial_space = {
+384 let mut space = PacketSpace::new(now, SpaceId::Initial, &mut rng);
+385 space.crypto = Some(crypto.initial_keys(init_cid, side));
+386 space
+387 };
+388 let handshake_space = PacketSpace::new(now, SpaceId::Handshake, &mut rng);
+389 #[cfg(test)]
+390 let data_space = match config.deterministic_packet_numbers {
+391 true => PacketSpace::new_deterministic(now, SpaceId::Data),
+392 false => PacketSpace::new(now, SpaceId::Data, &mut rng),
+393 };
+394 #[cfg(not(test))]
+395 let data_space = PacketSpace::new(now, SpaceId::Data, &mut rng);
+396 let state = State::handshake(state::Handshake {
+397 rem_cid_set: side.is_server(),
+398 expected_token: Bytes::new(),
+399 client_hello: None,
+400 allow_server_migration: side.is_client(),
+401 });
+402 let local_cid_state = FxHashMap::from_iter([(
+403 PathId::ZERO,
+404 CidState::new(
+405 cid_gen.cid_len(),
+406 cid_gen.cid_lifetime(),
+407 now,
+408 if pref_addr_cid.is_some() { 2 } else { 1 },
+409 ),
+410 )]);
+411
+412 let mut path = PathData::new(network_path, allow_mtud, None, 0, now, &config);
+413 // TODO(@divma): consider if we want to delay this until the path is validated
+414 path.open = true;
+415 let mut this = Self {
+416 endpoint_config,
+417 crypto,
+418 handshake_cid: loc_cid,
+419 rem_handshake_cid: rem_cid,
+420 local_cid_state,
+421 paths: BTreeMap::from_iter([(
+422 PathId::ZERO,
+423 PathState {
+424 data: path,
+425 prev: None,
+426 },
+427 )]),
+428 path_generation_counter: 0,
+429 allow_mtud,
+430 state,
+431 side: connection_side,
+432 zero_rtt_enabled: false,
+433 zero_rtt_crypto: None,
+434 key_phase: false,
+435 // A small initial key phase size ensures peers that don't handle key updates correctly
+436 // fail sooner rather than later. It's okay for both peers to do this, as the first one
+437 // to perform an update will reset the other's key phase size in `update_keys`, and a
+438 // simultaneous key update by both is just like a regular key update with a really fast
+439 // response. Inspired by quic-go's similar behavior of performing the first key update
+440 // at the 100th short-header packet.
+441 key_phase_size: rng.random_range(10..1000),
+442 peer_params: TransportParameters::default(),
+443 orig_rem_cid: rem_cid,
+444 initial_dst_cid: init_cid,
+445 retry_src_cid: None,
+446 events: VecDeque::new(),
+447 endpoint_events: VecDeque::new(),
+448 spin_enabled: config.allow_spin && rng.random_ratio(7, 8),
+449 spin: false,
+450 spaces: [initial_space, handshake_space, data_space],
+451 highest_space: SpaceId::Initial,
+452 prev_crypto: None,
+453 next_crypto: None,
+454 accepted_0rtt: false,
+455 permit_idle_reset: true,
+456 idle_timeout: match config.max_idle_timeout {
+457 None | Some(VarInt(0)) => None,
+458 Some(dur) => Some(Duration::from_millis(dur.0)),
+459 },
+460 timers: TimerTable::default(),
+461 authentication_failures: 0,
+462 close: false,
+463
+464 ack_frequency: AckFrequencyState::new(get_max_ack_delay(
+465 &TransportParameters::default(),
+466 )),
+467
+468 app_limited: false,
+469 receiving_ecn: false,
+470 total_authed_packets: 0,
+471
+472 next_observed_addr_seq_no: 0u32.into(),
473
-474 next_observed_addr_seq_no: 0u32.into(),
-475
-476 streams: StreamsState::new(
-477 side,
-478 config.max_concurrent_uni_streams,
-479 config.max_concurrent_bidi_streams,
-480 config.send_window,
-481 config.receive_window,
-482 config.stream_receive_window,
-483 ),
-484 datagrams: DatagramState::default(),
-485 config,
-486 rem_cids: FxHashMap::from_iter([(PathId::ZERO, CidQueue::new(rem_cid))]),
-487 rng,
-488 stats: ConnectionStats::default(),
-489 path_stats: Default::default(),
-490 version,
-491
-492 // peer params are not yet known, so multipath is not enabled
-493 max_concurrent_paths: NonZeroU32::MIN,
-494 local_max_path_id: PathId::ZERO,
-495 remote_max_path_id: PathId::ZERO,
-496 max_path_id_with_cids: PathId::ZERO,
-497 abandoned_paths: Default::default(),
-498
-499 // iroh's nat traversal
-500 iroh_hp: Default::default(),
-501 qlog,
-502 };
-503 if path_validated {
-504 this.on_path_validated(PathId::ZERO);
-505 }
-506 if side.is_client() {
-507 // Kick off the connection
-508 this.write_crypto();
-509 this.init_0rtt(now);
-510 }
-511 this.qlog
-512 .emit_tuple_assigned(PathId::ZERO, network_path, now);
-513 this
-514 }
-515
-516 /// Returns the next time at which `handle_timeout` should be called
-517 ///
-518 /// The value returned may change after:
-519 /// - the application performed some I/O on the connection
-520 /// - a call was made to `handle_event`
-521 /// - a call to `poll_transmit` returned `Some`
-522 /// - a call was made to `handle_timeout`
-523 #[must_use]
-524 pub fn poll_timeout(&mut self) -> Option<Instant> {
-525 self.timers.peek()
-526 }
-527
-528 /// Returns application-facing events
-529 ///
-530 /// Connections should be polled for events after:
-531 /// - a call was made to `handle_event`
-532 /// - a call was made to `handle_timeout`
-533 #[must_use]
-534 pub fn poll(&mut self) -> Option<Event> {
-535 if let Some(x) = self.events.pop_front() {
-536 return Some(x);
-537 }
-538
-539 if let Some(event) = self.streams.poll() {
-540 return Some(Event::Stream(event));
-541 }
-542
-543 if let Some(reason) = self.state.take_error() {
-544 return Some(Event::ConnectionLost { reason });
-545 }
-546
-547 None
-548 }
-549
-550 /// Return endpoint-facing events
-551 #[must_use]
-552 pub fn poll_endpoint_events(&mut self) -> Option<EndpointEvent> {
-553 self.endpoint_events.pop_front().map(EndpointEvent)
-554 }
-555
-556 /// Provide control over streams
-557 #[must_use]
-558 pub fn streams(&mut self) -> Streams<'_> {
-559 Streams {
-560 state: &mut self.streams,
-561 conn_state: &self.state,
-562 }
-563 }
-564
-565 /// Provide control over streams
-566 #[must_use]
-567 pub fn recv_stream(&mut self, id: StreamId) -> RecvStream<'_> {
-568 assert!(id.dir() == Dir::Bi || id.initiator() != self.side.side());
-569 RecvStream {
-570 id,
-571 state: &mut self.streams,
-572 pending: &mut self.spaces[SpaceId::Data].pending,
-573 }
-574 }
-575
-576 /// Provide control over streams
-577 #[must_use]
-578 pub fn send_stream(&mut self, id: StreamId) -> SendStream<'_> {
-579 assert!(id.dir() == Dir::Bi || id.initiator() == self.side.side());
-580 SendStream {
-581 id,
-582 state: &mut self.streams,
-583 pending: &mut self.spaces[SpaceId::Data].pending,
-584 conn_state: &self.state,
-585 }
-586 }
-587
-588 /// Opens a new path only if no path on the same network path currently exists.
-589 ///
-590 /// This comparison will use [`FourTuple::is_probably_same_path`] on the given `network_path`
-591 /// and pass it existing path's network paths.
-592 ///
-593 /// This means that you can pass `local_ip: None` to make the comparison only compare
-594 /// remote addresses.
-595 ///
-596 /// This avoids having to guess which local interface will be used to communicate with the
-597 /// remote, should it not be known yet. We assume that if we already have a path to the remote,
-598 /// the OS is likely to use the same interface to talk to said remote.
-599 ///
-600 /// See also [`open_path`]. Returns `(path_id, true)` if the path already existed. `(path_id,
-601 /// false)` if was opened.
-602 ///
-603 /// [`open_path`]: Connection::open_path
-604 pub fn open_path_ensure(
-605 &mut self,
-606 network_path: FourTuple,
-607 initial_status: PathStatus,
-608 now: Instant,
-609 ) -> Result<(PathId, bool), PathError> {
-610 let existing_open_path = self.paths.iter().find(|(id, path)| {
-611 network_path.is_probably_same_path(&path.data.network_path)
-612 && !self.abandoned_paths.contains(*id)
-613 });
-614 match existing_open_path {
-615 Some((path_id, _state)) => Ok((*path_id, true)),
-616 None => Ok((self.open_path(network_path, initial_status, now)?, false)),
-617 }
-618 }
-619
-620 /// Opens a new path
-621 ///
-622 /// Further errors might occur and they will be emitted in [`PathEvent::LocallyClosed`] events.
-623 /// When the path is opened it will be reported as an [`PathEvent::Opened`].
-624 pub fn open_path(
-625 &mut self,
-626 network_path: FourTuple,
-627 initial_status: PathStatus,
-628 now: Instant,
-629 ) -> Result<PathId, PathError> {
-630 if !self.is_multipath_negotiated() {
-631 return Err(PathError::MultipathNotNegotiated);
-632 }
-633 if self.side().is_server() {
-634 return Err(PathError::ServerSideNotAllowed);
-635 }
-636
-637 let max_abandoned = self.abandoned_paths.iter().max().copied();
-638 let max_used = self.paths.keys().last().copied();
-639 let path_id = max_abandoned
-640 .max(max_used)
-641 .unwrap_or(PathId::ZERO)
-642 .saturating_add(1u8);
-643
-644 if Some(path_id) > self.max_path_id() {
-645 return Err(PathError::MaxPathIdReached);
-646 }
-647 if path_id > self.remote_max_path_id {
-648 self.spaces[SpaceId::Data].pending.paths_blocked = true;
-649 return Err(PathError::MaxPathIdReached);
-650 }
-651 if self.rem_cids.get(&path_id).map(CidQueue::active).is_none() {
-652 self.spaces[SpaceId::Data]
-653 .pending
-654 .path_cids_blocked
-655 .insert(path_id);
-656 return Err(PathError::RemoteCidsExhausted);
-657 }
-658
-659 let path = self.ensure_path(path_id, network_path, now, None);
-660 path.status.local_update(initial_status);
-661
-662 Ok(path_id)
-663 }
-664
-665 /// Closes a path by sending a PATH_ABANDON frame
-666 ///
-667 /// This will not allow closing the last path. It does allow closing paths which have
-668 /// not yet been opened, as e.g. is the case when receiving a PATH_ABANDON from the peer
-669 /// for a path that was never opened locally.
-670 pub fn close_path(
-671 &mut self,
-672 now: Instant,
-673 path_id: PathId,
-674 error_code: VarInt,
-675 ) -> Result<(), ClosePathError> {
-676 if self.abandoned_paths.contains(&path_id)
-677 || Some(path_id) > self.max_path_id()
-678 || !self.paths.contains_key(&path_id)
-679 {
-680 return Err(ClosePathError::ClosedPath);
-681 }
-682 if self
-683 .paths
-684 .iter()
-685 // Would there be any remaining, non-abandoned, validated paths
-686 .any(|(id, path)| {
-687 *id != path_id && !self.abandoned_paths.contains(id) && path.data.validated
-688 })
-689 .not()
-690 {
-691 return Err(ClosePathError::LastOpenPath);
-692 }
-693
-694 // Send PATH_ABANDON
-695 self.spaces[SpaceId::Data]
-696 .pending
-697 .path_abandon
-698 .insert(path_id, error_code.into());
-699
-700 // Remove pending NEW CIDs for this path
-701 let pending_space = &mut self.spaces[SpaceId::Data].pending;
-702 pending_space.new_cids.retain(|cid| cid.path_id != path_id);
-703 pending_space.path_cids_blocked.retain(|&id| id != path_id);
-704 pending_space.path_status.retain(|&id| id != path_id);
-705
-706 // Cleanup retransmits across ALL paths (CIDs for path_id may have been transmitted on other paths)
-707 for space in self.spaces[SpaceId::Data].iter_paths_mut() {
-708 for sent_packet in space.sent_packets.values_mut() {
-709 if let Some(retransmits) = sent_packet.retransmits.get_mut() {
-710 retransmits.new_cids.retain(|cid| cid.path_id != path_id);
-711 retransmits.path_cids_blocked.retain(|&id| id != path_id);
-712 retransmits.path_status.retain(|&id| id != path_id);
-713 }
-714 }
-715 }
-716
-717 // Consider remotely issued CIDs as retired.
-718 // Technically we don't have to do this just yet. We only need to do this *after*
-719 // the ABANDON_PATH frame is sent, allowing us to still send it on the
-720 // to-be-abandoned path. However it is recommended to send it on another path, and
-721 // we do not allow abandoning the last path anyway.
-722 self.rem_cids.remove(&path_id);
-723 self.endpoint_events
-724 .push_back(EndpointEventInner::RetireResetToken(path_id));
+474 streams: StreamsState::new(
+475 side,
+476 config.max_concurrent_uni_streams,
+477 config.max_concurrent_bidi_streams,
+478 config.send_window,
+479 config.receive_window,
+480 config.stream_receive_window,
+481 ),
+482 datagrams: DatagramState::default(),
+483 config,
+484 rem_cids: FxHashMap::from_iter([(PathId::ZERO, CidQueue::new(rem_cid))]),
+485 rng,
+486 stats: ConnectionStats::default(),
+487 path_stats: Default::default(),
+488 version,
+489
+490 // peer params are not yet known, so multipath is not enabled
+491 max_concurrent_paths: NonZeroU32::MIN,
+492 local_max_path_id: PathId::ZERO,
+493 remote_max_path_id: PathId::ZERO,
+494 max_path_id_with_cids: PathId::ZERO,
+495 abandoned_paths: Default::default(),
+496
+497 // iroh's nat traversal
+498 iroh_hp: Default::default(),
+499 qlog,
+500 };
+501 if path_validated {
+502 this.on_path_validated(PathId::ZERO);
+503 }
+504 if side.is_client() {
+505 // Kick off the connection
+506 this.write_crypto();
+507 this.init_0rtt(now);
+508 }
+509 this.qlog
+510 .emit_tuple_assigned(PathId::ZERO, network_path, now);
+511 this
+512 }
+513
+514 /// Returns the next time at which `handle_timeout` should be called
+515 ///
+516 /// The value returned may change after:
+517 /// - the application performed some I/O on the connection
+518 /// - a call was made to `handle_event`
+519 /// - a call to `poll_transmit` returned `Some`
+520 /// - a call was made to `handle_timeout`
+521 #[must_use]
+522 pub fn poll_timeout(&mut self) -> Option<Instant> {
+523 self.timers.peek()
+524 }
+525
+526 /// Returns application-facing events
+527 ///
+528 /// Connections should be polled for events after:
+529 /// - a call was made to `handle_event`
+530 /// - a call was made to `handle_timeout`
+531 #[must_use]
+532 pub fn poll(&mut self) -> Option<Event> {
+533 if let Some(x) = self.events.pop_front() {
+534 return Some(x);
+535 }
+536
+537 if let Some(event) = self.streams.poll() {
+538 return Some(Event::Stream(event));
+539 }
+540
+541 if let Some(reason) = self.state.take_error() {
+542 return Some(Event::ConnectionLost { reason });
+543 }
+544
+545 None
+546 }
+547
+548 /// Return endpoint-facing events
+549 #[must_use]
+550 pub fn poll_endpoint_events(&mut self) -> Option<EndpointEvent> {
+551 self.endpoint_events.pop_front().map(EndpointEvent)
+552 }
+553
+554 /// Provide control over streams
+555 #[must_use]
+556 pub fn streams(&mut self) -> Streams<'_> {
+557 Streams {
+558 state: &mut self.streams,
+559 conn_state: &self.state,
+560 }
+561 }
+562
+563 /// Provide control over streams
+564 #[must_use]
+565 pub fn recv_stream(&mut self, id: StreamId) -> RecvStream<'_> {
+566 assert!(id.dir() == Dir::Bi || id.initiator() != self.side.side());
+567 RecvStream {
+568 id,
+569 state: &mut self.streams,
+570 pending: &mut self.spaces[SpaceId::Data].pending,
+571 }
+572 }
+573
+574 /// Provide control over streams
+575 #[must_use]
+576 pub fn send_stream(&mut self, id: StreamId) -> SendStream<'_> {
+577 assert!(id.dir() == Dir::Bi || id.initiator() == self.side.side());
+578 SendStream {
+579 id,
+580 state: &mut self.streams,
+581 pending: &mut self.spaces[SpaceId::Data].pending,
+582 conn_state: &self.state,
+583 }
+584 }
+585
+586 /// Opens a new path only if no path on the same network path currently exists.
+587 ///
+588 /// This comparison will use [`FourTuple::is_probably_same_path`] on the given `network_path`
+589 /// and pass it existing path's network paths.
+590 ///
+591 /// This means that you can pass `local_ip: None` to make the comparison only compare
+592 /// remote addresses.
+593 ///
+594 /// This avoids having to guess which local interface will be used to communicate with the
+595 /// remote, should it not be known yet. We assume that if we already have a path to the remote,
+596 /// the OS is likely to use the same interface to talk to said remote.
+597 ///
+598 /// See also [`open_path`]. Returns `(path_id, true)` if the path already existed. `(path_id,
+599 /// false)` if was opened.
+600 ///
+601 /// [`open_path`]: Connection::open_path
+602 pub fn open_path_ensure(
+603 &mut self,
+604 network_path: FourTuple,
+605 initial_status: PathStatus,
+606 now: Instant,
+607 ) -> Result<(PathId, bool), PathError> {
+608 let existing_open_path = self.paths.iter().find(|(id, path)| {
+609 network_path.is_probably_same_path(&path.data.network_path)
+610 && !self.abandoned_paths.contains(*id)
+611 });
+612 match existing_open_path {
+613 Some((path_id, _state)) => Ok((*path_id, true)),
+614 None => Ok((self.open_path(network_path, initial_status, now)?, false)),
+615 }
+616 }
+617
+618 /// Opens a new path
+619 ///
+620 /// Further errors might occur and they will be emitted in [`PathEvent::LocallyClosed`] events.
+621 /// When the path is opened it will be reported as an [`PathEvent::Opened`].
+622 pub fn open_path(
+623 &mut self,
+624 network_path: FourTuple,
+625 initial_status: PathStatus,
+626 now: Instant,
+627 ) -> Result<PathId, PathError> {
+628 if !self.is_multipath_negotiated() {
+629 return Err(PathError::MultipathNotNegotiated);
+630 }
+631 if self.side().is_server() {
+632 return Err(PathError::ServerSideNotAllowed);
+633 }
+634
+635 let max_abandoned = self.abandoned_paths.iter().max().copied();
+636 let max_used = self.paths.keys().last().copied();
+637 let path_id = max_abandoned
+638 .max(max_used)
+639 .unwrap_or(PathId::ZERO)
+640 .saturating_add(1u8);
+641
+642 if Some(path_id) > self.max_path_id() {
+643 return Err(PathError::MaxPathIdReached);
+644 }
+645 if path_id > self.remote_max_path_id {
+646 self.spaces[SpaceId::Data].pending.paths_blocked = true;
+647 return Err(PathError::MaxPathIdReached);
+648 }
+649 if self.rem_cids.get(&path_id).map(CidQueue::active).is_none() {
+650 self.spaces[SpaceId::Data]
+651 .pending
+652 .path_cids_blocked
+653 .insert(path_id);
+654 return Err(PathError::RemoteCidsExhausted);
+655 }
+656
+657 let path = self.ensure_path(path_id, network_path, now, None);
+658 path.status.local_update(initial_status);
+659
+660 Ok(path_id)
+661 }
+662
+663 /// Closes a path by sending a PATH_ABANDON frame
+664 ///
+665 /// This will not allow closing the last path. It does allow closing paths which have
+666 /// not yet been opened, as e.g. is the case when receiving a PATH_ABANDON from the peer
+667 /// for a path that was never opened locally.
+668 pub fn close_path(
+669 &mut self,
+670 now: Instant,
+671 path_id: PathId,
+672 error_code: VarInt,
+673 ) -> Result<(), ClosePathError> {
+674 if self.abandoned_paths.contains(&path_id)
+675 || Some(path_id) > self.max_path_id()
+676 || !self.paths.contains_key(&path_id)
+677 {
+678 return Err(ClosePathError::ClosedPath);
+679 }
+680 if self
+681 .paths
+682 .iter()
+683 // Would there be any remaining, non-abandoned, validated paths
+684 .any(|(id, path)| {
+685 *id != path_id && !self.abandoned_paths.contains(id) && path.data.validated
+686 })
+687 .not()
+688 {
+689 return Err(ClosePathError::LastOpenPath);
+690 }
+691
+692 // Send PATH_ABANDON
+693 self.spaces[SpaceId::Data]
+694 .pending
+695 .path_abandon
+696 .insert(path_id, error_code.into());
+697
+698 // Remove pending NEW CIDs for this path
+699 let pending_space = &mut self.spaces[SpaceId::Data].pending;
+700 pending_space.new_cids.retain(|cid| cid.path_id != path_id);
+701 pending_space.path_cids_blocked.retain(|&id| id != path_id);
+702 pending_space.path_status.retain(|&id| id != path_id);
+703
+704 // Cleanup retransmits across ALL paths (CIDs for path_id may have been transmitted on other paths)
+705 for space in self.spaces[SpaceId::Data].iter_paths_mut() {
+706 for sent_packet in space.sent_packets.values_mut() {
+707 if let Some(retransmits) = sent_packet.retransmits.get_mut() {
+708 retransmits.new_cids.retain(|cid| cid.path_id != path_id);
+709 retransmits.path_cids_blocked.retain(|&id| id != path_id);
+710 retransmits.path_status.retain(|&id| id != path_id);
+711 }
+712 }
+713 }
+714
+715 // Consider remotely issued CIDs as retired.
+716 // Technically we don't have to do this just yet. We only need to do this *after*
+717 // the ABANDON_PATH frame is sent, allowing us to still send it on the
+718 // to-be-abandoned path. However it is recommended to send it on another path, and
+719 // we do not allow abandoning the last path anyway.
+720 self.rem_cids.remove(&path_id);
+721 self.endpoint_events
+722 .push_back(EndpointEventInner::RetireResetToken(path_id));
+723
+724 self.abandoned_paths.insert(path_id);
725
-726 self.abandoned_paths.insert(path_id);
+726 self.set_max_path_id(now, self.local_max_path_id.saturating_add(1u8));
727
-728 self.set_max_path_id(now, self.local_max_path_id.saturating_add(1u8));
-729
-730 Ok(())
-731 }
-732
-733 /// Gets the [`PathData`] for a known [`PathId`].
-734 ///
-735 /// Will panic if the path_id does not reference any known path.
-736 #[track_caller]
-737 fn path_data(&self, path_id: PathId) -> &PathData {
-738 if let Some(data) = self.paths.get(&path_id) {
-739 &data.data
-740 } else {
-741 panic!(
-742 "unknown path: {path_id}, currently known paths: {:?}",
-743 self.paths.keys().collect::<Vec<_>>()
-744 );
-745 }
-746 }
-747
-748 /// Gets a reference to the [`PathData`] for a [`PathId`]
-749 fn path(&self, path_id: PathId) -> Option<&PathData> {
-750 self.paths.get(&path_id).map(|path_state| &path_state.data)
-751 }
-752
-753 /// Gets a mutable reference to the [`PathData`] for a [`PathId`]
-754 fn path_mut(&mut self, path_id: PathId) -> Option<&mut PathData> {
-755 self.paths
-756 .get_mut(&path_id)
-757 .map(|path_state| &mut path_state.data)
-758 }
-759
-760 /// Returns all known paths.
-761 ///
-762 /// There is no guarantee any of these paths are open or usable.
-763 pub fn paths(&self) -> Vec<PathId> {
-764 self.paths.keys().copied().collect()
-765 }
-766
-767 /// Gets the local [`PathStatus`] for a known [`PathId`]
-768 pub fn path_status(&self, path_id: PathId) -> Result<PathStatus, ClosedPath> {
-769 self.path(path_id)
-770 .map(PathData::local_status)
-771 .ok_or(ClosedPath { _private: () })
-772 }
-773
-774 /// Returns the path's network path represented as a 4-tuple.
-775 pub fn network_path(&self, path_id: PathId) -> Result<FourTuple, ClosedPath> {
-776 self.path(path_id)
-777 .map(|path| path.network_path)
-778 .ok_or(ClosedPath { _private: () })
-779 }
-780
-781 /// Sets the [`PathStatus`] for a known [`PathId`]
-782 ///
-783 /// Returns the previous path status on success.
-784 pub fn set_path_status(
-785 &mut self,
-786 path_id: PathId,
-787 status: PathStatus,
-788 ) -> Result<PathStatus, SetPathStatusError> {
-789 if !self.is_multipath_negotiated() {
-790 return Err(SetPathStatusError::MultipathNotNegotiated);
-791 }
-792 let path = self
-793 .path_mut(path_id)
-794 .ok_or(SetPathStatusError::ClosedPath)?;
-795 let prev = match path.status.local_update(status) {
-796 Some(prev) => {
-797 self.spaces[SpaceId::Data]
-798 .pending
-799 .path_status
-800 .insert(path_id);
-801 prev
-802 }
-803 None => path.local_status(),
-804 };
-805 Ok(prev)
-806 }
-807
-808 /// Returns the remote path status
-809 // TODO(flub): Probably should also be some kind of path event? Not even sure if I like
-810 // this as an API, but for now it allows me to write a test easily.
-811 // TODO(flub): Technically this should be a Result<Option<PathSTatus>>?
-812 pub fn remote_path_status(&self, path_id: PathId) -> Option<PathStatus> {
-813 self.path(path_id).and_then(|path| path.remote_status())
-814 }
-815
-816 /// Sets the max_idle_timeout for a specific path
+728 Ok(())
+729 }
+730
+731 /// Gets the [`PathData`] for a known [`PathId`].
+732 ///
+733 /// Will panic if the path_id does not reference any known path.
+734 #[track_caller]
+735 fn path_data(&self, path_id: PathId) -> &PathData {
+736 if let Some(data) = self.paths.get(&path_id) {
+737 &data.data
+738 } else {
+739 panic!(
+740 "unknown path: {path_id}, currently known paths: {:?}",
+741 self.paths.keys().collect::<Vec<_>>()
+742 );
+743 }
+744 }
+745
+746 /// Gets a reference to the [`PathData`] for a [`PathId`]
+747 fn path(&self, path_id: PathId) -> Option<&PathData> {
+748 self.paths.get(&path_id).map(|path_state| &path_state.data)
+749 }
+750
+751 /// Gets a mutable reference to the [`PathData`] for a [`PathId`]
+752 fn path_mut(&mut self, path_id: PathId) -> Option<&mut PathData> {
+753 self.paths
+754 .get_mut(&path_id)
+755 .map(|path_state| &mut path_state.data)
+756 }
+757
+758 /// Returns all known paths.
+759 ///
+760 /// There is no guarantee any of these paths are open or usable.
+761 pub fn paths(&self) -> Vec<PathId> {
+762 self.paths.keys().copied().collect()
+763 }
+764
+765 /// Gets the local [`PathStatus`] for a known [`PathId`]
+766 pub fn path_status(&self, path_id: PathId) -> Result<PathStatus, ClosedPath> {
+767 self.path(path_id)
+768 .map(PathData::local_status)
+769 .ok_or(ClosedPath { _private: () })
+770 }
+771
+772 /// Returns the path's network path represented as a 4-tuple.
+773 pub fn network_path(&self, path_id: PathId) -> Result<FourTuple, ClosedPath> {
+774 self.path(path_id)
+775 .map(|path| path.network_path)
+776 .ok_or(ClosedPath { _private: () })
+777 }
+778
+779 /// Sets the [`PathStatus`] for a known [`PathId`]
+780 ///
+781 /// Returns the previous path status on success.
+782 pub fn set_path_status(
+783 &mut self,
+784 path_id: PathId,
+785 status: PathStatus,
+786 ) -> Result<PathStatus, SetPathStatusError> {
+787 if !self.is_multipath_negotiated() {
+788 return Err(SetPathStatusError::MultipathNotNegotiated);
+789 }
+790 let path = self
+791 .path_mut(path_id)
+792 .ok_or(SetPathStatusError::ClosedPath)?;
+793 let prev = match path.status.local_update(status) {
+794 Some(prev) => {
+795 self.spaces[SpaceId::Data]
+796 .pending
+797 .path_status
+798 .insert(path_id);
+799 prev
+800 }
+801 None => path.local_status(),
+802 };
+803 Ok(prev)
+804 }
+805
+806 /// Returns the remote path status
+807 // TODO(flub): Probably should also be some kind of path event? Not even sure if I like
+808 // this as an API, but for now it allows me to write a test easily.
+809 // TODO(flub): Technically this should be a Result<Option<PathSTatus>>?
+810 pub fn remote_path_status(&self, path_id: PathId) -> Option<PathStatus> {
+811 self.path(path_id).and_then(|path| path.remote_status())
+812 }
+813
+814 /// Sets the max_idle_timeout for a specific path
+815 ///
+816 /// See [`TransportConfig::default_path_max_idle_timeout`] for details.
817 ///
-818 /// See [`TransportConfig::default_path_max_idle_timeout`] for details.
-819 ///
-820 /// Returns the previous value of the setting.
-821 pub fn set_path_max_idle_timeout(
-822 &mut self,
-823 path_id: PathId,
-824 timeout: Option<Duration>,
-825 ) -> Result<Option<Duration>, ClosedPath> {
-826 let path = self
-827 .paths
-828 .get_mut(&path_id)
-829 .ok_or(ClosedPath { _private: () })?;
-830 Ok(std::mem::replace(&mut path.data.idle_timeout, timeout))
-831 }
-832
-833 /// Sets the keep_alive_interval for a specific path
+818 /// Returns the previous value of the setting.
+819 pub fn set_path_max_idle_timeout(
+820 &mut self,
+821 path_id: PathId,
+822 timeout: Option<Duration>,
+823 ) -> Result<Option<Duration>, ClosedPath> {
+824 let path = self
+825 .paths
+826 .get_mut(&path_id)
+827 .ok_or(ClosedPath { _private: () })?;
+828 Ok(std::mem::replace(&mut path.data.idle_timeout, timeout))
+829 }
+830
+831 /// Sets the keep_alive_interval for a specific path
+832 ///
+833 /// See [`TransportConfig::default_path_keep_alive_interval`] for details.
834 ///
-835 /// See [`TransportConfig::default_path_keep_alive_interval`] for details.
-836 ///
-837 /// Returns the previous value of the setting.
-838 pub fn set_path_keep_alive_interval(
-839 &mut self,
-840 path_id: PathId,
-841 interval: Option<Duration>,
-842 ) -> Result<Option<Duration>, ClosedPath> {
-843 let path = self
-844 .paths
-845 .get_mut(&path_id)
-846 .ok_or(ClosedPath { _private: () })?;
-847 Ok(std::mem::replace(&mut path.data.keep_alive, interval))
-848 }
-849
-850 /// Gets the [`PathData`] for a known [`PathId`].
-851 ///
-852 /// Will panic if the path_id does not reference any known path.
-853 #[track_caller]
-854 fn path_data_mut(&mut self, path_id: PathId) -> &mut PathData {
-855 &mut self.paths.get_mut(&path_id).expect("known path").data
-856 }
-857
-858 /// Find an open, validated path that's on the same network path as the given network path.
-859 ///
-860 /// Returns the first path matching, even if there's multiple.
-861 fn find_validated_path_on_network_path(
-862 &self,
-863 network_path: FourTuple,
-864 ) -> Option<(&PathId, &PathState)> {
-865 self.paths.iter().find(|(path_id, path_state)| {
-866 path_state.data.validated
-867 // Would this use the same network path, if network_path were used to send right now?
-868 && network_path.is_probably_same_path(&path_state.data.network_path)
-869 && !self.abandoned_paths.contains(path_id)
-870 })
-871 // TODO(@divma): we might want to ensure the path has been recently active to consider the
-872 // address validated
-873 // matheus23: Perhaps looking at !self.abandoned_paths.contains(path_id) is enough, given keep-alives?
-874 }
-875
-876 fn ensure_path(
-877 &mut self,
-878 path_id: PathId,
-879 network_path: FourTuple,
-880 now: Instant,
-881 pn: Option<u64>,
-882 ) -> &mut PathData {
-883 let valid_path = self.find_validated_path_on_network_path(network_path);
-884 let validated = valid_path.is_some();
-885 let initial_rtt = valid_path.map(|(_, path)| path.data.rtt.conservative());
-886 let vacant_entry = match self.paths.entry(path_id) {
-887 btree_map::Entry::Vacant(vacant_entry) => vacant_entry,
-888 btree_map::Entry::Occupied(occupied_entry) => {
-889 return &mut occupied_entry.into_mut().data;
-890 }
-891 };
-892
-893 debug!(%validated, %path_id, %network_path, "path added");
-894 let peer_max_udp_payload_size =
-895 u16::try_from(self.peer_params.max_udp_payload_size.into_inner()).unwrap_or(u16::MAX);
-896 self.path_generation_counter = self.path_generation_counter.wrapping_add(1);
-897 let mut data = PathData::new(
-898 network_path,
-899 self.allow_mtud,
-900 Some(peer_max_udp_payload_size),
-901 self.path_generation_counter,
-902 now,
-903 &self.config,
-904 );
-905
-906 data.validated = validated;
-907 if let Some(initial_rtt) = initial_rtt {
-908 data.rtt.reset_initial_rtt(initial_rtt);
-909 }
-910
-911 let pto = self.ack_frequency.max_ack_delay_for_pto() + data.rtt.pto_base();
-912 self.timers.set(
-913 Timer::PerPath(path_id, PathTimer::PathOpen),
-914 now + 3 * pto,
-915 self.qlog.with_time(now),
-916 );
-917
-918 // for the path to be opened we need to send a packet on the path. Sending a challenge
-919 // guarantees this
-920 data.send_new_challenge = true;
+835 /// Returns the previous value of the setting.
+836 pub fn set_path_keep_alive_interval(
+837 &mut self,
+838 path_id: PathId,
+839 interval: Option<Duration>,
+840 ) -> Result<Option<Duration>, ClosedPath> {
+841 let path = self
+842 .paths
+843 .get_mut(&path_id)
+844 .ok_or(ClosedPath { _private: () })?;
+845 Ok(std::mem::replace(&mut path.data.keep_alive, interval))
+846 }
+847
+848 /// Gets the [`PathData`] for a known [`PathId`].
+849 ///
+850 /// Will panic if the path_id does not reference any known path.
+851 #[track_caller]
+852 fn path_data_mut(&mut self, path_id: PathId) -> &mut PathData {
+853 &mut self.paths.get_mut(&path_id).expect("known path").data
+854 }
+855
+856 /// Find an open, validated path that's on the same network path as the given network path.
+857 ///
+858 /// Returns the first path matching, even if there's multiple.
+859 fn find_validated_path_on_network_path(
+860 &self,
+861 network_path: FourTuple,
+862 ) -> Option<(&PathId, &PathState)> {
+863 self.paths.iter().find(|(path_id, path_state)| {
+864 path_state.data.validated
+865 // Would this use the same network path, if network_path were used to send right now?
+866 && network_path.is_probably_same_path(&path_state.data.network_path)
+867 && !self.abandoned_paths.contains(path_id)
+868 })
+869 // TODO(@divma): we might want to ensure the path has been recently active to consider the
+870 // address validated
+871 // matheus23: Perhaps looking at !self.abandoned_paths.contains(path_id) is enough, given keep-alives?
+872 }
+873
+874 fn ensure_path(
+875 &mut self,
+876 path_id: PathId,
+877 network_path: FourTuple,
+878 now: Instant,
+879 pn: Option<u64>,
+880 ) -> &mut PathData {
+881 let valid_path = self.find_validated_path_on_network_path(network_path);
+882 let validated = valid_path.is_some();
+883 let initial_rtt = valid_path.map(|(_, path)| path.data.rtt.conservative());
+884 let vacant_entry = match self.paths.entry(path_id) {
+885 btree_map::Entry::Vacant(vacant_entry) => vacant_entry,
+886 btree_map::Entry::Occupied(occupied_entry) => {
+887 return &mut occupied_entry.into_mut().data;
+888 }
+889 };
+890
+891 debug!(%validated, %path_id, %network_path, "path added");
+892 let peer_max_udp_payload_size =
+893 u16::try_from(self.peer_params.max_udp_payload_size.into_inner()).unwrap_or(u16::MAX);
+894 self.path_generation_counter = self.path_generation_counter.wrapping_add(1);
+895 let mut data = PathData::new(
+896 network_path,
+897 self.allow_mtud,
+898 Some(peer_max_udp_payload_size),
+899 self.path_generation_counter,
+900 now,
+901 &self.config,
+902 );
+903
+904 data.validated = validated;
+905 if let Some(initial_rtt) = initial_rtt {
+906 data.rtt.reset_initial_rtt(initial_rtt);
+907 }
+908
+909 let pto = self.ack_frequency.max_ack_delay_for_pto() + data.rtt.pto_base();
+910 self.timers.set(
+911 Timer::PerPath(path_id, PathTimer::PathOpen),
+912 now + 3 * pto,
+913 self.qlog.with_time(now),
+914 );
+915
+916 // for the path to be opened we need to send a packet on the path. Sending a challenge
+917 // guarantees this
+918 data.send_new_challenge = true;
+919
+920 let path = vacant_entry.insert(PathState { data, prev: None });
921
-922 let path = vacant_entry.insert(PathState { data, prev: None });
-923
-924 let mut pn_space = spaces::PacketNumberSpace::new(now, SpaceId::Data, &mut self.rng);
-925 if let Some(pn) = pn {
-926 pn_space.dedup.insert(pn);
-927 }
-928 self.spaces[SpaceId::Data]
-929 .number_spaces
-930 .insert(path_id, pn_space);
-931 self.qlog.emit_tuple_assigned(path_id, network_path, now);
-932 &mut path.data
-933 }
-934
-935 /// Returns packets to transmit
-936 ///
-937 /// Connections should be polled for transmit after:
-938 /// - the application performed some I/O on the connection
-939 /// - a call was made to `handle_event`
-940 /// - a call was made to `handle_timeout`
-941 ///
-942 /// `max_datagrams` specifies how many datagrams can be returned inside a
-943 /// single Transmit using GSO. This must be at least 1.
-944 #[must_use]
-945 pub fn poll_transmit(
-946 &mut self,
-947 now: Instant,
-948 max_datagrams: NonZeroUsize,
-949 buf: &mut Vec<u8>,
-950 ) -> Option<Transmit> {
-951 if let Some(probing) = self
-952 .iroh_hp
-953 .server_side_mut()
-954 .ok()
-955 .and_then(iroh_hp::ServerState::next_probe)
-956 {
-957 let destination = probing.remote();
-958 trace!(%destination, "RAND_DATA packet");
-959 let token: u64 = self.rng.random();
-960 buf.put_u64(token);
-961 probing.finish(token);
-962 return Some(Transmit {
-963 destination,
-964 ecn: None,
-965 size: 8,
-966 segment_size: None,
-967 src_ip: None,
-968 });
-969 }
-970
-971 let max_datagrams = match self.config.enable_segmentation_offload {
-972 false => NonZeroUsize::MIN,
-973 true => max_datagrams,
-974 };
-975
-976 // Each call to poll_transmit can only send datagrams to one destination, because
-977 // all datagrams in a GSO batch are for the same destination. Therefore only
-978 // datagrams for one Path ID are produced for each poll_transmit call.
-979
-980 // TODO(flub): this is wishful thinking and not actually implemented, but perhaps it
-981 // should be:
-982
-983 // First, if we have to send a close, select a path for that.
-984 // Next, all paths that have a PATH_CHALLENGE or PATH_RESPONSE pending.
-985
-986 // For all, open, validated and AVAILABLE paths:
-987 // - Is the path congestion blocked or pacing blocked?
-988 // - call maybe_queue_ to ensure a tail-loss probe would be sent?
-989 // - do we need to send a close message?
-990 // - call can_send
-991 // Once there's nothing more to send on the AVAILABLE paths, do the same for BACKUP paths
-992
-993 // Check whether we need to send a close message
-994 let close = match self.state.as_type() {
-995 StateType::Drained => {
-996 self.app_limited = true;
-997 return None;
-998 }
-999 StateType::Draining | StateType::Closed => {
-1000 // self.close is only reset once the associated packet had been
-1001 // encoded successfully
-1002 if !self.close {
-1003 self.app_limited = true;
-1004 return None;
-1005 }
-1006 true
-1007 }
-1008 _ => false,
-1009 };
-1010
-1011 // Check whether we need to send an ACK_FREQUENCY frame
-1012 if let Some(config) = &self.config.ack_frequency_config {
-1013 let rtt = self
-1014 .paths
-1015 .values()
-1016 .map(|p| p.data.rtt.get())
-1017 .min()
-1018 .expect("one path exists");
-1019 self.spaces[SpaceId::Data].pending.ack_frequency = self
-1020 .ack_frequency
-1021 .should_send_ack_frequency(rtt, config, &self.peer_params)
-1022 && self.highest_space == SpaceId::Data
-1023 && self.peer_supports_ack_frequency();
-1024 }
-1025
-1026 // If there is any open, validated and available path we only want to send frames to
-1027 // any backup path that must be sent on that backup path exclusively.
-1028 let have_available_path = self.paths.iter().any(|(id, path)| {
-1029 path.data.validated
-1030 && path.data.local_status() == PathStatus::Available
-1031 && self.rem_cids.contains_key(id)
-1032 });
-1033
-1034 // TODO: how to avoid the allocation? Cannot use a for loop because of borrowing
-1035 let path_ids: Vec<_> = self.paths.keys().copied().collect();
-1036
-1037 let mut transmit = TransmitBuf::new(
-1038 buf,
-1039 max_datagrams,
-1040 self.path_data(path_ids[0]).current_mtu().into(),
-1041 );
+922 let mut pn_space = spaces::PacketNumberSpace::new(now, SpaceId::Data, &mut self.rng);
+923 if let Some(pn) = pn {
+924 pn_space.dedup.insert(pn);
+925 }
+926 self.spaces[SpaceId::Data]
+927 .number_spaces
+928 .insert(path_id, pn_space);
+929 self.qlog.emit_tuple_assigned(path_id, network_path, now);
+930 &mut path.data
+931 }
+932
+933 /// Returns packets to transmit
+934 ///
+935 /// Connections should be polled for transmit after:
+936 /// - the application performed some I/O on the connection
+937 /// - a call was made to `handle_event`
+938 /// - a call was made to `handle_timeout`
+939 ///
+940 /// `max_datagrams` specifies how many datagrams can be returned inside a
+941 /// single Transmit using GSO. This must be at least 1.
+942 #[must_use]
+943 pub fn poll_transmit(
+944 &mut self,
+945 now: Instant,
+946 max_datagrams: NonZeroUsize,
+947 buf: &mut Vec<u8>,
+948 ) -> Option<Transmit> {
+949 if let Some(probing) = self
+950 .iroh_hp
+951 .server_side_mut()
+952 .ok()
+953 .and_then(iroh_hp::ServerState::next_probe)
+954 {
+955 let destination = probing.remote();
+956 trace!(%destination, "RAND_DATA packet");
+957 let token: u64 = self.rng.random();
+958 buf.put_u64(token);
+959 probing.finish(token);
+960 return Some(Transmit {
+961 destination,
+962 ecn: None,
+963 size: 8,
+964 segment_size: None,
+965 src_ip: None,
+966 });
+967 }
+968
+969 let max_datagrams = match self.config.enable_segmentation_offload {
+970 false => NonZeroUsize::MIN,
+971 true => max_datagrams,
+972 };
+973
+974 // Each call to poll_transmit can only send datagrams to one destination, because
+975 // all datagrams in a GSO batch are for the same destination. Therefore only
+976 // datagrams for one Path ID are produced for each poll_transmit call.
+977
+978 // TODO(flub): this is wishful thinking and not actually implemented, but perhaps it
+979 // should be:
+980
+981 // First, if we have to send a close, select a path for that.
+982 // Next, all paths that have a PATH_CHALLENGE or PATH_RESPONSE pending.
+983
+984 // For all, open, validated and AVAILABLE paths:
+985 // - Is the path congestion blocked or pacing blocked?
+986 // - call maybe_queue_ to ensure a tail-loss probe would be sent?
+987 // - do we need to send a close message?
+988 // - call can_send
+989 // Once there's nothing more to send on the AVAILABLE paths, do the same for BACKUP paths
+990
+991 // Check whether we need to send a close message
+992 let close = match self.state.as_type() {
+993 StateType::Drained => {
+994 self.app_limited = true;
+995 return None;
+996 }
+997 StateType::Draining | StateType::Closed => {
+998 // self.close is only reset once the associated packet had been
+999 // encoded successfully
+1000 if !self.close {
+1001 self.app_limited = true;
+1002 return None;
+1003 }
+1004 true
+1005 }
+1006 _ => false,
+1007 };
+1008
+1009 // Check whether we need to send an ACK_FREQUENCY frame
+1010 if let Some(config) = &self.config.ack_frequency_config {
+1011 let rtt = self
+1012 .paths
+1013 .values()
+1014 .map(|p| p.data.rtt.get())
+1015 .min()
+1016 .expect("one path exists");
+1017 self.spaces[SpaceId::Data].pending.ack_frequency = self
+1018 .ack_frequency
+1019 .should_send_ack_frequency(rtt, config, &self.peer_params)
+1020 && self.highest_space == SpaceId::Data
+1021 && self.peer_supports_ack_frequency();
+1022 }
+1023
+1024 // If there is any open, validated and available path we only want to send frames to
+1025 // any backup path that must be sent on that backup path exclusively.
+1026 let have_available_path = self.paths.iter().any(|(id, path)| {
+1027 path.data.validated
+1028 && path.data.local_status() == PathStatus::Available
+1029 && self.rem_cids.contains_key(id)
+1030 });
+1031
+1032 // TODO: how to avoid the allocation? Cannot use a for loop because of borrowing
+1033 let path_ids: Vec<_> = self.paths.keys().copied().collect();
+1034
+1035 let mut transmit = TransmitBuf::new(
+1036 buf,
+1037 max_datagrams,
+1038 self.path_data(path_ids[0]).current_mtu().into(),
+1039 );
+1040
+1041 let mut congestion_blocked = false;
1042
-1043 let mut congestion_blocked = false;
-1044
-1045 for &path_id in &path_ids {
-1046 // Update per path state
-1047 transmit.set_segment_size(self.path_data(path_id).current_mtu().into());
-1048
-1049 if let Some(challenge) = self.send_prev_path_challenge(now, &mut transmit, path_id) {
-1050 return Some(challenge);
-1051 }
-1052
-1053 match self.poll_transmit_path(now, &mut transmit, path_id, have_available_path, close) {
-1054 PollPathStatus::SendTransmit { transmit } => {
-1055 return Some(transmit);
-1056 }
-1057 PollPathStatus::Send => {
-1058 let transmit = self.build_transmit(path_id, transmit);
-1059 return Some(transmit);
-1060 }
-1061 PollPathStatus::NothingToSend {
-1062 congestion_blocked: cb,
-1063 } => {
-1064 // TODO: congestion blocked should be per path? how to deal with this better
-1065
-1066 // move to next path
-1067 if cb {
-1068 // note congestion block
-1069 congestion_blocked = true;
-1070 }
-1071 }
-1072 }
-1073 // Nothing more to send.
-1074 trace!(%path_id, "nothing to send on path");
-1075 }
-1076
-1077 // We didn't produce any application data packet
-1078 debug_assert!(
-1079 transmit.is_empty(),
-1080 "there was data in the transmit, but it was not sent"
-1081 );
-1082 self.app_limited = !congestion_blocked;
-1083
-1084 if self.state.is_established() {
-1085 // Try MTU probing now
-1086 for path_id in path_ids {
-1087 // Update per path state
-1088 transmit.set_segment_size(self.path_data(path_id).current_mtu().into());
-1089 self.poll_transmit_mtu_probe(now, &mut transmit, path_id);
-1090 if !transmit.is_empty() {
-1091 let transmit = self.build_transmit(path_id, transmit);
-1092 return Some(transmit);
-1093 }
-1094 }
-1095 }
-1096
-1097 None
-1098 }
-1099
-1100 fn build_transmit(&mut self, path_id: PathId, transmit: TransmitBuf<'_>) -> Transmit {
-1101 debug_assert!(
-1102 !transmit.is_empty(),
-1103 "must not be called with an empty transmit buffer"
-1104 );
-1105
-1106 let network_path = self.path_data(path_id).network_path;
-1107 trace!(
-1108 segment_size = transmit.segment_size(),
-1109 last_datagram_len = transmit.len() % transmit.segment_size(),
-1110 %network_path,
-1111 "sending {} bytes in {} datagrams",
-1112 transmit.len(),
-1113 transmit.num_datagrams()
-1114 );
-1115 self.path_data_mut(path_id)
-1116 .inc_total_sent(transmit.len() as u64);
-1117
-1118 self.stats
-1119 .udp_tx
-1120 .on_sent(transmit.num_datagrams() as u64, transmit.len());
-1121
-1122 Transmit {
-1123 destination: network_path.remote,
-1124 size: transmit.len(),
-1125 ecn: if self.path_data(path_id).sending_ecn {
-1126 Some(EcnCodepoint::Ect0)
-1127 } else {
-1128 None
-1129 },
-1130 segment_size: match transmit.num_datagrams() {
-1131 1 => None,
-1132 _ => Some(transmit.segment_size()),
-1133 },
-1134 src_ip: network_path.local_ip,
-1135 }
-1136 }
-1137
-1138 /// poll_transmit logic for a specific path
-1139 #[must_use]
-1140 fn poll_transmit_path(
-1141 &mut self,
-1142 now: Instant,
-1143 transmit: &mut TransmitBuf<'_>,
-1144 path_id: PathId,
-1145 have_available_path: bool,
-1146 close: bool,
-1147 ) -> PollPathStatus {
-1148 // Check if there is at least one active CID to use for sending
-1149 let Some(remote_cid) = self.rem_cids.get(&path_id).map(CidQueue::active) else {
-1150 self.on_remote_cids_exhausted(now, path_id);
-1151
-1152 return PollPathStatus::NothingToSend {
-1153 congestion_blocked: false,
-1154 };
-1155 };
-1156
-1157 // Whether this packet can be coalesced with another one in the same datagram.
-1158 let mut coalesce = true;
-1159
-1160 // Whether the last packet in the datagram must be padded so the datagram takes up
-1161 // to at least MIN_INITIAL_SIZE, or to the maximum segment size if this is smaller.
-1162 let mut pad_datagram = PadDatagram::No;
-1163
-1164 // The packet number of the last built packet.
-1165 let mut last_packet_number = None;
+1043 for &path_id in &path_ids {
+1044 // Update per path state
+1045 transmit.set_segment_size(self.path_data(path_id).current_mtu().into());
+1046
+1047 if let Some(challenge) = self.send_prev_path_challenge(now, &mut transmit, path_id) {
+1048 return Some(challenge);
+1049 }
+1050
+1051 match self.poll_transmit_path(now, &mut transmit, path_id, have_available_path, close) {
+1052 PollPathStatus::SendTransmit { transmit } => {
+1053 return Some(transmit);
+1054 }
+1055 PollPathStatus::Send => {
+1056 let transmit = self.build_transmit(path_id, transmit);
+1057 return Some(transmit);
+1058 }
+1059 PollPathStatus::NothingToSend {
+1060 congestion_blocked: cb,
+1061 } => {
+1062 // TODO: congestion blocked should be per path? how to deal with this better
+1063
+1064 // move to next path
+1065 if cb {
+1066 // note congestion block
+1067 congestion_blocked = true;
+1068 }
+1069 }
+1070 }
+1071 // Nothing more to send.
+1072 trace!(%path_id, "nothing to send on path");
+1073 }
+1074
+1075 // We didn't produce any application data packet
+1076 debug_assert!(
+1077 transmit.is_empty(),
+1078 "there was data in the transmit, but it was not sent"
+1079 );
+1080 self.app_limited = !congestion_blocked;
+1081
+1082 if self.state.is_established() {
+1083 // Try MTU probing now
+1084 for path_id in path_ids {
+1085 // Update per path state
+1086 transmit.set_segment_size(self.path_data(path_id).current_mtu().into());
+1087 self.poll_transmit_mtu_probe(now, &mut transmit, path_id);
+1088 if !transmit.is_empty() {
+1089 let transmit = self.build_transmit(path_id, transmit);
+1090 return Some(transmit);
+1091 }
+1092 }
+1093 }
+1094
+1095 None
+1096 }
+1097
+1098 fn build_transmit(&mut self, path_id: PathId, transmit: TransmitBuf<'_>) -> Transmit {
+1099 debug_assert!(
+1100 !transmit.is_empty(),
+1101 "must not be called with an empty transmit buffer"
+1102 );
+1103
+1104 let network_path = self.path_data(path_id).network_path;
+1105 trace!(
+1106 segment_size = transmit.segment_size(),
+1107 last_datagram_len = transmit.len() % transmit.segment_size(),
+1108 %network_path,
+1109 "sending {} bytes in {} datagrams",
+1110 transmit.len(),
+1111 transmit.num_datagrams()
+1112 );
+1113 self.path_data_mut(path_id)
+1114 .inc_total_sent(transmit.len() as u64);
+1115
+1116 self.stats
+1117 .udp_tx
+1118 .on_sent(transmit.num_datagrams() as u64, transmit.len());
+1119
+1120 Transmit {
+1121 destination: network_path.remote,
+1122 size: transmit.len(),
+1123 ecn: if self.path_data(path_id).sending_ecn {
+1124 Some(EcnCodepoint::Ect0)
+1125 } else {
+1126 None
+1127 },
+1128 segment_size: match transmit.num_datagrams() {
+1129 1 => None,
+1130 _ => Some(transmit.segment_size()),
+1131 },
+1132 src_ip: network_path.local_ip,
+1133 }
+1134 }
+1135
+1136 /// poll_transmit logic for a specific path
+1137 #[must_use]
+1138 fn poll_transmit_path(
+1139 &mut self,
+1140 now: Instant,
+1141 transmit: &mut TransmitBuf<'_>,
+1142 path_id: PathId,
+1143 have_available_path: bool,
+1144 close: bool,
+1145 ) -> PollPathStatus {
+1146 // Check if there is at least one active CID to use for sending
+1147 let Some(remote_cid) = self.rem_cids.get(&path_id).map(CidQueue::active) else {
+1148 self.on_remote_cids_exhausted(now, path_id);
+1149
+1150 return PollPathStatus::NothingToSend {
+1151 congestion_blocked: false,
+1152 };
+1153 };
+1154
+1155 // Whether the last packet in the datagram must be padded so the datagram takes up
+1156 // to at least MIN_INITIAL_SIZE, or to the maximum segment size if this is smaller.
+1157 let mut pad_datagram = PadDatagram::No;
+1158
+1159 // The packet number of the last built packet.
+1160 let mut last_packet_number = None;
+1161
+1162 // Whether this packet can be coalesced with another one in the same datagram.
+1163 let mut coalesce = true;
+1164
+1165 let mut congestion_blocked = false;
1166
1167 // Iterate over the available spaces
1168 for space_id in SpaceId::iter() {
@@ -1170,5668 +1170,5796 @@
1170 if path_id != PathId::ZERO && space_id != SpaceId::Data {
1171 continue;
1172 }
-1173
-1174 let res = self.poll_transmit_path_space(
-1175 now,
-1176 transmit,
-1177 path_id,
-1178 space_id,
-1179 remote_cid,
-1180 have_available_path,
-1181 close,
-1182 &mut coalesce,
-1183 &mut pad_datagram,
-1184 );
-1185 match res {
-1186 PollPathSpaceStatus::Send {
-1187 last_packet_number: lp,
-1188 } => {
-1189 last_packet_number = lp;
-1190 break;
-1191 }
-1192 PollPathSpaceStatus::SendTransmit { transmit } => {
-1193 return PollPathStatus::SendTransmit { transmit };
-1194 }
-1195 PollPathSpaceStatus::NothingToSend { congestion_blocked } => {
-1196 return PollPathStatus::NothingToSend { congestion_blocked };
-1197 }
-1198 PollPathSpaceStatus::NextSpace => {
-1199 // moving onto the next space
-1200 }
-1201 }
-1202 }
-1203
-1204 if let Some(last_packet_number) = last_packet_number {
-1205 // Note that when sending in multiple packet spaces the last packet number will
-1206 // be the one from the highest packet space.
-1207 self.path_data_mut(path_id).congestion.on_sent(
-1208 now,
-1209 transmit.len() as u64,
-1210 last_packet_number,
-1211 );
-1212 }
-1213
-1214 self.qlog.emit_recovery_metrics(
-1215 path_id,
-1216 &mut self.paths.get_mut(&path_id).unwrap().data,
-1217 now,
-1218 );
-1219
-1220 if transmit.is_empty() {
-1221 PollPathStatus::NothingToSend {
-1222 congestion_blocked: false,
+1173 // We only send up to the highest space
+1174 if space_id >= self.highest_space {
+1175 break;
+1176 }
+1177
+1178 // Check if we can send in this space for lower spaces.
+1179 let sendable_frames = if space_id < SpaceId::Data {
+1180 let sendable_frames = self.space_can_send_non_data(space_id, path_id, close);
+1181 let path_should_send =
+1182 self.path_should_send(path_id, space_id, have_available_path, &sendable_frames);
+1183
+1184 if !path_should_send {
+1185 if self.spaces[space_id].crypto.is_some() {
+1186 trace!(?space_id, %path_id, "nothing to send in space");
+1187 }
+1188 continue;
+1189 }
+1190 Some(sendable_frames)
+1191 } else {
+1192 None
+1193 };
+1194
+1195 let res = self.poll_transmit_path_space(
+1196 now,
+1197 transmit,
+1198 path_id,
+1199 space_id,
+1200 remote_cid,
+1201 have_available_path,
+1202 close,
+1203 sendable_frames,
+1204 &mut coalesce,
+1205 &mut pad_datagram,
+1206 );
+1207 match res {
+1208 PollPathSpaceStatus::Send {
+1209 last_packet_number: lp,
+1210 } => {
+1211 last_packet_number = lp;
+1212 break;
+1213 }
+1214 PollPathSpaceStatus::SendTransmit { transmit } => {
+1215 return PollPathStatus::SendTransmit { transmit };
+1216 }
+1217 PollPathSpaceStatus::NothingToSend {
+1218 congestion_blocked: cb,
+1219 } => {
+1220 congestion_blocked |= cb;
+1221 continue;
+1222 }
1223 }
-1224 } else {
-1225 PollPathStatus::Send
-1226 }
-1227 }
-1228
-1229 /// poll_transmit logic for a path_id - space_id combination
-1230 #[must_use]
-1231 fn poll_transmit_path_space(
-1232 &mut self,
-1233 now: Instant,
-1234 transmit: &mut TransmitBuf<'_>,
-1235 path_id: PathId,
-1236 space_id: SpaceId,
-1237 remote_cid: ConnectionId,
-1238 have_available_path: bool,
-1239 close: bool,
-1240 coalesce: &mut bool,
-1241 pad_datagram: &mut PadDatagram,
-1242 ) -> PollPathSpaceStatus {
-1243 let mut last_packet_number = None;
-1244
-1245 loop {
-1246 // Determine if anything can be sent in this packet number space (SpaceId + PathId).
-1247 let max_packet_size = if transmit.datagram_remaining_mut() > 0 {
-1248 // We are trying to coalesce another packet into this datagram.
-1249 transmit.datagram_remaining_mut()
-1250 } else {
-1251 // A new datagram needs to be started.
-1252 transmit.segment_size()
-1253 };
-1254
-1255 let can_send = self.space_can_send(space_id, path_id, max_packet_size, close);
-1256
-1257 let path_should_send = {
-1258 let path_exclusive_only = space_id == SpaceId::Data
-1259 && have_available_path
-1260 && self.path_data(path_id).local_status() == PathStatus::Backup;
-1261 let path_should_send = if path_exclusive_only {
-1262 can_send.path_exclusive
-1263 } else {
-1264 !can_send.is_empty()
-1265 };
-1266 let needs_loss_probe = self.spaces[space_id].for_path(path_id).loss_probes > 0;
-1267 path_should_send || needs_loss_probe
-1268 };
-1269
-1270 if !path_should_send && space_id < SpaceId::Data {
-1271 if self.spaces[space_id].crypto.is_some() {
-1272 trace!(?space_id, %path_id, "nothing to send in space");
-1273 }
-1274 return PollPathSpaceStatus::NextSpace;
-1275 }
-1276
-1277 let send_blocked = if path_should_send && transmit.datagram_remaining_mut() == 0 {
-1278 // Only check congestion control if a new datagram is needed.
-1279 self.path_congestion_check(space_id, path_id, transmit, &can_send, now)
-1280 } else {
-1281 PathBlocked::No
-1282 };
-1283
-1284 // Whether congestion control stopped the next packet from being sent. Further
-1285 // packets could still be built, as e.g. tail-loss probes are not congestion
-1286 // limited.
-1287 let congestion_blocked = if send_blocked != PathBlocked::No {
-1288 trace!(?space_id, %path_id, ?send_blocked, "congestion blocked");
-1289 true
-1290 } else {
-1291 false
-1292 };
-1293 if send_blocked != PathBlocked::No && space_id < SpaceId::Data {
-1294 // Higher spaces might still have tail-loss probes to send, which are not
-1295 // congestion blocked.
-1296 return PollPathSpaceStatus::NextSpace;
-1297 }
-1298 if !path_should_send || send_blocked != PathBlocked::No {
-1299 // Nothing more to send on this path, check the next path if possible.
-1300
-1301 // If there are any datagrams in the transmit, packets for another path can
-1302 // not be built.
-1303 if transmit.num_datagrams() > 0 {
-1304 return PollPathSpaceStatus::Send { last_packet_number };
-1305 }
-1306
-1307 return PollPathSpaceStatus::NothingToSend { congestion_blocked };
-1308 }
-1309
-1310 if transmit.datagram_remaining_mut() == 0 {
-1311 if transmit.num_datagrams() >= transmit.max_datagrams().get() {
-1312 // No more datagrams allowed
-1313 return PollPathSpaceStatus::Send { last_packet_number };
-1314 }
-1315
-1316 match self.spaces[space_id].for_path(path_id).loss_probes {
-1317 0 => transmit.start_new_datagram(),
-1318 _ => {
-1319 // We need something to send for a tail-loss probe.
-1320 let request_immediate_ack =
-1321 space_id == SpaceId::Data && self.peer_supports_ack_frequency();
-1322 self.spaces[space_id].maybe_queue_probe(
-1323 path_id,
-1324 request_immediate_ack,
-1325 &self.streams,
-1326 );
-1327
-1328 self.spaces[space_id].for_path(path_id).loss_probes -= 1;
-1329
-1330 // Clamp the datagram to at most the minimum MTU to ensure that loss
-1331 // probes can get through and enable recovery even if the path MTU
-1332 // has shrank unexpectedly.
-1333 transmit.start_new_datagram_with_size(std::cmp::min(
-1334 usize::from(INITIAL_MTU),
-1335 transmit.segment_size(),
-1336 ));
-1337 }
-1338 }
-1339 trace!(count = transmit.num_datagrams(), "new datagram started");
-1340 *coalesce = true;
-1341 *pad_datagram = PadDatagram::No;
-1342 }
-1343
-1344 // If coalescing another packet into the existing datagram, there should
-1345 // still be enough space for a whole packet.
-1346 if transmit.datagram_start_offset() < transmit.len() {
-1347 debug_assert!(transmit.datagram_remaining_mut() >= MIN_PACKET_SPACE);
-1348 }
-1349
-1350 //
-1351 // From here on, we've determined that a packet will definitely be sent.
-1352 //
+1224 }
+1225
+1226 if let Some(last_packet_number) = last_packet_number {
+1227 // Note that when sending in multiple packet spaces the last packet number will
+1228 // be the one from the highest packet space.
+1229 self.path_data_mut(path_id).congestion.on_sent(
+1230 now,
+1231 transmit.len() as u64,
+1232 last_packet_number,
+1233 );
+1234 }
+1235
+1236 self.qlog.emit_recovery_metrics(
+1237 path_id,
+1238 &mut self.paths.get_mut(&path_id).unwrap().data,
+1239 now,
+1240 );
+1241
+1242 if transmit.is_empty() {
+1243 PollPathStatus::NothingToSend { congestion_blocked }
+1244 } else {
+1245 PollPathStatus::Send
+1246 }
+1247 }
+1248
+1249 /// Determine if anything can be sent in this packet number space (SpaceId + PathId).
+1250 /// poll_transmit logic for a path_id - space_id combination
+1251 #[must_use]
+1252 fn poll_transmit_path_space(
+1253 &mut self,
+1254 now: Instant,
+1255 transmit: &mut TransmitBuf<'_>,
+1256 path_id: PathId,
+1257 space_id: SpaceId,
+1258 remote_cid: ConnectionId,
+1259 have_available_path: bool,
+1260 close: bool,
+1261 sendable_frames: Option<SendableFrames>,
+1262 coalesce: &mut bool,
+1263 pad_datagram: &mut PadDatagram,
+1264 ) -> PollPathSpaceStatus {
+1265 let mut last_packet_number = None;
+1266
+1267 // Build datagrams until the transmit is full.
+1268 while transmit.num_datagrams() < transmit.max_datagrams().get() {
+1269 let max_packet_size = if transmit.datagram_remaining_mut() > 0 {
+1270 // We are trying to coalesce another packet into this datagram.
+1271 debug_assert!(*coalesce, "coalescing must be set");
+1272 transmit.datagram_remaining_mut()
+1273 } else {
+1274 // A new datagram needs to be started.
+1275 transmit.segment_size()
+1276 };
+1277
+1278 let sendable_frames = sendable_frames
+1279 .unwrap_or_else(|| self.space_can_send(space_id, path_id, max_packet_size, close));
+1280 let path_should_send =
+1281 self.path_should_send(path_id, space_id, have_available_path, &sendable_frames);
+1282
+1283 let send_blocked = if path_should_send && transmit.datagram_remaining_mut() == 0 {
+1284 // Only check congestion control if a new datagram is needed.
+1285 self.path_congestion_check(space_id, path_id, transmit, &sendable_frames, now)
+1286 } else {
+1287 PathBlocked::No
+1288 };
+1289
+1290 // Whether congestion control stopped the next packet from being sent. Further
+1291 // packets could still be built, as e.g. tail-loss probes are not congestion limited.
+1292 let congestion_blocked = if send_blocked != PathBlocked::No {
+1293 trace!(?space_id, %path_id, ?send_blocked, "congestion blocked");
+1294 true
+1295 } else {
+1296 false
+1297 };
+1298 if send_blocked != PathBlocked::No && space_id < SpaceId::Data {
+1299 // Higher spaces might still have tail-loss probes to send, which are not
+1300 // congestion blocked.
+1301 return PollPathSpaceStatus::NothingToSend { congestion_blocked };
+1302 }
+1303 if !path_should_send || send_blocked != PathBlocked::No {
+1304 // Nothing more to send on this path, check the next path if possible.
+1305
+1306 // If there are any datagrams in the transmit, packets for another path can
+1307 // not be built.
+1308 if transmit.num_datagrams() > 0 {
+1309 return PollPathSpaceStatus::Send { last_packet_number };
+1310 }
+1311
+1312 return PollPathSpaceStatus::NothingToSend { congestion_blocked };
+1313 }
+1314
+1315 // If coalescing another packet into the existing datagram, there should
+1316 // still be enough space for a whole packet.
+1317 if *coalesce && transmit.datagram_start_offset() < transmit.len() {
+1318 debug_assert!(transmit.datagram_remaining_mut() >= MIN_PACKET_SPACE);
+1319 }
+1320
+1321 if transmit.datagram_remaining_mut() == 0 {
+1322 // If the datagram is full, start a new datagram.
+1323 // Only needed if we are building a new datagram
+1324 match self.spaces[space_id].for_path(path_id).loss_probes {
+1325 0 => transmit.start_new_datagram(),
+1326 _ => {
+1327 // We need something to send for a tail-loss probe.
+1328 let request_immediate_ack =
+1329 space_id == SpaceId::Data && self.peer_supports_ack_frequency();
+1330 self.spaces[space_id].maybe_queue_probe(
+1331 path_id,
+1332 request_immediate_ack,
+1333 &self.streams,
+1334 );
+1335
+1336 self.spaces[space_id].for_path(path_id).loss_probes -= 1;
+1337
+1338 // Clamp the datagram to at most the minimum MTU to ensure that loss
+1339 // probes can get through and enable recovery even if the path MTU
+1340 // has shrank unexpectedly.
+1341 transmit.start_new_datagram_with_size(std::cmp::min(
+1342 usize::from(INITIAL_MTU),
+1343 transmit.segment_size(),
+1344 ));
+1345 }
+1346 }
+1347 trace!(count = transmit.num_datagrams(), "new datagram started");
+1348 *coalesce = true;
+1349 *pad_datagram = PadDatagram::No;
+1350 }
+1351
+1352 let is_close = sendable_frames.close;
1353
-1354 if self.spaces[SpaceId::Initial].crypto.is_some()
-1355 && space_id == SpaceId::Handshake
-1356 && self.side.is_client()
-1357 {
-1358 // A client stops both sending and processing Initial packets when it
-1359 // sends its first Handshake packet.
-1360 self.discard_space(now, SpaceId::Initial);
-1361 }
-1362 if let Some(ref mut prev) = self.prev_crypto {
-1363 prev.update_unacked = false;
-1364 }
-1365
-1366 let Some(mut builder) = PacketBuilder::new(
-1367 now,
-1368 space_id,
-1369 path_id,
-1370 remote_cid,
+1354 // Send an off-path PATH_RESPONSE. Prioritized over on-path data to ensure that
+1355 // path validation can occur while the link is saturated.
+1356 if !is_close && space_id == SpaceId::Data {
+1357 if let Some(transmit) = self.build_off_path_challenge(
+1358 now,
+1359 path_id,
+1360 space_id,
+1361 remote_cid,
+1362 sendable_frames.other,
+1363 transmit,
+1364 ) {
+1365 return PollPathSpaceStatus::SendTransmit { transmit };
+1366 }
+1367 }
+1368
+1369 let n = self.poll_transmit_path_space_datagram(
+1370 now,
1371 transmit,
-1372 can_send.other,
-1373 self,
-1374 ) else {
-1375 return PollPathSpaceStatus::NothingToSend { congestion_blocked };
-1376 };
-1377 last_packet_number = Some(builder.exact_number);
-1378 *coalesce = *coalesce && !builder.short_header;
-1379
-1380 if space_id == SpaceId::Initial && (self.side.is_client() || can_send.other) {
-1381 // https://www.rfc-editor.org/rfc/rfc9000.html#section-14.1
-1382 *pad_datagram |= PadDatagram::ToMinMtu;
-1383 }
-1384 if space_id == SpaceId::Data && self.config.pad_to_mtu {
-1385 *pad_datagram |= PadDatagram::ToSegmentSize;
+1372 path_id,
+1373 space_id,
+1374 remote_cid,
+1375 have_available_path,
+1376 close,
+1377 sendable_frames,
+1378 coalesce,
+1379 pad_datagram,
+1380 );
+1381 if let Some(n) = n {
+1382 last_packet_number.replace(n);
+1383 } else {
+1384 // error, unable to send
+1385 return PollPathSpaceStatus::NothingToSend { congestion_blocked };
1386 }
1387
-1388 if can_send.close {
-1389 trace!("sending CONNECTION_CLOSE");
-1390 // Encode ACKs before the ConnectionClose message, to give the receiver
-1391 // a better approximate on what data has been processed. This is
-1392 // especially important with ack delay, since the peer might not
-1393 // have gotten any other ACK for the data earlier on.
-1394 let is_multipath_negotiated = self.is_multipath_negotiated();
-1395 for path_id in self.spaces[space_id]
-1396 .number_spaces
-1397 .iter()
-1398 .filter(|(_, pns)| !pns.pending_acks.ranges().is_empty())
-1399 .map(|(&path_id, _)| path_id)
-1400 .collect::<Vec<_>>()
-1401 {
-1402 Self::populate_acks(
-1403 now,
-1404 self.receiving_ecn,
-1405 path_id,
-1406 space_id,
-1407 &mut self.spaces[space_id],
-1408 is_multipath_negotiated,
-1409 &mut builder,
-1410 &mut self.stats.frame_tx,
-1411 );
-1412 }
-1413
-1414 // Since there only 64 ACK frames there will always be enough space
-1415 // to encode the ConnectionClose frame too. However we still have the
-1416 // check here to prevent crashes if something changes.
-1417 debug_assert!(
-1418 builder.frame_space_remaining() > frame::ConnectionClose::SIZE_BOUND,
-1419 "ACKs should leave space for ConnectionClose"
-1420 );
-1421 let stats = &mut self.stats.frame_tx;
-1422 if frame::ConnectionClose::SIZE_BOUND < builder.frame_space_remaining() {
-1423 let max_frame_size = builder.frame_space_remaining();
-1424 let close: Close = match self.state.as_type() {
-1425 StateType::Closed => {
-1426 let reason: Close =
-1427 self.state.as_closed().expect("checked").clone().into();
-1428 if space_id == SpaceId::Data || reason.is_transport_layer() {
-1429 reason
-1430 } else {
-1431 TransportError::APPLICATION_ERROR("").into()
-1432 }
-1433 }
-1434 StateType::Draining => TransportError::NO_ERROR("").into(),
-1435 _ => unreachable!(
-1436 "tried to make a close packet when the connection wasn't closed"
-1437 ),
-1438 };
-1439 builder.write_frame(close.encoder(max_frame_size), stats);
-1440 }
-1441 builder.finish_and_track(now, self, path_id, *pad_datagram);
-1442 if space_id == self.highest_space {
-1443 // Don't send another close packet. Even with multipath we only send
-1444 // CONNECTION_CLOSE on a single path since we expect our paths to work.
-1445 self.close = false;
-1446 // `CONNECTION_CLOSE` is the final packet
-1447 return PollPathSpaceStatus::Send { last_packet_number };
-1448 } else {
-1449 // Send a close frame in every possible space for robustness, per
-1450 // RFC9000 "Immediate Close during the Handshake". Don't bother trying
-1451 // to send anything else.
-1452 return PollPathSpaceStatus::NextSpace;
-1453 }
-1454 }
-1455
-1456 // Send an off-path PATH_RESPONSE. Prioritized over on-path data to ensure that
-1457 // path validation can occur while the link is saturated.
-1458 if space_id == SpaceId::Data && builder.buf.num_datagrams() == 1 {
-1459 let path = self.path_data_mut(path_id);
-1460 if let Some((token, network_path)) =
-1461 path.path_responses.pop_off_path(path.network_path)
-1462 {
-1463 // TODO(flub): We need to use the right CID! We shouldn't use the same
-1464 // CID as the current active one for the path. Though see also
-1465 // https://github.com/quinn-rs/quinn/issues/2184
-1466 //
-1467 let stats = &mut self.stats.frame_tx;
-1468 let frame = frame::PathResponse(token);
-1469 builder.write_frame_with_log_msg(frame, stats, Some("(off-path)"));
-1470 builder.finish_and_track(now, self, path_id, PadDatagram::ToMinMtu);
-1471 self.stats.udp_tx.on_sent(1, transmit.len());
-1472 return PollPathSpaceStatus::SendTransmit {
-1473 transmit: Transmit {
-1474 destination: network_path.remote,
-1475 size: transmit.len(),
-1476 ecn: None,
-1477 segment_size: None,
-1478 src_ip: network_path.local_ip,
-1479 },
-1480 };
-1481 }
-1482 }
-1483
-1484 let path_exclusive_only =
-1485 have_available_path && self.path_data(path_id).local_status() == PathStatus::Backup;
-1486 self.populate_packet(now, space_id, path_id, path_exclusive_only, &mut builder);
-1487
-1488 // ACK-only packets should only be sent when explicitly allowed. If we write them due to
-1489 // any other reason, there is a bug which leads to one component announcing write
-1490 // readiness while not writing any data. This degrades performance. The condition is
-1491 // only checked if the full MTU is available and when potentially large fixed-size
-1492 // frames aren't queued, so that lack of space in the datagram isn't the reason for just
-1493 // writing ACKs.
-1494 debug_assert!(
-1495 !(builder.sent_frames().is_ack_only(&self.streams)
-1496 && !can_send.acks
-1497 && can_send.other
-1498 && builder.buf.segment_size()
-1499 == self.path_data(path_id).current_mtu() as usize
-1500 && self.datagrams.outgoing.is_empty()),
-1501 "SendableFrames was {can_send:?}, but only ACKs have been written"
-1502 );
-1503 if builder.sent_frames().requires_padding {
-1504 *pad_datagram |= PadDatagram::ToMinMtu;
-1505 }
-1506
-1507 for (path_id, _pn) in builder.sent_frames().largest_acked.iter() {
-1508 self.spaces[space_id]
-1509 .for_path(*path_id)
-1510 .pending_acks
-1511 .acks_sent();
-1512 self.timers.stop(
-1513 Timer::PerPath(*path_id, PathTimer::MaxAckDelay),
-1514 self.qlog.with_time(now),
-1515 );
-1516 }
-1517
-1518 // Now we need to finish the packet. Before we do so we need to know if we will
-1519 // be coalescing the next packet into this one, or will be ending the datagram
-1520 // as well. Because if this is the last packet in the datagram more padding
-1521 // might be needed because of the packet type, or to fill the GSO segment size.
-1522
-1523 // Are we allowed to coalesce AND is there enough space for another *packet* in
-1524 // this datagram AND is there another packet to send in this or the next space?
-1525 if *coalesce
-1526 && builder
-1527 .buf
-1528 .datagram_remaining_mut()
-1529 .saturating_sub(builder.predict_packet_end())
-1530 > MIN_PACKET_SPACE
-1531 && self.has_next_send_space(space_id, path_id, builder.buf, close)
-1532 {
-1533 // We can append/coalesce the next packet into the current
-1534 // datagram. Finish the current packet without adding extra padding.
-1535 builder.finish_and_track(now, self, path_id, PadDatagram::No);
-1536 } else {
-1537 // We need a new datagram for the next packet. Finish the current
-1538 // packet with padding.
-1539 if builder.buf.num_datagrams() > 1 && matches!(pad_datagram, PadDatagram::No) {
-1540 // If too many padding bytes would be required to continue the
-1541 // GSO batch after this packet, end the GSO batch here. Ensures
-1542 // that fixed-size frames with heterogeneous sizes
-1543 // (e.g. application datagrams) won't inadvertently waste large
-1544 // amounts of bandwidth. The exact threshold is a bit arbitrary
-1545 // and might benefit from further tuning, though there's no
-1546 // universally optimal value.
-1547 const MAX_PADDING: usize = 32;
-1548 if builder.buf.datagram_remaining_mut()
-1549 > builder.predict_packet_end() + MAX_PADDING
-1550 {
-1551 trace!(
-1552 "GSO truncated by demand for {} padding bytes",
-1553 builder.buf.datagram_remaining_mut() - builder.predict_packet_end()
-1554 );
-1555 builder.finish_and_track(now, self, path_id, PadDatagram::No);
-1556 return PollPathSpaceStatus::Send { last_packet_number };
-1557 }
-1558
-1559 // Pad the current datagram to GSO segment size so it can be
-1560 // included in the GSO batch.
-1561 builder.finish_and_track(now, self, path_id, PadDatagram::ToSegmentSize);
-1562 } else {
-1563 builder.finish_and_track(now, self, path_id, *pad_datagram);
-1564 }
-1565 if transmit.num_datagrams() == 1 {
-1566 transmit.clip_datagram_size();
-1567 }
-1568 }
-1569 }
-1570 }
-1571
-1572 fn on_remote_cids_exhausted(&mut self, now: Instant, path_id: PathId) {
-1573 if self.abandoned_paths.contains(&path_id) {
-1574 trace!(%path_id, "remote CIDs retired for abandoned path");
-1575 return;
-1576 }
-1577
-1578 let error = PathError::RemoteCidsExhausted;
-1579 debug!(?error, %path_id, "no active CID for path");
-1580 self.events
-1581 .push_back(Event::Path(PathEvent::LocallyClosed { id: path_id, error }));
-1582 // Locally we should have refused to open this path, the remote should
-1583 // have given us CIDs for this path before opening it. So we can always
-1584 // abandon this here.
-1585 self.close_path(
-1586 now,
-1587 path_id,
-1588 TransportErrorCode::NO_CID_AVAILABLE_FOR_PATH.into(),
-1589 )
-1590 .ok();
-1591 self.spaces[SpaceId::Data]
-1592 .pending
-1593 .path_cids_blocked
-1594 .insert(path_id);
-1595 }
-1596
-1597 fn poll_transmit_mtu_probe(
-1598 &mut self,
-1599 now: Instant,
-1600 transmit: &mut TransmitBuf<'_>,
-1601 path_id: PathId,
-1602 ) {
-1603 let Some((active_cid, probe_size)) = self.get_mtu_probe_data(now, path_id) else {
-1604 return;
-1605 };
-1606
-1607 // We are definitely sending a DPLPMTUD probe.
-1608 debug_assert_eq!(transmit.num_datagrams(), 0);
-1609 transmit.start_new_datagram_with_size(probe_size as usize);
-1610
-1611 let Some(mut builder) = PacketBuilder::new(
-1612 now,
-1613 SpaceId::Data,
-1614 path_id,
-1615 active_cid,
-1616 transmit,
-1617 true,
-1618 self,
-1619 ) else {
-1620 return;
-1621 };
-1622
-1623 // We implement MTU probes as ping packets padded up to the probe size
-1624 trace!(?probe_size, "writing MTUD probe");
-1625 builder.write_frame(frame::Ping, &mut self.stats.frame_tx);
-1626
-1627 // If supported by the peer, we want no delays to the probe's ACK
-1628 if self.peer_supports_ack_frequency() {
-1629 builder.write_frame(frame::ImmediateAck, &mut self.stats.frame_tx);
-1630 }
-1631
-1632 builder.finish_and_track(now, self, path_id, PadDatagram::ToSize(probe_size));
-1633
-1634 self.path_stats
-1635 .entry(path_id)
-1636 .or_default()
-1637 .sent_plpmtud_probes += 1;
-1638 }
-1639
-1640 fn get_mtu_probe_data(&mut self, now: Instant, path_id: PathId) -> Option<(ConnectionId, u16)> {
-1641 // We MTU probe all paths for which all of the following is true:
-1642 // - We have an active destination CID for the path.
-1643 // - The remote address *and* path are validated.
-1644 // - The path is not abandoned.
-1645 // - The MTU Discovery subsystem wants to probe the path.
-1646 let active_cid = self.rem_cids.get(&path_id).map(CidQueue::active)?;
-1647 let is_eligible = self.path_data(path_id).validated
-1648 && !self.path_data(path_id).is_validating_path()
-1649 && !self.abandoned_paths.contains(&path_id);
-1650
-1651 if !is_eligible {
-1652 return None;
+1388 if is_close {
+1389 if space_id < self.highest_space {
+1390 // Send a close frame in every possible space for robustness, per
+1391 // RFC9000 "Immediate Close during the Handshake". Don't bother trying
+1392 // to send anything else.
+1393 continue;
+1394 } else {
+1395 // Don't send another close packet, we are in the highest space
+1396
+1397 // Even with multipath we only send
+1398 // CONNECTION_CLOSE on a single path since we expect our paths to work.
+1399 self.close = false;
+1400 // `CONNECTION_CLOSE` is the final packet
+1401 break;
+1402 }
+1403 }
+1404
+1405 // Clip the first datagram
+1406 if transmit.num_datagrams() == 1 {
+1407 transmit.clip_datagram_size();
+1408 }
+1409 }
+1410
+1411 if last_packet_number.is_some() {
+1412 // No more datagrams allowed
+1413 PollPathSpaceStatus::Send { last_packet_number }
+1414 } else {
+1415 PollPathSpaceStatus::NothingToSend {
+1416 congestion_blocked: false,
+1417 }
+1418 }
+1419 }
+1420
+1421 /// Tries to build a single datagram
+1422 fn poll_transmit_path_space_datagram(
+1423 &mut self,
+1424 now: Instant,
+1425 transmit: &mut TransmitBuf<'_>,
+1426 path_id: PathId,
+1427 space_id: SpaceId,
+1428 remote_cid: ConnectionId,
+1429 have_available_path: bool,
+1430 close: bool,
+1431 sendable_frames: SendableFrames,
+1432 coalesce: &mut bool,
+1433 pad_datagram: &mut PadDatagram,
+1434 ) -> Option<u64> {
+1435 if self.spaces[SpaceId::Initial].crypto.is_some()
+1436 && space_id == SpaceId::Handshake
+1437 && self.side.is_client()
+1438 {
+1439 // A client stops both sending and processing Initial packets when it
+1440 // sends its first Handshake packet.
+1441 self.discard_space(now, SpaceId::Initial);
+1442 }
+1443 if let Some(ref mut prev) = self.prev_crypto {
+1444 prev.update_unacked = false;
+1445 }
+1446
+1447 let mut builder = PacketBuilder::new(
+1448 now,
+1449 space_id,
+1450 path_id,
+1451 remote_cid,
+1452 transmit,
+1453 sendable_frames.other,
+1454 self,
+1455 )?;
+1456 *coalesce = *coalesce && !builder.short_header;
+1457
+1458 if space_id == SpaceId::Initial && (self.side.is_client() || sendable_frames.other) {
+1459 // https://www.rfc-editor.org/rfc/rfc9000.html#section-14.1
+1460 *pad_datagram |= PadDatagram::ToMinMtu;
+1461 }
+1462 if space_id == SpaceId::Data && self.config.pad_to_mtu {
+1463 *pad_datagram |= PadDatagram::ToSegmentSize;
+1464 }
+1465
+1466 if sendable_frames.close {
+1467 trace!("sending CONNECTION_CLOSE");
+1468 // Encode ACKs before the ConnectionClose message, to give the receiver
+1469 // a better approximate on what data has been processed. This is
+1470 // especially important with ack delay, since the peer might not
+1471 // have gotten any other ACK for the data earlier on.
+1472 let is_multipath_negotiated = self.is_multipath_negotiated();
+1473 for path_id in self.spaces[space_id]
+1474 .number_spaces
+1475 .iter()
+1476 .filter(|(_, pns)| !pns.pending_acks.ranges().is_empty())
+1477 .map(|(&path_id, _)| path_id)
+1478 .collect::<Vec<_>>()
+1479 {
+1480 Self::populate_acks(
+1481 now,
+1482 self.receiving_ecn,
+1483 path_id,
+1484 space_id,
+1485 &mut self.spaces[space_id],
+1486 is_multipath_negotiated,
+1487 &mut builder,
+1488 &mut self.stats.frame_tx,
+1489 );
+1490 }
+1491
+1492 // Since there only 64 ACK frames there will always be enough space
+1493 // to encode the ConnectionClose frame too. However we still have the
+1494 // check here to prevent crashes if something changes.
+1495 debug_assert!(
+1496 builder.frame_space_remaining() > frame::ConnectionClose::SIZE_BOUND,
+1497 "ACKs should leave space for ConnectionClose"
+1498 );
+1499 let stats = &mut self.stats.frame_tx;
+1500 if frame::ConnectionClose::SIZE_BOUND < builder.frame_space_remaining() {
+1501 let max_frame_size = builder.frame_space_remaining();
+1502 let close: Close = match self.state.as_type() {
+1503 StateType::Closed => {
+1504 let reason: Close = self.state.as_closed().expect("checked").clone().into();
+1505 if space_id == SpaceId::Data || reason.is_transport_layer() {
+1506 reason
+1507 } else {
+1508 TransportError::APPLICATION_ERROR("").into()
+1509 }
+1510 }
+1511 StateType::Draining => TransportError::NO_ERROR("").into(),
+1512 _ => unreachable!(
+1513 "tried to make a close packet when the connection wasn't closed"
+1514 ),
+1515 };
+1516 builder.write_frame(close.encoder(max_frame_size), stats);
+1517 }
+1518 return Some(builder.finish_and_track(now, self, path_id, *pad_datagram));
+1519 }
+1520
+1521 let path_exclusive_only =
+1522 have_available_path && self.path_data(path_id).local_status() == PathStatus::Backup;
+1523 self.populate_packet(now, space_id, path_id, path_exclusive_only, &mut builder);
+1524
+1525 // ACK-only packets should only be sent when explicitly allowed. If we write them due to
+1526 // any other reason, there is a bug which leads to one component announcing write
+1527 // readiness while not writing any data. This degrades performance. The condition is
+1528 // only checked if the full MTU is available and when potentially large fixed-size
+1529 // frames aren't queued, so that lack of space in the datagram isn't the reason for just
+1530 // writing ACKs.
+1531 debug_assert!(
+1532 !(builder.sent_frames().is_ack_only(&self.streams)
+1533 && !sendable_frames.acks
+1534 && sendable_frames.other
+1535 && builder.buf.segment_size() == self.path_data(path_id).current_mtu() as usize
+1536 && self.datagrams.outgoing.is_empty()),
+1537 "SendableFrames was {sendable_frames:?}, but only ACKs have been written"
+1538 );
+1539 if builder.sent_frames().requires_padding {
+1540 *pad_datagram |= PadDatagram::ToMinMtu;
+1541 }
+1542
+1543 for (path_id, _pn) in builder.sent_frames().largest_acked.iter() {
+1544 self.spaces[space_id]
+1545 .for_path(*path_id)
+1546 .pending_acks
+1547 .acks_sent();
+1548 self.timers.stop(
+1549 Timer::PerPath(*path_id, PathTimer::MaxAckDelay),
+1550 self.qlog.with_time(now),
+1551 );
+1552 }
+1553
+1554 // Now we need to finish the packet. Before we do so we need to know if we will
+1555 // be coalescing the next packet into this one, or will be ending the datagram
+1556 // as well. Because if this is the last packet in the datagram more padding
+1557 // might be needed because of the packet type, or to fill the GSO segment size.
+1558 let padding =
+1559 self.calculate_padding(path_id, space_id, *coalesce, close, *pad_datagram, &builder);
+1560
+1561 Some(builder.finish_and_track(now, self, path_id, padding))
+1562 }
+1563
+1564 fn build_off_path_challenge(
+1565 &mut self,
+1566 now: Instant,
+1567 path_id: PathId,
+1568 space_id: SpaceId,
+1569 remote_cid: ConnectionId,
+1570 ack_eliciting: bool,
+1571 transmit: &mut TransmitBuf<'_>,
+1572 ) -> Option<Transmit> {
+1573 let mut builder = PacketBuilder::new(
+1574 now,
+1575 space_id,
+1576 path_id,
+1577 remote_cid,
+1578 transmit,
+1579 ack_eliciting,
+1580 self,
+1581 )?;
+1582 let path = self.path_data_mut(path_id);
+1583 if let Some((token, network_path)) = path.path_responses.pop_off_path(path.network_path) {
+1584 // TODO(flub): We need to use the right CID! We shouldn't use the same
+1585 // CID as the current active one for the path. Though see also
+1586 // https://github.com/quinn-rs/quinn/issues/2184
+1587 //
+1588 let stats = &mut self.stats.frame_tx;
+1589 let frame = frame::PathResponse(token);
+1590 builder.write_frame_with_log_msg(frame, stats, Some("(off-path)"));
+1591 builder.finish_and_track(now, self, path_id, PadDatagram::ToMinMtu);
+1592 self.stats.udp_tx.on_sent(1, transmit.len());
+1593 return Some(Transmit {
+1594 destination: network_path.remote,
+1595 size: transmit.len(),
+1596 ecn: None,
+1597 segment_size: None,
+1598 src_ip: network_path.local_ip,
+1599 });
+1600 }
+1601 None
+1602 }
+1603
+1604 fn calculate_padding(
+1605 &mut self,
+1606 path_id: PathId,
+1607 space_id: SpaceId,
+1608 coalesce: bool,
+1609 close: bool,
+1610 pad_datagram: PadDatagram,
+1611 builder: &PacketBuilder<'_, '_>,
+1612 ) -> PadDatagram {
+1613 // Are we allowed to coalesce AND is there enough space for another *packet* in
+1614 // this datagram AND is there another packet to send in this or the next space?
+1615 if coalesce
+1616 && builder
+1617 .buf
+1618 .datagram_remaining_mut()
+1619 .saturating_sub(builder.predict_packet_end())
+1620 > MIN_PACKET_SPACE
+1621 && self.has_next_send_space(space_id, path_id, builder.buf, close)
+1622 {
+1623 // We can append/coalesce the next packet into the current
+1624 // datagram. Finish the current packet without adding extra padding.
+1625 PadDatagram::No
+1626 } else {
+1627 // We need a new datagram for the next packet. Finish the current
+1628 // packet with padding.
+1629 if builder.buf.num_datagrams() > 1 && matches!(pad_datagram, PadDatagram::No) {
+1630 // If too many padding bytes would be required to continue the
+1631 // GSO batch after this packet, end the GSO batch here. Ensures
+1632 // that fixed-size frames with heterogeneous sizes
+1633 // (e.g. application datagrams) won't inadvertently waste large
+1634 // amounts of bandwidth. The exact threshold is a bit arbitrary
+1635 // and might benefit from further tuning, though there's no
+1636 // universally optimal value.
+1637 const MAX_PADDING: usize = 32;
+1638 if builder.buf.datagram_remaining_mut() > builder.predict_packet_end() + MAX_PADDING
+1639 {
+1640 trace!(
+1641 "GSO truncated by demand for {} padding bytes",
+1642 builder.buf.datagram_remaining_mut() - builder.predict_packet_end()
+1643 );
+1644 PadDatagram::No
+1645 } else {
+1646 // Pad the current datagram to GSO segment size so it can be
+1647 // included in the GSO batch.
+1648 PadDatagram::ToSegmentSize
+1649 }
+1650 } else {
+1651 pad_datagram
+1652 }
1653 }
-1654 let next_pn = self.spaces[SpaceId::Data]
-1655 .for_path(path_id)
-1656 .peek_tx_number();
-1657 let probe_size = self
-1658 .path_data_mut(path_id)
-1659 .mtud
-1660 .poll_transmit(now, next_pn)?;
-1661
-1662 Some((active_cid, probe_size))
-1663 }
-1664
-1665 /// Returns if there is anext packet space which has data to send
-1666 ///
-1667 /// This takes into account the space available to frames in the next datagram.
-1668 // TODO(flub): This duplication is not nice.
-1669 fn has_next_send_space(
-1670 &mut self,
-1671 current_space_id: SpaceId,
-1672 path_id: PathId,
-1673 buf: &TransmitBuf<'_>,
-1674 close: bool,
-1675 ) -> bool {
-1676 // Number of bytes available for frames if this is a 1-RTT packet. We're guaranteed
-1677 // to be able to send an individual frame at least this large in the next 1-RTT
-1678 // packet. This could be generalized to support every space, but it's only needed to
-1679 // handle large fixed-size frames, which only exist in 1-RTT (application
-1680 // datagrams). We don't account for coalesced packets potentially occupying space
-1681 // because frames can always spill into the next datagram.
-1682 let mut space_id = current_space_id;
-1683 loop {
-1684 let can_send = self.space_can_send(space_id, path_id, buf.segment_size(), close);
-1685 if !can_send.is_empty() || (close && self.spaces[space_id].crypto.is_some()) {
-1686 return true;
-1687 }
-1688 let Some(next_space) = space_id.next() else {
-1689 break;
-1690 };
-1691 space_id = next_space;
-1692 }
-1693 false
-1694 }
-1695
-1696 /// Checks if creating a new datagram would be blocked by congestion control
-1697 fn path_congestion_check(
-1698 &mut self,
-1699 space_id: SpaceId,
-1700 path_id: PathId,
-1701 transmit: &TransmitBuf<'_>,
-1702 can_send: &SendableFrames,
-1703 now: Instant,
-1704 ) -> PathBlocked {
-1705 // Anti-amplification is only based on `total_sent`, which gets updated after
-1706 // the transmit is sent. Therefore we pass the amount of bytes for datagrams
-1707 // that are already created, as well as 1 byte for starting another datagram. If
-1708 // there is any anti-amplification budget left, we always allow a full MTU to be
-1709 // sent (see https://github.com/quinn-rs/quinn/issues/1082).
-1710 if self.side().is_server()
-1711 && self
-1712 .path_data(path_id)
-1713 .anti_amplification_blocked(transmit.len() as u64 + 1)
-1714 {
-1715 trace!(?space_id, %path_id, "blocked by anti-amplification");
-1716 return PathBlocked::AntiAmplification;
-1717 }
-1718
-1719 // Congestion control check.
-1720 // Tail loss probes must not be blocked by congestion, or a deadlock could arise.
-1721 let bytes_to_send = transmit.segment_size() as u64;
-1722 let need_loss_probe = self.spaces[space_id].for_path(path_id).loss_probes > 0;
-1723
-1724 if can_send.other && !need_loss_probe && !can_send.close {
-1725 let path = self.path_data(path_id);
-1726 if path.in_flight.bytes + bytes_to_send >= path.congestion.window() {
-1727 trace!(?space_id, %path_id, "blocked by congestion control");
-1728 return PathBlocked::Congestion;
-1729 }
-1730 }
-1731
-1732 // Pacing check.
-1733 if let Some(delay) = self.path_data_mut(path_id).pacing_delay(bytes_to_send, now) {
-1734 self.timers.set(
-1735 Timer::PerPath(path_id, PathTimer::Pacing),
-1736 delay,
-1737 self.qlog.with_time(now),
-1738 );
-1739 // Loss probes and CONNECTION_CLOSE should be subject to pacing, even though
-1740 // they are not congestion controlled.
-1741 trace!(?space_id, %path_id, "blocked by pacing");
-1742 return PathBlocked::Pacing;
-1743 }
-1744
-1745 PathBlocked::No
-1746 }
-1747
-1748 /// Send PATH_CHALLENGE for a previous path if necessary
-1749 ///
-1750 /// QUIC-TRANSPORT section 9.3.3
-1751 /// <https://www.rfc-editor.org/rfc/rfc9000.html#name-off-path-packet-forwarding>
-1752 fn send_prev_path_challenge(
-1753 &mut self,
-1754 now: Instant,
-1755 buf: &mut TransmitBuf<'_>,
-1756 path_id: PathId,
-1757 ) -> Option<Transmit> {
-1758 let (prev_cid, prev_path) = self.paths.get_mut(&path_id)?.prev.as_mut()?;
-1759 // TODO (matheus23): We could use !prev_path.is_validating() here instead to
-1760 // (possibly) also re-send challenges when they get lost.
-1761 if !prev_path.send_new_challenge {
-1762 return None;
-1763 };
-1764 prev_path.send_new_challenge = false;
-1765 let network_path = prev_path.network_path;
-1766 let token = self.rng.random();
-1767 let info = paths::SentChallengeInfo {
-1768 sent_instant: now,
-1769 network_path,
-1770 };
-1771 prev_path.challenges_sent.insert(token, info);
-1772 debug_assert_eq!(
-1773 self.highest_space,
-1774 SpaceId::Data,
-1775 "PATH_CHALLENGE queued without 1-RTT keys"
-1776 );
-1777 buf.start_new_datagram_with_size(MIN_INITIAL_SIZE as usize);
-1778
-1779 // Use the previous CID to avoid linking the new path with the previous path. We
-1780 // don't bother accounting for possible retirement of that prev_cid because this is
-1781 // sent once, immediately after migration, when the CID is known to be valid. Even
-1782 // if a post-migration packet caused the CID to be retired, it's fair to pretend
-1783 // this is sent first.
-1784 debug_assert_eq!(buf.datagram_start_offset(), 0);
-1785 let mut builder =
-1786 PacketBuilder::new(now, SpaceId::Data, path_id, *prev_cid, buf, false, self)?;
-1787 let challenge = frame::PathChallenge(token);
-1788 let stats = &mut self.stats.frame_tx;
-1789 builder.write_frame_with_log_msg(challenge, stats, Some("validating previous path"));
-1790
-1791 // An endpoint MUST expand datagrams that contain a PATH_CHALLENGE frame
-1792 // to at least the smallest allowed maximum datagram size of 1200 bytes,
-1793 // unless the anti-amplification limit for the path does not permit
-1794 // sending a datagram of this size
-1795 builder.pad_to(MIN_INITIAL_SIZE);
-1796
-1797 builder.finish(self, now);
-1798 self.stats.udp_tx.on_sent(1, buf.len());
-1799
-1800 Some(Transmit {
-1801 destination: network_path.remote,
-1802 size: buf.len(),
-1803 ecn: None,
-1804 segment_size: None,
-1805 src_ip: network_path.local_ip,
-1806 })
-1807 }
-1808
-1809 /// Indicate what types of frames are ready to send for the given space
-1810 ///
-1811 /// *packet_size* is the number of bytes available to build the next packet. *close*
-1812 /// *indicates whether a CONNECTION_CLOSE frame needs to be sent.
-1813 fn space_can_send(
-1814 &mut self,
-1815 space_id: SpaceId,
-1816 path_id: PathId,
-1817 packet_size: usize,
-1818 close: bool,
-1819 ) -> SendableFrames {
-1820 let pn = self.spaces[SpaceId::Data]
-1821 .for_path(path_id)
-1822 .peek_tx_number();
-1823 let frame_space_1rtt = packet_size.saturating_sub(self.predict_1rtt_overhead(pn, path_id));
-1824 if self.spaces[space_id].crypto.is_none()
-1825 && (space_id != SpaceId::Data
-1826 || self.zero_rtt_crypto.is_none()
-1827 || self.side.is_server())
-1828 {
-1829 // No keys available for this space
-1830 return SendableFrames::empty();
-1831 }
-1832 let mut can_send = self.spaces[space_id].can_send(path_id, &self.streams);
-1833 if space_id == SpaceId::Data {
-1834 can_send |= self.can_send_1rtt(path_id, frame_space_1rtt);
-1835 }
-1836
-1837 can_send.close = close && self.spaces[space_id].crypto.is_some();
-1838
-1839 can_send
-1840 }
-1841
-1842 /// Process `ConnectionEvent`s generated by the associated `Endpoint`
-1843 ///
-1844 /// Will execute protocol logic upon receipt of a connection event, in turn preparing signals
-1845 /// (including application `Event`s, `EndpointEvent`s and outgoing datagrams) that should be
-1846 /// extracted through the relevant methods.
-1847 pub fn handle_event(&mut self, event: ConnectionEvent) {
-1848 use ConnectionEventInner::*;
-1849 match event.0 {
-1850 Datagram(DatagramConnectionEvent {
-1851 now,
-1852 network_path,
-1853 path_id,
-1854 ecn,
-1855 first_decode,
-1856 remaining,
-1857 }) => {
-1858 let span = trace_span!("pkt", %path_id);
-1859 let _guard = span.enter();
-1860
-1861 if self.update_network_path_or_discard(network_path, path_id) {
-1862 // A return value of true indicates we should discard this packet.
-1863 return;
-1864 }
-1865
-1866 let was_anti_amplification_blocked = self
-1867 .path(path_id)
-1868 .map(|path| path.anti_amplification_blocked(1))
-1869 .unwrap_or(true); // if we don't know about this path it's eagerly considered as unvalidated
-1870 // TODO(@divma): revisit this
-1871
-1872 self.stats.udp_rx.datagrams += 1;
-1873 self.stats.udp_rx.bytes += first_decode.len() as u64;
-1874 let data_len = first_decode.len();
-1875
-1876 self.handle_decode(now, network_path, path_id, ecn, first_decode);
-1877 // The current `path` might have changed inside `handle_decode` since the packet
-1878 // could have triggered a migration. The packet might also belong to an unknown
-1879 // path and have been rejected. Make sure the data received is accounted for the
-1880 // most recent path by accessing `path` after `handle_decode`.
-1881 if let Some(path) = self.path_mut(path_id) {
-1882 path.inc_total_recvd(data_len as u64);
-1883 }
-1884
-1885 if let Some(data) = remaining {
-1886 self.stats.udp_rx.bytes += data.len() as u64;
-1887 self.handle_coalesced(now, network_path, path_id, ecn, data);
-1888 }
-1889
-1890 if let Some(path) = self.paths.get_mut(&path_id) {
-1891 self.qlog
-1892 .emit_recovery_metrics(path_id, &mut path.data, now);
-1893 }
-1894
-1895 if was_anti_amplification_blocked {
-1896 // A prior attempt to set the loss detection timer may have failed due to
-1897 // anti-amplification, so ensure it's set now. Prevents a handshake deadlock if
-1898 // the server's first flight is lost.
-1899 self.set_loss_detection_timer(now, path_id);
-1900 }
-1901 }
-1902 NewIdentifiers(ids, now, cid_len, cid_lifetime) => {
-1903 let path_id = ids.first().map(|issued| issued.path_id).unwrap_or_default();
-1904 debug_assert!(ids.iter().all(|issued| issued.path_id == path_id));
-1905 let cid_state = self
-1906 .local_cid_state
-1907 .entry(path_id)
-1908 .or_insert_with(|| CidState::new(cid_len, cid_lifetime, now, 0));
-1909 cid_state.new_cids(&ids, now);
-1910
-1911 ids.into_iter().rev().for_each(|frame| {
-1912 self.spaces[SpaceId::Data].pending.new_cids.push(frame);
-1913 });
-1914 // Always update Timer::PushNewCid
-1915 self.reset_cid_retirement(now);
-1916 }
-1917 }
-1918 }
-1919
-1920 /// Updates the network path for `path_id`.
-1921 ///
-1922 /// Returns true if a packet coming in for this `path_id` over given `network_path` should be discarded.
-1923 /// Returns false if the path was updated and the packet doesn't need to be discarded.
-1924 fn update_network_path_or_discard(&mut self, network_path: FourTuple, path_id: PathId) -> bool {
-1925 let remote_may_migrate = self.side.remote_may_migrate(&self.state);
-1926 let local_ip_may_migrate = self.side.is_client();
-1927 // If this packet could initiate a migration and we're a client or a server that
-1928 // forbids migration, drop the datagram. This could be relaxed to heuristically
-1929 // permit NAT-rebinding-like migration.
-1930 if let Some(known_path) = self.path_mut(path_id) {
-1931 if network_path.remote != known_path.network_path.remote && !remote_may_migrate {
-1932 trace!(
-1933 %path_id,
-1934 %network_path,
-1935 %known_path.network_path,
-1936 "discarding packet from unrecognized peer"
-1937 );
-1938 return true;
-1939 }
-1940
-1941 if known_path.network_path.local_ip.is_some()
-1942 && network_path.local_ip.is_some()
-1943 && known_path.network_path.local_ip != network_path.local_ip
-1944 && !local_ip_may_migrate
-1945 {
-1946 trace!(
-1947 %path_id,
-1948 %network_path,
-1949 %known_path.network_path,
-1950 "discarding packet sent to incorrect interface"
-1951 );
-1952 return true;
-1953 }
-1954 // If the datagram indicates that we've changed our local IP, we update it.
-1955 // This is alluded to in Section 5.2 of the Multipath RFC draft 18:
-1956 // https://www.ietf.org/archive/id/draft-ietf-quic-multipath-18.html#name-using-multiple-paths-on-the
-1957 // > Client receives the packet, recognizes a path migration, updates the source address of path 2 to 192.0.2.1.
-1958 if let Some(local_ip) = network_path.local_ip {
-1959 if known_path
-1960 .network_path
-1961 .local_ip
-1962 .is_some_and(|ip| ip != local_ip)
-1963 {
-1964 debug!(
-1965 %path_id,
-1966 %network_path,
-1967 %known_path.network_path,
-1968 "path's local address seemingly migrated"
-1969 );
-1970 }
-1971 // We update the address without path validation on the client side.
-1972 // https://www.ietf.org/archive/id/draft-ietf-quic-multipath-18.html#section-5.1
-1973 // > Servers observing a 4-tuple change will perform path validation (see Section 9 of [QUIC-TRANSPORT]).
-1974 // This sounds like it's *only* the server endpoints that do this.
-1975 // TODO(matheus23): We should still consider doing a proper migration on the client side in the future.
-1976 // For now, this preserves the behavior of this code pre 4-tuple tracking.
-1977 known_path.network_path.local_ip = Some(local_ip);
-1978 }
-1979 }
-1980 false
-1981 }
-1982
-1983 /// Process timer expirations
-1984 ///
-1985 /// Executes protocol logic, potentially preparing signals (including application `Event`s,
-1986 /// `EndpointEvent`s and outgoing datagrams) that should be extracted through the relevant
-1987 /// methods.
-1988 ///
-1989 /// It is most efficient to call this immediately after the system clock reaches the latest
-1990 /// `Instant` that was output by `poll_timeout`; however spurious extra calls will simply
-1991 /// no-op and therefore are safe.
-1992 pub fn handle_timeout(&mut self, now: Instant) {
-1993 while let Some((timer, _time)) = self.timers.expire_before(now, &self.qlog) {
-1994 // TODO(@divma): remove `at` when the unicorn is born
-1995 trace!(?timer, at=?now, "timeout");
-1996 match timer {
-1997 Timer::Conn(timer) => match timer {
-1998 ConnTimer::Close => {
-1999 self.state.move_to_drained(None);
-2000 self.endpoint_events.push_back(EndpointEventInner::Drained);
-2001 }
-2002 ConnTimer::Idle => {
-2003 self.kill(ConnectionError::TimedOut);
-2004 }
-2005 ConnTimer::KeepAlive => {
-2006 trace!("sending keep-alive");
-2007 self.ping();
-2008 }
-2009 ConnTimer::KeyDiscard => {
-2010 self.zero_rtt_crypto = None;
-2011 self.prev_crypto = None;
-2012 }
-2013 ConnTimer::PushNewCid => {
-2014 while let Some((path_id, when)) = self.next_cid_retirement() {
-2015 if when > now {
-2016 break;
-2017 }
-2018 match self.local_cid_state.get_mut(&path_id) {
-2019 None => error!(%path_id, "No local CID state for path"),
-2020 Some(cid_state) => {
-2021 // Update `retire_prior_to` field in NEW_CONNECTION_ID frame
-2022 let num_new_cid = cid_state.on_cid_timeout().into();
-2023 if !self.state.is_closed() {
-2024 trace!(
-2025 "push a new CID to peer RETIRE_PRIOR_TO field {}",
-2026 cid_state.retire_prior_to()
-2027 );
-2028 self.endpoint_events.push_back(
-2029 EndpointEventInner::NeedIdentifiers(
-2030 path_id,
-2031 now,
-2032 num_new_cid,
-2033 ),
-2034 );
-2035 }
-2036 }
-2037 }
-2038 }
-2039 }
-2040 },
-2041 // TODO: add path_id as span somehow
-2042 Timer::PerPath(path_id, timer) => {
-2043 let span = trace_span!("per-path timer fired", %path_id, ?timer);
-2044 let _guard = span.enter();
-2045 match timer {
-2046 PathTimer::PathIdle => {
-2047 self.close_path(now, path_id, TransportErrorCode::NO_ERROR.into())
-2048 .ok();
-2049 }
-2050
-2051 PathTimer::PathKeepAlive => {
-2052 trace!("sending keep-alive on path");
-2053 self.ping_path(path_id).ok();
-2054 }
-2055 PathTimer::LossDetection => {
-2056 self.on_loss_detection_timeout(now, path_id);
-2057 self.qlog.emit_recovery_metrics(
-2058 path_id,
-2059 &mut self.paths.get_mut(&path_id).unwrap().data,
-2060 now,
-2061 );
-2062 }
-2063 PathTimer::PathValidation => {
-2064 let Some(path) = self.paths.get_mut(&path_id) else {
-2065 continue;
-2066 };
-2067 self.timers.stop(
-2068 Timer::PerPath(path_id, PathTimer::PathChallengeLost),
-2069 self.qlog.with_time(now),
-2070 );
-2071 debug!("path validation failed");
-2072 if let Some((_, prev)) = path.prev.take() {
-2073 path.data = prev;
-2074 }
-2075 path.data.challenges_sent.clear();
-2076 path.data.send_new_challenge = false;
-2077 }
-2078 PathTimer::PathChallengeLost => {
-2079 let Some(path) = self.paths.get_mut(&path_id) else {
-2080 continue;
-2081 };
-2082 trace!("path challenge deemed lost");
-2083 path.data.send_new_challenge = true;
-2084 }
-2085 PathTimer::PathOpen => {
-2086 let Some(path) = self.paths.get_mut(&path_id) else {
-2087 continue;
-2088 };
-2089 path.data.challenges_sent.clear();
-2090 path.data.send_new_challenge = false;
-2091 self.timers.stop(
-2092 Timer::PerPath(path_id, PathTimer::PathChallengeLost),
-2093 self.qlog.with_time(now),
-2094 );
-2095 debug!("new path validation failed");
-2096 if let Err(err) = self.close_path(
-2097 now,
-2098 path_id,
-2099 TransportErrorCode::PATH_UNSTABLE_OR_POOR.into(),
-2100 ) {
-2101 warn!(?err, "failed closing path");
-2102 }
-2103
-2104 self.events.push_back(Event::Path(PathEvent::LocallyClosed {
-2105 id: path_id,
-2106 error: PathError::ValidationFailed,
-2107 }));
-2108 }
-2109 PathTimer::Pacing => trace!("pacing timer expired"),
-2110 PathTimer::MaxAckDelay => {
-2111 trace!("max ack delay reached");
-2112 // This timer is only armed in the Data space
-2113 self.spaces[SpaceId::Data]
-2114 .for_path(path_id)
-2115 .pending_acks
-2116 .on_max_ack_delay_timeout()
-2117 }
-2118 PathTimer::DiscardPath => {
-2119 // The path was abandoned and 3*PTO has expired since. Clean up all
-2120 // remaining state and install stateless reset token.
-2121 self.timers.stop_per_path(path_id, self.qlog.with_time(now));
-2122 if let Some(loc_cid_state) = self.local_cid_state.remove(&path_id) {
-2123 let (min_seq, max_seq) = loc_cid_state.active_seq();
-2124 for seq in min_seq..=max_seq {
-2125 self.endpoint_events.push_back(
-2126 EndpointEventInner::RetireConnectionId(
-2127 now, path_id, seq, false,
-2128 ),
-2129 );
-2130 }
-2131 }
-2132 self.discard_path(path_id, now);
-2133 }
-2134 }
-2135 }
-2136 }
-2137 }
-2138 }
-2139
-2140 /// Close a connection immediately
-2141 ///
-2142 /// This does not ensure delivery of outstanding data. It is the application's responsibility to
-2143 /// call this only when all important communications have been completed, e.g. by calling
-2144 /// [`SendStream::finish`] on outstanding streams and waiting for the corresponding
-2145 /// [`StreamEvent::Finished`] event.
-2146 ///
-2147 /// If [`Streams::send_streams`] returns 0, all outstanding stream data has been
-2148 /// delivered. There may still be data from the peer that has not been received.
-2149 ///
-2150 /// [`StreamEvent::Finished`]: crate::StreamEvent::Finished
-2151 pub fn close(&mut self, now: Instant, error_code: VarInt, reason: Bytes) {
-2152 self.close_inner(
-2153 now,
-2154 Close::Application(frame::ApplicationClose { error_code, reason }),
-2155 )
-2156 }
-2157
-2158 fn close_inner(&mut self, now: Instant, reason: Close) {
-2159 let was_closed = self.state.is_closed();
-2160 if !was_closed {
-2161 self.close_common();
-2162 self.set_close_timer(now);
-2163 self.close = true;
-2164 self.state.move_to_closed_local(reason);
-2165 }
-2166 }
-2167
-2168 /// Control datagrams
-2169 pub fn datagrams(&mut self) -> Datagrams<'_> {
-2170 Datagrams { conn: self }
-2171 }
-2172
-2173 /// Returns connection statistics
-2174 pub fn stats(&mut self) -> ConnectionStats {
-2175 self.stats.clone()
-2176 }
-2177
-2178 /// Returns path statistics
-2179 pub fn path_stats(&mut self, path_id: PathId) -> Option<PathStats> {
-2180 let path = self.paths.get(&path_id)?;
-2181 let stats = self.path_stats.entry(path_id).or_default();
-2182 stats.rtt = path.data.rtt.get();
-2183 stats.cwnd = path.data.congestion.window();
-2184 stats.current_mtu = path.data.mtud.current_mtu();
-2185 Some(*stats)
-2186 }
-2187
-2188 /// Ping the remote endpoint
-2189 ///
-2190 /// Causes an ACK-eliciting packet to be transmitted on the connection.
-2191 pub fn ping(&mut self) {
-2192 // TODO(flub): This is very brute-force: it pings *all* the paths. Instead it would
-2193 // be nice if we could only send a single packet for this.
-2194 for path_data in self.spaces[self.highest_space].number_spaces.values_mut() {
-2195 path_data.ping_pending = true;
-2196 }
-2197 }
-2198
-2199 /// Ping the remote endpoint over a specific path
-2200 ///
-2201 /// Causes an ACK-eliciting packet to be transmitted on the path.
-2202 pub fn ping_path(&mut self, path: PathId) -> Result<(), ClosedPath> {
-2203 let path_data = self.spaces[self.highest_space]
-2204 .number_spaces
-2205 .get_mut(&path)
-2206 .ok_or(ClosedPath { _private: () })?;
-2207 path_data.ping_pending = true;
-2208 Ok(())
-2209 }
-2210
-2211 /// Update traffic keys spontaneously
-2212 ///
-2213 /// This can be useful for testing key updates, as they otherwise only happen infrequently.
-2214 pub fn force_key_update(&mut self) {
-2215 if !self.state.is_established() {
-2216 debug!("ignoring forced key update in illegal state");
-2217 return;
-2218 }
-2219 if self.prev_crypto.is_some() {
-2220 // We already just updated, or are currently updating, the keys. Concurrent key updates
-2221 // are illegal.
-2222 debug!("ignoring redundant forced key update");
-2223 return;
-2224 }
-2225 self.update_keys(None, false);
-2226 }
-2227
-2228 /// Get a session reference
-2229 pub fn crypto_session(&self) -> &dyn crypto::Session {
-2230 &*self.crypto
-2231 }
-2232
-2233 /// Whether the connection is in the process of being established
-2234 ///
-2235 /// If this returns `false`, the connection may be either established or closed, signaled by the
-2236 /// emission of a `Connected` or `ConnectionLost` message respectively.
-2237 pub fn is_handshaking(&self) -> bool {
-2238 self.state.is_handshake()
-2239 }
-2240
-2241 /// Whether the connection is closed
-2242 ///
-2243 /// Closed connections cannot transport any further data. A connection becomes closed when
-2244 /// either peer application intentionally closes it, or when either transport layer detects an
-2245 /// error such as a time-out or certificate validation failure.
-2246 ///
-2247 /// A `ConnectionLost` event is emitted with details when the connection becomes closed.
-2248 pub fn is_closed(&self) -> bool {
-2249 self.state.is_closed()
-2250 }
-2251
-2252 /// Whether there is no longer any need to keep the connection around
-2253 ///
-2254 /// Closed connections become drained after a brief timeout to absorb any remaining in-flight
-2255 /// packets from the peer. All drained connections have been closed.
-2256 pub fn is_drained(&self) -> bool {
-2257 self.state.is_drained()
-2258 }
-2259
-2260 /// For clients, if the peer accepted the 0-RTT data packets
-2261 ///
-2262 /// The value is meaningless until after the handshake completes.
-2263 pub fn accepted_0rtt(&self) -> bool {
-2264 self.accepted_0rtt
-2265 }
-2266
-2267 /// Whether 0-RTT is/was possible during the handshake
-2268 pub fn has_0rtt(&self) -> bool {
-2269 self.zero_rtt_enabled
-2270 }
-2271
-2272 /// Whether there are any pending retransmits
-2273 pub fn has_pending_retransmits(&self) -> bool {
-2274 !self.spaces[SpaceId::Data].pending.is_empty(&self.streams)
-2275 }
-2276
-2277 /// Look up whether we're the client or server of this Connection
-2278 pub fn side(&self) -> Side {
-2279 self.side.side()
-2280 }
-2281
-2282 /// Get the address observed by the remote over the given path
-2283 pub fn path_observed_address(&self, path_id: PathId) -> Result<Option<SocketAddr>, ClosedPath> {
-2284 self.path(path_id)
-2285 .map(|path_data| {
-2286 path_data
-2287 .last_observed_addr_report
-2288 .as_ref()
-2289 .map(|observed| observed.socket_addr())
-2290 })
-2291 .ok_or(ClosedPath { _private: () })
-2292 }
-2293
-2294 /// Current best estimate of this connection's latency (round-trip-time)
-2295 pub fn rtt(&self, path_id: PathId) -> Option<Duration> {
-2296 self.path(path_id).map(|d| d.rtt.get())
-2297 }
-2298
-2299 /// Current state of this connection's congestion controller, for debugging purposes
-2300 pub fn congestion_state(&self, path_id: PathId) -> Option<&dyn Controller> {
-2301 self.path(path_id).map(|d| d.congestion.as_ref())
-2302 }
-2303
-2304 /// Modify the number of remotely initiated streams that may be concurrently open
-2305 ///
-2306 /// No streams may be opened by the peer unless fewer than `count` are already open. Large
-2307 /// `count`s increase both minimum and worst-case memory consumption.
-2308 pub fn set_max_concurrent_streams(&mut self, dir: Dir, count: VarInt) {
-2309 self.streams.set_max_concurrent(dir, count);
-2310 // If the limit was reduced, then a flow control update previously deemed insignificant may
-2311 // now be significant.
-2312 let pending = &mut self.spaces[SpaceId::Data].pending;
-2313 self.streams.queue_max_stream_id(pending);
+1654 }
+1655
+1656 fn path_should_send(
+1657 &mut self,
+1658 path_id: PathId,
+1659 space_id: SpaceId,
+1660 have_available_path: bool,
+1661 sendable_frames: &SendableFrames,
+1662 ) -> bool {
+1663 let path_exclusive_only = space_id == SpaceId::Data
+1664 && have_available_path
+1665 && self.path_data(path_id).local_status() == PathStatus::Backup;
+1666 let path_should_send = if path_exclusive_only {
+1667 sendable_frames.path_exclusive
+1668 } else {
+1669 !sendable_frames.is_empty()
+1670 };
+1671 let needs_loss_probe = self.spaces[space_id].for_path(path_id).loss_probes > 0;
+1672 path_should_send || needs_loss_probe
+1673 }
+1674
+1675 fn on_remote_cids_exhausted(&mut self, now: Instant, path_id: PathId) {
+1676 if self.abandoned_paths.contains(&path_id) {
+1677 trace!(%path_id, "remote CIDs retired for abandoned path");
+1678 return;
+1679 }
+1680
+1681 let error = PathError::RemoteCidsExhausted;
+1682 debug!(?error, %path_id, "no active CID for path");
+1683 self.events
+1684 .push_back(Event::Path(PathEvent::LocallyClosed { id: path_id, error }));
+1685 // Locally we should have refused to open this path, the remote should
+1686 // have given us CIDs for this path before opening it. So we can always
+1687 // abandon this here.
+1688 self.close_path(
+1689 now,
+1690 path_id,
+1691 TransportErrorCode::NO_CID_AVAILABLE_FOR_PATH.into(),
+1692 )
+1693 .ok();
+1694 self.spaces[SpaceId::Data]
+1695 .pending
+1696 .path_cids_blocked
+1697 .insert(path_id);
+1698 }
+1699
+1700 fn poll_transmit_mtu_probe(
+1701 &mut self,
+1702 now: Instant,
+1703 transmit: &mut TransmitBuf<'_>,
+1704 path_id: PathId,
+1705 ) {
+1706 let Some((active_cid, probe_size)) = self.get_mtu_probe_data(now, path_id) else {
+1707 return;
+1708 };
+1709
+1710 // We are definitely sending a DPLPMTUD probe.
+1711 debug_assert_eq!(transmit.num_datagrams(), 0);
+1712 transmit.start_new_datagram_with_size(probe_size as usize);
+1713
+1714 let Some(mut builder) = PacketBuilder::new(
+1715 now,
+1716 SpaceId::Data,
+1717 path_id,
+1718 active_cid,
+1719 transmit,
+1720 true,
+1721 self,
+1722 ) else {
+1723 return;
+1724 };
+1725
+1726 // We implement MTU probes as ping packets padded up to the probe size
+1727 trace!(?probe_size, "writing MTUD probe");
+1728 builder.write_frame(frame::Ping, &mut self.stats.frame_tx);
+1729
+1730 // If supported by the peer, we want no delays to the probe's ACK
+1731 if self.peer_supports_ack_frequency() {
+1732 builder.write_frame(frame::ImmediateAck, &mut self.stats.frame_tx);
+1733 }
+1734
+1735 builder.finish_and_track(now, self, path_id, PadDatagram::ToSize(probe_size));
+1736
+1737 self.path_stats
+1738 .entry(path_id)
+1739 .or_default()
+1740 .sent_plpmtud_probes += 1;
+1741 }
+1742
+1743 fn get_mtu_probe_data(&mut self, now: Instant, path_id: PathId) -> Option<(ConnectionId, u16)> {
+1744 // We MTU probe all paths for which all of the following is true:
+1745 // - We have an active destination CID for the path.
+1746 // - The remote address *and* path are validated.
+1747 // - The path is not abandoned.
+1748 // - The MTU Discovery subsystem wants to probe the path.
+1749 let active_cid = self.rem_cids.get(&path_id).map(CidQueue::active)?;
+1750 let is_eligible = self.path_data(path_id).validated
+1751 && !self.path_data(path_id).is_validating_path()
+1752 && !self.abandoned_paths.contains(&path_id);
+1753
+1754 if !is_eligible {
+1755 return None;
+1756 }
+1757 let next_pn = self.spaces[SpaceId::Data]
+1758 .for_path(path_id)
+1759 .peek_tx_number();
+1760 let probe_size = self
+1761 .path_data_mut(path_id)
+1762 .mtud
+1763 .poll_transmit(now, next_pn)?;
+1764
+1765 Some((active_cid, probe_size))
+1766 }
+1767
+1768 /// Returns if there is anext packet space which has data to send
+1769 ///
+1770 /// This takes into account the space available to frames in the next datagram.
+1771 // TODO(flub): This duplication is not nice.
+1772 fn has_next_send_space(
+1773 &mut self,
+1774 current_space_id: SpaceId,
+1775 path_id: PathId,
+1776 buf: &TransmitBuf<'_>,
+1777 close: bool,
+1778 ) -> bool {
+1779 // Number of bytes available for frames if this is a 1-RTT packet. We're guaranteed
+1780 // to be able to send an individual frame at least this large in the next 1-RTT
+1781 // packet. This could be generalized to support every space, but it's only needed to
+1782 // handle large fixed-size frames, which only exist in 1-RTT (application
+1783 // datagrams). We don't account for coalesced packets potentially occupying space
+1784 // because frames can always spill into the next datagram.
+1785 let mut space_id = current_space_id;
+1786 loop {
+1787 let can_send = self.space_can_send(space_id, path_id, buf.segment_size(), close);
+1788 if !can_send.is_empty() || (close && self.spaces[space_id].crypto.is_some()) {
+1789 return true;
+1790 }
+1791 let Some(next_space) = space_id.next() else {
+1792 break;
+1793 };
+1794 space_id = next_space;
+1795 }
+1796 false
+1797 }
+1798
+1799 /// Checks if creating a new datagram would be blocked by congestion control
+1800 fn path_congestion_check(
+1801 &mut self,
+1802 space_id: SpaceId,
+1803 path_id: PathId,
+1804 transmit: &TransmitBuf<'_>,
+1805 can_send: &SendableFrames,
+1806 now: Instant,
+1807 ) -> PathBlocked {
+1808 // Anti-amplification is only based on `total_sent`, which gets updated after
+1809 // the transmit is sent. Therefore we pass the amount of bytes for datagrams
+1810 // that are already created, as well as 1 byte for starting another datagram. If
+1811 // there is any anti-amplification budget left, we always allow a full MTU to be
+1812 // sent (see https://github.com/quinn-rs/quinn/issues/1082).
+1813 if self.side().is_server()
+1814 && self
+1815 .path_data(path_id)
+1816 .anti_amplification_blocked(transmit.len() as u64 + 1)
+1817 {
+1818 trace!(?space_id, %path_id, "blocked by anti-amplification");
+1819 return PathBlocked::AntiAmplification;
+1820 }
+1821
+1822 // Congestion control check.
+1823 // Tail loss probes must not be blocked by congestion, or a deadlock could arise.
+1824 let bytes_to_send = transmit.segment_size() as u64;
+1825 let need_loss_probe = self.spaces[space_id].for_path(path_id).loss_probes > 0;
+1826
+1827 if can_send.other && !need_loss_probe && !can_send.close {
+1828 let path = self.path_data(path_id);
+1829 if path.in_flight.bytes + bytes_to_send >= path.congestion.window() {
+1830 trace!(?space_id, %path_id, "blocked by congestion control");
+1831 return PathBlocked::Congestion;
+1832 }
+1833 }
+1834
+1835 // Pacing check.
+1836 if let Some(delay) = self.path_data_mut(path_id).pacing_delay(bytes_to_send, now) {
+1837 self.timers.set(
+1838 Timer::PerPath(path_id, PathTimer::Pacing),
+1839 delay,
+1840 self.qlog.with_time(now),
+1841 );
+1842 // Loss probes and CONNECTION_CLOSE should be subject to pacing, even though
+1843 // they are not congestion controlled.
+1844 trace!(?space_id, %path_id, "blocked by pacing");
+1845 return PathBlocked::Pacing;
+1846 }
+1847
+1848 PathBlocked::No
+1849 }
+1850
+1851 /// Send PATH_CHALLENGE for a previous path if necessary
+1852 ///
+1853 /// QUIC-TRANSPORT section 9.3.3
+1854 /// <https://www.rfc-editor.org/rfc/rfc9000.html#name-off-path-packet-forwarding>
+1855 fn send_prev_path_challenge(
+1856 &mut self,
+1857 now: Instant,
+1858 buf: &mut TransmitBuf<'_>,
+1859 path_id: PathId,
+1860 ) -> Option<Transmit> {
+1861 let (prev_cid, prev_path) = self.paths.get_mut(&path_id)?.prev.as_mut()?;
+1862 // TODO (matheus23): We could use !prev_path.is_validating() here instead to
+1863 // (possibly) also re-send challenges when they get lost.
+1864 if !prev_path.send_new_challenge {
+1865 return None;
+1866 };
+1867 prev_path.send_new_challenge = false;
+1868 let network_path = prev_path.network_path;
+1869 let token = self.rng.random();
+1870 let info = paths::SentChallengeInfo {
+1871 sent_instant: now,
+1872 network_path,
+1873 };
+1874 prev_path.challenges_sent.insert(token, info);
+1875 debug_assert_eq!(
+1876 self.highest_space,
+1877 SpaceId::Data,
+1878 "PATH_CHALLENGE queued without 1-RTT keys"
+1879 );
+1880 buf.start_new_datagram_with_size(MIN_INITIAL_SIZE as usize);
+1881
+1882 // Use the previous CID to avoid linking the new path with the previous path. We
+1883 // don't bother accounting for possible retirement of that prev_cid because this is
+1884 // sent once, immediately after migration, when the CID is known to be valid. Even
+1885 // if a post-migration packet caused the CID to be retired, it's fair to pretend
+1886 // this is sent first.
+1887 debug_assert_eq!(buf.datagram_start_offset(), 0);
+1888 let mut builder =
+1889 PacketBuilder::new(now, SpaceId::Data, path_id, *prev_cid, buf, false, self)?;
+1890 let challenge = frame::PathChallenge(token);
+1891 let stats = &mut self.stats.frame_tx;
+1892 builder.write_frame_with_log_msg(challenge, stats, Some("validating previous path"));
+1893
+1894 // An endpoint MUST expand datagrams that contain a PATH_CHALLENGE frame
+1895 // to at least the smallest allowed maximum datagram size of 1200 bytes,
+1896 // unless the anti-amplification limit for the path does not permit
+1897 // sending a datagram of this size
+1898 builder.pad_to(MIN_INITIAL_SIZE);
+1899
+1900 builder.finish(self, now);
+1901 self.stats.udp_tx.on_sent(1, buf.len());
+1902
+1903 Some(Transmit {
+1904 destination: network_path.remote,
+1905 size: buf.len(),
+1906 ecn: None,
+1907 segment_size: None,
+1908 src_ip: network_path.local_ip,
+1909 })
+1910 }
+1911
+1912 /// Indicate what types of frames are ready to send for the given space
+1913 ///
+1914 /// *packet_size* is the number of bytes available to build the next packet.
+1915 /// *close* indicates whether a CONNECTION_CLOSE frame needs to be sent.
+1916 fn space_can_send(
+1917 &mut self,
+1918 space_id: SpaceId,
+1919 path_id: PathId,
+1920 packet_size: usize,
+1921 close: bool,
+1922 ) -> SendableFrames {
+1923 let space = &mut self.spaces[space_id];
+1924 let space_has_crypto = space.crypto.is_none();
+1925
+1926 if !space_has_crypto
+1927 && (space_id != SpaceId::Data
+1928 || self.zero_rtt_crypto.is_none()
+1929 || self.side.is_server())
+1930 {
+1931 // Nothing send in this space
+1932 return SendableFrames::empty();
+1933 }
+1934
+1935 let mut can_send = space.can_send(path_id, &self.streams);
+1936
+1937 if space_id == SpaceId::Data {
+1938 // Check for 1RTT space.
+1939 let pn = space.for_path(path_id).peek_tx_number();
+1940 let frame_space_1rtt =
+1941 packet_size.saturating_sub(self.predict_1rtt_overhead(pn, path_id));
+1942 can_send |= self.can_send_1rtt(path_id, frame_space_1rtt);
+1943 }
+1944
+1945 can_send.close = close && space_has_crypto;
+1946
+1947 can_send
+1948 }
+1949
+1950 fn space_can_send_non_data(
+1951 &mut self,
+1952 space_id: SpaceId,
+1953 path_id: PathId,
+1954 close: bool,
+1955 ) -> SendableFrames {
+1956 debug_assert!(space_id != SpaceId::Data);
+1957 let space = &mut self.spaces[space_id];
+1958 let space_has_crypto = space.crypto.is_none();
+1959
+1960 if !space_has_crypto && (self.zero_rtt_crypto.is_none() || self.side.is_server()) {
+1961 // Nothing send in this space
+1962 return SendableFrames::empty();
+1963 }
+1964
+1965 let mut can_send = space.can_send(path_id, &self.streams);
+1966 can_send.close = close && space_has_crypto;
+1967 can_send
+1968 }
+1969
+1970 /// Process `ConnectionEvent`s generated by the associated `Endpoint`
+1971 ///
+1972 /// Will execute protocol logic upon receipt of a connection event, in turn preparing signals
+1973 /// (including application `Event`s, `EndpointEvent`s and outgoing datagrams) that should be
+1974 /// extracted through the relevant methods.
+1975 pub fn handle_event(&mut self, event: ConnectionEvent) {
+1976 use ConnectionEventInner::*;
+1977 match event.0 {
+1978 Datagram(DatagramConnectionEvent {
+1979 now,
+1980 network_path,
+1981 path_id,
+1982 ecn,
+1983 first_decode,
+1984 remaining,
+1985 }) => {
+1986 let span = trace_span!("pkt", %path_id);
+1987 let _guard = span.enter();
+1988
+1989 if self.update_network_path_or_discard(network_path, path_id) {
+1990 // A return value of true indicates we should discard this packet.
+1991 return;
+1992 }
+1993
+1994 let was_anti_amplification_blocked = self
+1995 .path(path_id)
+1996 .map(|path| path.anti_amplification_blocked(1))
+1997 .unwrap_or(true); // if we don't know about this path it's eagerly considered as unvalidated
+1998 // TODO(@divma): revisit this
+1999
+2000 self.stats.udp_rx.datagrams += 1;
+2001 self.stats.udp_rx.bytes += first_decode.len() as u64;
+2002 let data_len = first_decode.len();
+2003
+2004 self.handle_decode(now, network_path, path_id, ecn, first_decode);
+2005 // The current `path` might have changed inside `handle_decode` since the packet
+2006 // could have triggered a migration. The packet might also belong to an unknown
+2007 // path and have been rejected. Make sure the data received is accounted for the
+2008 // most recent path by accessing `path` after `handle_decode`.
+2009 if let Some(path) = self.path_mut(path_id) {
+2010 path.inc_total_recvd(data_len as u64);
+2011 }
+2012
+2013 if let Some(data) = remaining {
+2014 self.stats.udp_rx.bytes += data.len() as u64;
+2015 self.handle_coalesced(now, network_path, path_id, ecn, data);
+2016 }
+2017
+2018 if let Some(path) = self.paths.get_mut(&path_id) {
+2019 self.qlog
+2020 .emit_recovery_metrics(path_id, &mut path.data, now);
+2021 }
+2022
+2023 if was_anti_amplification_blocked {
+2024 // A prior attempt to set the loss detection timer may have failed due to
+2025 // anti-amplification, so ensure it's set now. Prevents a handshake deadlock if
+2026 // the server's first flight is lost.
+2027 self.set_loss_detection_timer(now, path_id);
+2028 }
+2029 }
+2030 NewIdentifiers(ids, now, cid_len, cid_lifetime) => {
+2031 let path_id = ids.first().map(|issued| issued.path_id).unwrap_or_default();
+2032 debug_assert!(ids.iter().all(|issued| issued.path_id == path_id));
+2033 let cid_state = self
+2034 .local_cid_state
+2035 .entry(path_id)
+2036 .or_insert_with(|| CidState::new(cid_len, cid_lifetime, now, 0));
+2037 cid_state.new_cids(&ids, now);
+2038
+2039 ids.into_iter().rev().for_each(|frame| {
+2040 self.spaces[SpaceId::Data].pending.new_cids.push(frame);
+2041 });
+2042 // Always update Timer::PushNewCid
+2043 self.reset_cid_retirement(now);
+2044 }
+2045 }
+2046 }
+2047
+2048 /// Updates the network path for `path_id`.
+2049 ///
+2050 /// Returns true if a packet coming in for this `path_id` over given `network_path` should be discarded.
+2051 /// Returns false if the path was updated and the packet doesn't need to be discarded.
+2052 fn update_network_path_or_discard(&mut self, network_path: FourTuple, path_id: PathId) -> bool {
+2053 let remote_may_migrate = self.side.remote_may_migrate(&self.state);
+2054 let local_ip_may_migrate = self.side.is_client();
+2055 // If this packet could initiate a migration and we're a client or a server that
+2056 // forbids migration, drop the datagram. This could be relaxed to heuristically
+2057 // permit NAT-rebinding-like migration.
+2058 if let Some(known_path) = self.path_mut(path_id) {
+2059 if network_path.remote != known_path.network_path.remote && !remote_may_migrate {
+2060 trace!(
+2061 %path_id,
+2062 %network_path,
+2063 %known_path.network_path,
+2064 "discarding packet from unrecognized peer"
+2065 );
+2066 return true;
+2067 }
+2068
+2069 if known_path.network_path.local_ip.is_some()
+2070 && network_path.local_ip.is_some()
+2071 && known_path.network_path.local_ip != network_path.local_ip
+2072 && !local_ip_may_migrate
+2073 {
+2074 trace!(
+2075 %path_id,
+2076 %network_path,
+2077 %known_path.network_path,
+2078 "discarding packet sent to incorrect interface"
+2079 );
+2080 return true;
+2081 }
+2082 // If the datagram indicates that we've changed our local IP, we update it.
+2083 // This is alluded to in Section 5.2 of the Multipath RFC draft 18:
+2084 // https://www.ietf.org/archive/id/draft-ietf-quic-multipath-18.html#name-using-multiple-paths-on-the
+2085 // > Client receives the packet, recognizes a path migration, updates the source address of path 2 to 192.0.2.1.
+2086 if let Some(local_ip) = network_path.local_ip {
+2087 if known_path
+2088 .network_path
+2089 .local_ip
+2090 .is_some_and(|ip| ip != local_ip)
+2091 {
+2092 debug!(
+2093 %path_id,
+2094 %network_path,
+2095 %known_path.network_path,
+2096 "path's local address seemingly migrated"
+2097 );
+2098 }
+2099 // We update the address without path validation on the client side.
+2100 // https://www.ietf.org/archive/id/draft-ietf-quic-multipath-18.html#section-5.1
+2101 // > Servers observing a 4-tuple change will perform path validation (see Section 9 of [QUIC-TRANSPORT]).
+2102 // This sounds like it's *only* the server endpoints that do this.
+2103 // TODO(matheus23): We should still consider doing a proper migration on the client side in the future.
+2104 // For now, this preserves the behavior of this code pre 4-tuple tracking.
+2105 known_path.network_path.local_ip = Some(local_ip);
+2106 }
+2107 }
+2108 false
+2109 }
+2110
+2111 /// Process timer expirations
+2112 ///
+2113 /// Executes protocol logic, potentially preparing signals (including application `Event`s,
+2114 /// `EndpointEvent`s and outgoing datagrams) that should be extracted through the relevant
+2115 /// methods.
+2116 ///
+2117 /// It is most efficient to call this immediately after the system clock reaches the latest
+2118 /// `Instant` that was output by `poll_timeout`; however spurious extra calls will simply
+2119 /// no-op and therefore are safe.
+2120 pub fn handle_timeout(&mut self, now: Instant) {
+2121 while let Some((timer, _time)) = self.timers.expire_before(now, &self.qlog) {
+2122 // TODO(@divma): remove `at` when the unicorn is born
+2123 trace!(?timer, at=?now, "timeout");
+2124 match timer {
+2125 Timer::Conn(timer) => match timer {
+2126 ConnTimer::Close => {
+2127 self.state.move_to_drained(None);
+2128 self.endpoint_events.push_back(EndpointEventInner::Drained);
+2129 }
+2130 ConnTimer::Idle => {
+2131 self.kill(ConnectionError::TimedOut);
+2132 }
+2133 ConnTimer::KeepAlive => {
+2134 trace!("sending keep-alive");
+2135 self.ping();
+2136 }
+2137 ConnTimer::KeyDiscard => {
+2138 self.zero_rtt_crypto = None;
+2139 self.prev_crypto = None;
+2140 }
+2141 ConnTimer::PushNewCid => {
+2142 while let Some((path_id, when)) = self.next_cid_retirement() {
+2143 if when > now {
+2144 break;
+2145 }
+2146 match self.local_cid_state.get_mut(&path_id) {
+2147 None => error!(%path_id, "No local CID state for path"),
+2148 Some(cid_state) => {
+2149 // Update `retire_prior_to` field in NEW_CONNECTION_ID frame
+2150 let num_new_cid = cid_state.on_cid_timeout().into();
+2151 if !self.state.is_closed() {
+2152 trace!(
+2153 "push a new CID to peer RETIRE_PRIOR_TO field {}",
+2154 cid_state.retire_prior_to()
+2155 );
+2156 self.endpoint_events.push_back(
+2157 EndpointEventInner::NeedIdentifiers(
+2158 path_id,
+2159 now,
+2160 num_new_cid,
+2161 ),
+2162 );
+2163 }
+2164 }
+2165 }
+2166 }
+2167 }
+2168 },
+2169 // TODO: add path_id as span somehow
+2170 Timer::PerPath(path_id, timer) => {
+2171 let span = trace_span!("per-path timer fired", %path_id, ?timer);
+2172 let _guard = span.enter();
+2173 match timer {
+2174 PathTimer::PathIdle => {
+2175 self.close_path(now, path_id, TransportErrorCode::NO_ERROR.into())
+2176 .ok();
+2177 }
+2178
+2179 PathTimer::PathKeepAlive => {
+2180 trace!("sending keep-alive on path");
+2181 self.ping_path(path_id).ok();
+2182 }
+2183 PathTimer::LossDetection => {
+2184 self.on_loss_detection_timeout(now, path_id);
+2185 self.qlog.emit_recovery_metrics(
+2186 path_id,
+2187 &mut self.paths.get_mut(&path_id).unwrap().data,
+2188 now,
+2189 );
+2190 }
+2191 PathTimer::PathValidation => {
+2192 let Some(path) = self.paths.get_mut(&path_id) else {
+2193 continue;
+2194 };
+2195 self.timers.stop(
+2196 Timer::PerPath(path_id, PathTimer::PathChallengeLost),
+2197 self.qlog.with_time(now),
+2198 );
+2199 debug!("path validation failed");
+2200 if let Some((_, prev)) = path.prev.take() {
+2201 path.data = prev;
+2202 }
+2203 path.data.challenges_sent.clear();
+2204 path.data.send_new_challenge = false;
+2205 }
+2206 PathTimer::PathChallengeLost => {
+2207 let Some(path) = self.paths.get_mut(&path_id) else {
+2208 continue;
+2209 };
+2210 trace!("path challenge deemed lost");
+2211 path.data.send_new_challenge = true;
+2212 }
+2213 PathTimer::PathOpen => {
+2214 let Some(path) = self.paths.get_mut(&path_id) else {
+2215 continue;
+2216 };
+2217 path.data.challenges_sent.clear();
+2218 path.data.send_new_challenge = false;
+2219 self.timers.stop(
+2220 Timer::PerPath(path_id, PathTimer::PathChallengeLost),
+2221 self.qlog.with_time(now),
+2222 );
+2223 debug!("new path validation failed");
+2224 if let Err(err) = self.close_path(
+2225 now,
+2226 path_id,
+2227 TransportErrorCode::PATH_UNSTABLE_OR_POOR.into(),
+2228 ) {
+2229 warn!(?err, "failed closing path");
+2230 }
+2231
+2232 self.events.push_back(Event::Path(PathEvent::LocallyClosed {
+2233 id: path_id,
+2234 error: PathError::ValidationFailed,
+2235 }));
+2236 }
+2237 PathTimer::Pacing => trace!("pacing timer expired"),
+2238 PathTimer::MaxAckDelay => {
+2239 trace!("max ack delay reached");
+2240 // This timer is only armed in the Data space
+2241 self.spaces[SpaceId::Data]
+2242 .for_path(path_id)
+2243 .pending_acks
+2244 .on_max_ack_delay_timeout()
+2245 }
+2246 PathTimer::DiscardPath => {
+2247 // The path was abandoned and 3*PTO has expired since. Clean up all
+2248 // remaining state and install stateless reset token.
+2249 self.timers.stop_per_path(path_id, self.qlog.with_time(now));
+2250 if let Some(loc_cid_state) = self.local_cid_state.remove(&path_id) {
+2251 let (min_seq, max_seq) = loc_cid_state.active_seq();
+2252 for seq in min_seq..=max_seq {
+2253 self.endpoint_events.push_back(
+2254 EndpointEventInner::RetireConnectionId(
+2255 now, path_id, seq, false,
+2256 ),
+2257 );
+2258 }
+2259 }
+2260 self.discard_path(path_id, now);
+2261 }
+2262 }
+2263 }
+2264 }
+2265 }
+2266 }
+2267
+2268 /// Close a connection immediately
+2269 ///
+2270 /// This does not ensure delivery of outstanding data. It is the application's responsibility to
+2271 /// call this only when all important communications have been completed, e.g. by calling
+2272 /// [`SendStream::finish`] on outstanding streams and waiting for the corresponding
+2273 /// [`StreamEvent::Finished`] event.
+2274 ///
+2275 /// If [`Streams::send_streams`] returns 0, all outstanding stream data has been
+2276 /// delivered. There may still be data from the peer that has not been received.
+2277 ///
+2278 /// [`StreamEvent::Finished`]: crate::StreamEvent::Finished
+2279 pub fn close(&mut self, now: Instant, error_code: VarInt, reason: Bytes) {
+2280 self.close_inner(
+2281 now,
+2282 Close::Application(frame::ApplicationClose { error_code, reason }),
+2283 )
+2284 }
+2285
+2286 fn close_inner(&mut self, now: Instant, reason: Close) {
+2287 let was_closed = self.state.is_closed();
+2288 if !was_closed {
+2289 self.close_common();
+2290 self.set_close_timer(now);
+2291 self.close = true;
+2292 self.state.move_to_closed_local(reason);
+2293 }
+2294 }
+2295
+2296 /// Control datagrams
+2297 pub fn datagrams(&mut self) -> Datagrams<'_> {
+2298 Datagrams { conn: self }
+2299 }
+2300
+2301 /// Returns connection statistics
+2302 pub fn stats(&mut self) -> ConnectionStats {
+2303 self.stats.clone()
+2304 }
+2305
+2306 /// Returns path statistics
+2307 pub fn path_stats(&mut self, path_id: PathId) -> Option<PathStats> {
+2308 let path = self.paths.get(&path_id)?;
+2309 let stats = self.path_stats.entry(path_id).or_default();
+2310 stats.rtt = path.data.rtt.get();
+2311 stats.cwnd = path.data.congestion.window();
+2312 stats.current_mtu = path.data.mtud.current_mtu();
+2313 Some(*stats)
2314 }
2315
-2316 /// Modify the number of open paths allowed when multipath is enabled
+2316 /// Ping the remote endpoint
2317 ///
-2318 /// When reducing the number of concurrent paths this will only affect delaying sending
-2319 /// new MAX_PATH_ID frames until fewer than this number of paths are possible. To
-2320 /// actively reduce paths they must be closed using [`Connection::close_path`], which
-2321 /// can also be used to close not-yet-opened paths.
-2322 ///
-2323 /// If multipath is not negotiated (see the [`TransportConfig`]) this can not enable
-2324 /// multipath and will fail.
-2325 pub fn set_max_concurrent_paths(
-2326 &mut self,
-2327 now: Instant,
-2328 count: NonZeroU32,
-2329 ) -> Result<(), MultipathNotNegotiated> {
-2330 if !self.is_multipath_negotiated() {
-2331 return Err(MultipathNotNegotiated { _private: () });
-2332 }
-2333 self.max_concurrent_paths = count;
-2334
-2335 let in_use_count = self
-2336 .local_max_path_id
-2337 .next()
-2338 .saturating_sub(self.abandoned_paths.len() as u32)
-2339 .as_u32();
-2340 let extra_needed = count.get().saturating_sub(in_use_count);
-2341 let new_max_path_id = self.local_max_path_id.saturating_add(extra_needed);
-2342
-2343 self.set_max_path_id(now, new_max_path_id);
-2344
-2345 Ok(())
-2346 }
-2347
-2348 /// If needed, issues a new MAX_PATH_ID frame and new CIDs for any newly allowed paths
-2349 fn set_max_path_id(&mut self, now: Instant, max_path_id: PathId) {
-2350 if max_path_id <= self.local_max_path_id {
+2318 /// Causes an ACK-eliciting packet to be transmitted on the connection.
+2319 pub fn ping(&mut self) {
+2320 // TODO(flub): This is very brute-force: it pings *all* the paths. Instead it would
+2321 // be nice if we could only send a single packet for this.
+2322 for path_data in self.spaces[self.highest_space].number_spaces.values_mut() {
+2323 path_data.ping_pending = true;
+2324 }
+2325 }
+2326
+2327 /// Ping the remote endpoint over a specific path
+2328 ///
+2329 /// Causes an ACK-eliciting packet to be transmitted on the path.
+2330 pub fn ping_path(&mut self, path: PathId) -> Result<(), ClosedPath> {
+2331 let path_data = self.spaces[self.highest_space]
+2332 .number_spaces
+2333 .get_mut(&path)
+2334 .ok_or(ClosedPath { _private: () })?;
+2335 path_data.ping_pending = true;
+2336 Ok(())
+2337 }
+2338
+2339 /// Update traffic keys spontaneously
+2340 ///
+2341 /// This can be useful for testing key updates, as they otherwise only happen infrequently.
+2342 pub fn force_key_update(&mut self) {
+2343 if !self.state.is_established() {
+2344 debug!("ignoring forced key update in illegal state");
+2345 return;
+2346 }
+2347 if self.prev_crypto.is_some() {
+2348 // We already just updated, or are currently updating, the keys. Concurrent key updates
+2349 // are illegal.
+2350 debug!("ignoring redundant forced key update");
2351 return;
2352 }
-2353
-2354 self.local_max_path_id = max_path_id;
-2355 self.spaces[SpaceId::Data].pending.max_path_id = true;
-2356
-2357 self.issue_first_path_cids(now);
-2358 }
-2359
-2360 /// Current number of remotely initiated streams that may be concurrently open
-2361 ///
-2362 /// If the target for this limit is reduced using [`set_max_concurrent_streams`](Self::set_max_concurrent_streams),
-2363 /// it will not change immediately, even if fewer streams are open. Instead, it will
-2364 /// decrement by one for each time a remotely initiated stream of matching directionality is closed.
-2365 pub fn max_concurrent_streams(&self, dir: Dir) -> u64 {
-2366 self.streams.max_concurrent(dir)
+2353 self.update_keys(None, false);
+2354 }
+2355
+2356 /// Get a session reference
+2357 pub fn crypto_session(&self) -> &dyn crypto::Session {
+2358 &*self.crypto
+2359 }
+2360
+2361 /// Whether the connection is in the process of being established
+2362 ///
+2363 /// If this returns `false`, the connection may be either established or closed, signaled by the
+2364 /// emission of a `Connected` or `ConnectionLost` message respectively.
+2365 pub fn is_handshaking(&self) -> bool {
+2366 self.state.is_handshake()
2367 }
2368
-2369 /// See [`TransportConfig::send_window()`]
-2370 pub fn set_send_window(&mut self, send_window: u64) {
-2371 self.streams.set_send_window(send_window);
-2372 }
-2373
-2374 /// See [`TransportConfig::receive_window()`]
-2375 pub fn set_receive_window(&mut self, receive_window: VarInt) {
-2376 if self.streams.set_receive_window(receive_window) {
-2377 self.spaces[SpaceId::Data].pending.max_data = true;
-2378 }
-2379 }
-2380
-2381 /// Whether the Multipath for QUIC extension is enabled.
-2382 ///
-2383 /// Multipath is only enabled after the handshake is completed and if it was enabled by both
-2384 /// peers.
-2385 pub fn is_multipath_negotiated(&self) -> bool {
-2386 !self.is_handshaking()
-2387 && self.config.max_concurrent_multipath_paths.is_some()
-2388 && self.peer_params.initial_max_path_id.is_some()
-2389 }
-2390
-2391 fn on_ack_received(
-2392 &mut self,
-2393 now: Instant,
-2394 space: SpaceId,
-2395 ack: frame::Ack,
-2396 ) -> Result<(), TransportError> {
-2397 // All ACKs are referencing path 0
-2398 let path = PathId::ZERO;
-2399 self.inner_on_ack_received(now, space, path, ack)
-2400 }
-2401
-2402 fn on_path_ack_received(
-2403 &mut self,
-2404 now: Instant,
-2405 space: SpaceId,
-2406 path_ack: frame::PathAck,
-2407 ) -> Result<(), TransportError> {
-2408 let (ack, path) = path_ack.into_ack();
-2409 self.inner_on_ack_received(now, space, path, ack)
-2410 }
-2411
-2412 /// Handles an ACK frame acknowledging packets sent on *path*.
-2413 fn inner_on_ack_received(
-2414 &mut self,
-2415 now: Instant,
-2416 space: SpaceId,
-2417 path: PathId,
-2418 ack: frame::Ack,
-2419 ) -> Result<(), TransportError> {
-2420 if self.abandoned_paths.contains(&path) {
-2421 // See also https://www.ietf.org/archive/id/draft-ietf-quic-multipath-17.html#section-3.4.3-3
-2422 // > PATH_ACK frames received with an abandoned path ID are silently ignored, as specified in Section 4.
-2423 trace!("silently ignoring PATH_ACK on abandoned path");
-2424 return Ok(());
-2425 }
-2426 if ack.largest >= self.spaces[space].for_path(path).next_packet_number {
-2427 return Err(TransportError::PROTOCOL_VIOLATION("unsent packet acked"));
-2428 }
-2429 let new_largest = {
-2430 let space = &mut self.spaces[space].for_path(path);
-2431 if space.largest_acked_packet.is_none_or(|pn| ack.largest > pn) {
-2432 space.largest_acked_packet = Some(ack.largest);
-2433 if let Some(info) = space.sent_packets.get(ack.largest) {
-2434 // This should always succeed, but a misbehaving peer might ACK a packet we
-2435 // haven't sent. At worst, that will result in us spuriously reducing the
-2436 // congestion window.
-2437 space.largest_acked_packet_sent = info.time_sent;
-2438 }
-2439 true
-2440 } else {
-2441 false
-2442 }
-2443 };
-2444
-2445 if self.detect_spurious_loss(&ack, space, path) {
-2446 self.path_data_mut(path)
-2447 .congestion
-2448 .on_spurious_congestion_event();
-2449 }
-2450
-2451 // Avoid DoS from unreasonably huge ack ranges by filtering out just the new acks.
-2452 let mut newly_acked = ArrayRangeSet::new();
-2453 for range in ack.iter() {
-2454 self.spaces[space].for_path(path).check_ack(range.clone())?;
-2455 for (pn, _) in self.spaces[space]
-2456 .for_path(path)
-2457 .sent_packets
-2458 .iter_range(range)
-2459 {
-2460 newly_acked.insert_one(pn);
-2461 }
-2462 }
-2463
-2464 if newly_acked.is_empty() {
-2465 return Ok(());
-2466 }
-2467
-2468 let mut ack_eliciting_acked = false;
-2469 for packet in newly_acked.elts() {
-2470 if let Some(info) = self.spaces[space].for_path(path).take(packet) {
-2471 for (acked_path_id, acked_pn) in info.largest_acked.iter() {
-2472 // Assume ACKs for all packets below the largest acknowledged in
-2473 // `packet` have been received. This can cause the peer to spuriously
-2474 // retransmit if some of our earlier ACKs were lost, but allows for
-2475 // simpler state tracking. See discussion at
-2476 // https://www.rfc-editor.org/rfc/rfc9000.html#name-limiting-ranges-by-tracking
-2477 if let Some(pns) = self.spaces[space].path_space_mut(*acked_path_id) {
-2478 pns.pending_acks.subtract_below(*acked_pn);
-2479 }
-2480 }
-2481 ack_eliciting_acked |= info.ack_eliciting;
-2482
-2483 // Notify MTU discovery that a packet was acked, because it might be an MTU probe
-2484 let path_data = self.path_data_mut(path);
-2485 let mtu_updated = path_data.mtud.on_acked(space, packet, info.size);
-2486 if mtu_updated {
-2487 path_data
-2488 .congestion
-2489 .on_mtu_update(path_data.mtud.current_mtu());
-2490 }
-2491
-2492 // Notify ack frequency that a packet was acked, because it might contain an ACK_FREQUENCY frame
-2493 self.ack_frequency.on_acked(path, packet);
-2494
-2495 self.on_packet_acked(now, path, info);
-2496 }
-2497 }
-2498
-2499 let largest_ackd = self.spaces[space].for_path(path).largest_acked_packet;
-2500 let app_limited = self.app_limited;
-2501 let path_data = self.path_data_mut(path);
-2502 let in_flight = path_data.in_flight.bytes;
-2503
-2504 path_data
-2505 .congestion
-2506 .on_end_acks(now, in_flight, app_limited, largest_ackd);
-2507
-2508 if new_largest && ack_eliciting_acked {
-2509 let ack_delay = if space != SpaceId::Data {
-2510 Duration::from_micros(0)
-2511 } else {
-2512 cmp::min(
-2513 self.ack_frequency.peer_max_ack_delay,
-2514 Duration::from_micros(ack.delay << self.peer_params.ack_delay_exponent.0),
-2515 )
-2516 };
-2517 let rtt = now.saturating_duration_since(
-2518 self.spaces[space].for_path(path).largest_acked_packet_sent,
-2519 );
-2520
-2521 let next_pn = self.spaces[space].for_path(path).next_packet_number;
-2522 let path_data = self.path_data_mut(path);
-2523 // TODO(@divma): should be a method of path, should be contained in a single place
-2524 path_data.rtt.update(ack_delay, rtt);
-2525 if path_data.first_packet_after_rtt_sample.is_none() {
-2526 path_data.first_packet_after_rtt_sample = Some((space, next_pn));
-2527 }
-2528 }
+2369 /// Whether the connection is closed
+2370 ///
+2371 /// Closed connections cannot transport any further data. A connection becomes closed when
+2372 /// either peer application intentionally closes it, or when either transport layer detects an
+2373 /// error such as a time-out or certificate validation failure.
+2374 ///
+2375 /// A `ConnectionLost` event is emitted with details when the connection becomes closed.
+2376 pub fn is_closed(&self) -> bool {
+2377 self.state.is_closed()
+2378 }
+2379
+2380 /// Whether there is no longer any need to keep the connection around
+2381 ///
+2382 /// Closed connections become drained after a brief timeout to absorb any remaining in-flight
+2383 /// packets from the peer. All drained connections have been closed.
+2384 pub fn is_drained(&self) -> bool {
+2385 self.state.is_drained()
+2386 }
+2387
+2388 /// For clients, if the peer accepted the 0-RTT data packets
+2389 ///
+2390 /// The value is meaningless until after the handshake completes.
+2391 pub fn accepted_0rtt(&self) -> bool {
+2392 self.accepted_0rtt
+2393 }
+2394
+2395 /// Whether 0-RTT is/was possible during the handshake
+2396 pub fn has_0rtt(&self) -> bool {
+2397 self.zero_rtt_enabled
+2398 }
+2399
+2400 /// Whether there are any pending retransmits
+2401 pub fn has_pending_retransmits(&self) -> bool {
+2402 !self.spaces[SpaceId::Data].pending.is_empty(&self.streams)
+2403 }
+2404
+2405 /// Look up whether we're the client or server of this Connection
+2406 pub fn side(&self) -> Side {
+2407 self.side.side()
+2408 }
+2409
+2410 /// Get the address observed by the remote over the given path
+2411 pub fn path_observed_address(&self, path_id: PathId) -> Result<Option<SocketAddr>, ClosedPath> {
+2412 self.path(path_id)
+2413 .map(|path_data| {
+2414 path_data
+2415 .last_observed_addr_report
+2416 .as_ref()
+2417 .map(|observed| observed.socket_addr())
+2418 })
+2419 .ok_or(ClosedPath { _private: () })
+2420 }
+2421
+2422 /// Current best estimate of this connection's latency (round-trip-time)
+2423 pub fn rtt(&self, path_id: PathId) -> Option<Duration> {
+2424 self.path(path_id).map(|d| d.rtt.get())
+2425 }
+2426
+2427 /// Current state of this connection's congestion controller, for debugging purposes
+2428 pub fn congestion_state(&self, path_id: PathId) -> Option<&dyn Controller> {
+2429 self.path(path_id).map(|d| d.congestion.as_ref())
+2430 }
+2431
+2432 /// Modify the number of remotely initiated streams that may be concurrently open
+2433 ///
+2434 /// No streams may be opened by the peer unless fewer than `count` are already open. Large
+2435 /// `count`s increase both minimum and worst-case memory consumption.
+2436 pub fn set_max_concurrent_streams(&mut self, dir: Dir, count: VarInt) {
+2437 self.streams.set_max_concurrent(dir, count);
+2438 // If the limit was reduced, then a flow control update previously deemed insignificant may
+2439 // now be significant.
+2440 let pending = &mut self.spaces[SpaceId::Data].pending;
+2441 self.streams.queue_max_stream_id(pending);
+2442 }
+2443
+2444 /// Modify the number of open paths allowed when multipath is enabled
+2445 ///
+2446 /// When reducing the number of concurrent paths this will only affect delaying sending
+2447 /// new MAX_PATH_ID frames until fewer than this number of paths are possible. To
+2448 /// actively reduce paths they must be closed using [`Connection::close_path`], which
+2449 /// can also be used to close not-yet-opened paths.
+2450 ///
+2451 /// If multipath is not negotiated (see the [`TransportConfig`]) this can not enable
+2452 /// multipath and will fail.
+2453 pub fn set_max_concurrent_paths(
+2454 &mut self,
+2455 now: Instant,
+2456 count: NonZeroU32,
+2457 ) -> Result<(), MultipathNotNegotiated> {
+2458 if !self.is_multipath_negotiated() {
+2459 return Err(MultipathNotNegotiated { _private: () });
+2460 }
+2461 self.max_concurrent_paths = count;
+2462
+2463 let in_use_count = self
+2464 .local_max_path_id
+2465 .next()
+2466 .saturating_sub(self.abandoned_paths.len() as u32)
+2467 .as_u32();
+2468 let extra_needed = count.get().saturating_sub(in_use_count);
+2469 let new_max_path_id = self.local_max_path_id.saturating_add(extra_needed);
+2470
+2471 self.set_max_path_id(now, new_max_path_id);
+2472
+2473 Ok(())
+2474 }
+2475
+2476 /// If needed, issues a new MAX_PATH_ID frame and new CIDs for any newly allowed paths
+2477 fn set_max_path_id(&mut self, now: Instant, max_path_id: PathId) {
+2478 if max_path_id <= self.local_max_path_id {
+2479 return;
+2480 }
+2481
+2482 self.local_max_path_id = max_path_id;
+2483 self.spaces[SpaceId::Data].pending.max_path_id = true;
+2484
+2485 self.issue_first_path_cids(now);
+2486 }
+2487
+2488 /// Current number of remotely initiated streams that may be concurrently open
+2489 ///
+2490 /// If the target for this limit is reduced using [`set_max_concurrent_streams`](Self::set_max_concurrent_streams),
+2491 /// it will not change immediately, even if fewer streams are open. Instead, it will
+2492 /// decrement by one for each time a remotely initiated stream of matching directionality is closed.
+2493 pub fn max_concurrent_streams(&self, dir: Dir) -> u64 {
+2494 self.streams.max_concurrent(dir)
+2495 }
+2496
+2497 /// See [`TransportConfig::send_window()`]
+2498 pub fn set_send_window(&mut self, send_window: u64) {
+2499 self.streams.set_send_window(send_window);
+2500 }
+2501
+2502 /// See [`TransportConfig::receive_window()`]
+2503 pub fn set_receive_window(&mut self, receive_window: VarInt) {
+2504 if self.streams.set_receive_window(receive_window) {
+2505 self.spaces[SpaceId::Data].pending.max_data = true;
+2506 }
+2507 }
+2508
+2509 /// Whether the Multipath for QUIC extension is enabled.
+2510 ///
+2511 /// Multipath is only enabled after the handshake is completed and if it was enabled by both
+2512 /// peers.
+2513 pub fn is_multipath_negotiated(&self) -> bool {
+2514 !self.is_handshaking()
+2515 && self.config.max_concurrent_multipath_paths.is_some()
+2516 && self.peer_params.initial_max_path_id.is_some()
+2517 }
+2518
+2519 fn on_ack_received(
+2520 &mut self,
+2521 now: Instant,
+2522 space: SpaceId,
+2523 ack: frame::Ack,
+2524 ) -> Result<(), TransportError> {
+2525 // All ACKs are referencing path 0
+2526 let path = PathId::ZERO;
+2527 self.inner_on_ack_received(now, space, path, ack)
+2528 }
2529
-2530 // Must be called before crypto/pto_count are clobbered
-2531 self.detect_lost_packets(now, space, path, true);
-2532
-2533 if self.peer_completed_address_validation(path) {
-2534 self.path_data_mut(path).pto_count = 0;
-2535 }
-2536
-2537 // Explicit congestion notification
-2538 // TODO(@divma): this code is a good example of logic that should be contained in a single
-2539 // place but it's split between the path data and the packet number space data, we should
-2540 // find a way to make this work without two lookups
-2541 if self.path_data(path).sending_ecn {
-2542 if let Some(ecn) = ack.ecn {
-2543 // We only examine ECN counters from ACKs that we are certain we received in transmit
-2544 // order, allowing us to compute an increase in ECN counts to compare against the number
-2545 // of newly acked packets that remains well-defined in the presence of arbitrary packet
-2546 // reordering.
-2547 if new_largest {
-2548 let sent = self.spaces[space].for_path(path).largest_acked_packet_sent;
-2549 self.process_ecn(now, space, path, newly_acked.len() as u64, ecn, sent);
-2550 }
-2551 } else {
-2552 // We always start out sending ECN, so any ack that doesn't acknowledge it disables it.
-2553 debug!("ECN not acknowledged by peer");
-2554 self.path_data_mut(path).sending_ecn = false;
-2555 }
+2530 fn on_path_ack_received(
+2531 &mut self,
+2532 now: Instant,
+2533 space: SpaceId,
+2534 path_ack: frame::PathAck,
+2535 ) -> Result<(), TransportError> {
+2536 let (ack, path) = path_ack.into_ack();
+2537 self.inner_on_ack_received(now, space, path, ack)
+2538 }
+2539
+2540 /// Handles an ACK frame acknowledging packets sent on *path*.
+2541 fn inner_on_ack_received(
+2542 &mut self,
+2543 now: Instant,
+2544 space: SpaceId,
+2545 path: PathId,
+2546 ack: frame::Ack,
+2547 ) -> Result<(), TransportError> {
+2548 if self.abandoned_paths.contains(&path) {
+2549 // See also https://www.ietf.org/archive/id/draft-ietf-quic-multipath-17.html#section-3.4.3-3
+2550 // > PATH_ACK frames received with an abandoned path ID are silently ignored, as specified in Section 4.
+2551 trace!("silently ignoring PATH_ACK on abandoned path");
+2552 return Ok(());
+2553 }
+2554 if ack.largest >= self.spaces[space].for_path(path).next_packet_number {
+2555 return Err(TransportError::PROTOCOL_VIOLATION("unsent packet acked"));
2556 }
-2557
-2558 self.set_loss_detection_timer(now, path);
-2559 Ok(())
-2560 }
-2561
-2562 fn detect_spurious_loss(&mut self, ack: &frame::Ack, space: SpaceId, path: PathId) -> bool {
-2563 let lost_packets = &mut self.spaces[space].for_path(path).lost_packets;
-2564
-2565 if lost_packets.is_empty() {
-2566 return false;
-2567 }
-2568
-2569 for range in ack.iter() {
-2570 let spurious_losses: Vec<u64> = lost_packets
-2571 .iter_range(range.clone())
-2572 .map(|(pn, _info)| pn)
-2573 .collect();
-2574
-2575 for pn in spurious_losses {
-2576 lost_packets.remove(pn);
-2577 }
-2578 }
-2579
-2580 // If this ACK frame acknowledged all deemed lost packets,
-2581 // then we have raised a spurious congestion event in the past.
-2582 // We cannot conclude when there are remaining packets,
-2583 // but future ACK frames might indicate a spurious loss detection.
-2584 lost_packets.is_empty()
-2585 }
-2586
-2587 /// Drain lost packets that we reasonably think will never arrive
-2588 ///
-2589 /// The current criterion is copied from `msquic`:
-2590 /// discard packets that were sent earlier than 2 probe timeouts ago.
-2591 fn drain_lost_packets(&mut self, now: Instant, space: SpaceId, path: PathId) {
-2592 let two_pto = 2 * self.path_data(path).rtt.pto_base();
-2593
-2594 let lost_packets = &mut self.spaces[space].for_path(path).lost_packets;
-2595 lost_packets.retain(|_pn, info| now.saturating_duration_since(info.time_sent) <= two_pto);
-2596 }
-2597
-2598 /// Process a new ECN block from an in-order ACK
-2599 fn process_ecn(
-2600 &mut self,
-2601 now: Instant,
-2602 space: SpaceId,
-2603 path: PathId,
-2604 newly_acked: u64,
-2605 ecn: frame::EcnCounts,
-2606 largest_sent_time: Instant,
-2607 ) {
-2608 match self.spaces[space]
-2609 .for_path(path)
-2610 .detect_ecn(newly_acked, ecn)
-2611 {
-2612 Err(e) => {
-2613 debug!("halting ECN due to verification failure: {}", e);
-2614
-2615 self.path_data_mut(path).sending_ecn = false;
-2616 // Wipe out the existing value because it might be garbage and could interfere with
-2617 // future attempts to use ECN on new paths.
-2618 self.spaces[space].for_path(path).ecn_feedback = frame::EcnCounts::ZERO;
-2619 }
-2620 Ok(false) => {}
-2621 Ok(true) => {
-2622 self.path_stats.entry(path).or_default().congestion_events += 1;
-2623 self.path_data_mut(path).congestion.on_congestion_event(
-2624 now,
-2625 largest_sent_time,
-2626 false,
-2627 true,
-2628 0,
-2629 );
-2630 }
-2631 }
-2632 }
-2633
-2634 // Not timing-aware, so it's safe to call this for inferred acks, such as arise from
-2635 // high-latency handshakes
-2636 fn on_packet_acked(&mut self, now: Instant, path_id: PathId, info: SentPacket) {
-2637 self.paths
-2638 .get_mut(&path_id)
-2639 .expect("known path")
-2640 .remove_in_flight(&info);
-2641 let app_limited = self.app_limited;
-2642 let path = self.path_data_mut(path_id);
-2643 if info.ack_eliciting && !path.is_validating_path() {
-2644 // Only pass ACKs to the congestion controller if we are not validating the current
-2645 // path, so as to ignore any ACKs from older paths still coming in.
-2646 let rtt = path.rtt;
-2647 path.congestion
-2648 .on_ack(now, info.time_sent, info.size.into(), app_limited, &rtt);
-2649 }
-2650
-2651 // Update state for confirmed delivery of frames
-2652 if let Some(retransmits) = info.retransmits.get() {
-2653 for (id, _) in retransmits.reset_stream.iter() {
-2654 self.streams.reset_acked(*id);
+2557 let new_largest = {
+2558 let space = &mut self.spaces[space].for_path(path);
+2559 if space.largest_acked_packet.is_none_or(|pn| ack.largest > pn) {
+2560 space.largest_acked_packet = Some(ack.largest);
+2561 if let Some(info) = space.sent_packets.get(ack.largest) {
+2562 // This should always succeed, but a misbehaving peer might ACK a packet we
+2563 // haven't sent. At worst, that will result in us spuriously reducing the
+2564 // congestion window.
+2565 space.largest_acked_packet_sent = info.time_sent;
+2566 }
+2567 true
+2568 } else {
+2569 false
+2570 }
+2571 };
+2572
+2573 if self.detect_spurious_loss(&ack, space, path) {
+2574 self.path_data_mut(path)
+2575 .congestion
+2576 .on_spurious_congestion_event();
+2577 }
+2578
+2579 // Avoid DoS from unreasonably huge ack ranges by filtering out just the new acks.
+2580 let mut newly_acked = ArrayRangeSet::new();
+2581 for range in ack.iter() {
+2582 self.spaces[space].for_path(path).check_ack(range.clone())?;
+2583 for (pn, _) in self.spaces[space]
+2584 .for_path(path)
+2585 .sent_packets
+2586 .iter_range(range)
+2587 {
+2588 newly_acked.insert_one(pn);
+2589 }
+2590 }
+2591
+2592 if newly_acked.is_empty() {
+2593 return Ok(());
+2594 }
+2595
+2596 let mut ack_eliciting_acked = false;
+2597 for packet in newly_acked.elts() {
+2598 if let Some(info) = self.spaces[space].for_path(path).take(packet) {
+2599 for (acked_path_id, acked_pn) in info.largest_acked.iter() {
+2600 // Assume ACKs for all packets below the largest acknowledged in
+2601 // `packet` have been received. This can cause the peer to spuriously
+2602 // retransmit if some of our earlier ACKs were lost, but allows for
+2603 // simpler state tracking. See discussion at
+2604 // https://www.rfc-editor.org/rfc/rfc9000.html#name-limiting-ranges-by-tracking
+2605 if let Some(pns) = self.spaces[space].path_space_mut(*acked_path_id) {
+2606 pns.pending_acks.subtract_below(*acked_pn);
+2607 }
+2608 }
+2609 ack_eliciting_acked |= info.ack_eliciting;
+2610
+2611 // Notify MTU discovery that a packet was acked, because it might be an MTU probe
+2612 let path_data = self.path_data_mut(path);
+2613 let mtu_updated = path_data.mtud.on_acked(space, packet, info.size);
+2614 if mtu_updated {
+2615 path_data
+2616 .congestion
+2617 .on_mtu_update(path_data.mtud.current_mtu());
+2618 }
+2619
+2620 // Notify ack frequency that a packet was acked, because it might contain an ACK_FREQUENCY frame
+2621 self.ack_frequency.on_acked(path, packet);
+2622
+2623 self.on_packet_acked(now, path, info);
+2624 }
+2625 }
+2626
+2627 let largest_ackd = self.spaces[space].for_path(path).largest_acked_packet;
+2628 let app_limited = self.app_limited;
+2629 let path_data = self.path_data_mut(path);
+2630 let in_flight = path_data.in_flight.bytes;
+2631
+2632 path_data
+2633 .congestion
+2634 .on_end_acks(now, in_flight, app_limited, largest_ackd);
+2635
+2636 if new_largest && ack_eliciting_acked {
+2637 let ack_delay = if space != SpaceId::Data {
+2638 Duration::from_micros(0)
+2639 } else {
+2640 cmp::min(
+2641 self.ack_frequency.peer_max_ack_delay,
+2642 Duration::from_micros(ack.delay << self.peer_params.ack_delay_exponent.0),
+2643 )
+2644 };
+2645 let rtt = now.saturating_duration_since(
+2646 self.spaces[space].for_path(path).largest_acked_packet_sent,
+2647 );
+2648
+2649 let next_pn = self.spaces[space].for_path(path).next_packet_number;
+2650 let path_data = self.path_data_mut(path);
+2651 // TODO(@divma): should be a method of path, should be contained in a single place
+2652 path_data.rtt.update(ack_delay, rtt);
+2653 if path_data.first_packet_after_rtt_sample.is_none() {
+2654 path_data.first_packet_after_rtt_sample = Some((space, next_pn));
2655 }
2656 }
2657
-2658 for frame in info.stream_frames {
-2659 self.streams.received_ack_of(frame);
-2660 }
-2661 }
-2662
-2663 fn set_key_discard_timer(&mut self, now: Instant, space: SpaceId) {
-2664 let start = if self.zero_rtt_crypto.is_some() {
-2665 now
-2666 } else {
-2667 self.prev_crypto
-2668 .as_ref()
-2669 .expect("no previous keys")
-2670 .end_packet
-2671 .as_ref()
-2672 .expect("update not acknowledged yet")
-2673 .1
-2674 };
-2675
-2676 // QUIC-MULTIPATH § 2.5 Key Phase Update Process: use largest PTO off all paths.
-2677 self.timers.set(
-2678 Timer::Conn(ConnTimer::KeyDiscard),
-2679 start + self.pto_max_path(space, false) * 3,
-2680 self.qlog.with_time(now),
-2681 );
-2682 }
-2683
-2684 /// Handle a [`PathTimer::LossDetection`] timeout.
-2685 ///
-2686 /// This timer expires for two reasons:
-2687 /// - An ACK-eliciting packet we sent should be considered lost.
-2688 /// - The PTO may have expired and a tail-loss probe needs to be scheduled.
-2689 ///
-2690 /// The former needs us to schedule re-transmission of the lost data.
-2691 ///
-2692 /// The latter means we have not received an ACK for an ack-eliciting packet we sent
-2693 /// within the PTO time-window. We need to schedule a tail-loss probe, an ack-eliciting
-2694 /// packet, to try and elicit new acknowledgements. These new acknowledgements will
-2695 /// indicate whether the previously sent packets were lost or not.
-2696 fn on_loss_detection_timeout(&mut self, now: Instant, path_id: PathId) {
-2697 if let Some((_, pn_space)) = self.loss_time_and_space(path_id) {
-2698 // Time threshold loss Detection
-2699 self.detect_lost_packets(now, pn_space, path_id, false);
-2700 self.set_loss_detection_timer(now, path_id);
-2701 return;
-2702 }
-2703
-2704 let (_, space) = match self.pto_time_and_space(now, path_id) {
-2705 Some(x) => x,
-2706 None => {
-2707 error!(%path_id, "PTO expired while unset");
-2708 return;
-2709 }
-2710 };
-2711 trace!(
-2712 in_flight = self.path_data(path_id).in_flight.bytes,
-2713 count = self.path_data(path_id).pto_count,
-2714 ?space,
-2715 %path_id,
-2716 "PTO fired"
-2717 );
-2718
-2719 let count = match self.path_data(path_id).in_flight.ack_eliciting {
-2720 // A PTO when we're not expecting any ACKs must be due to handshake anti-amplification
-2721 // deadlock preventions
-2722 0 => {
-2723 debug_assert!(!self.peer_completed_address_validation(path_id));
-2724 1
-2725 }
-2726 // Conventional loss probe
-2727 _ => 2,
-2728 };
-2729 let pns = self.spaces[space].for_path(path_id);
-2730 pns.loss_probes = pns.loss_probes.saturating_add(count);
-2731 let path_data = self.path_data_mut(path_id);
-2732 path_data.pto_count = path_data.pto_count.saturating_add(1);
-2733 self.set_loss_detection_timer(now, path_id);
-2734 }
-2735
-2736 /// Detect any lost packets
-2737 ///
-2738 /// There are two cases in which we detects lost packets:
-2739 ///
-2740 /// - We received an ACK packet.
-2741 /// - The [`PathTimer::LossDetection`] timer expired. So there is an un-acknowledged packet
-2742 /// that was followed by an acknowledged packet. The loss timer for this
-2743 /// un-acknowledged packet expired and we need to detect that packet as lost.
-2744 ///
-2745 /// Packets are lost if they are both (See RFC9002 §6.1):
-2746 ///
-2747 /// - Unacknowledged, in flight and sent prior to an acknowledged packet.
-2748 /// - Old enough by either:
-2749 /// - Having a packet number [`TransportConfig::packet_threshold`] lower then the last
-2750 /// acknowledged packet.
-2751 /// - Being sent [`TransportConfig::time_threshold`] * RTT in the past.
-2752 fn detect_lost_packets(
-2753 &mut self,
-2754 now: Instant,
-2755 pn_space: SpaceId,
-2756 path_id: PathId,
-2757 due_to_ack: bool,
-2758 ) {
-2759 let mut lost_packets = Vec::<u64>::new();
-2760 let mut lost_mtu_probe = None;
-2761 let mut in_persistent_congestion = false;
-2762 let mut size_of_lost_packets = 0u64;
-2763 self.spaces[pn_space].for_path(path_id).loss_time = None;
-2764
-2765 // Find all the lost packets, populating all variables initialised above.
-2766
-2767 let path = self.path_data(path_id);
-2768 let in_flight_mtu_probe = path.mtud.in_flight_mtu_probe();
-2769 let loss_delay = path
-2770 .rtt
-2771 .conservative()
-2772 .mul_f32(self.config.time_threshold)
-2773 .max(TIMER_GRANULARITY);
-2774 let first_packet_after_rtt_sample = path.first_packet_after_rtt_sample;
-2775
-2776 let largest_acked_packet = self.spaces[pn_space]
-2777 .for_path(path_id)
-2778 .largest_acked_packet
-2779 .expect("detect_lost_packets only to be called if path received at least one ACK");
-2780 let packet_threshold = self.config.packet_threshold as u64;
-2781
-2782 // InPersistentCongestion: Determine if all packets in the time period before the newest
-2783 // lost packet, including the edges, are marked lost. PTO computation must always
-2784 // include max ACK delay, i.e. operate as if in Data space (see RFC9001 §7.6.1).
-2785 let congestion_period = self
-2786 .pto(SpaceId::Data, path_id)
-2787 .saturating_mul(self.config.persistent_congestion_threshold);
-2788 let mut persistent_congestion_start: Option<Instant> = None;
-2789 let mut prev_packet = None;
-2790 let space = self.spaces[pn_space].for_path(path_id);
-2791
-2792 for (packet, info) in space.sent_packets.iter_range(0..largest_acked_packet) {
-2793 if prev_packet != Some(packet.wrapping_sub(1)) {
-2794 // An intervening packet was acknowledged
-2795 persistent_congestion_start = None;
-2796 }
-2797
-2798 // Packets sent before now - loss_delay are deemed lost.
-2799 // However, we avoid subtraction as it can panic and there's no
-2800 // saturating equivalent of this subtraction operation with a Duration.
-2801 let packet_too_old = now.saturating_duration_since(info.time_sent) >= loss_delay;
-2802 if packet_too_old || largest_acked_packet >= packet + packet_threshold {
-2803 // The packet should be declared lost.
-2804 if Some(packet) == in_flight_mtu_probe {
-2805 // Lost MTU probes are not included in `lost_packets`, because they
-2806 // should not trigger a congestion control response
-2807 lost_mtu_probe = in_flight_mtu_probe;
-2808 } else {
-2809 lost_packets.push(packet);
-2810 size_of_lost_packets += info.size as u64;
-2811 if info.ack_eliciting && due_to_ack {
-2812 match persistent_congestion_start {
-2813 // Two ACK-eliciting packets lost more than
-2814 // congestion_period apart, with no ACKed packets in between
-2815 Some(start) if info.time_sent - start > congestion_period => {
-2816 in_persistent_congestion = true;
-2817 }
-2818 // Persistent congestion must start after the first RTT sample
-2819 None if first_packet_after_rtt_sample
-2820 .is_some_and(|x| x < (pn_space, packet)) =>
-2821 {
-2822 persistent_congestion_start = Some(info.time_sent);
-2823 }
-2824 _ => {}
-2825 }
-2826 }
-2827 }
-2828 } else {
-2829 // The packet should not yet be declared lost.
-2830 if space.loss_time.is_none() {
-2831 // Since we iterate in order the lowest packet number's loss time will
-2832 // always be the earliest.
-2833 space.loss_time = Some(info.time_sent + loss_delay);
-2834 }
-2835 persistent_congestion_start = None;
-2836 }
-2837
-2838 prev_packet = Some(packet);
-2839 }
-2840
-2841 self.handle_lost_packets(
-2842 pn_space,
-2843 path_id,
-2844 now,
-2845 lost_packets,
-2846 lost_mtu_probe,
-2847 loss_delay,
-2848 in_persistent_congestion,
-2849 size_of_lost_packets,
-2850 );
-2851 }
-2852
-2853 /// Drops the path state, declaring any remaining in-flight packets as lost
-2854 fn discard_path(&mut self, path_id: PathId, now: Instant) {
-2855 trace!(%path_id, "dropping path state");
-2856 let path = self.path_data(path_id);
-2857 let in_flight_mtu_probe = path.mtud.in_flight_mtu_probe();
-2858
-2859 let mut size_of_lost_packets = 0u64; // add to path_stats.lost_bytes;
-2860 let lost_pns: Vec<_> = self.spaces[SpaceId::Data]
-2861 .for_path(path_id)
-2862 .sent_packets
-2863 .iter()
-2864 .filter(|(pn, _info)| Some(*pn) != in_flight_mtu_probe)
-2865 .map(|(pn, info)| {
-2866 size_of_lost_packets += info.size as u64;
-2867 pn
-2868 })
-2869 .collect();
-2870
-2871 if !lost_pns.is_empty() {
-2872 trace!(
-2873 %path_id,
-2874 count = lost_pns.len(),
-2875 lost_bytes = size_of_lost_packets,
-2876 "packets lost on path abandon"
-2877 );
-2878 self.handle_lost_packets(
-2879 SpaceId::Data,
-2880 path_id,
-2881 now,
-2882 lost_pns,
-2883 in_flight_mtu_probe,
-2884 Duration::ZERO,
-2885 false,
-2886 size_of_lost_packets,
-2887 );
-2888 }
-2889 self.paths.remove(&path_id);
-2890 self.spaces[SpaceId::Data].number_spaces.remove(&path_id);
-2891
-2892 let path_stats = self.path_stats.remove(&path_id).unwrap_or_default();
-2893 self.events.push_back(
-2894 PathEvent::Abandoned {
-2895 id: path_id,
-2896 path_stats,
-2897 }
-2898 .into(),
-2899 );
-2900 }
-2901
-2902 fn handle_lost_packets(
-2903 &mut self,
-2904 pn_space: SpaceId,
-2905 path_id: PathId,
-2906 now: Instant,
-2907 lost_packets: Vec<u64>,
-2908 lost_mtu_probe: Option<u64>,
-2909 loss_delay: Duration,
-2910 in_persistent_congestion: bool,
-2911 size_of_lost_packets: u64,
-2912 ) {
-2913 debug_assert!(
-2914 {
-2915 let mut sorted = lost_packets.clone();
-2916 sorted.sort();
-2917 sorted == lost_packets
-2918 },
-2919 "lost_packets must be sorted"
-2920 );
-2921
-2922 self.drain_lost_packets(now, pn_space, path_id);
-2923
-2924 // OnPacketsLost
-2925 if let Some(largest_lost) = lost_packets.last().cloned() {
-2926 let old_bytes_in_flight = self.path_data_mut(path_id).in_flight.bytes;
-2927 let largest_lost_sent = self.spaces[pn_space]
-2928 .for_path(path_id)
-2929 .sent_packets
-2930 .get(largest_lost)
-2931 .unwrap()
-2932 .time_sent;
-2933 let path_stats = self.path_stats.entry(path_id).or_default();
-2934 path_stats.lost_packets += lost_packets.len() as u64;
-2935 path_stats.lost_bytes += size_of_lost_packets;
-2936 trace!(
-2937 %path_id,
-2938 count = lost_packets.len(),
-2939 lost_bytes = size_of_lost_packets,
-2940 "packets lost",
-2941 );
-2942
-2943 for &packet in &lost_packets {
-2944 let Some(info) = self.spaces[pn_space].for_path(path_id).take(packet) else {
-2945 continue;
-2946 };
-2947 self.qlog
-2948 .emit_packet_lost(packet, &info, loss_delay, pn_space, now);
-2949 self.paths
-2950 .get_mut(&path_id)
-2951 .unwrap()
-2952 .remove_in_flight(&info);
-2953
-2954 for frame in info.stream_frames {
-2955 self.streams.retransmit(frame);
-2956 }
-2957 self.spaces[pn_space].pending |= info.retransmits;
-2958 self.path_data_mut(path_id)
-2959 .mtud
-2960 .on_non_probe_lost(packet, info.size);
-2961
-2962 self.spaces[pn_space].for_path(path_id).lost_packets.insert(
-2963 packet,
-2964 LostPacket {
-2965 time_sent: info.time_sent,
-2966 },
-2967 );
-2968 }
-2969
-2970 let path = self.path_data_mut(path_id);
-2971 if path.mtud.black_hole_detected(now) {
-2972 path.congestion.on_mtu_update(path.mtud.current_mtu());
-2973 if let Some(max_datagram_size) = self.datagrams().max_size() {
-2974 if self.datagrams.drop_oversized(max_datagram_size)
-2975 && self.datagrams.send_blocked
-2976 {
-2977 self.datagrams.send_blocked = false;
-2978 self.events.push_back(Event::DatagramsUnblocked);
-2979 }
-2980 }
-2981 self.path_stats
-2982 .entry(path_id)
-2983 .or_default()
-2984 .black_holes_detected += 1;
-2985 }
+2658 // Must be called before crypto/pto_count are clobbered
+2659 self.detect_lost_packets(now, space, path, true);
+2660
+2661 if self.peer_completed_address_validation(path) {
+2662 self.path_data_mut(path).pto_count = 0;
+2663 }
+2664
+2665 // Explicit congestion notification
+2666 // TODO(@divma): this code is a good example of logic that should be contained in a single
+2667 // place but it's split between the path data and the packet number space data, we should
+2668 // find a way to make this work without two lookups
+2669 if self.path_data(path).sending_ecn {
+2670 if let Some(ecn) = ack.ecn {
+2671 // We only examine ECN counters from ACKs that we are certain we received in transmit
+2672 // order, allowing us to compute an increase in ECN counts to compare against the number
+2673 // of newly acked packets that remains well-defined in the presence of arbitrary packet
+2674 // reordering.
+2675 if new_largest {
+2676 let sent = self.spaces[space].for_path(path).largest_acked_packet_sent;
+2677 self.process_ecn(now, space, path, newly_acked.len() as u64, ecn, sent);
+2678 }
+2679 } else {
+2680 // We always start out sending ECN, so any ack that doesn't acknowledge it disables it.
+2681 debug!("ECN not acknowledged by peer");
+2682 self.path_data_mut(path).sending_ecn = false;
+2683 }
+2684 }
+2685
+2686 self.set_loss_detection_timer(now, path);
+2687 Ok(())
+2688 }
+2689
+2690 fn detect_spurious_loss(&mut self, ack: &frame::Ack, space: SpaceId, path: PathId) -> bool {
+2691 let lost_packets = &mut self.spaces[space].for_path(path).lost_packets;
+2692
+2693 if lost_packets.is_empty() {
+2694 return false;
+2695 }
+2696
+2697 for range in ack.iter() {
+2698 let spurious_losses: Vec<u64> = lost_packets
+2699 .iter_range(range.clone())
+2700 .map(|(pn, _info)| pn)
+2701 .collect();
+2702
+2703 for pn in spurious_losses {
+2704 lost_packets.remove(pn);
+2705 }
+2706 }
+2707
+2708 // If this ACK frame acknowledged all deemed lost packets,
+2709 // then we have raised a spurious congestion event in the past.
+2710 // We cannot conclude when there are remaining packets,
+2711 // but future ACK frames might indicate a spurious loss detection.
+2712 lost_packets.is_empty()
+2713 }
+2714
+2715 /// Drain lost packets that we reasonably think will never arrive
+2716 ///
+2717 /// The current criterion is copied from `msquic`:
+2718 /// discard packets that were sent earlier than 2 probe timeouts ago.
+2719 fn drain_lost_packets(&mut self, now: Instant, space: SpaceId, path: PathId) {
+2720 let two_pto = 2 * self.path_data(path).rtt.pto_base();
+2721
+2722 let lost_packets = &mut self.spaces[space].for_path(path).lost_packets;
+2723 lost_packets.retain(|_pn, info| now.saturating_duration_since(info.time_sent) <= two_pto);
+2724 }
+2725
+2726 /// Process a new ECN block from an in-order ACK
+2727 fn process_ecn(
+2728 &mut self,
+2729 now: Instant,
+2730 space: SpaceId,
+2731 path: PathId,
+2732 newly_acked: u64,
+2733 ecn: frame::EcnCounts,
+2734 largest_sent_time: Instant,
+2735 ) {
+2736 match self.spaces[space]
+2737 .for_path(path)
+2738 .detect_ecn(newly_acked, ecn)
+2739 {
+2740 Err(e) => {
+2741 debug!("halting ECN due to verification failure: {}", e);
+2742
+2743 self.path_data_mut(path).sending_ecn = false;
+2744 // Wipe out the existing value because it might be garbage and could interfere with
+2745 // future attempts to use ECN on new paths.
+2746 self.spaces[space].for_path(path).ecn_feedback = frame::EcnCounts::ZERO;
+2747 }
+2748 Ok(false) => {}
+2749 Ok(true) => {
+2750 self.path_stats.entry(path).or_default().congestion_events += 1;
+2751 self.path_data_mut(path).congestion.on_congestion_event(
+2752 now,
+2753 largest_sent_time,
+2754 false,
+2755 true,
+2756 0,
+2757 );
+2758 }
+2759 }
+2760 }
+2761
+2762 // Not timing-aware, so it's safe to call this for inferred acks, such as arise from
+2763 // high-latency handshakes
+2764 fn on_packet_acked(&mut self, now: Instant, path_id: PathId, info: SentPacket) {
+2765 self.paths
+2766 .get_mut(&path_id)
+2767 .expect("known path")
+2768 .remove_in_flight(&info);
+2769 let app_limited = self.app_limited;
+2770 let path = self.path_data_mut(path_id);
+2771 if info.ack_eliciting && !path.is_validating_path() {
+2772 // Only pass ACKs to the congestion controller if we are not validating the current
+2773 // path, so as to ignore any ACKs from older paths still coming in.
+2774 let rtt = path.rtt;
+2775 path.congestion
+2776 .on_ack(now, info.time_sent, info.size.into(), app_limited, &rtt);
+2777 }
+2778
+2779 // Update state for confirmed delivery of frames
+2780 if let Some(retransmits) = info.retransmits.get() {
+2781 for (id, _) in retransmits.reset_stream.iter() {
+2782 self.streams.reset_acked(*id);
+2783 }
+2784 }
+2785
+2786 for frame in info.stream_frames {
+2787 self.streams.received_ack_of(frame);
+2788 }
+2789 }
+2790
+2791 fn set_key_discard_timer(&mut self, now: Instant, space: SpaceId) {
+2792 let start = if self.zero_rtt_crypto.is_some() {
+2793 now
+2794 } else {
+2795 self.prev_crypto
+2796 .as_ref()
+2797 .expect("no previous keys")
+2798 .end_packet
+2799 .as_ref()
+2800 .expect("update not acknowledged yet")
+2801 .1
+2802 };
+2803
+2804 // QUIC-MULTIPATH § 2.5 Key Phase Update Process: use largest PTO off all paths.
+2805 self.timers.set(
+2806 Timer::Conn(ConnTimer::KeyDiscard),
+2807 start + self.pto_max_path(space, false) * 3,
+2808 self.qlog.with_time(now),
+2809 );
+2810 }
+2811
+2812 /// Handle a [`PathTimer::LossDetection`] timeout.
+2813 ///
+2814 /// This timer expires for two reasons:
+2815 /// - An ACK-eliciting packet we sent should be considered lost.
+2816 /// - The PTO may have expired and a tail-loss probe needs to be scheduled.
+2817 ///
+2818 /// The former needs us to schedule re-transmission of the lost data.
+2819 ///
+2820 /// The latter means we have not received an ACK for an ack-eliciting packet we sent
+2821 /// within the PTO time-window. We need to schedule a tail-loss probe, an ack-eliciting
+2822 /// packet, to try and elicit new acknowledgements. These new acknowledgements will
+2823 /// indicate whether the previously sent packets were lost or not.
+2824 fn on_loss_detection_timeout(&mut self, now: Instant, path_id: PathId) {
+2825 if let Some((_, pn_space)) = self.loss_time_and_space(path_id) {
+2826 // Time threshold loss Detection
+2827 self.detect_lost_packets(now, pn_space, path_id, false);
+2828 self.set_loss_detection_timer(now, path_id);
+2829 return;
+2830 }
+2831
+2832 let (_, space) = match self.pto_time_and_space(now, path_id) {
+2833 Some(x) => x,
+2834 None => {
+2835 error!(%path_id, "PTO expired while unset");
+2836 return;
+2837 }
+2838 };
+2839 trace!(
+2840 in_flight = self.path_data(path_id).in_flight.bytes,
+2841 count = self.path_data(path_id).pto_count,
+2842 ?space,
+2843 %path_id,
+2844 "PTO fired"
+2845 );
+2846
+2847 let count = match self.path_data(path_id).in_flight.ack_eliciting {
+2848 // A PTO when we're not expecting any ACKs must be due to handshake anti-amplification
+2849 // deadlock preventions
+2850 0 => {
+2851 debug_assert!(!self.peer_completed_address_validation(path_id));
+2852 1
+2853 }
+2854 // Conventional loss probe
+2855 _ => 2,
+2856 };
+2857 let pns = self.spaces[space].for_path(path_id);
+2858 pns.loss_probes = pns.loss_probes.saturating_add(count);
+2859 let path_data = self.path_data_mut(path_id);
+2860 path_data.pto_count = path_data.pto_count.saturating_add(1);
+2861 self.set_loss_detection_timer(now, path_id);
+2862 }
+2863
+2864 /// Detect any lost packets
+2865 ///
+2866 /// There are two cases in which we detects lost packets:
+2867 ///
+2868 /// - We received an ACK packet.
+2869 /// - The [`PathTimer::LossDetection`] timer expired. So there is an un-acknowledged packet
+2870 /// that was followed by an acknowledged packet. The loss timer for this
+2871 /// un-acknowledged packet expired and we need to detect that packet as lost.
+2872 ///
+2873 /// Packets are lost if they are both (See RFC9002 §6.1):
+2874 ///
+2875 /// - Unacknowledged, in flight and sent prior to an acknowledged packet.
+2876 /// - Old enough by either:
+2877 /// - Having a packet number [`TransportConfig::packet_threshold`] lower then the last
+2878 /// acknowledged packet.
+2879 /// - Being sent [`TransportConfig::time_threshold`] * RTT in the past.
+2880 fn detect_lost_packets(
+2881 &mut self,
+2882 now: Instant,
+2883 pn_space: SpaceId,
+2884 path_id: PathId,
+2885 due_to_ack: bool,
+2886 ) {
+2887 let mut lost_packets = Vec::<u64>::new();
+2888 let mut lost_mtu_probe = None;
+2889 let mut in_persistent_congestion = false;
+2890 let mut size_of_lost_packets = 0u64;
+2891 self.spaces[pn_space].for_path(path_id).loss_time = None;
+2892
+2893 // Find all the lost packets, populating all variables initialised above.
+2894
+2895 let path = self.path_data(path_id);
+2896 let in_flight_mtu_probe = path.mtud.in_flight_mtu_probe();
+2897 let loss_delay = path
+2898 .rtt
+2899 .conservative()
+2900 .mul_f32(self.config.time_threshold)
+2901 .max(TIMER_GRANULARITY);
+2902 let first_packet_after_rtt_sample = path.first_packet_after_rtt_sample;
+2903
+2904 let largest_acked_packet = self.spaces[pn_space]
+2905 .for_path(path_id)
+2906 .largest_acked_packet
+2907 .expect("detect_lost_packets only to be called if path received at least one ACK");
+2908 let packet_threshold = self.config.packet_threshold as u64;
+2909
+2910 // InPersistentCongestion: Determine if all packets in the time period before the newest
+2911 // lost packet, including the edges, are marked lost. PTO computation must always
+2912 // include max ACK delay, i.e. operate as if in Data space (see RFC9001 §7.6.1).
+2913 let congestion_period = self
+2914 .pto(SpaceId::Data, path_id)
+2915 .saturating_mul(self.config.persistent_congestion_threshold);
+2916 let mut persistent_congestion_start: Option<Instant> = None;
+2917 let mut prev_packet = None;
+2918 let space = self.spaces[pn_space].for_path(path_id);
+2919
+2920 for (packet, info) in space.sent_packets.iter_range(0..largest_acked_packet) {
+2921 if prev_packet != Some(packet.wrapping_sub(1)) {
+2922 // An intervening packet was acknowledged
+2923 persistent_congestion_start = None;
+2924 }
+2925
+2926 // Packets sent before now - loss_delay are deemed lost.
+2927 // However, we avoid subtraction as it can panic and there's no
+2928 // saturating equivalent of this subtraction operation with a Duration.
+2929 let packet_too_old = now.saturating_duration_since(info.time_sent) >= loss_delay;
+2930 if packet_too_old || largest_acked_packet >= packet + packet_threshold {
+2931 // The packet should be declared lost.
+2932 if Some(packet) == in_flight_mtu_probe {
+2933 // Lost MTU probes are not included in `lost_packets`, because they
+2934 // should not trigger a congestion control response
+2935 lost_mtu_probe = in_flight_mtu_probe;
+2936 } else {
+2937 lost_packets.push(packet);
+2938 size_of_lost_packets += info.size as u64;
+2939 if info.ack_eliciting && due_to_ack {
+2940 match persistent_congestion_start {
+2941 // Two ACK-eliciting packets lost more than
+2942 // congestion_period apart, with no ACKed packets in between
+2943 Some(start) if info.time_sent - start > congestion_period => {
+2944 in_persistent_congestion = true;
+2945 }
+2946 // Persistent congestion must start after the first RTT sample
+2947 None if first_packet_after_rtt_sample
+2948 .is_some_and(|x| x < (pn_space, packet)) =>
+2949 {
+2950 persistent_congestion_start = Some(info.time_sent);
+2951 }
+2952 _ => {}
+2953 }
+2954 }
+2955 }
+2956 } else {
+2957 // The packet should not yet be declared lost.
+2958 if space.loss_time.is_none() {
+2959 // Since we iterate in order the lowest packet number's loss time will
+2960 // always be the earliest.
+2961 space.loss_time = Some(info.time_sent + loss_delay);
+2962 }
+2963 persistent_congestion_start = None;
+2964 }
+2965
+2966 prev_packet = Some(packet);
+2967 }
+2968
+2969 self.handle_lost_packets(
+2970 pn_space,
+2971 path_id,
+2972 now,
+2973 lost_packets,
+2974 lost_mtu_probe,
+2975 loss_delay,
+2976 in_persistent_congestion,
+2977 size_of_lost_packets,
+2978 );
+2979 }
+2980
+2981 /// Drops the path state, declaring any remaining in-flight packets as lost
+2982 fn discard_path(&mut self, path_id: PathId, now: Instant) {
+2983 trace!(%path_id, "dropping path state");
+2984 let path = self.path_data(path_id);
+2985 let in_flight_mtu_probe = path.mtud.in_flight_mtu_probe();
2986
-2987 // Don't apply congestion penalty for lost ack-only packets
-2988 let lost_ack_eliciting =
-2989 old_bytes_in_flight != self.path_data_mut(path_id).in_flight.bytes;
-2990
-2991 if lost_ack_eliciting {
-2992 self.path_stats
-2993 .entry(path_id)
-2994 .or_default()
-2995 .congestion_events += 1;
-2996 self.path_data_mut(path_id).congestion.on_congestion_event(
-2997 now,
-2998 largest_lost_sent,
-2999 in_persistent_congestion,
-3000 false,
-3001 size_of_lost_packets,
-3002 );
-3003 }
-3004 }
-3005
-3006 // Handle a lost MTU probe
-3007 if let Some(packet) = lost_mtu_probe {
-3008 let info = self.spaces[SpaceId::Data]
-3009 .for_path(path_id)
-3010 .take(packet)
-3011 .unwrap(); // safe: lost_mtu_probe is omitted from lost_packets, and
-3012 // therefore must not have been removed yet
-3013 self.paths
-3014 .get_mut(&path_id)
-3015 .unwrap()
-3016 .remove_in_flight(&info);
-3017 self.path_data_mut(path_id).mtud.on_probe_lost();
-3018 self.path_stats
-3019 .entry(path_id)
-3020 .or_default()
-3021 .lost_plpmtud_probes += 1;
-3022 }
-3023 }
-3024
-3025 /// Returns the earliest time packets should be declared lost for all spaces on a path.
-3026 ///
-3027 /// If a path has an acknowledged packet with any prior un-acknowledged packets, the
-3028 /// earliest un-acknowledged packet can be declared lost after a timeout has elapsed.
-3029 /// The time returned is when this packet should be declared lost.
-3030 fn loss_time_and_space(&self, path_id: PathId) -> Option<(Instant, SpaceId)> {
-3031 SpaceId::iter()
-3032 .filter_map(|id| {
-3033 self.spaces[id]
-3034 .number_spaces
-3035 .get(&path_id)
-3036 .and_then(|pns| pns.loss_time)
-3037 .map(|time| (time, id))
-3038 })
-3039 .min_by_key(|&(time, _)| time)
-3040 }
-3041
-3042 /// Returns the earliest next PTO should fire for all spaces on a path.
-3043 fn pto_time_and_space(&mut self, now: Instant, path_id: PathId) -> Option<(Instant, SpaceId)> {
-3044 let path = self.path(path_id)?;
-3045 let pto_count = path.pto_count;
-3046 let backoff = 2u32.pow(pto_count.min(MAX_BACKOFF_EXPONENT));
-3047 let mut duration = path.rtt.pto_base() * backoff;
-3048
-3049 if path_id == PathId::ZERO
-3050 && path.in_flight.ack_eliciting == 0
-3051 && !self.peer_completed_address_validation(PathId::ZERO)
-3052 {
-3053 // Address Validation during Connection Establishment:
-3054 // https://www.rfc-editor.org/rfc/rfc9000.html#section-8.1. To prevent a
-3055 // deadlock if an Initial or Handshake packet from the server is lost and the
-3056 // server can not send more due to its anti-amplification limit the client must
-3057 // send another packet on PTO.
-3058 let space = match self.highest_space {
-3059 SpaceId::Handshake => SpaceId::Handshake,
-3060 _ => SpaceId::Initial,
-3061 };
-3062
-3063 return Some((now + duration, space));
-3064 }
-3065
-3066 let mut result = None;
-3067 for space in SpaceId::iter() {
-3068 let Some(pns) = self.spaces[space].number_spaces.get(&path_id) else {
-3069 continue;
-3070 };
-3071
-3072 if !pns.has_in_flight() {
-3073 continue;
-3074 }
-3075 if space == SpaceId::Data {
-3076 // Skip ApplicationData until handshake completes.
-3077 if self.is_handshaking() {
-3078 return result;
-3079 }
-3080 // Include max_ack_delay and backoff for ApplicationData.
-3081 duration += self.ack_frequency.max_ack_delay_for_pto() * backoff;
-3082 }
-3083 let Some(last_ack_eliciting) = pns.time_of_last_ack_eliciting_packet else {
-3084 continue;
-3085 };
-3086 let pto = last_ack_eliciting + duration;
-3087 if result.is_none_or(|(earliest_pto, _)| pto < earliest_pto) {
-3088 if path.anti_amplification_blocked(1) {
-3089 // Nothing would be able to be sent.
-3090 continue;
-3091 }
-3092 if path.in_flight.ack_eliciting == 0 {
-3093 // Nothing ack-eliciting, no PTO to arm/fire.
-3094 continue;
-3095 }
-3096 result = Some((pto, space));
-3097 }
-3098 }
-3099 result
-3100 }
-3101
-3102 fn peer_completed_address_validation(&self, path: PathId) -> bool {
-3103 // TODO(flub): This logic needs updating for multipath
-3104 if self.side.is_server() || self.state.is_closed() {
-3105 return true;
-3106 }
-3107 // The server is guaranteed to have validated our address if any of our handshake or 1-RTT
-3108 // packets are acknowledged or we've seen HANDSHAKE_DONE and discarded handshake keys.
-3109 self.spaces[SpaceId::Handshake]
-3110 .path_space(PathId::ZERO)
-3111 .and_then(|pns| pns.largest_acked_packet)
-3112 .is_some()
-3113 || self.spaces[SpaceId::Data]
-3114 .path_space(path)
-3115 .and_then(|pns| pns.largest_acked_packet)
-3116 .is_some()
-3117 || (self.spaces[SpaceId::Data].crypto.is_some()
-3118 && self.spaces[SpaceId::Handshake].crypto.is_none())
-3119 }
-3120
-3121 /// Resets the the [`PathTimer::LossDetection`] timer to the next instant it may be needed
-3122 ///
-3123 /// The timer must fire if either:
-3124 /// - An ack-eliciting packet we sent needs to be declared lost.
-3125 /// - A tail-loss probe needs to be sent.
-3126 ///
-3127 /// See [`Connection::on_loss_detection_timeout`] for details.
-3128 fn set_loss_detection_timer(&mut self, now: Instant, path_id: PathId) {
-3129 if self.state.is_closed() {
-3130 // No loss detection takes place on closed connections, and `close_common` already
-3131 // stopped time timer. Ensure we don't restart it inadvertently, e.g. in response to a
-3132 // reordered packet being handled by state-insensitive code.
-3133 return;
-3134 }
-3135
-3136 if let Some((loss_time, _)) = self.loss_time_and_space(path_id) {
-3137 // Time threshold loss detection.
-3138 self.timers.set(
-3139 Timer::PerPath(path_id, PathTimer::LossDetection),
-3140 loss_time,
-3141 self.qlog.with_time(now),
-3142 );
-3143 return;
-3144 }
-3145
-3146 // Determine which PN space to arm PTO for.
-3147 // Calculate PTO duration
-3148 if let Some((timeout, _)) = self.pto_time_and_space(now, path_id) {
-3149 self.timers.set(
-3150 Timer::PerPath(path_id, PathTimer::LossDetection),
-3151 timeout,
-3152 self.qlog.with_time(now),
-3153 );
-3154 } else {
-3155 self.timers.stop(
-3156 Timer::PerPath(path_id, PathTimer::LossDetection),
-3157 self.qlog.with_time(now),
-3158 );
-3159 }
-3160 }
-3161
-3162 /// The maximum probe timeout across all paths
-3163 ///
-3164 /// If `is_closing` is set to `true` it will filter out paths that have not yet been used.
-3165 ///
-3166 /// See [`Connection::pto`]
-3167 fn pto_max_path(&self, space: SpaceId, is_closing: bool) -> Duration {
-3168 match space {
-3169 SpaceId::Initial | SpaceId::Handshake => self.pto(space, PathId::ZERO),
-3170 SpaceId::Data => self
-3171 .paths
-3172 .iter()
-3173 .filter_map(|(path_id, state)| {
-3174 if is_closing && state.data.total_sent == 0 && state.data.total_recvd == 0 {
-3175 // If we are closing and haven't sent anything yet, do not include
-3176 None
-3177 } else {
-3178 let pto = self.pto(space, *path_id);
-3179 Some(pto)
-3180 }
-3181 })
-3182 .max()
-3183 .expect("there should be at least one path"),
-3184 }
-3185 }
-3186
-3187 /// Probe Timeout
-3188 ///
-3189 /// The PTO is logically the time in which you'd expect to receive an acknowledgement
-3190 /// for a packet. So approximately RTT + max_ack_delay.
-3191 fn pto(&self, space: SpaceId, path_id: PathId) -> Duration {
-3192 let max_ack_delay = match space {
-3193 SpaceId::Initial | SpaceId::Handshake => Duration::ZERO,
-3194 SpaceId::Data => self.ack_frequency.max_ack_delay_for_pto(),
-3195 };
-3196 self.path_data(path_id).rtt.pto_base() + max_ack_delay
-3197 }
-3198
-3199 fn on_packet_authenticated(
-3200 &mut self,
-3201 now: Instant,
-3202 space_id: SpaceId,
-3203 path_id: PathId,
-3204 ecn: Option<EcnCodepoint>,
-3205 packet: Option<u64>,
-3206 spin: bool,
-3207 is_1rtt: bool,
-3208 ) {
-3209 self.total_authed_packets += 1;
-3210 if let Some(AbandonState::ExpectingPathAbandon { deadline }) = self
-3211 .paths
-3212 .get(&path_id)
-3213 .map(|path| &path.data.abandon_state)
-3214 {
-3215 if now > *deadline {
-3216 warn!("received data on path which we abandoned more than 3 * PTO ago");
-3217 // The peer failed to respond with a PATH_ABANDON in time.
-3218 if !self.state.is_closed() {
-3219 self.state
-3220 .move_to_closed(TransportError::PROTOCOL_VIOLATION(
-3221 "peer failed to respond with PATH_ABANDON in time",
-3222 ));
-3223 self.close_common();
-3224 self.set_close_timer(now);
-3225 self.close = true;
-3226 }
-3227 return;
-3228 }
-3229 }
-3230
-3231 self.reset_keep_alive(path_id, now);
-3232 self.reset_idle_timeout(now, space_id, path_id);
-3233 self.permit_idle_reset = true;
-3234 self.receiving_ecn |= ecn.is_some();
-3235 if let Some(x) = ecn {
-3236 let space = &mut self.spaces[space_id];
-3237 space.for_path(path_id).ecn_counters += x;
-3238
-3239 if x.is_ce() {
-3240 space
-3241 .for_path(path_id)
-3242 .pending_acks
-3243 .set_immediate_ack_required();
-3244 }
-3245 }
-3246
-3247 let packet = match packet {
-3248 Some(x) => x,
-3249 None => return,
-3250 };
-3251 match &self.side {
-3252 ConnectionSide::Client { .. } => {
-3253 // If we received a handshake packet that authenticated, then we're talking to
-3254 // the real server. From now on we should no longer allow the server to migrate
-3255 // its address.
-3256 if space_id == SpaceId::Handshake {
-3257 if let Some(hs) = self.state.as_handshake_mut() {
-3258 hs.allow_server_migration = false;
-3259 }
-3260 }
-3261 }
-3262 ConnectionSide::Server { .. } => {
-3263 if self.spaces[SpaceId::Initial].crypto.is_some() && space_id == SpaceId::Handshake
-3264 {
-3265 // A server stops sending and processing Initial packets when it receives its first Handshake packet.
-3266 self.discard_space(now, SpaceId::Initial);
-3267 }
-3268 if self.zero_rtt_crypto.is_some() && is_1rtt {
-3269 // Discard 0-RTT keys soon after receiving a 1-RTT packet
-3270 self.set_key_discard_timer(now, space_id)
-3271 }
-3272 }
-3273 }
-3274 let space = self.spaces[space_id].for_path(path_id);
-3275 space.pending_acks.insert_one(packet, now);
-3276 if packet >= space.rx_packet.unwrap_or_default() {
-3277 space.rx_packet = Some(packet);
-3278 // Update outgoing spin bit, inverting iff we're the client
-3279 self.spin = self.side.is_client() ^ spin;
-3280 }
-3281 }
-3282
-3283 /// Resets the idle timeout timers
-3284 ///
-3285 /// Without multipath there is only the connection-wide idle timeout. When multipath is
-3286 /// enabled there is an additional per-path idle timeout.
-3287 fn reset_idle_timeout(&mut self, now: Instant, space: SpaceId, path_id: PathId) {
-3288 // First reset the global idle timeout.
-3289 if let Some(timeout) = self.idle_timeout {
-3290 if self.state.is_closed() {
-3291 self.timers
-3292 .stop(Timer::Conn(ConnTimer::Idle), self.qlog.with_time(now));
-3293 } else {
-3294 let dt = cmp::max(timeout, 3 * self.pto_max_path(space, false));
-3295 self.timers.set(
-3296 Timer::Conn(ConnTimer::Idle),
-3297 now + dt,
-3298 self.qlog.with_time(now),
-3299 );
-3300 }
-3301 }
-3302
-3303 // Now handle the per-path state
-3304 if let Some(timeout) = self.path_data(path_id).idle_timeout {
-3305 if self.state.is_closed() {
-3306 self.timers.stop(
-3307 Timer::PerPath(path_id, PathTimer::PathIdle),
-3308 self.qlog.with_time(now),
-3309 );
-3310 } else {
-3311 let dt = cmp::max(timeout, 3 * self.pto(space, path_id));
-3312 self.timers.set(
-3313 Timer::PerPath(path_id, PathTimer::PathIdle),
-3314 now + dt,
-3315 self.qlog.with_time(now),
-3316 );
-3317 }
-3318 }
-3319 }
-3320
-3321 /// Resets both the [`ConnTimer::KeepAlive`] and [`PathTimer::PathKeepAlive`] timers
-3322 fn reset_keep_alive(&mut self, path_id: PathId, now: Instant) {
-3323 if !self.state.is_established() {
-3324 return;
-3325 }
+2987 let mut size_of_lost_packets = 0u64; // add to path_stats.lost_bytes;
+2988 let lost_pns: Vec<_> = self.spaces[SpaceId::Data]
+2989 .for_path(path_id)
+2990 .sent_packets
+2991 .iter()
+2992 .filter(|(pn, _info)| Some(*pn) != in_flight_mtu_probe)
+2993 .map(|(pn, info)| {
+2994 size_of_lost_packets += info.size as u64;
+2995 pn
+2996 })
+2997 .collect();
+2998
+2999 if !lost_pns.is_empty() {
+3000 trace!(
+3001 %path_id,
+3002 count = lost_pns.len(),
+3003 lost_bytes = size_of_lost_packets,
+3004 "packets lost on path abandon"
+3005 );
+3006 self.handle_lost_packets(
+3007 SpaceId::Data,
+3008 path_id,
+3009 now,
+3010 lost_pns,
+3011 in_flight_mtu_probe,
+3012 Duration::ZERO,
+3013 false,
+3014 size_of_lost_packets,
+3015 );
+3016 }
+3017 self.paths.remove(&path_id);
+3018 self.spaces[SpaceId::Data].number_spaces.remove(&path_id);
+3019
+3020 let path_stats = self.path_stats.remove(&path_id).unwrap_or_default();
+3021 self.events.push_back(
+3022 PathEvent::Abandoned {
+3023 id: path_id,
+3024 path_stats,
+3025 }
+3026 .into(),
+3027 );
+3028 }
+3029
+3030 fn handle_lost_packets(
+3031 &mut self,
+3032 pn_space: SpaceId,
+3033 path_id: PathId,
+3034 now: Instant,
+3035 lost_packets: Vec<u64>,
+3036 lost_mtu_probe: Option<u64>,
+3037 loss_delay: Duration,
+3038 in_persistent_congestion: bool,
+3039 size_of_lost_packets: u64,
+3040 ) {
+3041 debug_assert!(
+3042 {
+3043 let mut sorted = lost_packets.clone();
+3044 sorted.sort();
+3045 sorted == lost_packets
+3046 },
+3047 "lost_packets must be sorted"
+3048 );
+3049
+3050 self.drain_lost_packets(now, pn_space, path_id);
+3051
+3052 // OnPacketsLost
+3053 if let Some(largest_lost) = lost_packets.last().cloned() {
+3054 let old_bytes_in_flight = self.path_data_mut(path_id).in_flight.bytes;
+3055 let largest_lost_sent = self.spaces[pn_space]
+3056 .for_path(path_id)
+3057 .sent_packets
+3058 .get(largest_lost)
+3059 .unwrap()
+3060 .time_sent;
+3061 let path_stats = self.path_stats.entry(path_id).or_default();
+3062 path_stats.lost_packets += lost_packets.len() as u64;
+3063 path_stats.lost_bytes += size_of_lost_packets;
+3064 trace!(
+3065 %path_id,
+3066 count = lost_packets.len(),
+3067 lost_bytes = size_of_lost_packets,
+3068 "packets lost",
+3069 );
+3070
+3071 for &packet in &lost_packets {
+3072 let Some(info) = self.spaces[pn_space].for_path(path_id).take(packet) else {
+3073 continue;
+3074 };
+3075 self.qlog
+3076 .emit_packet_lost(packet, &info, loss_delay, pn_space, now);
+3077 self.paths
+3078 .get_mut(&path_id)
+3079 .unwrap()
+3080 .remove_in_flight(&info);
+3081
+3082 for frame in info.stream_frames {
+3083 self.streams.retransmit(frame);
+3084 }
+3085 self.spaces[pn_space].pending |= info.retransmits;
+3086 self.path_data_mut(path_id)
+3087 .mtud
+3088 .on_non_probe_lost(packet, info.size);
+3089
+3090 self.spaces[pn_space].for_path(path_id).lost_packets.insert(
+3091 packet,
+3092 LostPacket {
+3093 time_sent: info.time_sent,
+3094 },
+3095 );
+3096 }
+3097
+3098 let path = self.path_data_mut(path_id);
+3099 if path.mtud.black_hole_detected(now) {
+3100 path.congestion.on_mtu_update(path.mtud.current_mtu());
+3101 if let Some(max_datagram_size) = self.datagrams().max_size() {
+3102 if self.datagrams.drop_oversized(max_datagram_size)
+3103 && self.datagrams.send_blocked
+3104 {
+3105 self.datagrams.send_blocked = false;
+3106 self.events.push_back(Event::DatagramsUnblocked);
+3107 }
+3108 }
+3109 self.path_stats
+3110 .entry(path_id)
+3111 .or_default()
+3112 .black_holes_detected += 1;
+3113 }
+3114
+3115 // Don't apply congestion penalty for lost ack-only packets
+3116 let lost_ack_eliciting =
+3117 old_bytes_in_flight != self.path_data_mut(path_id).in_flight.bytes;
+3118
+3119 if lost_ack_eliciting {
+3120 self.path_stats
+3121 .entry(path_id)
+3122 .or_default()
+3123 .congestion_events += 1;
+3124 self.path_data_mut(path_id).congestion.on_congestion_event(
+3125 now,
+3126 largest_lost_sent,
+3127 in_persistent_congestion,
+3128 false,
+3129 size_of_lost_packets,
+3130 );
+3131 }
+3132 }
+3133
+3134 // Handle a lost MTU probe
+3135 if let Some(packet) = lost_mtu_probe {
+3136 let info = self.spaces[SpaceId::Data]
+3137 .for_path(path_id)
+3138 .take(packet)
+3139 .unwrap(); // safe: lost_mtu_probe is omitted from lost_packets, and
+3140 // therefore must not have been removed yet
+3141 self.paths
+3142 .get_mut(&path_id)
+3143 .unwrap()
+3144 .remove_in_flight(&info);
+3145 self.path_data_mut(path_id).mtud.on_probe_lost();
+3146 self.path_stats
+3147 .entry(path_id)
+3148 .or_default()
+3149 .lost_plpmtud_probes += 1;
+3150 }
+3151 }
+3152
+3153 /// Returns the earliest time packets should be declared lost for all spaces on a path.
+3154 ///
+3155 /// If a path has an acknowledged packet with any prior un-acknowledged packets, the
+3156 /// earliest un-acknowledged packet can be declared lost after a timeout has elapsed.
+3157 /// The time returned is when this packet should be declared lost.
+3158 fn loss_time_and_space(&self, path_id: PathId) -> Option<(Instant, SpaceId)> {
+3159 SpaceId::iter()
+3160 .filter_map(|id| {
+3161 self.spaces[id]
+3162 .number_spaces
+3163 .get(&path_id)
+3164 .and_then(|pns| pns.loss_time)
+3165 .map(|time| (time, id))
+3166 })
+3167 .min_by_key(|&(time, _)| time)
+3168 }
+3169
+3170 /// Returns the earliest next PTO should fire for all spaces on a path.
+3171 fn pto_time_and_space(&mut self, now: Instant, path_id: PathId) -> Option<(Instant, SpaceId)> {
+3172 let path = self.path(path_id)?;
+3173 let pto_count = path.pto_count;
+3174 let backoff = 2u32.pow(pto_count.min(MAX_BACKOFF_EXPONENT));
+3175 let mut duration = path.rtt.pto_base() * backoff;
+3176
+3177 if path_id == PathId::ZERO
+3178 && path.in_flight.ack_eliciting == 0
+3179 && !self.peer_completed_address_validation(PathId::ZERO)
+3180 {
+3181 // Address Validation during Connection Establishment:
+3182 // https://www.rfc-editor.org/rfc/rfc9000.html#section-8.1. To prevent a
+3183 // deadlock if an Initial or Handshake packet from the server is lost and the
+3184 // server can not send more due to its anti-amplification limit the client must
+3185 // send another packet on PTO.
+3186 let space = match self.highest_space {
+3187 SpaceId::Handshake => SpaceId::Handshake,
+3188 _ => SpaceId::Initial,
+3189 };
+3190
+3191 return Some((now + duration, space));
+3192 }
+3193
+3194 let mut result = None;
+3195 for space in SpaceId::iter() {
+3196 let Some(pns) = self.spaces[space].number_spaces.get(&path_id) else {
+3197 continue;
+3198 };
+3199
+3200 if !pns.has_in_flight() {
+3201 continue;
+3202 }
+3203 if space == SpaceId::Data {
+3204 // Skip ApplicationData until handshake completes.
+3205 if self.is_handshaking() {
+3206 return result;
+3207 }
+3208 // Include max_ack_delay and backoff for ApplicationData.
+3209 duration += self.ack_frequency.max_ack_delay_for_pto() * backoff;
+3210 }
+3211 let Some(last_ack_eliciting) = pns.time_of_last_ack_eliciting_packet else {
+3212 continue;
+3213 };
+3214 let pto = last_ack_eliciting + duration;
+3215 if result.is_none_or(|(earliest_pto, _)| pto < earliest_pto) {
+3216 if path.anti_amplification_blocked(1) {
+3217 // Nothing would be able to be sent.
+3218 continue;
+3219 }
+3220 if path.in_flight.ack_eliciting == 0 {
+3221 // Nothing ack-eliciting, no PTO to arm/fire.
+3222 continue;
+3223 }
+3224 result = Some((pto, space));
+3225 }
+3226 }
+3227 result
+3228 }
+3229
+3230 fn peer_completed_address_validation(&self, path: PathId) -> bool {
+3231 // TODO(flub): This logic needs updating for multipath
+3232 if self.side.is_server() || self.state.is_closed() {
+3233 return true;
+3234 }
+3235 // The server is guaranteed to have validated our address if any of our handshake or 1-RTT
+3236 // packets are acknowledged or we've seen HANDSHAKE_DONE and discarded handshake keys.
+3237 self.spaces[SpaceId::Handshake]
+3238 .path_space(PathId::ZERO)
+3239 .and_then(|pns| pns.largest_acked_packet)
+3240 .is_some()
+3241 || self.spaces[SpaceId::Data]
+3242 .path_space(path)
+3243 .and_then(|pns| pns.largest_acked_packet)
+3244 .is_some()
+3245 || (self.spaces[SpaceId::Data].crypto.is_some()
+3246 && self.spaces[SpaceId::Handshake].crypto.is_none())
+3247 }
+3248
+3249 /// Resets the the [`PathTimer::LossDetection`] timer to the next instant it may be needed
+3250 ///
+3251 /// The timer must fire if either:
+3252 /// - An ack-eliciting packet we sent needs to be declared lost.
+3253 /// - A tail-loss probe needs to be sent.
+3254 ///
+3255 /// See [`Connection::on_loss_detection_timeout`] for details.
+3256 fn set_loss_detection_timer(&mut self, now: Instant, path_id: PathId) {
+3257 if self.state.is_closed() {
+3258 // No loss detection takes place on closed connections, and `close_common` already
+3259 // stopped time timer. Ensure we don't restart it inadvertently, e.g. in response to a
+3260 // reordered packet being handled by state-insensitive code.
+3261 return;
+3262 }
+3263
+3264 if let Some((loss_time, _)) = self.loss_time_and_space(path_id) {
+3265 // Time threshold loss detection.
+3266 self.timers.set(
+3267 Timer::PerPath(path_id, PathTimer::LossDetection),
+3268 loss_time,
+3269 self.qlog.with_time(now),
+3270 );
+3271 return;
+3272 }
+3273
+3274 // Determine which PN space to arm PTO for.
+3275 // Calculate PTO duration
+3276 if let Some((timeout, _)) = self.pto_time_and_space(now, path_id) {
+3277 self.timers.set(
+3278 Timer::PerPath(path_id, PathTimer::LossDetection),
+3279 timeout,
+3280 self.qlog.with_time(now),
+3281 );
+3282 } else {
+3283 self.timers.stop(
+3284 Timer::PerPath(path_id, PathTimer::LossDetection),
+3285 self.qlog.with_time(now),
+3286 );
+3287 }
+3288 }
+3289
+3290 /// The maximum probe timeout across all paths
+3291 ///
+3292 /// If `is_closing` is set to `true` it will filter out paths that have not yet been used.
+3293 ///
+3294 /// See [`Connection::pto`]
+3295 fn pto_max_path(&self, space: SpaceId, is_closing: bool) -> Duration {
+3296 match space {
+3297 SpaceId::Initial | SpaceId::Handshake => self.pto(space, PathId::ZERO),
+3298 SpaceId::Data => self
+3299 .paths
+3300 .iter()
+3301 .filter_map(|(path_id, state)| {
+3302 if is_closing && state.data.total_sent == 0 && state.data.total_recvd == 0 {
+3303 // If we are closing and haven't sent anything yet, do not include
+3304 None
+3305 } else {
+3306 let pto = self.pto(space, *path_id);
+3307 Some(pto)
+3308 }
+3309 })
+3310 .max()
+3311 .expect("there should be at least one path"),
+3312 }
+3313 }
+3314
+3315 /// Probe Timeout
+3316 ///
+3317 /// The PTO is logically the time in which you'd expect to receive an acknowledgement
+3318 /// for a packet. So approximately RTT + max_ack_delay.
+3319 fn pto(&self, space: SpaceId, path_id: PathId) -> Duration {
+3320 let max_ack_delay = match space {
+3321 SpaceId::Initial | SpaceId::Handshake => Duration::ZERO,
+3322 SpaceId::Data => self.ack_frequency.max_ack_delay_for_pto(),
+3323 };
+3324 self.path_data(path_id).rtt.pto_base() + max_ack_delay
+3325 }
3326
-3327 if let Some(interval) = self.config.keep_alive_interval {
-3328 self.timers.set(
-3329 Timer::Conn(ConnTimer::KeepAlive),
-3330 now + interval,
-3331 self.qlog.with_time(now),
-3332 );
-3333 }
-3334
-3335 if let Some(interval) = self.path_data(path_id).keep_alive {
-3336 self.timers.set(
-3337 Timer::PerPath(path_id, PathTimer::PathKeepAlive),
-3338 now + interval,
-3339 self.qlog.with_time(now),
-3340 );
-3341 }
-3342 }
-3343
-3344 /// Sets the timer for when a previously issued CID should be retired next
-3345 fn reset_cid_retirement(&mut self, now: Instant) {
-3346 if let Some((_path, t)) = self.next_cid_retirement() {
-3347 self.timers.set(
-3348 Timer::Conn(ConnTimer::PushNewCid),
-3349 t,
-3350 self.qlog.with_time(now),
-3351 );
-3352 }
-3353 }
-3354
-3355 /// The next time when a previously issued CID should be retired
-3356 fn next_cid_retirement(&self) -> Option<(PathId, Instant)> {
-3357 self.local_cid_state
-3358 .iter()
-3359 .filter_map(|(path_id, cid_state)| cid_state.next_timeout().map(|t| (*path_id, t)))
-3360 .min_by_key(|(_path_id, timeout)| *timeout)
-3361 }
-3362
-3363 /// Handle the already-decrypted first packet from the client
-3364 ///
-3365 /// Decrypting the first packet in the `Endpoint` allows stateless packet handling to be more
-3366 /// efficient.
-3367 pub(crate) fn handle_first_packet(
-3368 &mut self,
-3369 now: Instant,
-3370 network_path: FourTuple,
-3371 ecn: Option<EcnCodepoint>,
-3372 packet_number: u64,
-3373 packet: InitialPacket,
-3374 remaining: Option<BytesMut>,
-3375 ) -> Result<(), ConnectionError> {
-3376 let span = trace_span!("first recv");
-3377 let _guard = span.enter();
-3378 debug_assert!(self.side.is_server());
-3379 let len = packet.header_data.len() + packet.payload.len();
-3380 let path_id = PathId::ZERO;
-3381 self.path_data_mut(path_id).total_recvd = len as u64;
-3382
-3383 if let Some(hs) = self.state.as_handshake_mut() {
-3384 hs.expected_token = packet.header.token.clone();
-3385 } else {
-3386 unreachable!("first packet must be delivered in Handshake state");
-3387 }
-3388
-3389 // The first packet is always on PathId::ZERO
-3390 self.on_packet_authenticated(
-3391 now,
-3392 SpaceId::Initial,
-3393 path_id,
-3394 ecn,
-3395 Some(packet_number),
-3396 false,
-3397 false,
-3398 );
-3399
-3400 let packet: Packet = packet.into();
-3401
-3402 let mut qlog = QlogRecvPacket::new(len);
-3403 qlog.header(&packet.header, Some(packet_number), path_id);
-3404
-3405 self.process_decrypted_packet(
-3406 now,
-3407 network_path,
-3408 path_id,
-3409 Some(packet_number),
-3410 packet,
-3411 &mut qlog,
-3412 )?;
-3413 self.qlog.emit_packet_received(qlog, now);
-3414 if let Some(data) = remaining {
-3415 self.handle_coalesced(now, network_path, path_id, ecn, data);
-3416 }
-3417
-3418 self.qlog.emit_recovery_metrics(
-3419 path_id,
-3420 &mut self.paths.get_mut(&path_id).unwrap().data,
-3421 now,
-3422 );
-3423
-3424 Ok(())
-3425 }
-3426
-3427 fn init_0rtt(&mut self, now: Instant) {
-3428 let (header, packet) = match self.crypto.early_crypto() {
-3429 Some(x) => x,
-3430 None => return,
-3431 };
-3432 if self.side.is_client() {
-3433 match self.crypto.transport_parameters() {
-3434 Ok(params) => {
-3435 let params = params
-3436 .expect("crypto layer didn't supply transport parameters with ticket");
-3437 // Certain values must not be cached
-3438 let params = TransportParameters {
-3439 initial_src_cid: None,
-3440 original_dst_cid: None,
-3441 preferred_address: None,
-3442 retry_src_cid: None,
-3443 stateless_reset_token: None,
-3444 min_ack_delay: None,
-3445 ack_delay_exponent: TransportParameters::default().ack_delay_exponent,
-3446 max_ack_delay: TransportParameters::default().max_ack_delay,
-3447 initial_max_path_id: None,
-3448 ..params
-3449 };
-3450 self.set_peer_params(params);
-3451 self.qlog.emit_peer_transport_params_restored(self, now);
-3452 }
-3453 Err(e) => {
-3454 error!("session ticket has malformed transport parameters: {}", e);
-3455 return;
-3456 }
-3457 }
-3458 }
-3459 trace!("0-RTT enabled");
-3460 self.zero_rtt_enabled = true;
-3461 self.zero_rtt_crypto = Some(ZeroRttCrypto { header, packet });
-3462 }
-3463
-3464 fn read_crypto(
-3465 &mut self,
-3466 space: SpaceId,
-3467 crypto: &frame::Crypto,
-3468 payload_len: usize,
-3469 ) -> Result<(), TransportError> {
-3470 let expected = if !self.state.is_handshake() {
-3471 SpaceId::Data
-3472 } else if self.highest_space == SpaceId::Initial {
-3473 SpaceId::Initial
-3474 } else {
-3475 // On the server, self.highest_space can be Data after receiving the client's first
-3476 // flight, but we expect Handshake CRYPTO until the handshake is complete.
-3477 SpaceId::Handshake
-3478 };
-3479 // We can't decrypt Handshake packets when highest_space is Initial, CRYPTO frames in 0-RTT
-3480 // packets are illegal, and we don't process 1-RTT packets until the handshake is
-3481 // complete. Therefore, we will never see CRYPTO data from a later-than-expected space.
-3482 debug_assert!(space <= expected, "received out-of-order CRYPTO data");
-3483
-3484 let end = crypto.offset + crypto.data.len() as u64;
-3485 if space < expected && end > self.spaces[space].crypto_stream.bytes_read() {
-3486 warn!(
-3487 "received new {:?} CRYPTO data when expecting {:?}",
-3488 space, expected
-3489 );
-3490 return Err(TransportError::PROTOCOL_VIOLATION(
-3491 "new data at unexpected encryption level",
-3492 ));
-3493 }
-3494
-3495 let space = &mut self.spaces[space];
-3496 let max = end.saturating_sub(space.crypto_stream.bytes_read());
-3497 if max > self.config.crypto_buffer_size as u64 {
-3498 return Err(TransportError::CRYPTO_BUFFER_EXCEEDED(""));
-3499 }
-3500
-3501 space
-3502 .crypto_stream
-3503 .insert(crypto.offset, crypto.data.clone(), payload_len);
-3504 while let Some(chunk) = space.crypto_stream.read(usize::MAX, true) {
-3505 trace!("consumed {} CRYPTO bytes", chunk.bytes.len());
-3506 if self.crypto.read_handshake(&chunk.bytes)? {
-3507 self.events.push_back(Event::HandshakeDataReady);
-3508 }
-3509 }
+3327 fn on_packet_authenticated(
+3328 &mut self,
+3329 now: Instant,
+3330 space_id: SpaceId,
+3331 path_id: PathId,
+3332 ecn: Option<EcnCodepoint>,
+3333 packet: Option<u64>,
+3334 spin: bool,
+3335 is_1rtt: bool,
+3336 ) {
+3337 self.total_authed_packets += 1;
+3338 if let Some(AbandonState::ExpectingPathAbandon { deadline }) = self
+3339 .paths
+3340 .get(&path_id)
+3341 .map(|path| &path.data.abandon_state)
+3342 {
+3343 if now > *deadline {
+3344 warn!("received data on path which we abandoned more than 3 * PTO ago");
+3345 // The peer failed to respond with a PATH_ABANDON in time.
+3346 if !self.state.is_closed() {
+3347 self.state
+3348 .move_to_closed(TransportError::PROTOCOL_VIOLATION(
+3349 "peer failed to respond with PATH_ABANDON in time",
+3350 ));
+3351 self.close_common();
+3352 self.set_close_timer(now);
+3353 self.close = true;
+3354 }
+3355 return;
+3356 }
+3357 }
+3358
+3359 self.reset_keep_alive(path_id, now);
+3360 self.reset_idle_timeout(now, space_id, path_id);
+3361 self.permit_idle_reset = true;
+3362 self.receiving_ecn |= ecn.is_some();
+3363 if let Some(x) = ecn {
+3364 let space = &mut self.spaces[space_id];
+3365 space.for_path(path_id).ecn_counters += x;
+3366
+3367 if x.is_ce() {
+3368 space
+3369 .for_path(path_id)
+3370 .pending_acks
+3371 .set_immediate_ack_required();
+3372 }
+3373 }
+3374
+3375 let packet = match packet {
+3376 Some(x) => x,
+3377 None => return,
+3378 };
+3379 match &self.side {
+3380 ConnectionSide::Client { .. } => {
+3381 // If we received a handshake packet that authenticated, then we're talking to
+3382 // the real server. From now on we should no longer allow the server to migrate
+3383 // its address.
+3384 if space_id == SpaceId::Handshake {
+3385 if let Some(hs) = self.state.as_handshake_mut() {
+3386 hs.allow_server_migration = false;
+3387 }
+3388 }
+3389 }
+3390 ConnectionSide::Server { .. } => {
+3391 if self.spaces[SpaceId::Initial].crypto.is_some() && space_id == SpaceId::Handshake
+3392 {
+3393 // A server stops sending and processing Initial packets when it receives its first Handshake packet.
+3394 self.discard_space(now, SpaceId::Initial);
+3395 }
+3396 if self.zero_rtt_crypto.is_some() && is_1rtt {
+3397 // Discard 0-RTT keys soon after receiving a 1-RTT packet
+3398 self.set_key_discard_timer(now, space_id)
+3399 }
+3400 }
+3401 }
+3402 let space = self.spaces[space_id].for_path(path_id);
+3403 space.pending_acks.insert_one(packet, now);
+3404 if packet >= space.rx_packet.unwrap_or_default() {
+3405 space.rx_packet = Some(packet);
+3406 // Update outgoing spin bit, inverting iff we're the client
+3407 self.spin = self.side.is_client() ^ spin;
+3408 }
+3409 }
+3410
+3411 /// Resets the idle timeout timers
+3412 ///
+3413 /// Without multipath there is only the connection-wide idle timeout. When multipath is
+3414 /// enabled there is an additional per-path idle timeout.
+3415 fn reset_idle_timeout(&mut self, now: Instant, space: SpaceId, path_id: PathId) {
+3416 // First reset the global idle timeout.
+3417 if let Some(timeout) = self.idle_timeout {
+3418 if self.state.is_closed() {
+3419 self.timers
+3420 .stop(Timer::Conn(ConnTimer::Idle), self.qlog.with_time(now));
+3421 } else {
+3422 let dt = cmp::max(timeout, 3 * self.pto_max_path(space, false));
+3423 self.timers.set(
+3424 Timer::Conn(ConnTimer::Idle),
+3425 now + dt,
+3426 self.qlog.with_time(now),
+3427 );
+3428 }
+3429 }
+3430
+3431 // Now handle the per-path state
+3432 if let Some(timeout) = self.path_data(path_id).idle_timeout {
+3433 if self.state.is_closed() {
+3434 self.timers.stop(
+3435 Timer::PerPath(path_id, PathTimer::PathIdle),
+3436 self.qlog.with_time(now),
+3437 );
+3438 } else {
+3439 let dt = cmp::max(timeout, 3 * self.pto(space, path_id));
+3440 self.timers.set(
+3441 Timer::PerPath(path_id, PathTimer::PathIdle),
+3442 now + dt,
+3443 self.qlog.with_time(now),
+3444 );
+3445 }
+3446 }
+3447 }
+3448
+3449 /// Resets both the [`ConnTimer::KeepAlive`] and [`PathTimer::PathKeepAlive`] timers
+3450 fn reset_keep_alive(&mut self, path_id: PathId, now: Instant) {
+3451 if !self.state.is_established() {
+3452 return;
+3453 }
+3454
+3455 if let Some(interval) = self.config.keep_alive_interval {
+3456 self.timers.set(
+3457 Timer::Conn(ConnTimer::KeepAlive),
+3458 now + interval,
+3459 self.qlog.with_time(now),
+3460 );
+3461 }
+3462
+3463 if let Some(interval) = self.path_data(path_id).keep_alive {
+3464 self.timers.set(
+3465 Timer::PerPath(path_id, PathTimer::PathKeepAlive),
+3466 now + interval,
+3467 self.qlog.with_time(now),
+3468 );
+3469 }
+3470 }
+3471
+3472 /// Sets the timer for when a previously issued CID should be retired next
+3473 fn reset_cid_retirement(&mut self, now: Instant) {
+3474 if let Some((_path, t)) = self.next_cid_retirement() {
+3475 self.timers.set(
+3476 Timer::Conn(ConnTimer::PushNewCid),
+3477 t,
+3478 self.qlog.with_time(now),
+3479 );
+3480 }
+3481 }
+3482
+3483 /// The next time when a previously issued CID should be retired
+3484 fn next_cid_retirement(&self) -> Option<(PathId, Instant)> {
+3485 self.local_cid_state
+3486 .iter()
+3487 .filter_map(|(path_id, cid_state)| cid_state.next_timeout().map(|t| (*path_id, t)))
+3488 .min_by_key(|(_path_id, timeout)| *timeout)
+3489 }
+3490
+3491 /// Handle the already-decrypted first packet from the client
+3492 ///
+3493 /// Decrypting the first packet in the `Endpoint` allows stateless packet handling to be more
+3494 /// efficient.
+3495 pub(crate) fn handle_first_packet(
+3496 &mut self,
+3497 now: Instant,
+3498 network_path: FourTuple,
+3499 ecn: Option<EcnCodepoint>,
+3500 packet_number: u64,
+3501 packet: InitialPacket,
+3502 remaining: Option<BytesMut>,
+3503 ) -> Result<(), ConnectionError> {
+3504 let span = trace_span!("first recv");
+3505 let _guard = span.enter();
+3506 debug_assert!(self.side.is_server());
+3507 let len = packet.header_data.len() + packet.payload.len();
+3508 let path_id = PathId::ZERO;
+3509 self.path_data_mut(path_id).total_recvd = len as u64;
3510
-3511 Ok(())
-3512 }
-3513
-3514 fn write_crypto(&mut self) {
-3515 loop {
-3516 let space = self.highest_space;
-3517 let mut outgoing = Vec::new();
-3518 if let Some(crypto) = self.crypto.write_handshake(&mut outgoing) {
-3519 match space {
-3520 SpaceId::Initial => {
-3521 self.upgrade_crypto(SpaceId::Handshake, crypto);
-3522 }
-3523 SpaceId::Handshake => {
-3524 self.upgrade_crypto(SpaceId::Data, crypto);
-3525 }
-3526 _ => unreachable!("got updated secrets during 1-RTT"),
-3527 }
-3528 }
-3529 if outgoing.is_empty() {
-3530 if space == self.highest_space {
-3531 break;
-3532 } else {
-3533 // Keys updated, check for more data to send
-3534 continue;
-3535 }
-3536 }
-3537 let offset = self.spaces[space].crypto_offset;
-3538 let outgoing = Bytes::from(outgoing);
-3539 if let Some(hs) = self.state.as_handshake_mut() {
-3540 if space == SpaceId::Initial && offset == 0 && self.side.is_client() {
-3541 hs.client_hello = Some(outgoing.clone());
-3542 }
-3543 }
-3544 self.spaces[space].crypto_offset += outgoing.len() as u64;
-3545 trace!("wrote {} {:?} CRYPTO bytes", outgoing.len(), space);
-3546 self.spaces[space].pending.crypto.push_back(frame::Crypto {
-3547 offset,
-3548 data: outgoing,
-3549 });
-3550 }
-3551 }
-3552
-3553 /// Switch to stronger cryptography during handshake
-3554 fn upgrade_crypto(&mut self, space: SpaceId, crypto: Keys) {
-3555 debug_assert!(
-3556 self.spaces[space].crypto.is_none(),
-3557 "already reached packet space {space:?}"
-3558 );
-3559 trace!("{:?} keys ready", space);
-3560 if space == SpaceId::Data {
-3561 // Precompute the first key update
-3562 self.next_crypto = Some(
-3563 self.crypto
-3564 .next_1rtt_keys()
-3565 .expect("handshake should be complete"),
-3566 );
-3567 }
-3568
-3569 self.spaces[space].crypto = Some(crypto);
-3570 debug_assert!(space as usize > self.highest_space as usize);
-3571 self.highest_space = space;
-3572 if space == SpaceId::Data && self.side.is_client() {
-3573 // Discard 0-RTT keys because 1-RTT keys are available.
-3574 self.zero_rtt_crypto = None;
-3575 }
-3576 }
-3577
-3578 fn discard_space(&mut self, now: Instant, space_id: SpaceId) {
-3579 debug_assert!(space_id != SpaceId::Data);
-3580 trace!("discarding {:?} keys", space_id);
-3581 if space_id == SpaceId::Initial {
-3582 // No longer needed
-3583 if let ConnectionSide::Client { token, .. } = &mut self.side {
-3584 *token = Bytes::new();
+3511 if let Some(hs) = self.state.as_handshake_mut() {
+3512 hs.expected_token = packet.header.token.clone();
+3513 } else {
+3514 unreachable!("first packet must be delivered in Handshake state");
+3515 }
+3516
+3517 // The first packet is always on PathId::ZERO
+3518 self.on_packet_authenticated(
+3519 now,
+3520 SpaceId::Initial,
+3521 path_id,
+3522 ecn,
+3523 Some(packet_number),
+3524 false,
+3525 false,
+3526 );
+3527
+3528 let packet: Packet = packet.into();
+3529
+3530 let mut qlog = QlogRecvPacket::new(len);
+3531 qlog.header(&packet.header, Some(packet_number), path_id);
+3532
+3533 self.process_decrypted_packet(
+3534 now,
+3535 network_path,
+3536 path_id,
+3537 Some(packet_number),
+3538 packet,
+3539 &mut qlog,
+3540 )?;
+3541 self.qlog.emit_packet_received(qlog, now);
+3542 if let Some(data) = remaining {
+3543 self.handle_coalesced(now, network_path, path_id, ecn, data);
+3544 }
+3545
+3546 self.qlog.emit_recovery_metrics(
+3547 path_id,
+3548 &mut self.paths.get_mut(&path_id).unwrap().data,
+3549 now,
+3550 );
+3551
+3552 Ok(())
+3553 }
+3554
+3555 fn init_0rtt(&mut self, now: Instant) {
+3556 let (header, packet) = match self.crypto.early_crypto() {
+3557 Some(x) => x,
+3558 None => return,
+3559 };
+3560 if self.side.is_client() {
+3561 match self.crypto.transport_parameters() {
+3562 Ok(params) => {
+3563 let params = params
+3564 .expect("crypto layer didn't supply transport parameters with ticket");
+3565 // Certain values must not be cached
+3566 let params = TransportParameters {
+3567 initial_src_cid: None,
+3568 original_dst_cid: None,
+3569 preferred_address: None,
+3570 retry_src_cid: None,
+3571 stateless_reset_token: None,
+3572 min_ack_delay: None,
+3573 ack_delay_exponent: TransportParameters::default().ack_delay_exponent,
+3574 max_ack_delay: TransportParameters::default().max_ack_delay,
+3575 initial_max_path_id: None,
+3576 ..params
+3577 };
+3578 self.set_peer_params(params);
+3579 self.qlog.emit_peer_transport_params_restored(self, now);
+3580 }
+3581 Err(e) => {
+3582 error!("session ticket has malformed transport parameters: {}", e);
+3583 return;
+3584 }
3585 }
3586 }
-3587 let space = &mut self.spaces[space_id];
-3588 space.crypto = None;
-3589 let pns = space.for_path(PathId::ZERO);
-3590 pns.time_of_last_ack_eliciting_packet = None;
-3591 pns.loss_time = None;
-3592 pns.loss_probes = 0;
-3593 let sent_packets = mem::take(&mut pns.sent_packets);
-3594 let path = self.paths.get_mut(&PathId::ZERO).unwrap();
-3595 for (_, packet) in sent_packets.into_iter() {
-3596 path.data.remove_in_flight(&packet);
-3597 }
-3598
-3599 self.set_loss_detection_timer(now, PathId::ZERO)
-3600 }
-3601
-3602 fn handle_coalesced(
-3603 &mut self,
-3604 now: Instant,
-3605 network_path: FourTuple,
-3606 path_id: PathId,
-3607 ecn: Option<EcnCodepoint>,
-3608 data: BytesMut,
-3609 ) {
-3610 self.path_data_mut(path_id)
-3611 .inc_total_recvd(data.len() as u64);
-3612 let mut remaining = Some(data);
-3613 let cid_len = self
-3614 .local_cid_state
-3615 .values()
-3616 .map(|cid_state| cid_state.cid_len())
-3617 .next()
-3618 .expect("one cid_state must exist");
-3619 while let Some(data) = remaining {
-3620 match PartialDecode::new(
-3621 data,
-3622 &FixedLengthConnectionIdParser::new(cid_len),
-3623 &[self.version],
-3624 self.endpoint_config.grease_quic_bit,
-3625 ) {
-3626 Ok((partial_decode, rest)) => {
-3627 remaining = rest;
-3628 self.handle_decode(now, network_path, path_id, ecn, partial_decode);
-3629 }
-3630 Err(e) => {
-3631 trace!("malformed header: {}", e);
-3632 return;
-3633 }
-3634 }
-3635 }
-3636 }
-3637
-3638 fn handle_decode(
-3639 &mut self,
-3640 now: Instant,
-3641 network_path: FourTuple,
-3642 path_id: PathId,
-3643 ecn: Option<EcnCodepoint>,
-3644 partial_decode: PartialDecode,
-3645 ) {
-3646 let qlog = QlogRecvPacket::new(partial_decode.len());
-3647 if let Some(decoded) = packet_crypto::unprotect_header(
-3648 partial_decode,
-3649 &self.spaces,
-3650 self.zero_rtt_crypto.as_ref(),
-3651 self.peer_params.stateless_reset_token,
-3652 ) {
-3653 self.handle_packet(
-3654 now,
-3655 network_path,
-3656 path_id,
-3657 ecn,
-3658 decoded.packet,
-3659 decoded.stateless_reset,
-3660 qlog,
-3661 );
-3662 }
-3663 }
-3664
-3665 fn handle_packet(
-3666 &mut self,
-3667 now: Instant,
-3668 network_path: FourTuple,
-3669 path_id: PathId,
-3670 ecn: Option<EcnCodepoint>,
-3671 packet: Option<Packet>,
-3672 stateless_reset: bool,
-3673 mut qlog: QlogRecvPacket,
-3674 ) {
-3675 self.stats.udp_rx.ios += 1;
-3676 if let Some(ref packet) = packet {
-3677 trace!(
-3678 "got {:?} packet ({} bytes) from {} using id {}",
-3679 packet.header.space(),
-3680 packet.payload.len() + packet.header_data.len(),
-3681 network_path,
-3682 packet.header.dst_cid(),
-3683 );
-3684 }
-3685
-3686 if self.is_handshaking() {
-3687 if path_id != PathId::ZERO {
-3688 debug!(%network_path, %path_id, "discarding multipath packet during handshake");
-3689 return;
-3690 }
-3691 if network_path != self.path_data_mut(path_id).network_path {
-3692 if let Some(hs) = self.state.as_handshake() {
-3693 if hs.allow_server_migration {
-3694 trace!(%network_path, prev = %self.path_data(path_id).network_path, "server migrated to new remote");
-3695 self.path_data_mut(path_id).network_path = network_path;
-3696 self.qlog.emit_tuple_assigned(path_id, network_path, now);
-3697 } else {
-3698 debug!("discarding packet with unexpected remote during handshake");
-3699 return;
-3700 }
-3701 } else {
-3702 debug!("discarding packet with unexpected remote during handshake");
-3703 return;
-3704 }
-3705 }
-3706 }
-3707
-3708 let was_closed = self.state.is_closed();
-3709 let was_drained = self.state.is_drained();
-3710
-3711 let decrypted = match packet {
-3712 None => Err(None),
-3713 Some(mut packet) => self
-3714 .decrypt_packet(now, path_id, &mut packet)
-3715 .map(move |number| (packet, number)),
-3716 };
-3717 let result = match decrypted {
-3718 _ if stateless_reset => {
-3719 debug!("got stateless reset");
-3720 Err(ConnectionError::Reset)
-3721 }
-3722 Err(Some(e)) => {
-3723 warn!("illegal packet: {}", e);
-3724 Err(e.into())
-3725 }
-3726 Err(None) => {
-3727 debug!("failed to authenticate packet");
-3728 self.authentication_failures += 1;
-3729 let integrity_limit = self.spaces[self.highest_space]
-3730 .crypto
-3731 .as_ref()
-3732 .unwrap()
-3733 .packet
-3734 .local
-3735 .integrity_limit();
-3736 if self.authentication_failures > integrity_limit {
-3737 Err(TransportError::AEAD_LIMIT_REACHED("integrity limit violated").into())
-3738 } else {
-3739 return;
-3740 }
-3741 }
-3742 Ok((packet, number)) => {
-3743 qlog.header(&packet.header, number, path_id);
-3744 let span = match number {
-3745 Some(pn) => trace_span!("recv", space = ?packet.header.space(), pn),
-3746 None => trace_span!("recv", space = ?packet.header.space()),
-3747 };
-3748 let _guard = span.enter();
-3749
-3750 let dedup = self.spaces[packet.header.space()]
-3751 .path_space_mut(path_id)
-3752 .map(|pns| &mut pns.dedup);
-3753 if number.zip(dedup).is_some_and(|(n, d)| d.insert(n)) {
-3754 debug!("discarding possible duplicate packet");
-3755 self.qlog.emit_packet_received(qlog, now);
-3756 return;
-3757 } else if self.state.is_handshake() && packet.header.is_short() {
-3758 // TODO: SHOULD buffer these to improve reordering tolerance.
-3759 trace!("dropping short packet during handshake");
-3760 self.qlog.emit_packet_received(qlog, now);
-3761 return;
-3762 } else {
-3763 if let Header::Initial(InitialHeader { ref token, .. }) = packet.header {
-3764 if let Some(hs) = self.state.as_handshake() {
-3765 if self.side.is_server() && token != &hs.expected_token {
-3766 // Clients must send the same retry token in every Initial. Initial
-3767 // packets can be spoofed, so we discard rather than killing the
-3768 // connection.
-3769 warn!("discarding Initial with invalid retry token");
-3770 self.qlog.emit_packet_received(qlog, now);
-3771 return;
-3772 }
-3773 }
-3774 }
-3775
-3776 if !self.state.is_closed() {
-3777 let spin = match packet.header {
-3778 Header::Short { spin, .. } => spin,
-3779 _ => false,
-3780 };
-3781
-3782 if self.side().is_server() && !self.abandoned_paths.contains(&path_id) {
-3783 // Only the client is allowed to open paths
-3784 self.ensure_path(path_id, network_path, now, number);
-3785 }
-3786 if self.paths.contains_key(&path_id) {
-3787 self.on_packet_authenticated(
-3788 now,
-3789 packet.header.space(),
-3790 path_id,
-3791 ecn,
-3792 number,
-3793 spin,
-3794 packet.header.is_1rtt(),
-3795 );
-3796 }
-3797 }
-3798
-3799 let res = self.process_decrypted_packet(
-3800 now,
-3801 network_path,
-3802 path_id,
-3803 number,
-3804 packet,
-3805 &mut qlog,
-3806 );
-3807
-3808 self.qlog.emit_packet_received(qlog, now);
-3809 res
-3810 }
-3811 }
-3812 };
+3587 trace!("0-RTT enabled");
+3588 self.zero_rtt_enabled = true;
+3589 self.zero_rtt_crypto = Some(ZeroRttCrypto { header, packet });
+3590 }
+3591
+3592 fn read_crypto(
+3593 &mut self,
+3594 space: SpaceId,
+3595 crypto: &frame::Crypto,
+3596 payload_len: usize,
+3597 ) -> Result<(), TransportError> {
+3598 let expected = if !self.state.is_handshake() {
+3599 SpaceId::Data
+3600 } else if self.highest_space == SpaceId::Initial {
+3601 SpaceId::Initial
+3602 } else {
+3603 // On the server, self.highest_space can be Data after receiving the client's first
+3604 // flight, but we expect Handshake CRYPTO until the handshake is complete.
+3605 SpaceId::Handshake
+3606 };
+3607 // We can't decrypt Handshake packets when highest_space is Initial, CRYPTO frames in 0-RTT
+3608 // packets are illegal, and we don't process 1-RTT packets until the handshake is
+3609 // complete. Therefore, we will never see CRYPTO data from a later-than-expected space.
+3610 debug_assert!(space <= expected, "received out-of-order CRYPTO data");
+3611
+3612 let end = crypto.offset + crypto.data.len() as u64;
+3613 if space < expected && end > self.spaces[space].crypto_stream.bytes_read() {
+3614 warn!(
+3615 "received new {:?} CRYPTO data when expecting {:?}",
+3616 space, expected
+3617 );
+3618 return Err(TransportError::PROTOCOL_VIOLATION(
+3619 "new data at unexpected encryption level",
+3620 ));
+3621 }
+3622
+3623 let space = &mut self.spaces[space];
+3624 let max = end.saturating_sub(space.crypto_stream.bytes_read());
+3625 if max > self.config.crypto_buffer_size as u64 {
+3626 return Err(TransportError::CRYPTO_BUFFER_EXCEEDED(""));
+3627 }
+3628
+3629 space
+3630 .crypto_stream
+3631 .insert(crypto.offset, crypto.data.clone(), payload_len);
+3632 while let Some(chunk) = space.crypto_stream.read(usize::MAX, true) {
+3633 trace!("consumed {} CRYPTO bytes", chunk.bytes.len());
+3634 if self.crypto.read_handshake(&chunk.bytes)? {
+3635 self.events.push_back(Event::HandshakeDataReady);
+3636 }
+3637 }
+3638
+3639 Ok(())
+3640 }
+3641
+3642 fn write_crypto(&mut self) {
+3643 loop {
+3644 let space = self.highest_space;
+3645 let mut outgoing = Vec::new();
+3646 if let Some(crypto) = self.crypto.write_handshake(&mut outgoing) {
+3647 match space {
+3648 SpaceId::Initial => {
+3649 self.upgrade_crypto(SpaceId::Handshake, crypto);
+3650 }
+3651 SpaceId::Handshake => {
+3652 self.upgrade_crypto(SpaceId::Data, crypto);
+3653 }
+3654 _ => unreachable!("got updated secrets during 1-RTT"),
+3655 }
+3656 }
+3657 if outgoing.is_empty() {
+3658 if space == self.highest_space {
+3659 break;
+3660 } else {
+3661 // Keys updated, check for more data to send
+3662 continue;
+3663 }
+3664 }
+3665 let offset = self.spaces[space].crypto_offset;
+3666 let outgoing = Bytes::from(outgoing);
+3667 if let Some(hs) = self.state.as_handshake_mut() {
+3668 if space == SpaceId::Initial && offset == 0 && self.side.is_client() {
+3669 hs.client_hello = Some(outgoing.clone());
+3670 }
+3671 }
+3672 self.spaces[space].crypto_offset += outgoing.len() as u64;
+3673 trace!("wrote {} {:?} CRYPTO bytes", outgoing.len(), space);
+3674 self.spaces[space].pending.crypto.push_back(frame::Crypto {
+3675 offset,
+3676 data: outgoing,
+3677 });
+3678 }
+3679 }
+3680
+3681 /// Switch to stronger cryptography during handshake
+3682 fn upgrade_crypto(&mut self, space: SpaceId, crypto: Keys) {
+3683 debug_assert!(
+3684 self.spaces[space].crypto.is_none(),
+3685 "already reached packet space {space:?}"
+3686 );
+3687 trace!("{:?} keys ready", space);
+3688 if space == SpaceId::Data {
+3689 // Precompute the first key update
+3690 self.next_crypto = Some(
+3691 self.crypto
+3692 .next_1rtt_keys()
+3693 .expect("handshake should be complete"),
+3694 );
+3695 }
+3696
+3697 self.spaces[space].crypto = Some(crypto);
+3698 debug_assert!(space as usize > self.highest_space as usize);
+3699 self.highest_space = space;
+3700 if space == SpaceId::Data && self.side.is_client() {
+3701 // Discard 0-RTT keys because 1-RTT keys are available.
+3702 self.zero_rtt_crypto = None;
+3703 }
+3704 }
+3705
+3706 fn discard_space(&mut self, now: Instant, space_id: SpaceId) {
+3707 debug_assert!(space_id != SpaceId::Data);
+3708 trace!("discarding {:?} keys", space_id);
+3709 if space_id == SpaceId::Initial {
+3710 // No longer needed
+3711 if let ConnectionSide::Client { token, .. } = &mut self.side {
+3712 *token = Bytes::new();
+3713 }
+3714 }
+3715 let space = &mut self.spaces[space_id];
+3716 space.crypto = None;
+3717 let pns = space.for_path(PathId::ZERO);
+3718 pns.time_of_last_ack_eliciting_packet = None;
+3719 pns.loss_time = None;
+3720 pns.loss_probes = 0;
+3721 let sent_packets = mem::take(&mut pns.sent_packets);
+3722 let path = self.paths.get_mut(&PathId::ZERO).unwrap();
+3723 for (_, packet) in sent_packets.into_iter() {
+3724 path.data.remove_in_flight(&packet);
+3725 }
+3726
+3727 self.set_loss_detection_timer(now, PathId::ZERO)
+3728 }
+3729
+3730 fn handle_coalesced(
+3731 &mut self,
+3732 now: Instant,
+3733 network_path: FourTuple,
+3734 path_id: PathId,
+3735 ecn: Option<EcnCodepoint>,
+3736 data: BytesMut,
+3737 ) {
+3738 self.path_data_mut(path_id)
+3739 .inc_total_recvd(data.len() as u64);
+3740 let mut remaining = Some(data);
+3741 let cid_len = self
+3742 .local_cid_state
+3743 .values()
+3744 .map(|cid_state| cid_state.cid_len())
+3745 .next()
+3746 .expect("one cid_state must exist");
+3747 while let Some(data) = remaining {
+3748 match PartialDecode::new(
+3749 data,
+3750 &FixedLengthConnectionIdParser::new(cid_len),
+3751 &[self.version],
+3752 self.endpoint_config.grease_quic_bit,
+3753 ) {
+3754 Ok((partial_decode, rest)) => {
+3755 remaining = rest;
+3756 self.handle_decode(now, network_path, path_id, ecn, partial_decode);
+3757 }
+3758 Err(e) => {
+3759 trace!("malformed header: {}", e);
+3760 return;
+3761 }
+3762 }
+3763 }
+3764 }
+3765
+3766 fn handle_decode(
+3767 &mut self,
+3768 now: Instant,
+3769 network_path: FourTuple,
+3770 path_id: PathId,
+3771 ecn: Option<EcnCodepoint>,
+3772 partial_decode: PartialDecode,
+3773 ) {
+3774 let qlog = QlogRecvPacket::new(partial_decode.len());
+3775 if let Some(decoded) = packet_crypto::unprotect_header(
+3776 partial_decode,
+3777 &self.spaces,
+3778 self.zero_rtt_crypto.as_ref(),
+3779 self.peer_params.stateless_reset_token,
+3780 ) {
+3781 self.handle_packet(
+3782 now,
+3783 network_path,
+3784 path_id,
+3785 ecn,
+3786 decoded.packet,
+3787 decoded.stateless_reset,
+3788 qlog,
+3789 );
+3790 }
+3791 }
+3792
+3793 fn handle_packet(
+3794 &mut self,
+3795 now: Instant,
+3796 network_path: FourTuple,
+3797 path_id: PathId,
+3798 ecn: Option<EcnCodepoint>,
+3799 packet: Option<Packet>,
+3800 stateless_reset: bool,
+3801 mut qlog: QlogRecvPacket,
+3802 ) {
+3803 self.stats.udp_rx.ios += 1;
+3804 if let Some(ref packet) = packet {
+3805 trace!(
+3806 "got {:?} packet ({} bytes) from {} using id {}",
+3807 packet.header.space(),
+3808 packet.payload.len() + packet.header_data.len(),
+3809 network_path,
+3810 packet.header.dst_cid(),
+3811 );
+3812 }
3813
-3814 // State transitions for error cases
-3815 if let Err(conn_err) = result {
-3816 match conn_err {
-3817 ConnectionError::ApplicationClosed(reason) => self.state.move_to_closed(reason),
-3818 ConnectionError::ConnectionClosed(reason) => self.state.move_to_closed(reason),
-3819 ConnectionError::Reset
-3820 | ConnectionError::TransportError(TransportError {
-3821 code: TransportErrorCode::AEAD_LIMIT_REACHED,
-3822 ..
-3823 }) => {
-3824 self.state.move_to_drained(Some(conn_err));
-3825 }
-3826 ConnectionError::TimedOut => {
-3827 unreachable!("timeouts aren't generated by packet processing");
-3828 }
-3829 ConnectionError::TransportError(err) => {
-3830 debug!("closing connection due to transport error: {}", err);
-3831 self.state.move_to_closed(err);
+3814 if self.is_handshaking() {
+3815 if path_id != PathId::ZERO {
+3816 debug!(%network_path, %path_id, "discarding multipath packet during handshake");
+3817 return;
+3818 }
+3819 if network_path != self.path_data_mut(path_id).network_path {
+3820 if let Some(hs) = self.state.as_handshake() {
+3821 if hs.allow_server_migration {
+3822 trace!(%network_path, prev = %self.path_data(path_id).network_path, "server migrated to new remote");
+3823 self.path_data_mut(path_id).network_path = network_path;
+3824 self.qlog.emit_tuple_assigned(path_id, network_path, now);
+3825 } else {
+3826 debug!("discarding packet with unexpected remote during handshake");
+3827 return;
+3828 }
+3829 } else {
+3830 debug!("discarding packet with unexpected remote during handshake");
+3831 return;
3832 }
-3833 ConnectionError::VersionMismatch => {
-3834 self.state.move_to_draining(Some(conn_err));
-3835 }
-3836 ConnectionError::LocallyClosed => {
-3837 unreachable!("LocallyClosed isn't generated by packet processing");
-3838 }
-3839 ConnectionError::CidsExhausted => {
-3840 unreachable!("CidsExhausted isn't generated by packet processing");
-3841 }
-3842 };
-3843 }
-3844
-3845 if !was_closed && self.state.is_closed() {
-3846 self.close_common();
-3847 if !self.state.is_drained() {
-3848 self.set_close_timer(now);
+3833 }
+3834 }
+3835
+3836 let was_closed = self.state.is_closed();
+3837 let was_drained = self.state.is_drained();
+3838
+3839 let decrypted = match packet {
+3840 None => Err(None),
+3841 Some(mut packet) => self
+3842 .decrypt_packet(now, path_id, &mut packet)
+3843 .map(move |number| (packet, number)),
+3844 };
+3845 let result = match decrypted {
+3846 _ if stateless_reset => {
+3847 debug!("got stateless reset");
+3848 Err(ConnectionError::Reset)
3849 }
-3850 }
-3851 if !was_drained && self.state.is_drained() {
-3852 self.endpoint_events.push_back(EndpointEventInner::Drained);
-3853 // Close timer may have been started previously, e.g. if we sent a close and got a
-3854 // stateless reset in response
-3855 self.timers
-3856 .stop(Timer::Conn(ConnTimer::Close), self.qlog.with_time(now));
-3857 }
-3858
-3859 // Transmit CONNECTION_CLOSE if necessary
-3860 if matches!(self.state.as_type(), StateType::Closed) {
-3861 // If there is no PathData for this PathId the packet was for a brand new
-3862 // path. It was a valid packet however, so the remote is valid and we want to
-3863 // send CONNECTION_CLOSE.
-3864 let path_remote = self
-3865 .paths
-3866 .get(&path_id)
-3867 .map(|p| p.data.network_path)
-3868 .unwrap_or(network_path);
-3869 self.close = network_path == path_remote;
-3870 }
-3871 }
-3872
-3873 fn process_decrypted_packet(
-3874 &mut self,
-3875 now: Instant,
-3876 network_path: FourTuple,
-3877 path_id: PathId,
-3878 number: Option<u64>,
-3879 packet: Packet,
-3880 qlog: &mut QlogRecvPacket,
-3881 ) -> Result<(), ConnectionError> {
-3882 if !self.paths.contains_key(&path_id) {
-3883 // There is a chance this is a server side, first (for this path) packet, which would
-3884 // be a protocol violation. It's more likely, however, that this is a packet of a
-3885 // pruned path
-3886 trace!(%path_id, ?number, "discarding packet for unknown path");
-3887 return Ok(());
-3888 }
-3889 let state = match self.state.as_type() {
-3890 StateType::Established => {
-3891 match packet.header.space() {
-3892 SpaceId::Data => self.process_payload(
-3893 now,
-3894 network_path,
-3895 path_id,
-3896 number.unwrap(),
-3897 packet,
-3898 qlog,
-3899 )?,
-3900 _ if packet.header.has_frames() => {
-3901 self.process_early_payload(now, path_id, packet, qlog)?
-3902 }
-3903 _ => {
-3904 trace!("discarding unexpected pre-handshake packet");
-3905 }
-3906 }
-3907 return Ok(());
-3908 }
-3909 StateType::Closed => {
-3910 for result in frame::Iter::new(packet.payload.freeze())? {
-3911 let frame = match result {
-3912 Ok(frame) => frame,
-3913 Err(err) => {
-3914 debug!("frame decoding error: {err:?}");
-3915 continue;
-3916 }
-3917 };
-3918 qlog.frame(&frame);
-3919
-3920 if let Frame::Padding = frame {
-3921 continue;
-3922 };
-3923
-3924 self.stats.frame_rx.record(frame.ty());
-3925
-3926 if let Frame::Close(_error) = frame {
-3927 self.state.move_to_draining(None);
-3928 break;
-3929 }
-3930 }
-3931 return Ok(());
-3932 }
-3933 StateType::Draining | StateType::Drained => return Ok(()),
-3934 StateType::Handshake => self.state.as_handshake_mut().expect("checked"),
-3935 };
-3936
-3937 match packet.header {
-3938 Header::Retry {
-3939 src_cid: rem_cid, ..
-3940 } => {
-3941 debug_assert_eq!(path_id, PathId::ZERO);
-3942 if self.side.is_server() {
-3943 return Err(TransportError::PROTOCOL_VIOLATION("client sent Retry").into());
-3944 }
-3945
-3946 let is_valid_retry = self
-3947 .rem_cids
-3948 .get(&path_id)
-3949 .map(|cids| cids.active())
-3950 .map(|orig_dst_cid| {
-3951 self.crypto.is_valid_retry(
-3952 orig_dst_cid,
-3953 &packet.header_data,
-3954 &packet.payload,
-3955 )
-3956 })
-3957 .unwrap_or_default();
-3958 if self.total_authed_packets > 1
-3959 || packet.payload.len() <= 16 // token + 16 byte tag
-3960 || !is_valid_retry
-3961 {
-3962 trace!("discarding invalid Retry");
-3963 // - After the client has received and processed an Initial or Retry
-3964 // packet from the server, it MUST discard any subsequent Retry
-3965 // packets that it receives.
-3966 // - A client MUST discard a Retry packet with a zero-length Retry Token
-3967 // field.
-3968 // - Clients MUST discard Retry packets that have a Retry Integrity Tag
-3969 // that cannot be validated
-3970 return Ok(());
-3971 }
+3850 Err(Some(e)) => {
+3851 warn!("illegal packet: {}", e);
+3852 Err(e.into())
+3853 }
+3854 Err(None) => {
+3855 debug!("failed to authenticate packet");
+3856 self.authentication_failures += 1;
+3857 let integrity_limit = self.spaces[self.highest_space]
+3858 .crypto
+3859 .as_ref()
+3860 .unwrap()
+3861 .packet
+3862 .local
+3863 .integrity_limit();
+3864 if self.authentication_failures > integrity_limit {
+3865 Err(TransportError::AEAD_LIMIT_REACHED("integrity limit violated").into())
+3866 } else {
+3867 return;
+3868 }
+3869 }
+3870 Ok((packet, number)) => {
+3871 qlog.header(&packet.header, number, path_id);
+3872 let span = match number {
+3873 Some(pn) => trace_span!("recv", space = ?packet.header.space(), pn),
+3874 None => trace_span!("recv", space = ?packet.header.space()),
+3875 };
+3876 let _guard = span.enter();
+3877
+3878 let dedup = self.spaces[packet.header.space()]
+3879 .path_space_mut(path_id)
+3880 .map(|pns| &mut pns.dedup);
+3881 if number.zip(dedup).is_some_and(|(n, d)| d.insert(n)) {
+3882 debug!("discarding possible duplicate packet");
+3883 self.qlog.emit_packet_received(qlog, now);
+3884 return;
+3885 } else if self.state.is_handshake() && packet.header.is_short() {
+3886 // TODO: SHOULD buffer these to improve reordering tolerance.
+3887 trace!("dropping short packet during handshake");
+3888 self.qlog.emit_packet_received(qlog, now);
+3889 return;
+3890 } else {
+3891 if let Header::Initial(InitialHeader { ref token, .. }) = packet.header {
+3892 if let Some(hs) = self.state.as_handshake() {
+3893 if self.side.is_server() && token != &hs.expected_token {
+3894 // Clients must send the same retry token in every Initial. Initial
+3895 // packets can be spoofed, so we discard rather than killing the
+3896 // connection.
+3897 warn!("discarding Initial with invalid retry token");
+3898 self.qlog.emit_packet_received(qlog, now);
+3899 return;
+3900 }
+3901 }
+3902 }
+3903
+3904 if !self.state.is_closed() {
+3905 let spin = match packet.header {
+3906 Header::Short { spin, .. } => spin,
+3907 _ => false,
+3908 };
+3909
+3910 if self.side().is_server() && !self.abandoned_paths.contains(&path_id) {
+3911 // Only the client is allowed to open paths
+3912 self.ensure_path(path_id, network_path, now, number);
+3913 }
+3914 if self.paths.contains_key(&path_id) {
+3915 self.on_packet_authenticated(
+3916 now,
+3917 packet.header.space(),
+3918 path_id,
+3919 ecn,
+3920 number,
+3921 spin,
+3922 packet.header.is_1rtt(),
+3923 );
+3924 }
+3925 }
+3926
+3927 let res = self.process_decrypted_packet(
+3928 now,
+3929 network_path,
+3930 path_id,
+3931 number,
+3932 packet,
+3933 &mut qlog,
+3934 );
+3935
+3936 self.qlog.emit_packet_received(qlog, now);
+3937 res
+3938 }
+3939 }
+3940 };
+3941
+3942 // State transitions for error cases
+3943 if let Err(conn_err) = result {
+3944 match conn_err {
+3945 ConnectionError::ApplicationClosed(reason) => self.state.move_to_closed(reason),
+3946 ConnectionError::ConnectionClosed(reason) => self.state.move_to_closed(reason),
+3947 ConnectionError::Reset
+3948 | ConnectionError::TransportError(TransportError {
+3949 code: TransportErrorCode::AEAD_LIMIT_REACHED,
+3950 ..
+3951 }) => {
+3952 self.state.move_to_drained(Some(conn_err));
+3953 }
+3954 ConnectionError::TimedOut => {
+3955 unreachable!("timeouts aren't generated by packet processing");
+3956 }
+3957 ConnectionError::TransportError(err) => {
+3958 debug!("closing connection due to transport error: {}", err);
+3959 self.state.move_to_closed(err);
+3960 }
+3961 ConnectionError::VersionMismatch => {
+3962 self.state.move_to_draining(Some(conn_err));
+3963 }
+3964 ConnectionError::LocallyClosed => {
+3965 unreachable!("LocallyClosed isn't generated by packet processing");
+3966 }
+3967 ConnectionError::CidsExhausted => {
+3968 unreachable!("CidsExhausted isn't generated by packet processing");
+3969 }
+3970 };
+3971 }
3972
-3973 trace!("retrying with CID {}", rem_cid);
-3974 let client_hello = state.client_hello.take().unwrap();
-3975 self.retry_src_cid = Some(rem_cid);
-3976 self.rem_cids
-3977 .get_mut(&path_id)
-3978 .expect("PathId::ZERO not yet abandoned, is_valid_retry would have been false")
-3979 .update_initial_cid(rem_cid);
-3980 self.rem_handshake_cid = rem_cid;
-3981
-3982 let space = &mut self.spaces[SpaceId::Initial];
-3983 if let Some(info) = space.for_path(PathId::ZERO).take(0) {
-3984 self.on_packet_acked(now, PathId::ZERO, info);
-3985 };
+3973 if !was_closed && self.state.is_closed() {
+3974 self.close_common();
+3975 if !self.state.is_drained() {
+3976 self.set_close_timer(now);
+3977 }
+3978 }
+3979 if !was_drained && self.state.is_drained() {
+3980 self.endpoint_events.push_back(EndpointEventInner::Drained);
+3981 // Close timer may have been started previously, e.g. if we sent a close and got a
+3982 // stateless reset in response
+3983 self.timers
+3984 .stop(Timer::Conn(ConnTimer::Close), self.qlog.with_time(now));
+3985 }
3986
-3987 self.discard_space(now, SpaceId::Initial); // Make sure we clean up after
-3988 // any retransmitted Initials
-3989 self.spaces[SpaceId::Initial] = {
-3990 let mut space = PacketSpace::new(now, SpaceId::Initial, &mut self.rng);
-3991 space.crypto = Some(self.crypto.initial_keys(rem_cid, self.side.side()));
-3992 space.crypto_offset = client_hello.len() as u64;
-3993 space.for_path(path_id).next_packet_number = self.spaces[SpaceId::Initial]
-3994 .for_path(path_id)
-3995 .next_packet_number;
-3996 space.pending.crypto.push_back(frame::Crypto {
-3997 offset: 0,
-3998 data: client_hello,
-3999 });
-4000 space
-4001 };
-4002
-4003 // Retransmit all 0-RTT data
-4004 let zero_rtt = mem::take(
-4005 &mut self.spaces[SpaceId::Data]
-4006 .for_path(PathId::ZERO)
-4007 .sent_packets,
-4008 );
-4009 for (_, info) in zero_rtt.into_iter() {
-4010 self.paths
-4011 .get_mut(&PathId::ZERO)
-4012 .unwrap()
-4013 .remove_in_flight(&info);
-4014 self.spaces[SpaceId::Data].pending |= info.retransmits;
-4015 }
-4016 self.streams.retransmit_all_for_0rtt();
-4017
-4018 let token_len = packet.payload.len() - 16;
-4019 let ConnectionSide::Client { ref mut token, .. } = self.side else {
-4020 unreachable!("we already short-circuited if we're server");
-4021 };
-4022 *token = packet.payload.freeze().split_to(token_len);
-4023
-4024 self.state = State::handshake(state::Handshake {
-4025 expected_token: Bytes::new(),
-4026 rem_cid_set: false,
-4027 client_hello: None,
-4028 allow_server_migration: true,
-4029 });
-4030 Ok(())
-4031 }
-4032 Header::Long {
-4033 ty: LongType::Handshake,
-4034 src_cid: rem_cid,
-4035 dst_cid: loc_cid,
-4036 ..
-4037 } => {
-4038 debug_assert_eq!(path_id, PathId::ZERO);
-4039 if rem_cid != self.rem_handshake_cid {
-4040 debug!(
-4041 "discarding packet with mismatched remote CID: {} != {}",
-4042 self.rem_handshake_cid, rem_cid
-4043 );
-4044 return Ok(());
-4045 }
-4046 self.on_path_validated(path_id);
+3987 // Transmit CONNECTION_CLOSE if necessary
+3988 if matches!(self.state.as_type(), StateType::Closed) {
+3989 // If there is no PathData for this PathId the packet was for a brand new
+3990 // path. It was a valid packet however, so the remote is valid and we want to
+3991 // send CONNECTION_CLOSE.
+3992 let path_remote = self
+3993 .paths
+3994 .get(&path_id)
+3995 .map(|p| p.data.network_path)
+3996 .unwrap_or(network_path);
+3997 self.close = network_path == path_remote;
+3998 }
+3999 }
+4000
+4001 fn process_decrypted_packet(
+4002 &mut self,
+4003 now: Instant,
+4004 network_path: FourTuple,
+4005 path_id: PathId,
+4006 number: Option<u64>,
+4007 packet: Packet,
+4008 qlog: &mut QlogRecvPacket,
+4009 ) -> Result<(), ConnectionError> {
+4010 if !self.paths.contains_key(&path_id) {
+4011 // There is a chance this is a server side, first (for this path) packet, which would
+4012 // be a protocol violation. It's more likely, however, that this is a packet of a
+4013 // pruned path
+4014 trace!(%path_id, ?number, "discarding packet for unknown path");
+4015 return Ok(());
+4016 }
+4017 let state = match self.state.as_type() {
+4018 StateType::Established => {
+4019 match packet.header.space() {
+4020 SpaceId::Data => self.process_payload(
+4021 now,
+4022 network_path,
+4023 path_id,
+4024 number.unwrap(),
+4025 packet,
+4026 qlog,
+4027 )?,
+4028 _ if packet.header.has_frames() => {
+4029 self.process_early_payload(now, path_id, packet, qlog)?
+4030 }
+4031 _ => {
+4032 trace!("discarding unexpected pre-handshake packet");
+4033 }
+4034 }
+4035 return Ok(());
+4036 }
+4037 StateType::Closed => {
+4038 for result in frame::Iter::new(packet.payload.freeze())? {
+4039 let frame = match result {
+4040 Ok(frame) => frame,
+4041 Err(err) => {
+4042 debug!("frame decoding error: {err:?}");
+4043 continue;
+4044 }
+4045 };
+4046 qlog.frame(&frame);
4047
-4048 self.process_early_payload(now, path_id, packet, qlog)?;
-4049 if self.state.is_closed() {
-4050 return Ok(());
-4051 }
-4052
-4053 if self.crypto.is_handshaking() {
-4054 trace!("handshake ongoing");
-4055 return Ok(());
-4056 }
-4057
-4058 if self.side.is_client() {
-4059 // Client-only because server params were set from the client's Initial
-4060 let params = self.crypto.transport_parameters()?.ok_or_else(|| {
-4061 TransportError::new(
-4062 TransportErrorCode::crypto(0x6d),
-4063 "transport parameters missing".to_owned(),
-4064 )
-4065 })?;
-4066
-4067 if self.has_0rtt() {
-4068 if !self.crypto.early_data_accepted().unwrap() {
-4069 debug_assert!(self.side.is_client());
-4070 debug!("0-RTT rejected");
-4071 self.accepted_0rtt = false;
-4072 self.streams.zero_rtt_rejected();
+4048 if let Frame::Padding = frame {
+4049 continue;
+4050 };
+4051
+4052 self.stats.frame_rx.record(frame.ty());
+4053
+4054 if let Frame::Close(_error) = frame {
+4055 self.state.move_to_draining(None);
+4056 break;
+4057 }
+4058 }
+4059 return Ok(());
+4060 }
+4061 StateType::Draining | StateType::Drained => return Ok(()),
+4062 StateType::Handshake => self.state.as_handshake_mut().expect("checked"),
+4063 };
+4064
+4065 match packet.header {
+4066 Header::Retry {
+4067 src_cid: rem_cid, ..
+4068 } => {
+4069 debug_assert_eq!(path_id, PathId::ZERO);
+4070 if self.side.is_server() {
+4071 return Err(TransportError::PROTOCOL_VIOLATION("client sent Retry").into());
+4072 }
4073
-4074 // Discard already-queued frames
-4075 self.spaces[SpaceId::Data].pending = Retransmits::default();
-4076
-4077 // Discard 0-RTT packets
-4078 let sent_packets = mem::take(
-4079 &mut self.spaces[SpaceId::Data].for_path(path_id).sent_packets,
-4080 );
-4081 for (_, packet) in sent_packets.into_iter() {
-4082 self.paths
-4083 .get_mut(&path_id)
-4084 .unwrap()
-4085 .remove_in_flight(&packet);
-4086 }
-4087 } else {
-4088 self.accepted_0rtt = true;
-4089 params.validate_resumption_from(&self.peer_params)?;
-4090 }
-4091 }
-4092 if let Some(token) = params.stateless_reset_token {
-4093 // TODO(matheus23): Reset token for a remote, or for a 4-tuple?
-4094 let remote = self.path_data(path_id).network_path.remote;
-4095 self.endpoint_events
-4096 .push_back(EndpointEventInner::ResetToken(path_id, remote, token));
-4097 }
-4098 self.handle_peer_params(params, loc_cid, rem_cid, now)?;
-4099 self.issue_first_cids(now);
-4100 } else {
-4101 // Server-only
-4102 self.spaces[SpaceId::Data].pending.handshake_done = true;
-4103 self.discard_space(now, SpaceId::Handshake);
-4104 self.events.push_back(Event::HandshakeConfirmed);
-4105 trace!("handshake confirmed");
-4106 }
-4107
-4108 self.events.push_back(Event::Connected);
-4109 self.state.move_to_established();
-4110 trace!("established");
-4111
-4112 // Multipath can only be enabled after the state has reached Established.
-4113 // So this can not happen any earlier.
-4114 self.issue_first_path_cids(now);
-4115 Ok(())
-4116 }
-4117 Header::Initial(InitialHeader {
-4118 src_cid: rem_cid,
-4119 dst_cid: loc_cid,
-4120 ..
-4121 }) => {
-4122 debug_assert_eq!(path_id, PathId::ZERO);
-4123 if !state.rem_cid_set {
-4124 trace!("switching remote CID to {}", rem_cid);
-4125 let mut state = state.clone();
-4126 self.rem_cids
-4127 .get_mut(&path_id)
-4128 .expect("PathId::ZERO not yet abandoned")
-4129 .update_initial_cid(rem_cid);
-4130 self.rem_handshake_cid = rem_cid;
-4131 self.orig_rem_cid = rem_cid;
-4132 state.rem_cid_set = true;
-4133 self.state.move_to_handshake(state);
-4134 } else if rem_cid != self.rem_handshake_cid {
-4135 debug!(
-4136 "discarding packet with mismatched remote CID: {} != {}",
-4137 self.rem_handshake_cid, rem_cid
-4138 );
-4139 return Ok(());
-4140 }
-4141
-4142 let starting_space = self.highest_space;
-4143 self.process_early_payload(now, path_id, packet, qlog)?;
-4144
-4145 if self.side.is_server()
-4146 && starting_space == SpaceId::Initial
-4147 && self.highest_space != SpaceId::Initial
-4148 {
-4149 let params = self.crypto.transport_parameters()?.ok_or_else(|| {
-4150 TransportError::new(
-4151 TransportErrorCode::crypto(0x6d),
-4152 "transport parameters missing".to_owned(),
-4153 )
-4154 })?;
-4155 self.handle_peer_params(params, loc_cid, rem_cid, now)?;
-4156 self.issue_first_cids(now);
-4157 self.init_0rtt(now);
-4158 }
-4159 Ok(())
-4160 }
-4161 Header::Long {
-4162 ty: LongType::ZeroRtt,
-4163 ..
-4164 } => {
-4165 self.process_payload(now, network_path, path_id, number.unwrap(), packet, qlog)?;
-4166 Ok(())
-4167 }
-4168 Header::VersionNegotiate { .. } => {
-4169 if self.total_authed_packets > 1 {
-4170 return Ok(());
-4171 }
-4172 let supported = packet
-4173 .payload
-4174 .chunks(4)
-4175 .any(|x| match <[u8; 4]>::try_from(x) {
-4176 Ok(version) => self.version == u32::from_be_bytes(version),
-4177 Err(_) => false,
-4178 });
-4179 if supported {
-4180 return Ok(());
-4181 }
-4182 debug!("remote doesn't support our version");
-4183 Err(ConnectionError::VersionMismatch)
-4184 }
-4185 Header::Short { .. } => unreachable!(
-4186 "short packets received during handshake are discarded in handle_packet"
-4187 ),
-4188 }
-4189 }
-4190
-4191 /// Process an Initial or Handshake packet payload
-4192 fn process_early_payload(
-4193 &mut self,
-4194 now: Instant,
-4195 path_id: PathId,
-4196 packet: Packet,
-4197 #[allow(unused)] qlog: &mut QlogRecvPacket,
-4198 ) -> Result<(), TransportError> {
-4199 debug_assert_ne!(packet.header.space(), SpaceId::Data);
-4200 debug_assert_eq!(path_id, PathId::ZERO);
-4201 let payload_len = packet.payload.len();
-4202 let mut ack_eliciting = false;
-4203 for result in frame::Iter::new(packet.payload.freeze())? {
-4204 let frame = result?;
-4205 qlog.frame(&frame);
-4206 let span = match frame {
-4207 Frame::Padding => continue,
-4208 _ => Some(trace_span!("frame", ty = %frame.ty(), path = tracing::field::Empty)),
-4209 };
-4210
-4211 self.stats.frame_rx.record(frame.ty());
-4212
-4213 let _guard = span.as_ref().map(|x| x.enter());
-4214 ack_eliciting |= frame.is_ack_eliciting();
-4215
-4216 // Process frames
-4217 if frame.is_1rtt() && packet.header.space() != SpaceId::Data {
-4218 return Err(TransportError::PROTOCOL_VIOLATION(
-4219 "illegal frame type in handshake",
-4220 ));
-4221 }
-4222
-4223 match frame {
-4224 Frame::Padding | Frame::Ping => {}
-4225 Frame::Crypto(frame) => {
-4226 self.read_crypto(packet.header.space(), &frame, payload_len)?;
-4227 }
-4228 Frame::Ack(ack) => {
-4229 self.on_ack_received(now, packet.header.space(), ack)?;
-4230 }
-4231 Frame::PathAck(ack) => {
-4232 span.as_ref()
-4233 .map(|span| span.record("path", tracing::field::debug(&ack.path_id)));
-4234 self.on_path_ack_received(now, packet.header.space(), ack)?;
-4235 }
-4236 Frame::Close(reason) => {
-4237 self.state.move_to_draining(Some(reason.into()));
-4238 return Ok(());
-4239 }
-4240 _ => {
-4241 let mut err =
-4242 TransportError::PROTOCOL_VIOLATION("illegal frame type in handshake");
-4243 err.frame = frame::MaybeFrame::Known(frame.ty());
-4244 return Err(err);
-4245 }
-4246 }
-4247 }
-4248
-4249 if ack_eliciting {
-4250 // In the initial and handshake spaces, ACKs must be sent immediately
-4251 self.spaces[packet.header.space()]
-4252 .for_path(path_id)
-4253 .pending_acks
-4254 .set_immediate_ack_required();
-4255 }
-4256
-4257 self.write_crypto();
-4258 Ok(())
-4259 }
-4260
-4261 /// Processes the packet payload, always in the data space.
-4262 fn process_payload(
-4263 &mut self,
-4264 now: Instant,
-4265 network_path: FourTuple,
-4266 path_id: PathId,
-4267 number: u64,
-4268 packet: Packet,
-4269 #[allow(unused)] qlog: &mut QlogRecvPacket,
-4270 ) -> Result<(), TransportError> {
-4271 let payload = packet.payload.freeze();
-4272 let mut is_probing_packet = true;
-4273 let mut close = None;
-4274 let payload_len = payload.len();
-4275 let mut ack_eliciting = false;
-4276 // if this packet triggers a path migration and includes a observed address frame, it's
-4277 // stored here
-4278 let mut migration_observed_addr = None;
-4279 for result in frame::Iter::new(payload)? {
-4280 let frame = result?;
-4281 qlog.frame(&frame);
-4282 let span = match frame {
-4283 Frame::Padding => continue,
-4284 _ => trace_span!("frame", ty = %frame.ty(), path = tracing::field::Empty),
-4285 };
-4286
-4287 self.stats.frame_rx.record(frame.ty());
-4288 // Crypto, Stream and Datagram frames are special cased in order no pollute
-4289 // the log with payload data
-4290 match &frame {
-4291 Frame::Crypto(f) => {
-4292 trace!(offset = f.offset, len = f.data.len(), "got crypto frame");
-4293 }
-4294 Frame::Stream(f) => {
-4295 trace!(id = %f.id, offset = f.offset, len = f.data.len(), fin = f.fin, "got stream frame");
-4296 }
-4297 Frame::Datagram(f) => {
-4298 trace!(len = f.data.len(), "got datagram frame");
+4074 let is_valid_retry = self
+4075 .rem_cids
+4076 .get(&path_id)
+4077 .map(|cids| cids.active())
+4078 .map(|orig_dst_cid| {
+4079 self.crypto.is_valid_retry(
+4080 orig_dst_cid,
+4081 &packet.header_data,
+4082 &packet.payload,
+4083 )
+4084 })
+4085 .unwrap_or_default();
+4086 if self.total_authed_packets > 1
+4087 || packet.payload.len() <= 16 // token + 16 byte tag
+4088 || !is_valid_retry
+4089 {
+4090 trace!("discarding invalid Retry");
+4091 // - After the client has received and processed an Initial or Retry
+4092 // packet from the server, it MUST discard any subsequent Retry
+4093 // packets that it receives.
+4094 // - A client MUST discard a Retry packet with a zero-length Retry Token
+4095 // field.
+4096 // - Clients MUST discard Retry packets that have a Retry Integrity Tag
+4097 // that cannot be validated
+4098 return Ok(());
+4099 }
+4100
+4101 trace!("retrying with CID {}", rem_cid);
+4102 let client_hello = state.client_hello.take().unwrap();
+4103 self.retry_src_cid = Some(rem_cid);
+4104 self.rem_cids
+4105 .get_mut(&path_id)
+4106 .expect("PathId::ZERO not yet abandoned, is_valid_retry would have been false")
+4107 .update_initial_cid(rem_cid);
+4108 self.rem_handshake_cid = rem_cid;
+4109
+4110 let space = &mut self.spaces[SpaceId::Initial];
+4111 if let Some(info) = space.for_path(PathId::ZERO).take(0) {
+4112 self.on_packet_acked(now, PathId::ZERO, info);
+4113 };
+4114
+4115 self.discard_space(now, SpaceId::Initial); // Make sure we clean up after
+4116 // any retransmitted Initials
+4117 self.spaces[SpaceId::Initial] = {
+4118 let mut space = PacketSpace::new(now, SpaceId::Initial, &mut self.rng);
+4119 space.crypto = Some(self.crypto.initial_keys(rem_cid, self.side.side()));
+4120 space.crypto_offset = client_hello.len() as u64;
+4121 space.for_path(path_id).next_packet_number = self.spaces[SpaceId::Initial]
+4122 .for_path(path_id)
+4123 .next_packet_number;
+4124 space.pending.crypto.push_back(frame::Crypto {
+4125 offset: 0,
+4126 data: client_hello,
+4127 });
+4128 space
+4129 };
+4130
+4131 // Retransmit all 0-RTT data
+4132 let zero_rtt = mem::take(
+4133 &mut self.spaces[SpaceId::Data]
+4134 .for_path(PathId::ZERO)
+4135 .sent_packets,
+4136 );
+4137 for (_, info) in zero_rtt.into_iter() {
+4138 self.paths
+4139 .get_mut(&PathId::ZERO)
+4140 .unwrap()
+4141 .remove_in_flight(&info);
+4142 self.spaces[SpaceId::Data].pending |= info.retransmits;
+4143 }
+4144 self.streams.retransmit_all_for_0rtt();
+4145
+4146 let token_len = packet.payload.len() - 16;
+4147 let ConnectionSide::Client { ref mut token, .. } = self.side else {
+4148 unreachable!("we already short-circuited if we're server");
+4149 };
+4150 *token = packet.payload.freeze().split_to(token_len);
+4151
+4152 self.state = State::handshake(state::Handshake {
+4153 expected_token: Bytes::new(),
+4154 rem_cid_set: false,
+4155 client_hello: None,
+4156 allow_server_migration: true,
+4157 });
+4158 Ok(())
+4159 }
+4160 Header::Long {
+4161 ty: LongType::Handshake,
+4162 src_cid: rem_cid,
+4163 dst_cid: loc_cid,
+4164 ..
+4165 } => {
+4166 debug_assert_eq!(path_id, PathId::ZERO);
+4167 if rem_cid != self.rem_handshake_cid {
+4168 debug!(
+4169 "discarding packet with mismatched remote CID: {} != {}",
+4170 self.rem_handshake_cid, rem_cid
+4171 );
+4172 return Ok(());
+4173 }
+4174 self.on_path_validated(path_id);
+4175
+4176 self.process_early_payload(now, path_id, packet, qlog)?;
+4177 if self.state.is_closed() {
+4178 return Ok(());
+4179 }
+4180
+4181 if self.crypto.is_handshaking() {
+4182 trace!("handshake ongoing");
+4183 return Ok(());
+4184 }
+4185
+4186 if self.side.is_client() {
+4187 // Client-only because server params were set from the client's Initial
+4188 let params = self.crypto.transport_parameters()?.ok_or_else(|| {
+4189 TransportError::new(
+4190 TransportErrorCode::crypto(0x6d),
+4191 "transport parameters missing".to_owned(),
+4192 )
+4193 })?;
+4194
+4195 if self.has_0rtt() {
+4196 if !self.crypto.early_data_accepted().unwrap() {
+4197 debug_assert!(self.side.is_client());
+4198 debug!("0-RTT rejected");
+4199 self.accepted_0rtt = false;
+4200 self.streams.zero_rtt_rejected();
+4201
+4202 // Discard already-queued frames
+4203 self.spaces[SpaceId::Data].pending = Retransmits::default();
+4204
+4205 // Discard 0-RTT packets
+4206 let sent_packets = mem::take(
+4207 &mut self.spaces[SpaceId::Data].for_path(path_id).sent_packets,
+4208 );
+4209 for (_, packet) in sent_packets.into_iter() {
+4210 self.paths
+4211 .get_mut(&path_id)
+4212 .unwrap()
+4213 .remove_in_flight(&packet);
+4214 }
+4215 } else {
+4216 self.accepted_0rtt = true;
+4217 params.validate_resumption_from(&self.peer_params)?;
+4218 }
+4219 }
+4220 if let Some(token) = params.stateless_reset_token {
+4221 // TODO(matheus23): Reset token for a remote, or for a 4-tuple?
+4222 let remote = self.path_data(path_id).network_path.remote;
+4223 self.endpoint_events
+4224 .push_back(EndpointEventInner::ResetToken(path_id, remote, token));
+4225 }
+4226 self.handle_peer_params(params, loc_cid, rem_cid, now)?;
+4227 self.issue_first_cids(now);
+4228 } else {
+4229 // Server-only
+4230 self.spaces[SpaceId::Data].pending.handshake_done = true;
+4231 self.discard_space(now, SpaceId::Handshake);
+4232 self.events.push_back(Event::HandshakeConfirmed);
+4233 trace!("handshake confirmed");
+4234 }
+4235
+4236 self.events.push_back(Event::Connected);
+4237 self.state.move_to_established();
+4238 trace!("established");
+4239
+4240 // Multipath can only be enabled after the state has reached Established.
+4241 // So this can not happen any earlier.
+4242 self.issue_first_path_cids(now);
+4243 Ok(())
+4244 }
+4245 Header::Initial(InitialHeader {
+4246 src_cid: rem_cid,
+4247 dst_cid: loc_cid,
+4248 ..
+4249 }) => {
+4250 debug_assert_eq!(path_id, PathId::ZERO);
+4251 if !state.rem_cid_set {
+4252 trace!("switching remote CID to {}", rem_cid);
+4253 let mut state = state.clone();
+4254 self.rem_cids
+4255 .get_mut(&path_id)
+4256 .expect("PathId::ZERO not yet abandoned")
+4257 .update_initial_cid(rem_cid);
+4258 self.rem_handshake_cid = rem_cid;
+4259 self.orig_rem_cid = rem_cid;
+4260 state.rem_cid_set = true;
+4261 self.state.move_to_handshake(state);
+4262 } else if rem_cid != self.rem_handshake_cid {
+4263 debug!(
+4264 "discarding packet with mismatched remote CID: {} != {}",
+4265 self.rem_handshake_cid, rem_cid
+4266 );
+4267 return Ok(());
+4268 }
+4269
+4270 let starting_space = self.highest_space;
+4271 self.process_early_payload(now, path_id, packet, qlog)?;
+4272
+4273 if self.side.is_server()
+4274 && starting_space == SpaceId::Initial
+4275 && self.highest_space != SpaceId::Initial
+4276 {
+4277 let params = self.crypto.transport_parameters()?.ok_or_else(|| {
+4278 TransportError::new(
+4279 TransportErrorCode::crypto(0x6d),
+4280 "transport parameters missing".to_owned(),
+4281 )
+4282 })?;
+4283 self.handle_peer_params(params, loc_cid, rem_cid, now)?;
+4284 self.issue_first_cids(now);
+4285 self.init_0rtt(now);
+4286 }
+4287 Ok(())
+4288 }
+4289 Header::Long {
+4290 ty: LongType::ZeroRtt,
+4291 ..
+4292 } => {
+4293 self.process_payload(now, network_path, path_id, number.unwrap(), packet, qlog)?;
+4294 Ok(())
+4295 }
+4296 Header::VersionNegotiate { .. } => {
+4297 if self.total_authed_packets > 1 {
+4298 return Ok(());
4299 }
-4300 f => {
-4301 trace!("got frame {f}");
-4302 }
-4303 }
-4304
-4305 let _guard = span.enter();
-4306 if packet.header.is_0rtt() {
-4307 match frame {
-4308 Frame::Crypto(_) | Frame::Close(Close::Application(_)) => {
-4309 return Err(TransportError::PROTOCOL_VIOLATION(
-4310 "illegal frame type in 0-RTT",
-4311 ));
-4312 }
-4313 _ => {
-4314 if frame.is_1rtt() {
-4315 return Err(TransportError::PROTOCOL_VIOLATION(
-4316 "illegal frame type in 0-RTT",
-4317 ));
-4318 }
-4319 }
-4320 }
-4321 }
-4322 ack_eliciting |= frame.is_ack_eliciting();
-4323
-4324 // Check whether this could be a probing packet
-4325 match frame {
-4326 Frame::Padding
-4327 | Frame::PathChallenge(_)
-4328 | Frame::PathResponse(_)
-4329 | Frame::NewConnectionId(_)
-4330 | Frame::ObservedAddr(_) => {}
-4331 _ => {
-4332 is_probing_packet = false;
-4333 }
-4334 }
-4335
-4336 match frame {
-4337 Frame::Crypto(frame) => {
-4338 self.read_crypto(SpaceId::Data, &frame, payload_len)?;
-4339 }
-4340 Frame::Stream(frame) => {
-4341 if self.streams.received(frame, payload_len)?.should_transmit() {
-4342 self.spaces[SpaceId::Data].pending.max_data = true;
-4343 }
-4344 }
-4345 Frame::Ack(ack) => {
-4346 self.on_ack_received(now, SpaceId::Data, ack)?;
-4347 }
-4348 Frame::PathAck(ack) => {
-4349 span.record("path", tracing::field::debug(&ack.path_id));
-4350 self.on_path_ack_received(now, SpaceId::Data, ack)?;
-4351 }
+4300 let supported = packet
+4301 .payload
+4302 .chunks(4)
+4303 .any(|x| match <[u8; 4]>::try_from(x) {
+4304 Ok(version) => self.version == u32::from_be_bytes(version),
+4305 Err(_) => false,
+4306 });
+4307 if supported {
+4308 return Ok(());
+4309 }
+4310 debug!("remote doesn't support our version");
+4311 Err(ConnectionError::VersionMismatch)
+4312 }
+4313 Header::Short { .. } => unreachable!(
+4314 "short packets received during handshake are discarded in handle_packet"
+4315 ),
+4316 }
+4317 }
+4318
+4319 /// Process an Initial or Handshake packet payload
+4320 fn process_early_payload(
+4321 &mut self,
+4322 now: Instant,
+4323 path_id: PathId,
+4324 packet: Packet,
+4325 #[allow(unused)] qlog: &mut QlogRecvPacket,
+4326 ) -> Result<(), TransportError> {
+4327 debug_assert_ne!(packet.header.space(), SpaceId::Data);
+4328 debug_assert_eq!(path_id, PathId::ZERO);
+4329 let payload_len = packet.payload.len();
+4330 let mut ack_eliciting = false;
+4331 for result in frame::Iter::new(packet.payload.freeze())? {
+4332 let frame = result?;
+4333 qlog.frame(&frame);
+4334 let span = match frame {
+4335 Frame::Padding => continue,
+4336 _ => Some(trace_span!("frame", ty = %frame.ty(), path = tracing::field::Empty)),
+4337 };
+4338
+4339 self.stats.frame_rx.record(frame.ty());
+4340
+4341 let _guard = span.as_ref().map(|x| x.enter());
+4342 ack_eliciting |= frame.is_ack_eliciting();
+4343
+4344 // Process frames
+4345 if frame.is_1rtt() && packet.header.space() != SpaceId::Data {
+4346 return Err(TransportError::PROTOCOL_VIOLATION(
+4347 "illegal frame type in handshake",
+4348 ));
+4349 }
+4350
+4351 match frame {
4352 Frame::Padding | Frame::Ping => {}
-4353 Frame::Close(reason) => {
-4354 close = Some(reason);
+4353 Frame::Crypto(frame) => {
+4354 self.read_crypto(packet.header.space(), &frame, payload_len)?;
4355 }
-4356 Frame::PathChallenge(challenge) => {
-4357 let path = &mut self
-4358 .path_mut(path_id)
-4359 .expect("payload is processed only after the path becomes known");
-4360 path.path_responses.push(number, challenge.0, network_path);
-4361 // At this point, update_network_path_or_discard was already called, so
-4362 // we don't need to be lenient about `local_ip` possibly mis-matching.
-4363 if network_path == path.network_path {
-4364 // PATH_CHALLENGE on active path, possible off-path packet forwarding
-4365 // attack. Send a non-probing packet to recover the active path.
-4366 // TODO(flub): No longer true! We now path_challege also to validate
-4367 // the path if the path is new, without an RFC9000-style
-4368 // migration involved. This means we add in an extra
-4369 // IMMEDIATE_ACK on some challenges. It isn't really wrong to do
-4370 // so, but it still is something untidy. We should instead
-4371 // suppress this when we know the remote is still validating the
-4372 // path.
-4373 match self.peer_supports_ack_frequency() {
-4374 true => self.immediate_ack(path_id),
-4375 false => {
-4376 self.ping_path(path_id).ok();
-4377 }
-4378 }
-4379 }
-4380 }
-4381 Frame::PathResponse(response) => {
-4382 let path = self
-4383 .paths
-4384 .get_mut(&path_id)
-4385 .expect("payload is processed only after the path becomes known");
-4386
-4387 use PathTimer::*;
-4388 use paths::OnPathResponseReceived::*;
-4389 match path
-4390 .data
-4391 .on_path_response_received(now, response.0, network_path)
-4392 {
-4393 OnPath { was_open } => {
-4394 let qlog = self.qlog.with_time(now);
-4395
-4396 self.timers
-4397 .stop(Timer::PerPath(path_id, PathValidation), qlog.clone());
-4398 self.timers
-4399 .stop(Timer::PerPath(path_id, PathOpen), qlog.clone());
-4400
-4401 let next_challenge = path
-4402 .data
-4403 .earliest_expiring_challenge()
-4404 .map(|time| time + self.ack_frequency.max_ack_delay_for_pto());
-4405 self.timers.set_or_stop(
-4406 Timer::PerPath(path_id, PathChallengeLost),
-4407 next_challenge,
-4408 qlog,
-4409 );
-4410
-4411 if !was_open {
-4412 self.events
-4413 .push_back(Event::Path(PathEvent::Opened { id: path_id }));
-4414 if let Some(observed) = path.data.last_observed_addr_report.as_ref()
-4415 {
-4416 self.events.push_back(Event::Path(PathEvent::ObservedAddr {
-4417 id: path_id,
-4418 addr: observed.socket_addr(),
-4419 }));
-4420 }
-4421 }
-4422 if let Some((_, ref mut prev)) = path.prev {
-4423 prev.challenges_sent.clear();
-4424 prev.send_new_challenge = false;
-4425 }
-4426 }
-4427 OffPath => {
-4428 debug!("Response to off-path PathChallenge!");
-4429 let next_challenge = path
-4430 .data
-4431 .earliest_expiring_challenge()
-4432 .map(|time| time + self.ack_frequency.max_ack_delay_for_pto());
-4433 self.timers.set_or_stop(
-4434 Timer::PerPath(path_id, PathChallengeLost),
-4435 next_challenge,
-4436 self.qlog.with_time(now),
-4437 );
-4438 }
-4439 Invalid { expected } => {
-4440 debug!(%response, %network_path, %expected, "ignoring invalid PATH_RESPONSE")
-4441 }
-4442 Unknown => debug!(%response, "ignoring invalid PATH_RESPONSE"),
-4443 }
-4444 }
-4445 Frame::MaxData(frame::MaxData(bytes)) => {
-4446 self.streams.received_max_data(bytes);
-4447 }
-4448 Frame::MaxStreamData(frame::MaxStreamData { id, offset }) => {
-4449 self.streams.received_max_stream_data(id, offset)?;
-4450 }
-4451 Frame::MaxStreams(frame::MaxStreams { dir, count }) => {
-4452 self.streams.received_max_streams(dir, count)?;
-4453 }
-4454 Frame::ResetStream(frame) => {
-4455 if self.streams.received_reset(frame)?.should_transmit() {
-4456 self.spaces[SpaceId::Data].pending.max_data = true;
-4457 }
-4458 }
-4459 Frame::DataBlocked { offset } => {
-4460 debug!(offset, "peer claims to be blocked at connection level");
+4356 Frame::Ack(ack) => {
+4357 self.on_ack_received(now, packet.header.space(), ack)?;
+4358 }
+4359 Frame::PathAck(ack) => {
+4360 span.as_ref()
+4361 .map(|span| span.record("path", tracing::field::debug(&ack.path_id)));
+4362 self.on_path_ack_received(now, packet.header.space(), ack)?;
+4363 }
+4364 Frame::Close(reason) => {
+4365 self.state.move_to_draining(Some(reason.into()));
+4366 return Ok(());
+4367 }
+4368 _ => {
+4369 let mut err =
+4370 TransportError::PROTOCOL_VIOLATION("illegal frame type in handshake");
+4371 err.frame = frame::MaybeFrame::Known(frame.ty());
+4372 return Err(err);
+4373 }
+4374 }
+4375 }
+4376
+4377 if ack_eliciting {
+4378 // In the initial and handshake spaces, ACKs must be sent immediately
+4379 self.spaces[packet.header.space()]
+4380 .for_path(path_id)
+4381 .pending_acks
+4382 .set_immediate_ack_required();
+4383 }
+4384
+4385 self.write_crypto();
+4386 Ok(())
+4387 }
+4388
+4389 /// Processes the packet payload, always in the data space.
+4390 fn process_payload(
+4391 &mut self,
+4392 now: Instant,
+4393 network_path: FourTuple,
+4394 path_id: PathId,
+4395 number: u64,
+4396 packet: Packet,
+4397 #[allow(unused)] qlog: &mut QlogRecvPacket,
+4398 ) -> Result<(), TransportError> {
+4399 let payload = packet.payload.freeze();
+4400 let mut is_probing_packet = true;
+4401 let mut close = None;
+4402 let payload_len = payload.len();
+4403 let mut ack_eliciting = false;
+4404 // if this packet triggers a path migration and includes a observed address frame, it's
+4405 // stored here
+4406 let mut migration_observed_addr = None;
+4407 for result in frame::Iter::new(payload)? {
+4408 let frame = result?;
+4409 qlog.frame(&frame);
+4410 let span = match frame {
+4411 Frame::Padding => continue,
+4412 _ => trace_span!("frame", ty = %frame.ty(), path = tracing::field::Empty),
+4413 };
+4414
+4415 self.stats.frame_rx.record(frame.ty());
+4416 // Crypto, Stream and Datagram frames are special cased in order no pollute
+4417 // the log with payload data
+4418 match &frame {
+4419 Frame::Crypto(f) => {
+4420 trace!(offset = f.offset, len = f.data.len(), "got crypto frame");
+4421 }
+4422 Frame::Stream(f) => {
+4423 trace!(id = %f.id, offset = f.offset, len = f.data.len(), fin = f.fin, "got stream frame");
+4424 }
+4425 Frame::Datagram(f) => {
+4426 trace!(len = f.data.len(), "got datagram frame");
+4427 }
+4428 f => {
+4429 trace!("got frame {f}");
+4430 }
+4431 }
+4432
+4433 let _guard = span.enter();
+4434 if packet.header.is_0rtt() {
+4435 match frame {
+4436 Frame::Crypto(_) | Frame::Close(Close::Application(_)) => {
+4437 return Err(TransportError::PROTOCOL_VIOLATION(
+4438 "illegal frame type in 0-RTT",
+4439 ));
+4440 }
+4441 _ => {
+4442 if frame.is_1rtt() {
+4443 return Err(TransportError::PROTOCOL_VIOLATION(
+4444 "illegal frame type in 0-RTT",
+4445 ));
+4446 }
+4447 }
+4448 }
+4449 }
+4450 ack_eliciting |= frame.is_ack_eliciting();
+4451
+4452 // Check whether this could be a probing packet
+4453 match frame {
+4454 Frame::Padding
+4455 | Frame::PathChallenge(_)
+4456 | Frame::PathResponse(_)
+4457 | Frame::NewConnectionId(_)
+4458 | Frame::ObservedAddr(_) => {}
+4459 _ => {
+4460 is_probing_packet = false;
4461 }
-4462 Frame::StreamDataBlocked { id, offset } => {
-4463 if id.initiator() == self.side.side() && id.dir() == Dir::Uni {
-4464 debug!("got STREAM_DATA_BLOCKED on send-only {}", id);
-4465 return Err(TransportError::STREAM_STATE_ERROR(
-4466 "STREAM_DATA_BLOCKED on send-only stream",
-4467 ));
-4468 }
-4469 debug!(
-4470 stream = %id,
-4471 offset, "peer claims to be blocked at stream level"
-4472 );
-4473 }
-4474 Frame::StreamsBlocked { dir, limit } => {
-4475 if limit > MAX_STREAM_COUNT {
-4476 return Err(TransportError::FRAME_ENCODING_ERROR(
-4477 "unrepresentable stream limit",
-4478 ));
-4479 }
-4480 debug!(
-4481 "peer claims to be blocked opening more than {} {} streams",
-4482 limit, dir
-4483 );
-4484 }
-4485 Frame::StopSending(frame::StopSending { id, error_code }) => {
-4486 if id.initiator() != self.side.side() {
-4487 if id.dir() == Dir::Uni {
-4488 debug!("got STOP_SENDING on recv-only {}", id);
-4489 return Err(TransportError::STREAM_STATE_ERROR(
-4490 "STOP_SENDING on recv-only stream",
-4491 ));
-4492 }
-4493 } else if self.streams.is_local_unopened(id) {
-4494 return Err(TransportError::STREAM_STATE_ERROR(
-4495 "STOP_SENDING on unopened stream",
-4496 ));
-4497 }
-4498 self.streams.received_stop_sending(id, error_code);
-4499 }
-4500 Frame::RetireConnectionId(frame::RetireConnectionId { path_id, sequence }) => {
-4501 if let Some(ref path_id) = path_id {
-4502 span.record("path", tracing::field::debug(&path_id));
-4503 }
-4504 let path_id = path_id.unwrap_or_default();
-4505 match self.local_cid_state.get_mut(&path_id) {
-4506 None => error!(?path_id, "RETIRE_CONNECTION_ID for unknown path"),
-4507 Some(cid_state) => {
-4508 let allow_more_cids = cid_state
-4509 .on_cid_retirement(sequence, self.peer_params.issue_cids_limit())?;
-4510
-4511 // If the path has closed, we do not issue more CIDs for this path
-4512 // For details see https://www.ietf.org/archive/id/draft-ietf-quic-multipath-17.html#section-3.2.2
-4513 // > an endpoint SHOULD provide new connection IDs for that path, if still open, using PATH_NEW_CONNECTION_ID frames.
-4514 let has_path = !self.abandoned_paths.contains(&path_id);
-4515 let allow_more_cids = allow_more_cids && has_path;
-4516
-4517 self.endpoint_events
-4518 .push_back(EndpointEventInner::RetireConnectionId(
-4519 now,
-4520 path_id,
-4521 sequence,
-4522 allow_more_cids,
-4523 ));
-4524 }
-4525 }
-4526 }
-4527 Frame::NewConnectionId(frame) => {
-4528 let path_id = if let Some(path_id) = frame.path_id {
-4529 if !self.is_multipath_negotiated() {
-4530 return Err(TransportError::PROTOCOL_VIOLATION(
-4531 "received PATH_NEW_CONNECTION_ID frame when multipath was not negotiated",
-4532 ));
-4533 }
-4534 if path_id > self.local_max_path_id {
-4535 return Err(TransportError::PROTOCOL_VIOLATION(
-4536 "PATH_NEW_CONNECTION_ID contains path_id exceeding current max",
-4537 ));
-4538 }
-4539 path_id
-4540 } else {
-4541 PathId::ZERO
-4542 };
-4543
-4544 if self.abandoned_paths.contains(&path_id) {
-4545 trace!("ignoring issued CID for abandoned path");
-4546 continue;
-4547 }
-4548 if let Some(ref path_id) = frame.path_id {
-4549 span.record("path", tracing::field::debug(&path_id));
-4550 }
-4551 let rem_cids = self
-4552 .rem_cids
-4553 .entry(path_id)
-4554 .or_insert_with(|| CidQueue::new(frame.id));
-4555 if rem_cids.active().is_empty() {
-4556 return Err(TransportError::PROTOCOL_VIOLATION(
-4557 "NEW_CONNECTION_ID when CIDs aren't in use",
-4558 ));
-4559 }
-4560 if frame.retire_prior_to > frame.sequence {
-4561 return Err(TransportError::PROTOCOL_VIOLATION(
-4562 "NEW_CONNECTION_ID retiring unissued CIDs",
-4563 ));
-4564 }
-4565
-4566 use crate::cid_queue::InsertError;
-4567 match rem_cids.insert(frame) {
-4568 Ok(None) if self.path(path_id).is_none() => {
-4569 // if this gives us CIDs to open a new path and a nat traversal attempt
-4570 // is underway we could try to probe a pending remote
-4571 self.continue_nat_traversal_round(now);
-4572 }
-4573 Ok(None) => {}
-4574 Ok(Some((retired, reset_token))) => {
-4575 let pending_retired =
-4576 &mut self.spaces[SpaceId::Data].pending.retire_cids;
-4577 /// Ensure `pending_retired` cannot grow without bound. Limit is
-4578 /// somewhat arbitrary but very permissive.
-4579 const MAX_PENDING_RETIRED_CIDS: u64 = CidQueue::LEN as u64 * 10;
-4580 // We don't bother counting in-flight frames because those are bounded
-4581 // by congestion control.
-4582 if (pending_retired.len() as u64)
-4583 .saturating_add(retired.end.saturating_sub(retired.start))
-4584 > MAX_PENDING_RETIRED_CIDS
-4585 {
-4586 return Err(TransportError::CONNECTION_ID_LIMIT_ERROR(
-4587 "queued too many retired CIDs",
-4588 ));
-4589 }
-4590 pending_retired.extend(retired.map(|seq| (path_id, seq)));
-4591 // TODO(matheus23): Reset token for a remote or a full 4-tuple?
-4592 self.set_reset_token(path_id, network_path.remote, reset_token);
-4593 }
-4594 Err(InsertError::ExceedsLimit) => {
-4595 return Err(TransportError::CONNECTION_ID_LIMIT_ERROR(""));
-4596 }
-4597 Err(InsertError::Retired) => {
-4598 trace!("discarding already-retired");
-4599 // RETIRE_CONNECTION_ID might not have been previously sent if e.g. a
-4600 // range of connection IDs larger than the active connection ID limit
-4601 // was retired all at once via retire_prior_to.
-4602 self.spaces[SpaceId::Data]
-4603 .pending
-4604 .retire_cids
-4605 .push((path_id, frame.sequence));
-4606 continue;
-4607 }
-4608 };
-4609
-4610 if self.side.is_server()
-4611 && path_id == PathId::ZERO
-4612 && self
-4613 .rem_cids
-4614 .get(&PathId::ZERO)
-4615 .map(|cids| cids.active_seq() == 0)
-4616 .unwrap_or_default()
-4617 {
-4618 // We're a server still using the initial remote CID for the client, so
-4619 // let's switch immediately to enable clientside stateless resets.
-4620 self.update_rem_cid(PathId::ZERO);
-4621 }
-4622 }
-4623 Frame::NewToken(NewToken { token }) => {
-4624 let ConnectionSide::Client {
-4625 token_store,
-4626 server_name,
-4627 ..
-4628 } = &self.side
-4629 else {
-4630 return Err(TransportError::PROTOCOL_VIOLATION("client sent NEW_TOKEN"));
-4631 };
-4632 if token.is_empty() {
-4633 return Err(TransportError::FRAME_ENCODING_ERROR("empty token"));
-4634 }
-4635 trace!("got new token");
-4636 token_store.insert(server_name, token);
-4637 }
-4638 Frame::Datagram(datagram) => {
-4639 if self
-4640 .datagrams
-4641 .received(datagram, &self.config.datagram_receive_buffer_size)?
-4642 {
-4643 self.events.push_back(Event::DatagramReceived);
-4644 }
-4645 }
-4646 Frame::AckFrequency(ack_frequency) => {
-4647 // This frame can only be sent in the Data space
-4648
-4649 if !self.ack_frequency.ack_frequency_received(&ack_frequency)? {
-4650 // The AckFrequency frame is stale (we have already received a more
-4651 // recent one)
-4652 continue;
+4462 }
+4463
+4464 match frame {
+4465 Frame::Crypto(frame) => {
+4466 self.read_crypto(SpaceId::Data, &frame, payload_len)?;
+4467 }
+4468 Frame::Stream(frame) => {
+4469 if self.streams.received(frame, payload_len)?.should_transmit() {
+4470 self.spaces[SpaceId::Data].pending.max_data = true;
+4471 }
+4472 }
+4473 Frame::Ack(ack) => {
+4474 self.on_ack_received(now, SpaceId::Data, ack)?;
+4475 }
+4476 Frame::PathAck(ack) => {
+4477 span.record("path", tracing::field::debug(&ack.path_id));
+4478 self.on_path_ack_received(now, SpaceId::Data, ack)?;
+4479 }
+4480 Frame::Padding | Frame::Ping => {}
+4481 Frame::Close(reason) => {
+4482 close = Some(reason);
+4483 }
+4484 Frame::PathChallenge(challenge) => {
+4485 let path = &mut self
+4486 .path_mut(path_id)
+4487 .expect("payload is processed only after the path becomes known");
+4488 path.path_responses.push(number, challenge.0, network_path);
+4489 // At this point, update_network_path_or_discard was already called, so
+4490 // we don't need to be lenient about `local_ip` possibly mis-matching.
+4491 if network_path == path.network_path {
+4492 // PATH_CHALLENGE on active path, possible off-path packet forwarding
+4493 // attack. Send a non-probing packet to recover the active path.
+4494 // TODO(flub): No longer true! We now path_challege also to validate
+4495 // the path if the path is new, without an RFC9000-style
+4496 // migration involved. This means we add in an extra
+4497 // IMMEDIATE_ACK on some challenges. It isn't really wrong to do
+4498 // so, but it still is something untidy. We should instead
+4499 // suppress this when we know the remote is still validating the
+4500 // path.
+4501 match self.peer_supports_ack_frequency() {
+4502 true => self.immediate_ack(path_id),
+4503 false => {
+4504 self.ping_path(path_id).ok();
+4505 }
+4506 }
+4507 }
+4508 }
+4509 Frame::PathResponse(response) => {
+4510 let path = self
+4511 .paths
+4512 .get_mut(&path_id)
+4513 .expect("payload is processed only after the path becomes known");
+4514
+4515 use PathTimer::*;
+4516 use paths::OnPathResponseReceived::*;
+4517 match path
+4518 .data
+4519 .on_path_response_received(now, response.0, network_path)
+4520 {
+4521 OnPath { was_open } => {
+4522 let qlog = self.qlog.with_time(now);
+4523
+4524 self.timers
+4525 .stop(Timer::PerPath(path_id, PathValidation), qlog.clone());
+4526 self.timers
+4527 .stop(Timer::PerPath(path_id, PathOpen), qlog.clone());
+4528
+4529 let next_challenge = path
+4530 .data
+4531 .earliest_expiring_challenge()
+4532 .map(|time| time + self.ack_frequency.max_ack_delay_for_pto());
+4533 self.timers.set_or_stop(
+4534 Timer::PerPath(path_id, PathChallengeLost),
+4535 next_challenge,
+4536 qlog,
+4537 );
+4538
+4539 if !was_open {
+4540 self.events
+4541 .push_back(Event::Path(PathEvent::Opened { id: path_id }));
+4542 if let Some(observed) = path.data.last_observed_addr_report.as_ref()
+4543 {
+4544 self.events.push_back(Event::Path(PathEvent::ObservedAddr {
+4545 id: path_id,
+4546 addr: observed.socket_addr(),
+4547 }));
+4548 }
+4549 }
+4550 if let Some((_, ref mut prev)) = path.prev {
+4551 prev.challenges_sent.clear();
+4552 prev.send_new_challenge = false;
+4553 }
+4554 }
+4555 OffPath => {
+4556 debug!("Response to off-path PathChallenge!");
+4557 let next_challenge = path
+4558 .data
+4559 .earliest_expiring_challenge()
+4560 .map(|time| time + self.ack_frequency.max_ack_delay_for_pto());
+4561 self.timers.set_or_stop(
+4562 Timer::PerPath(path_id, PathChallengeLost),
+4563 next_challenge,
+4564 self.qlog.with_time(now),
+4565 );
+4566 }
+4567 Invalid { expected } => {
+4568 debug!(%response, %network_path, %expected, "ignoring invalid PATH_RESPONSE")
+4569 }
+4570 Unknown => debug!(%response, "ignoring invalid PATH_RESPONSE"),
+4571 }
+4572 }
+4573 Frame::MaxData(frame::MaxData(bytes)) => {
+4574 self.streams.received_max_data(bytes);
+4575 }
+4576 Frame::MaxStreamData(frame::MaxStreamData { id, offset }) => {
+4577 self.streams.received_max_stream_data(id, offset)?;
+4578 }
+4579 Frame::MaxStreams(frame::MaxStreams { dir, count }) => {
+4580 self.streams.received_max_streams(dir, count)?;
+4581 }
+4582 Frame::ResetStream(frame) => {
+4583 if self.streams.received_reset(frame)?.should_transmit() {
+4584 self.spaces[SpaceId::Data].pending.max_data = true;
+4585 }
+4586 }
+4587 Frame::DataBlocked { offset } => {
+4588 debug!(offset, "peer claims to be blocked at connection level");
+4589 }
+4590 Frame::StreamDataBlocked { id, offset } => {
+4591 if id.initiator() == self.side.side() && id.dir() == Dir::Uni {
+4592 debug!("got STREAM_DATA_BLOCKED on send-only {}", id);
+4593 return Err(TransportError::STREAM_STATE_ERROR(
+4594 "STREAM_DATA_BLOCKED on send-only stream",
+4595 ));
+4596 }
+4597 debug!(
+4598 stream = %id,
+4599 offset, "peer claims to be blocked at stream level"
+4600 );
+4601 }
+4602 Frame::StreamsBlocked { dir, limit } => {
+4603 if limit > MAX_STREAM_COUNT {
+4604 return Err(TransportError::FRAME_ENCODING_ERROR(
+4605 "unrepresentable stream limit",
+4606 ));
+4607 }
+4608 debug!(
+4609 "peer claims to be blocked opening more than {} {} streams",
+4610 limit, dir
+4611 );
+4612 }
+4613 Frame::StopSending(frame::StopSending { id, error_code }) => {
+4614 if id.initiator() != self.side.side() {
+4615 if id.dir() == Dir::Uni {
+4616 debug!("got STOP_SENDING on recv-only {}", id);
+4617 return Err(TransportError::STREAM_STATE_ERROR(
+4618 "STOP_SENDING on recv-only stream",
+4619 ));
+4620 }
+4621 } else if self.streams.is_local_unopened(id) {
+4622 return Err(TransportError::STREAM_STATE_ERROR(
+4623 "STOP_SENDING on unopened stream",
+4624 ));
+4625 }
+4626 self.streams.received_stop_sending(id, error_code);
+4627 }
+4628 Frame::RetireConnectionId(frame::RetireConnectionId { path_id, sequence }) => {
+4629 if let Some(ref path_id) = path_id {
+4630 span.record("path", tracing::field::debug(&path_id));
+4631 }
+4632 let path_id = path_id.unwrap_or_default();
+4633 match self.local_cid_state.get_mut(&path_id) {
+4634 None => error!(?path_id, "RETIRE_CONNECTION_ID for unknown path"),
+4635 Some(cid_state) => {
+4636 let allow_more_cids = cid_state
+4637 .on_cid_retirement(sequence, self.peer_params.issue_cids_limit())?;
+4638
+4639 // If the path has closed, we do not issue more CIDs for this path
+4640 // For details see https://www.ietf.org/archive/id/draft-ietf-quic-multipath-17.html#section-3.2.2
+4641 // > an endpoint SHOULD provide new connection IDs for that path, if still open, using PATH_NEW_CONNECTION_ID frames.
+4642 let has_path = !self.abandoned_paths.contains(&path_id);
+4643 let allow_more_cids = allow_more_cids && has_path;
+4644
+4645 self.endpoint_events
+4646 .push_back(EndpointEventInner::RetireConnectionId(
+4647 now,
+4648 path_id,
+4649 sequence,
+4650 allow_more_cids,
+4651 ));
+4652 }
4653 }
-4654
-4655 // Update the params for all of our paths
-4656 for (path_id, space) in self.spaces[SpaceId::Data].number_spaces.iter_mut() {
-4657 space.pending_acks.set_ack_frequency_params(&ack_frequency);
-4658
-4659 // Our `max_ack_delay` has been updated, so we may need to adjust
-4660 // its associated timeout
-4661 if let Some(timeout) = space
-4662 .pending_acks
-4663 .max_ack_delay_timeout(self.ack_frequency.max_ack_delay)
-4664 {
-4665 self.timers.set(
-4666 Timer::PerPath(*path_id, PathTimer::MaxAckDelay),
-4667 timeout,
-4668 self.qlog.with_time(now),
-4669 );
-4670 }
-4671 }
-4672 }
-4673 Frame::ImmediateAck => {
-4674 // This frame can only be sent in the Data space
-4675 for pns in self.spaces[SpaceId::Data].iter_paths_mut() {
-4676 pns.pending_acks.set_immediate_ack_required();
-4677 }
-4678 }
-4679 Frame::HandshakeDone => {
-4680 if self.side.is_server() {
-4681 return Err(TransportError::PROTOCOL_VIOLATION(
-4682 "client sent HANDSHAKE_DONE",
-4683 ));
-4684 }
-4685 if self.spaces[SpaceId::Handshake].crypto.is_some() {
-4686 self.discard_space(now, SpaceId::Handshake);
+4654 }
+4655 Frame::NewConnectionId(frame) => {
+4656 let path_id = if let Some(path_id) = frame.path_id {
+4657 if !self.is_multipath_negotiated() {
+4658 return Err(TransportError::PROTOCOL_VIOLATION(
+4659 "received PATH_NEW_CONNECTION_ID frame when multipath was not negotiated",
+4660 ));
+4661 }
+4662 if path_id > self.local_max_path_id {
+4663 return Err(TransportError::PROTOCOL_VIOLATION(
+4664 "PATH_NEW_CONNECTION_ID contains path_id exceeding current max",
+4665 ));
+4666 }
+4667 path_id
+4668 } else {
+4669 PathId::ZERO
+4670 };
+4671
+4672 if self.abandoned_paths.contains(&path_id) {
+4673 trace!("ignoring issued CID for abandoned path");
+4674 continue;
+4675 }
+4676 if let Some(ref path_id) = frame.path_id {
+4677 span.record("path", tracing::field::debug(&path_id));
+4678 }
+4679 let rem_cids = self
+4680 .rem_cids
+4681 .entry(path_id)
+4682 .or_insert_with(|| CidQueue::new(frame.id));
+4683 if rem_cids.active().is_empty() {
+4684 return Err(TransportError::PROTOCOL_VIOLATION(
+4685 "NEW_CONNECTION_ID when CIDs aren't in use",
+4686 ));
4687 }
-4688 self.events.push_back(Event::HandshakeConfirmed);
-4689 trace!("handshake confirmed");
-4690 }
-4691 Frame::ObservedAddr(observed) => {
-4692 // check if params allows the peer to send report and this node to receive it
-4693 trace!(seq_no = %observed.seq_no, ip = %observed.ip, port = observed.port);
-4694 if !self
-4695 .peer_params
-4696 .address_discovery_role
-4697 .should_report(&self.config.address_discovery_role)
-4698 {
-4699 return Err(TransportError::PROTOCOL_VIOLATION(
-4700 "received OBSERVED_ADDRESS frame when not negotiated",
-4701 ));
-4702 }
-4703 // must only be sent in data space
-4704 if packet.header.space() != SpaceId::Data {
-4705 return Err(TransportError::PROTOCOL_VIOLATION(
-4706 "OBSERVED_ADDRESS frame outside data space",
-4707 ));
-4708 }
-4709
-4710 let path = self.path_data_mut(path_id);
-4711 if network_path == path.network_path {
-4712 if let Some(updated) = path.update_observed_addr_report(observed) {
-4713 if path.open {
-4714 self.events.push_back(Event::Path(PathEvent::ObservedAddr {
-4715 id: path_id,
-4716 addr: updated,
-4717 }));
-4718 }
-4719 // otherwise the event is reported when the path is deemed open
-4720 }
-4721 } else {
-4722 // include in migration
-4723 migration_observed_addr = Some(observed)
-4724 }
-4725 }
-4726 Frame::PathAbandon(frame::PathAbandon {
-4727 path_id,
-4728 error_code,
-4729 }) => {
-4730 span.record("path", tracing::field::debug(&path_id));
-4731 // TODO(flub): don't really know which error code to use here.
-4732 match self.close_path(now, path_id, error_code.into()) {
-4733 Ok(()) => {
-4734 trace!("peer abandoned path");
+4688 if frame.retire_prior_to > frame.sequence {
+4689 return Err(TransportError::PROTOCOL_VIOLATION(
+4690 "NEW_CONNECTION_ID retiring unissued CIDs",
+4691 ));
+4692 }
+4693
+4694 use crate::cid_queue::InsertError;
+4695 match rem_cids.insert(frame) {
+4696 Ok(None) if self.path(path_id).is_none() => {
+4697 // if this gives us CIDs to open a new path and a nat traversal attempt
+4698 // is underway we could try to probe a pending remote
+4699 self.continue_nat_traversal_round(now);
+4700 }
+4701 Ok(None) => {}
+4702 Ok(Some((retired, reset_token))) => {
+4703 let pending_retired =
+4704 &mut self.spaces[SpaceId::Data].pending.retire_cids;
+4705 /// Ensure `pending_retired` cannot grow without bound. Limit is
+4706 /// somewhat arbitrary but very permissive.
+4707 const MAX_PENDING_RETIRED_CIDS: u64 = CidQueue::LEN as u64 * 10;
+4708 // We don't bother counting in-flight frames because those are bounded
+4709 // by congestion control.
+4710 if (pending_retired.len() as u64)
+4711 .saturating_add(retired.end.saturating_sub(retired.start))
+4712 > MAX_PENDING_RETIRED_CIDS
+4713 {
+4714 return Err(TransportError::CONNECTION_ID_LIMIT_ERROR(
+4715 "queued too many retired CIDs",
+4716 ));
+4717 }
+4718 pending_retired.extend(retired.map(|seq| (path_id, seq)));
+4719 // TODO(matheus23): Reset token for a remote or a full 4-tuple?
+4720 self.set_reset_token(path_id, network_path.remote, reset_token);
+4721 }
+4722 Err(InsertError::ExceedsLimit) => {
+4723 return Err(TransportError::CONNECTION_ID_LIMIT_ERROR(""));
+4724 }
+4725 Err(InsertError::Retired) => {
+4726 trace!("discarding already-retired");
+4727 // RETIRE_CONNECTION_ID might not have been previously sent if e.g. a
+4728 // range of connection IDs larger than the active connection ID limit
+4729 // was retired all at once via retire_prior_to.
+4730 self.spaces[SpaceId::Data]
+4731 .pending
+4732 .retire_cids
+4733 .push((path_id, frame.sequence));
+4734 continue;
4735 }
-4736 Err(ClosePathError::LastOpenPath) => {
-4737 trace!("peer abandoned last path, closing connection");
-4738 return Err(TransportError::NO_VIABLE_PATH(
-4739 "last path abandoned by peer",
-4740 ));
-4741 }
-4742 Err(ClosePathError::ClosedPath) => {
-4743 trace!("peer abandoned already closed path");
-4744 }
-4745 };
-4746 // If we receive a retransmit of PATH_ABANDON then we may already have
-4747 // abandoned this path locally. In that case the DiscardPath timer
-4748 // may already have fired and we no longer have any state for this path.
-4749 // Only set this timer if we still have path state.
-4750 if let Some(path) = self.paths.get_mut(&path_id) {
-4751 if !matches!(path.data.abandon_state, AbandonState::ReceivedPathAbandon) {
-4752 let ack_delay = self.ack_frequency.max_ack_delay_for_pto();
-4753 let pto = path.data.rtt.pto_base() + ack_delay;
-4754 self.timers.set(
-4755 Timer::PerPath(path_id, PathTimer::DiscardPath),
-4756 now + 3 * pto,
-4757 self.qlog.with_time(now),
-4758 );
-4759 // We received a PATH_ABANDON, we don't expect another one by a certain time.
-4760 path.data.abandon_state = AbandonState::ReceivedPathAbandon;
-4761 }
+4736 };
+4737
+4738 if self.side.is_server()
+4739 && path_id == PathId::ZERO
+4740 && self
+4741 .rem_cids
+4742 .get(&PathId::ZERO)
+4743 .map(|cids| cids.active_seq() == 0)
+4744 .unwrap_or_default()
+4745 {
+4746 // We're a server still using the initial remote CID for the client, so
+4747 // let's switch immediately to enable clientside stateless resets.
+4748 self.update_rem_cid(PathId::ZERO);
+4749 }
+4750 }
+4751 Frame::NewToken(NewToken { token }) => {
+4752 let ConnectionSide::Client {
+4753 token_store,
+4754 server_name,
+4755 ..
+4756 } = &self.side
+4757 else {
+4758 return Err(TransportError::PROTOCOL_VIOLATION("client sent NEW_TOKEN"));
+4759 };
+4760 if token.is_empty() {
+4761 return Err(TransportError::FRAME_ENCODING_ERROR("empty token"));
4762 }
-4763 }
-4764 Frame::PathStatusAvailable(info) => {
-4765 span.record("path", tracing::field::debug(&info.path_id));
-4766 if self.is_multipath_negotiated() {
-4767 self.on_path_status(
-4768 info.path_id,
-4769 PathStatus::Available,
-4770 info.status_seq_no,
-4771 );
-4772 } else {
-4773 return Err(TransportError::PROTOCOL_VIOLATION(
-4774 "received PATH_STATUS_AVAILABLE frame when multipath was not negotiated",
-4775 ));
-4776 }
-4777 }
-4778 Frame::PathStatusBackup(info) => {
-4779 span.record("path", tracing::field::debug(&info.path_id));
-4780 if self.is_multipath_negotiated() {
-4781 self.on_path_status(info.path_id, PathStatus::Backup, info.status_seq_no);
-4782 } else {
-4783 return Err(TransportError::PROTOCOL_VIOLATION(
-4784 "received PATH_STATUS_BACKUP frame when multipath was not negotiated",
-4785 ));
-4786 }
-4787 }
-4788 Frame::MaxPathId(frame::MaxPathId(path_id)) => {
-4789 span.record("path", tracing::field::debug(&path_id));
-4790 if !self.is_multipath_negotiated() {
-4791 return Err(TransportError::PROTOCOL_VIOLATION(
-4792 "received MAX_PATH_ID frame when multipath was not negotiated",
-4793 ));
-4794 }
-4795 // frames that do not increase the path id are ignored
-4796 if path_id > self.remote_max_path_id {
-4797 self.remote_max_path_id = path_id;
-4798 self.issue_first_path_cids(now);
-4799 while let Some(true) = self.continue_nat_traversal_round(now) {}
-4800 }
-4801 }
-4802 Frame::PathsBlocked(frame::PathsBlocked(max_path_id)) => {
-4803 // Receipt of a value of Maximum Path Identifier or Path Identifier that is higher than the local maximum value MUST
-4804 // be treated as a connection error of type PROTOCOL_VIOLATION.
-4805 // Ref <https://www.ietf.org/archive/id/draft-ietf-quic-multipath-14.html#name-paths_blocked-and-path_cids>
-4806 if self.is_multipath_negotiated() {
-4807 if max_path_id > self.local_max_path_id {
-4808 return Err(TransportError::PROTOCOL_VIOLATION(
-4809 "PATHS_BLOCKED maximum path identifier was larger than local maximum",
-4810 ));
-4811 }
-4812 debug!("received PATHS_BLOCKED({:?})", max_path_id);
-4813 // TODO(@divma): ensure max concurrent paths
-4814 } else {
-4815 return Err(TransportError::PROTOCOL_VIOLATION(
-4816 "received PATHS_BLOCKED frame when not multipath was not negotiated",
-4817 ));
-4818 }
-4819 }
-4820 Frame::PathCidsBlocked(frame::PathCidsBlocked { path_id, next_seq }) => {
-4821 // Nothing to do. This is recorded in the frame stats, but otherwise we
-4822 // always issue all CIDs we're allowed to issue, so either this is an
-4823 // impatient peer or a bug on our side.
-4824
-4825 // Receipt of a value of Maximum Path Identifier or Path Identifier that is higher than the local maximum value MUST
-4826 // be treated as a connection error of type PROTOCOL_VIOLATION.
-4827 // Ref <https://www.ietf.org/archive/id/draft-ietf-quic-multipath-14.html#name-paths_blocked-and-path_cids>
-4828 if self.is_multipath_negotiated() {
-4829 if path_id > self.local_max_path_id {
-4830 return Err(TransportError::PROTOCOL_VIOLATION(
-4831 "PATH_CIDS_BLOCKED path identifier was larger than local maximum",
-4832 ));
-4833 }
-4834 if next_seq.0
-4835 > self
-4836 .local_cid_state
-4837 .get(&path_id)
-4838 .map(|cid_state| cid_state.active_seq().1 + 1)
-4839 .unwrap_or_default()
-4840 {
-4841 return Err(TransportError::PROTOCOL_VIOLATION(
-4842 "PATH_CIDS_BLOCKED next sequence number larger than in local state",
-4843 ));
-4844 }
-4845 debug!(%path_id, %next_seq, "received PATH_CIDS_BLOCKED");
-4846 } else {
-4847 return Err(TransportError::PROTOCOL_VIOLATION(
-4848 "received PATH_CIDS_BLOCKED frame when not multipath was not negotiated",
-4849 ));
-4850 }
-4851 }
-4852 Frame::AddAddress(addr) => {
-4853 let client_state = match self.iroh_hp.client_side_mut() {
-4854 Ok(state) => state,
-4855 Err(err) => {
-4856 return Err(TransportError::PROTOCOL_VIOLATION(format!(
-4857 "Nat traversal(ADD_ADDRESS): {err}"
-4858 )));
-4859 }
-4860 };
-4861
-4862 if !client_state.check_remote_address(&addr) {
-4863 // if the address is not valid we flag it, but update anyway
-4864 warn!(?addr, "server sent illegal ADD_ADDRESS frame");
-4865 }
-4866
-4867 match client_state.add_remote_address(addr) {
-4868 Ok(maybe_added) => {
-4869 if let Some(added) = maybe_added {
-4870 self.events.push_back(Event::NatTraversal(
-4871 iroh_hp::Event::AddressAdded(added),
-4872 ));
-4873 }
-4874 }
-4875 Err(e) => {
-4876 warn!(%e, "failed to add remote address")
-4877 }
-4878 }
-4879 }
-4880 Frame::RemoveAddress(addr) => {
-4881 let client_state = match self.iroh_hp.client_side_mut() {
-4882 Ok(state) => state,
-4883 Err(err) => {
-4884 return Err(TransportError::PROTOCOL_VIOLATION(format!(
-4885 "Nat traversal(REMOVE_ADDRESS): {err}"
-4886 )));
-4887 }
-4888 };
-4889 if let Some(removed_addr) = client_state.remove_remote_address(addr) {
-4890 self.events
-4891 .push_back(Event::NatTraversal(iroh_hp::Event::AddressRemoved(
-4892 removed_addr,
-4893 )));
-4894 }
-4895 }
-4896 Frame::ReachOut(reach_out) => {
-4897 let server_state = match self.iroh_hp.server_side_mut() {
-4898 Ok(state) => state,
-4899 Err(err) => {
-4900 return Err(TransportError::PROTOCOL_VIOLATION(format!(
-4901 "Nat traversal(REACH_OUT): {err}"
-4902 )));
-4903 }
-4904 };
-4905
-4906 if let Err(err) = server_state.handle_reach_out(reach_out) {
-4907 return Err(TransportError::PROTOCOL_VIOLATION(format!(
-4908 "Nat traversal(REACH_OUT): {err}"
-4909 )));
-4910 }
-4911 }
-4912 }
-4913 }
-4914
-4915 let space = self.spaces[SpaceId::Data].for_path(path_id);
-4916 if space
-4917 .pending_acks
-4918 .packet_received(now, number, ack_eliciting, &space.dedup)
-4919 {
-4920 if self.abandoned_paths.contains(&path_id) {
-4921 // § 3.4.3 QUIC-MULTIPATH: promptly send ACKs for packets received from
-4922 // abandoned paths.
-4923 space.pending_acks.set_immediate_ack_required();
-4924 } else {
-4925 self.timers.set(
-4926 Timer::PerPath(path_id, PathTimer::MaxAckDelay),
-4927 now + self.ack_frequency.max_ack_delay,
-4928 self.qlog.with_time(now),
-4929 );
-4930 }
-4931 }
-4932
-4933 // Issue stream ID credit due to ACKs of outgoing finish/resets and incoming finish/resets
-4934 // on stopped streams. Incoming finishes/resets on open streams are not handled here as they
-4935 // are only freed, and hence only issue credit, once the application has been notified
-4936 // during a read on the stream.
-4937 let pending = &mut self.spaces[SpaceId::Data].pending;
-4938 self.streams.queue_max_stream_id(pending);
-4939
-4940 if let Some(reason) = close {
-4941 self.state.move_to_draining(Some(reason.into()));
-4942 self.close = true;
-4943 }
-4944
-4945 if Some(number) == self.spaces[SpaceId::Data].for_path(path_id).rx_packet
-4946 && !is_probing_packet
-4947 && network_path != self.path_data(path_id).network_path
-4948 {
-4949 let ConnectionSide::Server { ref server_config } = self.side else {
-4950 panic!("packets from unknown remote should be dropped by clients");
-4951 };
-4952 debug_assert!(
-4953 server_config.migration,
-4954 "migration-initiating packets should have been dropped immediately"
-4955 );
-4956 self.migrate(path_id, now, network_path, migration_observed_addr);
-4957 // Break linkability, if possible
-4958 self.update_rem_cid(path_id);
-4959 self.spin = false;
-4960 }
-4961
-4962 Ok(())
-4963 }
-4964
-4965 fn migrate(
-4966 &mut self,
-4967 path_id: PathId,
-4968 now: Instant,
-4969 network_path: FourTuple,
-4970 observed_addr: Option<ObservedAddr>,
-4971 ) {
-4972 trace!(%network_path, %path_id, "migration initiated");
-4973 self.path_generation_counter = self.path_generation_counter.wrapping_add(1);
-4974 // TODO(@divma): conditions for path migration in multipath are very specific, check them
-4975 // again to prevent path migrations that should actually create a new path
-4976
-4977 // Reset rtt/congestion state for new path unless it looks like a NAT rebinding.
-4978 // Note that the congestion window will not grow until validation terminates. Helps mitigate
-4979 // amplification attacks performed by spoofing source addresses.
-4980 let prev_pto = self.pto(SpaceId::Data, path_id);
-4981 let known_path = self.paths.get_mut(&path_id).expect("known path");
-4982 let path = &mut known_path.data;
-4983 let mut new_path = if network_path.remote.is_ipv4()
-4984 && network_path.remote.ip() == path.network_path.remote.ip()
-4985 {
-4986 PathData::from_previous(network_path, path, self.path_generation_counter, now)
-4987 } else {
-4988 let peer_max_udp_payload_size =
-4989 u16::try_from(self.peer_params.max_udp_payload_size.into_inner())
-4990 .unwrap_or(u16::MAX);
-4991 PathData::new(
-4992 network_path,
-4993 self.allow_mtud,
-4994 Some(peer_max_udp_payload_size),
-4995 self.path_generation_counter,
-4996 now,
-4997 &self.config,
-4998 )
-4999 };
-5000 new_path.last_observed_addr_report = path.last_observed_addr_report.clone();
-5001 if let Some(report) = observed_addr {
-5002 if let Some(updated) = new_path.update_observed_addr_report(report) {
-5003 tracing::info!("adding observed addr event from migration");
-5004 self.events.push_back(Event::Path(PathEvent::ObservedAddr {
-5005 id: path_id,
-5006 addr: updated,
-5007 }));
-5008 }
-5009 }
-5010 new_path.send_new_challenge = true;
-5011
-5012 let mut prev = mem::replace(path, new_path);
-5013 // Don't clobber the original path if the previous one hasn't been validated yet
-5014 if !prev.is_validating_path() {
-5015 prev.send_new_challenge = true;
-5016 // We haven't updated the remote CID yet, this captures the remote CID we were using on
-5017 // the previous path.
-5018
-5019 known_path.prev = Some((self.rem_cids.get(&path_id).unwrap().active(), prev));
-5020 }
-5021
-5022 // We need to re-assign the correct remote to this path in qlog
-5023 self.qlog.emit_tuple_assigned(path_id, network_path, now);
-5024
-5025 self.timers.set(
-5026 Timer::PerPath(path_id, PathTimer::PathValidation),
-5027 now + 3 * cmp::max(self.pto(SpaceId::Data, path_id), prev_pto),
-5028 self.qlog.with_time(now),
-5029 );
-5030 }
-5031
-5032 /// Handle a change in the local address, i.e. an active migration
-5033 pub fn local_address_changed(&mut self) {
-5034 // TODO(flub): if multipath is enabled this needs to create a new path entirely.
-5035 self.update_rem_cid(PathId::ZERO);
-5036 self.ping();
-5037 }
-5038
-5039 /// Switch to a previously unused remote connection ID, if possible
-5040 fn update_rem_cid(&mut self, path_id: PathId) {
-5041 let Some((reset_token, retired)) =
-5042 self.rem_cids.get_mut(&path_id).and_then(|cids| cids.next())
-5043 else {
-5044 return;
-5045 };
-5046
-5047 // Retire the current remote CID and any CIDs we had to skip.
-5048 self.spaces[SpaceId::Data]
-5049 .pending
-5050 .retire_cids
-5051 .extend(retired.map(|seq| (path_id, seq)));
-5052 let remote = self.path_data(path_id).network_path.remote;
-5053 self.set_reset_token(path_id, remote, reset_token);
-5054 }
-5055
-5056 /// Sends this reset token to the endpoint
-5057 ///
-5058 /// The endpoint needs to know the reset tokens issued by the peer, so that if the peer
-5059 /// sends a reset token it knows to route it to this connection. See RFC 9000 section
-5060 /// 10.3. Stateless Reset.
-5061 ///
-5062 /// Reset tokens are different for each path, the endpoint identifies paths by peer
-5063 /// socket address however, not by path ID.
-5064 fn set_reset_token(&mut self, path_id: PathId, remote: SocketAddr, reset_token: ResetToken) {
-5065 self.endpoint_events
-5066 .push_back(EndpointEventInner::ResetToken(path_id, remote, reset_token));
+4763 trace!("got new token");
+4764 token_store.insert(server_name, token);
+4765 }
+4766 Frame::Datagram(datagram) => {
+4767 if self
+4768 .datagrams
+4769 .received(datagram, &self.config.datagram_receive_buffer_size)?
+4770 {
+4771 self.events.push_back(Event::DatagramReceived);
+4772 }
+4773 }
+4774 Frame::AckFrequency(ack_frequency) => {
+4775 // This frame can only be sent in the Data space
+4776
+4777 if !self.ack_frequency.ack_frequency_received(&ack_frequency)? {
+4778 // The AckFrequency frame is stale (we have already received a more
+4779 // recent one)
+4780 continue;
+4781 }
+4782
+4783 // Update the params for all of our paths
+4784 for (path_id, space) in self.spaces[SpaceId::Data].number_spaces.iter_mut() {
+4785 space.pending_acks.set_ack_frequency_params(&ack_frequency);
+4786
+4787 // Our `max_ack_delay` has been updated, so we may need to adjust
+4788 // its associated timeout
+4789 if let Some(timeout) = space
+4790 .pending_acks
+4791 .max_ack_delay_timeout(self.ack_frequency.max_ack_delay)
+4792 {
+4793 self.timers.set(
+4794 Timer::PerPath(*path_id, PathTimer::MaxAckDelay),
+4795 timeout,
+4796 self.qlog.with_time(now),
+4797 );
+4798 }
+4799 }
+4800 }
+4801 Frame::ImmediateAck => {
+4802 // This frame can only be sent in the Data space
+4803 for pns in self.spaces[SpaceId::Data].iter_paths_mut() {
+4804 pns.pending_acks.set_immediate_ack_required();
+4805 }
+4806 }
+4807 Frame::HandshakeDone => {
+4808 if self.side.is_server() {
+4809 return Err(TransportError::PROTOCOL_VIOLATION(
+4810 "client sent HANDSHAKE_DONE",
+4811 ));
+4812 }
+4813 if self.spaces[SpaceId::Handshake].crypto.is_some() {
+4814 self.discard_space(now, SpaceId::Handshake);
+4815 }
+4816 self.events.push_back(Event::HandshakeConfirmed);
+4817 trace!("handshake confirmed");
+4818 }
+4819 Frame::ObservedAddr(observed) => {
+4820 // check if params allows the peer to send report and this node to receive it
+4821 trace!(seq_no = %observed.seq_no, ip = %observed.ip, port = observed.port);
+4822 if !self
+4823 .peer_params
+4824 .address_discovery_role
+4825 .should_report(&self.config.address_discovery_role)
+4826 {
+4827 return Err(TransportError::PROTOCOL_VIOLATION(
+4828 "received OBSERVED_ADDRESS frame when not negotiated",
+4829 ));
+4830 }
+4831 // must only be sent in data space
+4832 if packet.header.space() != SpaceId::Data {
+4833 return Err(TransportError::PROTOCOL_VIOLATION(
+4834 "OBSERVED_ADDRESS frame outside data space",
+4835 ));
+4836 }
+4837
+4838 let path = self.path_data_mut(path_id);
+4839 if network_path == path.network_path {
+4840 if let Some(updated) = path.update_observed_addr_report(observed) {
+4841 if path.open {
+4842 self.events.push_back(Event::Path(PathEvent::ObservedAddr {
+4843 id: path_id,
+4844 addr: updated,
+4845 }));
+4846 }
+4847 // otherwise the event is reported when the path is deemed open
+4848 }
+4849 } else {
+4850 // include in migration
+4851 migration_observed_addr = Some(observed)
+4852 }
+4853 }
+4854 Frame::PathAbandon(frame::PathAbandon {
+4855 path_id,
+4856 error_code,
+4857 }) => {
+4858 span.record("path", tracing::field::debug(&path_id));
+4859 // TODO(flub): don't really know which error code to use here.
+4860 match self.close_path(now, path_id, error_code.into()) {
+4861 Ok(()) => {
+4862 trace!("peer abandoned path");
+4863 }
+4864 Err(ClosePathError::LastOpenPath) => {
+4865 trace!("peer abandoned last path, closing connection");
+4866 return Err(TransportError::NO_VIABLE_PATH(
+4867 "last path abandoned by peer",
+4868 ));
+4869 }
+4870 Err(ClosePathError::ClosedPath) => {
+4871 trace!("peer abandoned already closed path");
+4872 }
+4873 };
+4874 // If we receive a retransmit of PATH_ABANDON then we may already have
+4875 // abandoned this path locally. In that case the DiscardPath timer
+4876 // may already have fired and we no longer have any state for this path.
+4877 // Only set this timer if we still have path state.
+4878 if let Some(path) = self.paths.get_mut(&path_id) {
+4879 if !matches!(path.data.abandon_state, AbandonState::ReceivedPathAbandon) {
+4880 let ack_delay = self.ack_frequency.max_ack_delay_for_pto();
+4881 let pto = path.data.rtt.pto_base() + ack_delay;
+4882 self.timers.set(
+4883 Timer::PerPath(path_id, PathTimer::DiscardPath),
+4884 now + 3 * pto,
+4885 self.qlog.with_time(now),
+4886 );
+4887 // We received a PATH_ABANDON, we don't expect another one by a certain time.
+4888 path.data.abandon_state = AbandonState::ReceivedPathAbandon;
+4889 }
+4890 }
+4891 }
+4892 Frame::PathStatusAvailable(info) => {
+4893 span.record("path", tracing::field::debug(&info.path_id));
+4894 if self.is_multipath_negotiated() {
+4895 self.on_path_status(
+4896 info.path_id,
+4897 PathStatus::Available,
+4898 info.status_seq_no,
+4899 );
+4900 } else {
+4901 return Err(TransportError::PROTOCOL_VIOLATION(
+4902 "received PATH_STATUS_AVAILABLE frame when multipath was not negotiated",
+4903 ));
+4904 }
+4905 }
+4906 Frame::PathStatusBackup(info) => {
+4907 span.record("path", tracing::field::debug(&info.path_id));
+4908 if self.is_multipath_negotiated() {
+4909 self.on_path_status(info.path_id, PathStatus::Backup, info.status_seq_no);
+4910 } else {
+4911 return Err(TransportError::PROTOCOL_VIOLATION(
+4912 "received PATH_STATUS_BACKUP frame when multipath was not negotiated",
+4913 ));
+4914 }
+4915 }
+4916 Frame::MaxPathId(frame::MaxPathId(path_id)) => {
+4917 span.record("path", tracing::field::debug(&path_id));
+4918 if !self.is_multipath_negotiated() {
+4919 return Err(TransportError::PROTOCOL_VIOLATION(
+4920 "received MAX_PATH_ID frame when multipath was not negotiated",
+4921 ));
+4922 }
+4923 // frames that do not increase the path id are ignored
+4924 if path_id > self.remote_max_path_id {
+4925 self.remote_max_path_id = path_id;
+4926 self.issue_first_path_cids(now);
+4927 while let Some(true) = self.continue_nat_traversal_round(now) {}
+4928 }
+4929 }
+4930 Frame::PathsBlocked(frame::PathsBlocked(max_path_id)) => {
+4931 // Receipt of a value of Maximum Path Identifier or Path Identifier that is higher than the local maximum value MUST
+4932 // be treated as a connection error of type PROTOCOL_VIOLATION.
+4933 // Ref <https://www.ietf.org/archive/id/draft-ietf-quic-multipath-14.html#name-paths_blocked-and-path_cids>
+4934 if self.is_multipath_negotiated() {
+4935 if max_path_id > self.local_max_path_id {
+4936 return Err(TransportError::PROTOCOL_VIOLATION(
+4937 "PATHS_BLOCKED maximum path identifier was larger than local maximum",
+4938 ));
+4939 }
+4940 debug!("received PATHS_BLOCKED({:?})", max_path_id);
+4941 // TODO(@divma): ensure max concurrent paths
+4942 } else {
+4943 return Err(TransportError::PROTOCOL_VIOLATION(
+4944 "received PATHS_BLOCKED frame when not multipath was not negotiated",
+4945 ));
+4946 }
+4947 }
+4948 Frame::PathCidsBlocked(frame::PathCidsBlocked { path_id, next_seq }) => {
+4949 // Nothing to do. This is recorded in the frame stats, but otherwise we
+4950 // always issue all CIDs we're allowed to issue, so either this is an
+4951 // impatient peer or a bug on our side.
+4952
+4953 // Receipt of a value of Maximum Path Identifier or Path Identifier that is higher than the local maximum value MUST
+4954 // be treated as a connection error of type PROTOCOL_VIOLATION.
+4955 // Ref <https://www.ietf.org/archive/id/draft-ietf-quic-multipath-14.html#name-paths_blocked-and-path_cids>
+4956 if self.is_multipath_negotiated() {
+4957 if path_id > self.local_max_path_id {
+4958 return Err(TransportError::PROTOCOL_VIOLATION(
+4959 "PATH_CIDS_BLOCKED path identifier was larger than local maximum",
+4960 ));
+4961 }
+4962 if next_seq.0
+4963 > self
+4964 .local_cid_state
+4965 .get(&path_id)
+4966 .map(|cid_state| cid_state.active_seq().1 + 1)
+4967 .unwrap_or_default()
+4968 {
+4969 return Err(TransportError::PROTOCOL_VIOLATION(
+4970 "PATH_CIDS_BLOCKED next sequence number larger than in local state",
+4971 ));
+4972 }
+4973 debug!(%path_id, %next_seq, "received PATH_CIDS_BLOCKED");
+4974 } else {
+4975 return Err(TransportError::PROTOCOL_VIOLATION(
+4976 "received PATH_CIDS_BLOCKED frame when not multipath was not negotiated",
+4977 ));
+4978 }
+4979 }
+4980 Frame::AddAddress(addr) => {
+4981 let client_state = match self.iroh_hp.client_side_mut() {
+4982 Ok(state) => state,
+4983 Err(err) => {
+4984 return Err(TransportError::PROTOCOL_VIOLATION(format!(
+4985 "Nat traversal(ADD_ADDRESS): {err}"
+4986 )));
+4987 }
+4988 };
+4989
+4990 if !client_state.check_remote_address(&addr) {
+4991 // if the address is not valid we flag it, but update anyway
+4992 warn!(?addr, "server sent illegal ADD_ADDRESS frame");
+4993 }
+4994
+4995 match client_state.add_remote_address(addr) {
+4996 Ok(maybe_added) => {
+4997 if let Some(added) = maybe_added {
+4998 self.events.push_back(Event::NatTraversal(
+4999 iroh_hp::Event::AddressAdded(added),
+5000 ));
+5001 }
+5002 }
+5003 Err(e) => {
+5004 warn!(%e, "failed to add remote address")
+5005 }
+5006 }
+5007 }
+5008 Frame::RemoveAddress(addr) => {
+5009 let client_state = match self.iroh_hp.client_side_mut() {
+5010 Ok(state) => state,
+5011 Err(err) => {
+5012 return Err(TransportError::PROTOCOL_VIOLATION(format!(
+5013 "Nat traversal(REMOVE_ADDRESS): {err}"
+5014 )));
+5015 }
+5016 };
+5017 if let Some(removed_addr) = client_state.remove_remote_address(addr) {
+5018 self.events
+5019 .push_back(Event::NatTraversal(iroh_hp::Event::AddressRemoved(
+5020 removed_addr,
+5021 )));
+5022 }
+5023 }
+5024 Frame::ReachOut(reach_out) => {
+5025 let server_state = match self.iroh_hp.server_side_mut() {
+5026 Ok(state) => state,
+5027 Err(err) => {
+5028 return Err(TransportError::PROTOCOL_VIOLATION(format!(
+5029 "Nat traversal(REACH_OUT): {err}"
+5030 )));
+5031 }
+5032 };
+5033
+5034 if let Err(err) = server_state.handle_reach_out(reach_out) {
+5035 return Err(TransportError::PROTOCOL_VIOLATION(format!(
+5036 "Nat traversal(REACH_OUT): {err}"
+5037 )));
+5038 }
+5039 }
+5040 }
+5041 }
+5042
+5043 let space = self.spaces[SpaceId::Data].for_path(path_id);
+5044 if space
+5045 .pending_acks
+5046 .packet_received(now, number, ack_eliciting, &space.dedup)
+5047 {
+5048 if self.abandoned_paths.contains(&path_id) {
+5049 // § 3.4.3 QUIC-MULTIPATH: promptly send ACKs for packets received from
+5050 // abandoned paths.
+5051 space.pending_acks.set_immediate_ack_required();
+5052 } else {
+5053 self.timers.set(
+5054 Timer::PerPath(path_id, PathTimer::MaxAckDelay),
+5055 now + self.ack_frequency.max_ack_delay,
+5056 self.qlog.with_time(now),
+5057 );
+5058 }
+5059 }
+5060
+5061 // Issue stream ID credit due to ACKs of outgoing finish/resets and incoming finish/resets
+5062 // on stopped streams. Incoming finishes/resets on open streams are not handled here as they
+5063 // are only freed, and hence only issue credit, once the application has been notified
+5064 // during a read on the stream.
+5065 let pending = &mut self.spaces[SpaceId::Data].pending;
+5066 self.streams.queue_max_stream_id(pending);
5067
-5068 // During the handshake the server sends a reset token in the transport
-5069 // parameters. When we are the client and we receive the reset token during the
-5070 // handshake we want this to affect our peer transport parameters.
-5071 // TODO(flub): Pretty sure this is pointless, the entire params is overwritten
-5072 // shortly after this was called. And then the params don't have this anymore.
-5073 if path_id == PathId::ZERO {
-5074 self.peer_params.stateless_reset_token = Some(reset_token);
-5075 }
-5076 }
-5077
-5078 /// Issue an initial set of connection IDs to the peer upon connection
-5079 fn issue_first_cids(&mut self, now: Instant) {
-5080 if self
-5081 .local_cid_state
-5082 .get(&PathId::ZERO)
-5083 .expect("PathId::ZERO exists when the connection is created")
-5084 .cid_len()
-5085 == 0
-5086 {
-5087 return;
+5068 if let Some(reason) = close {
+5069 self.state.move_to_draining(Some(reason.into()));
+5070 self.close = true;
+5071 }
+5072
+5073 if Some(number) == self.spaces[SpaceId::Data].for_path(path_id).rx_packet
+5074 && !is_probing_packet
+5075 && network_path != self.path_data(path_id).network_path
+5076 {
+5077 let ConnectionSide::Server { ref server_config } = self.side else {
+5078 panic!("packets from unknown remote should be dropped by clients");
+5079 };
+5080 debug_assert!(
+5081 server_config.migration,
+5082 "migration-initiating packets should have been dropped immediately"
+5083 );
+5084 self.migrate(path_id, now, network_path, migration_observed_addr);
+5085 // Break linkability, if possible
+5086 self.update_rem_cid(path_id);
+5087 self.spin = false;
5088 }
5089
-5090 // Subtract 1 to account for the CID we supplied while handshaking
-5091 let mut n = self.peer_params.issue_cids_limit() - 1;
-5092 if let ConnectionSide::Server { server_config } = &self.side {
-5093 if server_config.has_preferred_address() {
-5094 // We also sent a CID in the transport parameters
-5095 n -= 1;
-5096 }
-5097 }
-5098 self.endpoint_events
-5099 .push_back(EndpointEventInner::NeedIdentifiers(PathId::ZERO, now, n));
-5100 }
-5101
-5102 /// Issues an initial set of CIDs for paths that have not yet had any CIDs issued
-5103 ///
-5104 /// Later CIDs are issued when CIDs expire or are retired by the peer.
-5105 fn issue_first_path_cids(&mut self, now: Instant) {
-5106 if let Some(max_path_id) = self.max_path_id() {
-5107 let mut path_id = self.max_path_id_with_cids.next();
-5108 while path_id <= max_path_id {
-5109 self.endpoint_events
-5110 .push_back(EndpointEventInner::NeedIdentifiers(
-5111 path_id,
-5112 now,
-5113 self.peer_params.issue_cids_limit(),
-5114 ));
-5115 path_id = path_id.next();
-5116 }
-5117 self.max_path_id_with_cids = max_path_id;
-5118 }
-5119 }
-5120
-5121 /// Populates a packet with frames
-5122 ///
-5123 /// This tries to fit as many frames as possible into the packet.
-5124 ///
-5125 /// *path_exclusive_only* means to only build frames which can only be sent on this
-5126 /// *path. This is used in multipath for backup paths while there is still an active
-5127 /// *path.
-5128 fn populate_packet<'a, 'b>(
-5129 &mut self,
-5130 now: Instant,
-5131 space_id: SpaceId,
-5132 path_id: PathId,
-5133 path_exclusive_only: bool,
-5134 builder: &mut PacketBuilder<'a, 'b>,
-5135 ) {
-5136 let pn = builder.exact_number;
-5137 let is_multipath_negotiated = self.is_multipath_negotiated();
-5138 let stats = &mut self.stats.frame_tx;
-5139 let space = &mut self.spaces[space_id];
-5140 let path = &mut self.paths.get_mut(&path_id).expect("known path").data;
-5141 let is_0rtt = space_id == SpaceId::Data && space.crypto.is_none();
-5142 space
-5143 .for_path(path_id)
-5144 .pending_acks
-5145 .maybe_ack_non_eliciting();
+5090 Ok(())
+5091 }
+5092
+5093 fn migrate(
+5094 &mut self,
+5095 path_id: PathId,
+5096 now: Instant,
+5097 network_path: FourTuple,
+5098 observed_addr: Option<ObservedAddr>,
+5099 ) {
+5100 trace!(%network_path, %path_id, "migration initiated");
+5101 self.path_generation_counter = self.path_generation_counter.wrapping_add(1);
+5102 // TODO(@divma): conditions for path migration in multipath are very specific, check them
+5103 // again to prevent path migrations that should actually create a new path
+5104
+5105 // Reset rtt/congestion state for new path unless it looks like a NAT rebinding.
+5106 // Note that the congestion window will not grow until validation terminates. Helps mitigate
+5107 // amplification attacks performed by spoofing source addresses.
+5108 let prev_pto = self.pto(SpaceId::Data, path_id);
+5109 let known_path = self.paths.get_mut(&path_id).expect("known path");
+5110 let path = &mut known_path.data;
+5111 let mut new_path = if network_path.remote.is_ipv4()
+5112 && network_path.remote.ip() == path.network_path.remote.ip()
+5113 {
+5114 PathData::from_previous(network_path, path, self.path_generation_counter, now)
+5115 } else {
+5116 let peer_max_udp_payload_size =
+5117 u16::try_from(self.peer_params.max_udp_payload_size.into_inner())
+5118 .unwrap_or(u16::MAX);
+5119 PathData::new(
+5120 network_path,
+5121 self.allow_mtud,
+5122 Some(peer_max_udp_payload_size),
+5123 self.path_generation_counter,
+5124 now,
+5125 &self.config,
+5126 )
+5127 };
+5128 new_path.last_observed_addr_report = path.last_observed_addr_report.clone();
+5129 if let Some(report) = observed_addr {
+5130 if let Some(updated) = new_path.update_observed_addr_report(report) {
+5131 tracing::info!("adding observed addr event from migration");
+5132 self.events.push_back(Event::Path(PathEvent::ObservedAddr {
+5133 id: path_id,
+5134 addr: updated,
+5135 }));
+5136 }
+5137 }
+5138 new_path.send_new_challenge = true;
+5139
+5140 let mut prev = mem::replace(path, new_path);
+5141 // Don't clobber the original path if the previous one hasn't been validated yet
+5142 if !prev.is_validating_path() {
+5143 prev.send_new_challenge = true;
+5144 // We haven't updated the remote CID yet, this captures the remote CID we were using on
+5145 // the previous path.
5146
-5147 // HANDSHAKE_DONE
-5148 if !is_0rtt && mem::replace(&mut space.pending.handshake_done, false) {
-5149 builder.write_frame(frame::HandshakeDone, stats);
-5150 }
-5151
-5152 // REACH_OUT
-5153 // TODO(@divma): path explusive considerations
-5154 if let Some((round, addresses)) = space.pending.reach_out.as_mut() {
-5155 while let Some(local_addr) = addresses.pop() {
-5156 let reach_out = frame::ReachOut::new(*round, local_addr);
-5157 if builder.frame_space_remaining() > reach_out.size() {
-5158 builder.write_frame(reach_out, stats);
-5159 } else {
-5160 addresses.push(local_addr);
-5161 break;
-5162 }
-5163 }
-5164 if addresses.is_empty() {
-5165 space.pending.reach_out = None;
-5166 }
-5167 }
-5168
-5169 // OBSERVED_ADDR
-5170 if !path_exclusive_only
-5171 && space_id == SpaceId::Data
-5172 && self
-5173 .config
-5174 .address_discovery_role
-5175 .should_report(&self.peer_params.address_discovery_role)
-5176 && (!path.observed_addr_sent || space.pending.observed_addr)
-5177 {
-5178 let frame =
-5179 frame::ObservedAddr::new(path.network_path.remote, self.next_observed_addr_seq_no);
-5180 if builder.frame_space_remaining() > frame.size() {
-5181 builder.write_frame(frame, stats);
-5182
-5183 self.next_observed_addr_seq_no = self.next_observed_addr_seq_no.saturating_add(1u8);
-5184 path.observed_addr_sent = true;
-5185
-5186 space.pending.observed_addr = false;
-5187 }
-5188 }
-5189
-5190 // PING
-5191 if mem::replace(&mut space.for_path(path_id).ping_pending, false) {
-5192 builder.write_frame(frame::Ping, stats);
-5193 }
-5194
-5195 // IMMEDIATE_ACK
-5196 if mem::replace(&mut space.for_path(path_id).immediate_ack_pending, false) {
-5197 debug_assert_eq!(
-5198 space_id,
-5199 SpaceId::Data,
-5200 "immediate acks must be sent in the data space"
-5201 );
-5202 builder.write_frame(frame::ImmediateAck, stats);
+5147 known_path.prev = Some((self.rem_cids.get(&path_id).unwrap().active(), prev));
+5148 }
+5149
+5150 // We need to re-assign the correct remote to this path in qlog
+5151 self.qlog.emit_tuple_assigned(path_id, network_path, now);
+5152
+5153 self.timers.set(
+5154 Timer::PerPath(path_id, PathTimer::PathValidation),
+5155 now + 3 * cmp::max(self.pto(SpaceId::Data, path_id), prev_pto),
+5156 self.qlog.with_time(now),
+5157 );
+5158 }
+5159
+5160 /// Handle a change in the local address, i.e. an active migration
+5161 pub fn local_address_changed(&mut self) {
+5162 // TODO(flub): if multipath is enabled this needs to create a new path entirely.
+5163 self.update_rem_cid(PathId::ZERO);
+5164 self.ping();
+5165 }
+5166
+5167 /// Switch to a previously unused remote connection ID, if possible
+5168 fn update_rem_cid(&mut self, path_id: PathId) {
+5169 let Some((reset_token, retired)) =
+5170 self.rem_cids.get_mut(&path_id).and_then(|cids| cids.next())
+5171 else {
+5172 return;
+5173 };
+5174
+5175 // Retire the current remote CID and any CIDs we had to skip.
+5176 self.spaces[SpaceId::Data]
+5177 .pending
+5178 .retire_cids
+5179 .extend(retired.map(|seq| (path_id, seq)));
+5180 let remote = self.path_data(path_id).network_path.remote;
+5181 self.set_reset_token(path_id, remote, reset_token);
+5182 }
+5183
+5184 /// Sends this reset token to the endpoint
+5185 ///
+5186 /// The endpoint needs to know the reset tokens issued by the peer, so that if the peer
+5187 /// sends a reset token it knows to route it to this connection. See RFC 9000 section
+5188 /// 10.3. Stateless Reset.
+5189 ///
+5190 /// Reset tokens are different for each path, the endpoint identifies paths by peer
+5191 /// socket address however, not by path ID.
+5192 fn set_reset_token(&mut self, path_id: PathId, remote: SocketAddr, reset_token: ResetToken) {
+5193 self.endpoint_events
+5194 .push_back(EndpointEventInner::ResetToken(path_id, remote, reset_token));
+5195
+5196 // During the handshake the server sends a reset token in the transport
+5197 // parameters. When we are the client and we receive the reset token during the
+5198 // handshake we want this to affect our peer transport parameters.
+5199 // TODO(flub): Pretty sure this is pointless, the entire params is overwritten
+5200 // shortly after this was called. And then the params don't have this anymore.
+5201 if path_id == PathId::ZERO {
+5202 self.peer_params.stateless_reset_token = Some(reset_token);
5203 }
-5204
-5205 // ACK
-5206 // TODO(flub): Should this send acks for this path anyway?
-5207
-5208 if !path_exclusive_only {
-5209 for path_id in space
-5210 .number_spaces
-5211 .iter_mut()
-5212 .filter(|(_, pns)| pns.pending_acks.can_send())
-5213 .map(|(&path_id, _)| path_id)
-5214 .collect::<Vec<_>>()
-5215 {
-5216 Self::populate_acks(
-5217 now,
-5218 self.receiving_ecn,
-5219 path_id,
-5220 space_id,
-5221 space,
-5222 is_multipath_negotiated,
-5223 builder,
-5224 stats,
-5225 );
-5226 }
-5227 }
-5228
-5229 // ACK_FREQUENCY
-5230 if !path_exclusive_only && mem::replace(&mut space.pending.ack_frequency, false) {
-5231 let sequence_number = self.ack_frequency.next_sequence_number();
-5232
-5233 // Safe to unwrap because this is always provided when ACK frequency is enabled
-5234 let config = self.config.ack_frequency_config.as_ref().unwrap();
-5235
-5236 // Ensure the delay is within bounds to avoid a PROTOCOL_VIOLATION error
-5237 let max_ack_delay = self.ack_frequency.candidate_max_ack_delay(
-5238 path.rtt.get(),
-5239 config,
-5240 &self.peer_params,
-5241 );
-5242
-5243 let frame = frame::AckFrequency {
-5244 sequence: sequence_number,
-5245 ack_eliciting_threshold: config.ack_eliciting_threshold,
-5246 request_max_ack_delay: max_ack_delay.as_micros().try_into().unwrap_or(VarInt::MAX),
-5247 reordering_threshold: config.reordering_threshold,
-5248 };
-5249 builder.write_frame(frame, stats);
-5250
-5251 self.ack_frequency
-5252 .ack_frequency_sent(path_id, pn, max_ack_delay);
-5253 }
-5254
-5255 // PATH_CHALLENGE
-5256 if builder.frame_space_remaining() > frame::PathChallenge::SIZE_BOUND
-5257 && space_id == SpaceId::Data
-5258 && path.send_new_challenge
-5259 && !self.state.is_closed()
-5260 // we don't want to send new challenges if we are already closing
-5261 {
-5262 path.send_new_challenge = false;
-5263
-5264 // Generate a new challenge every time we send a new PATH_CHALLENGE
-5265 let token = self.rng.random();
-5266 let info = paths::SentChallengeInfo {
-5267 sent_instant: now,
-5268 network_path: path.network_path,
-5269 };
-5270 path.challenges_sent.insert(token, info);
-5271 let challenge = frame::PathChallenge(token);
-5272 trace!(frame = %challenge);
-5273 builder.write_frame(challenge, stats);
-5274 builder.require_padding();
-5275 let pto = self.ack_frequency.max_ack_delay_for_pto() + path.rtt.pto_base();
-5276 self.timers.set(
-5277 Timer::PerPath(path_id, PathTimer::PathChallengeLost),
-5278 now + pto,
-5279 self.qlog.with_time(now),
-5280 );
-5281
-5282 if is_multipath_negotiated && !path.validated && path.send_new_challenge {
-5283 // queue informing the path status along with the challenge
-5284 space.pending.path_status.insert(path_id);
-5285 }
-5286
-5287 // Always include an OBSERVED_ADDR frame with a PATH_CHALLENGE, regardless
-5288 // of whether one has already been sent on this path.
-5289 if space_id == SpaceId::Data
-5290 && self
-5291 .config
-5292 .address_discovery_role
-5293 .should_report(&self.peer_params.address_discovery_role)
-5294 {
-5295 let frame = frame::ObservedAddr::new(
-5296 path.network_path.remote,
-5297 self.next_observed_addr_seq_no,
-5298 );
-5299 if builder.frame_space_remaining() > frame.size() {
-5300 builder.write_frame(frame, stats);
-5301
-5302 self.next_observed_addr_seq_no =
-5303 self.next_observed_addr_seq_no.saturating_add(1u8);
-5304 path.observed_addr_sent = true;
-5305
-5306 space.pending.observed_addr = false;
-5307 }
-5308 }
-5309 }
+5204 }
+5205
+5206 /// Issue an initial set of connection IDs to the peer upon connection
+5207 fn issue_first_cids(&mut self, now: Instant) {
+5208 if self
+5209 .local_cid_state
+5210 .get(&PathId::ZERO)
+5211 .expect("PathId::ZERO exists when the connection is created")
+5212 .cid_len()
+5213 == 0
+5214 {
+5215 return;
+5216 }
+5217
+5218 // Subtract 1 to account for the CID we supplied while handshaking
+5219 let mut n = self.peer_params.issue_cids_limit() - 1;
+5220 if let ConnectionSide::Server { server_config } = &self.side {
+5221 if server_config.has_preferred_address() {
+5222 // We also sent a CID in the transport parameters
+5223 n -= 1;
+5224 }
+5225 }
+5226 self.endpoint_events
+5227 .push_back(EndpointEventInner::NeedIdentifiers(PathId::ZERO, now, n));
+5228 }
+5229
+5230 /// Issues an initial set of CIDs for paths that have not yet had any CIDs issued
+5231 ///
+5232 /// Later CIDs are issued when CIDs expire or are retired by the peer.
+5233 fn issue_first_path_cids(&mut self, now: Instant) {
+5234 if let Some(max_path_id) = self.max_path_id() {
+5235 let mut path_id = self.max_path_id_with_cids.next();
+5236 while path_id <= max_path_id {
+5237 self.endpoint_events
+5238 .push_back(EndpointEventInner::NeedIdentifiers(
+5239 path_id,
+5240 now,
+5241 self.peer_params.issue_cids_limit(),
+5242 ));
+5243 path_id = path_id.next();
+5244 }
+5245 self.max_path_id_with_cids = max_path_id;
+5246 }
+5247 }
+5248
+5249 /// Populates a packet with frames
+5250 ///
+5251 /// This tries to fit as many frames as possible into the packet.
+5252 ///
+5253 /// *path_exclusive_only* means to only build frames which can only be sent on this
+5254 /// *path. This is used in multipath for backup paths while there is still an active
+5255 /// *path.
+5256 fn populate_packet<'a, 'b>(
+5257 &mut self,
+5258 now: Instant,
+5259 space_id: SpaceId,
+5260 path_id: PathId,
+5261 path_exclusive_only: bool,
+5262 builder: &mut PacketBuilder<'a, 'b>,
+5263 ) {
+5264 let pn = builder.exact_number;
+5265 let is_multipath_negotiated = self.is_multipath_negotiated();
+5266 let stats = &mut self.stats.frame_tx;
+5267 let space = &mut self.spaces[space_id];
+5268 let path = &mut self.paths.get_mut(&path_id).expect("known path").data;
+5269 let is_0rtt = space_id == SpaceId::Data && space.crypto.is_none();
+5270 space
+5271 .for_path(path_id)
+5272 .pending_acks
+5273 .maybe_ack_non_eliciting();
+5274
+5275 // HANDSHAKE_DONE
+5276 if !is_0rtt && mem::replace(&mut space.pending.handshake_done, false) {
+5277 builder.write_frame(frame::HandshakeDone, stats);
+5278 }
+5279
+5280 // REACH_OUT
+5281 // TODO(@divma): path explusive considerations
+5282 if let Some((round, addresses)) = space.pending.reach_out.as_mut() {
+5283 while let Some(local_addr) = addresses.pop() {
+5284 let reach_out = frame::ReachOut::new(*round, local_addr);
+5285 if builder.frame_space_remaining() > reach_out.size() {
+5286 builder.write_frame(reach_out, stats);
+5287 } else {
+5288 addresses.push(local_addr);
+5289 break;
+5290 }
+5291 }
+5292 if addresses.is_empty() {
+5293 space.pending.reach_out = None;
+5294 }
+5295 }
+5296
+5297 // OBSERVED_ADDR
+5298 if !path_exclusive_only
+5299 && space_id == SpaceId::Data
+5300 && self
+5301 .config
+5302 .address_discovery_role
+5303 .should_report(&self.peer_params.address_discovery_role)
+5304 && (!path.observed_addr_sent || space.pending.observed_addr)
+5305 {
+5306 let frame =
+5307 frame::ObservedAddr::new(path.network_path.remote, self.next_observed_addr_seq_no);
+5308 if builder.frame_space_remaining() > frame.size() {
+5309 builder.write_frame(frame, stats);
5310
-5311 // PATH_RESPONSE
-5312 if builder.frame_space_remaining() > frame::PathResponse::SIZE_BOUND
-5313 && space_id == SpaceId::Data
-5314 {
-5315 if let Some(token) = path.path_responses.pop_on_path(path.network_path) {
-5316 let response = frame::PathResponse(token);
-5317 trace!(frame = %response);
-5318 builder.write_frame(response, stats);
-5319 builder.require_padding();
-5320
-5321 // NOTE: this is technically not required but might be useful to ride the
-5322 // request/response nature of path challenges to refresh an observation
-5323 // Since PATH_RESPONSE is a probing frame, this is allowed by the spec.
-5324 if space_id == SpaceId::Data
-5325 && self
-5326 .config
-5327 .address_discovery_role
-5328 .should_report(&self.peer_params.address_discovery_role)
-5329 {
-5330 let frame = frame::ObservedAddr::new(
-5331 path.network_path.remote,
-5332 self.next_observed_addr_seq_no,
-5333 );
-5334 if builder.frame_space_remaining() > frame.size() {
-5335 builder.write_frame(frame, stats);
-5336
-5337 self.next_observed_addr_seq_no =
-5338 self.next_observed_addr_seq_no.saturating_add(1u8);
-5339 path.observed_addr_sent = true;
-5340
-5341 space.pending.observed_addr = false;
-5342 }
-5343 }
-5344 }
-5345 }
-5346
-5347 // CRYPTO
-5348 while !path_exclusive_only
-5349 && builder.frame_space_remaining() > frame::Crypto::SIZE_BOUND
-5350 && !is_0rtt
-5351 {
-5352 let mut frame = match space.pending.crypto.pop_front() {
-5353 Some(x) => x,
-5354 None => break,
-5355 };
+5311 self.next_observed_addr_seq_no = self.next_observed_addr_seq_no.saturating_add(1u8);
+5312 path.observed_addr_sent = true;
+5313
+5314 space.pending.observed_addr = false;
+5315 }
+5316 }
+5317
+5318 // PING
+5319 if mem::replace(&mut space.for_path(path_id).ping_pending, false) {
+5320 builder.write_frame(frame::Ping, stats);
+5321 }
+5322
+5323 // IMMEDIATE_ACK
+5324 if mem::replace(&mut space.for_path(path_id).immediate_ack_pending, false) {
+5325 debug_assert_eq!(
+5326 space_id,
+5327 SpaceId::Data,
+5328 "immediate acks must be sent in the data space"
+5329 );
+5330 builder.write_frame(frame::ImmediateAck, stats);
+5331 }
+5332
+5333 // ACK
+5334 // TODO(flub): Should this send acks for this path anyway?
+5335
+5336 if !path_exclusive_only {
+5337 for path_id in space
+5338 .number_spaces
+5339 .iter_mut()
+5340 .filter(|(_, pns)| pns.pending_acks.can_send())
+5341 .map(|(&path_id, _)| path_id)
+5342 .collect::<Vec<_>>()
+5343 {
+5344 Self::populate_acks(
+5345 now,
+5346 self.receiving_ecn,
+5347 path_id,
+5348 space_id,
+5349 space,
+5350 is_multipath_negotiated,
+5351 builder,
+5352 stats,
+5353 );
+5354 }
+5355 }
5356
-5357 // Calculate the maximum amount of crypto data we can store in the buffer.
-5358 // Since the offset is known, we can reserve the exact size required to encode it.
-5359 // For length we reserve 2bytes which allows to encode up to 2^14,
-5360 // which is more than what fits into normally sized QUIC frames.
-5361 let max_crypto_data_size = builder.frame_space_remaining()
-5362 - 1 // Frame Type
-5363 - VarInt::size(unsafe { VarInt::from_u64_unchecked(frame.offset) })
-5364 - 2; // Maximum encoded length for frame size, given we send less than 2^14 bytes
-5365
-5366 let len = frame
-5367 .data
-5368 .len()
-5369 .min(2usize.pow(14) - 1)
-5370 .min(max_crypto_data_size);
-5371
-5372 let data = frame.data.split_to(len);
-5373 let offset = frame.offset;
-5374 let truncated = frame::Crypto { offset, data };
-5375 builder.write_frame(truncated, stats);
-5376
-5377 if !frame.data.is_empty() {
-5378 frame.offset += len as u64;
-5379 space.pending.crypto.push_front(frame);
-5380 }
+5357 // ACK_FREQUENCY
+5358 if !path_exclusive_only && mem::replace(&mut space.pending.ack_frequency, false) {
+5359 let sequence_number = self.ack_frequency.next_sequence_number();
+5360
+5361 // Safe to unwrap because this is always provided when ACK frequency is enabled
+5362 let config = self.config.ack_frequency_config.as_ref().unwrap();
+5363
+5364 // Ensure the delay is within bounds to avoid a PROTOCOL_VIOLATION error
+5365 let max_ack_delay = self.ack_frequency.candidate_max_ack_delay(
+5366 path.rtt.get(),
+5367 config,
+5368 &self.peer_params,
+5369 );
+5370
+5371 let frame = frame::AckFrequency {
+5372 sequence: sequence_number,
+5373 ack_eliciting_threshold: config.ack_eliciting_threshold,
+5374 request_max_ack_delay: max_ack_delay.as_micros().try_into().unwrap_or(VarInt::MAX),
+5375 reordering_threshold: config.reordering_threshold,
+5376 };
+5377 builder.write_frame(frame, stats);
+5378
+5379 self.ack_frequency
+5380 .ack_frequency_sent(path_id, pn, max_ack_delay);
5381 }
5382
-5383 // TODO(flub): maybe this is much higher priority?
-5384 // PATH_ABANDON
-5385 while !path_exclusive_only
-5386 && space_id == SpaceId::Data
-5387 && frame::PathAbandon::SIZE_BOUND <= builder.frame_space_remaining()
-5388 {
-5389 let Some((abandoned_path_id, error_code)) = space.pending.path_abandon.pop_first()
-5390 else {
-5391 break;
-5392 };
-5393 let frame = frame::PathAbandon {
-5394 path_id: abandoned_path_id,
-5395 error_code,
-5396 };
-5397 builder.write_frame(frame, stats);
-5398
-5399 let ack_delay = self.ack_frequency.max_ack_delay_for_pto();
-5400 // We can't access path here anymore due to borrowing issues.
-5401 let send_pto = self.paths.get(&path_id).unwrap().data.rtt.pto_base() + ack_delay;
-5402 if let Some(abandoned_path) = self.paths.get_mut(&abandoned_path_id) {
-5403 // We only want to set the deadline on the *first* PATH_ABANDON we send.
-5404 // Retransmits shouldn't run this code again
-5405 if matches!(
-5406 abandoned_path.data.abandon_state,
-5407 AbandonState::NotAbandoned
-5408 ) {
-5409 // The peer MUST respond with a corresponding PATH_ABANDON frame.
-5410 // The other peer has 3 * PTO to do that.
-5411 // This uses the PTO of the path we send on!
-5412 // If the PATH_ABANDON comes in within the deadline we're giving here, then this
-5413 // state will be set to `AbandonState::ReceivedPathAbandon`, essentially clearing
-5414 // the deadline. If we receive a frame after the deadline, we error out with a
-5415 // protocol violation.
-5416 // Receiving other frames before the deadline is fine, as those might be packets
-5417 // that were still in-flight.
-5418 abandoned_path.data.abandon_state = AbandonState::ExpectingPathAbandon {
-5419 deadline: now + 3 * send_pto,
-5420 };
-5421
-5422 // At some point, we need to forget about the path.
-5423 // If we do so too early, then we'll have discarded the CIDs of that path and won't
-5424 // handle incoming packets on that path correctly.
-5425 // To give this path enough time, we assume that the peer will have received our
-5426 // PATH_ABANDON within 3 * PTO of the path we sent the abandon on,
-5427 // and then we give the path 3 * PTO time to make it very unlikely that there will
-5428 // still be packets incoming on the path at that point.
-5429 // This timer will actually get reset to a value that's likely to be even earlier
-5430 // once we actually receive the PATH_ABANDON frame itself.
-5431 let abandoned_pto =
-5432 self.paths.get(&path_id).unwrap().data.rtt.pto_base() + ack_delay;
-5433 self.timers.set(
-5434 Timer::PerPath(abandoned_path_id, PathTimer::DiscardPath),
-5435 now + 3 * send_pto + 3 * abandoned_pto,
-5436 self.qlog.with_time(now),
-5437 );
-5438 }
-5439 } else {
-5440 warn!("sent PATH_ABANDON after path was already discarded");
-5441 }
-5442 }
-5443
-5444 // PATH_STATUS_AVAILABLE & PATH_STATUS_BACKUP
-5445 while !path_exclusive_only
-5446 && space_id == SpaceId::Data
-5447 && frame::PathStatusAvailable::SIZE_BOUND <= builder.frame_space_remaining()
-5448 {
-5449 let Some(path_id) = space.pending.path_status.pop_first() else {
-5450 break;
-5451 };
-5452 let Some(path) = self.paths.get(&path_id).map(|path_state| &path_state.data) else {
-5453 trace!(%path_id, "discarding queued path status for unknown path");
-5454 continue;
-5455 };
-5456
-5457 let seq = path.status.seq();
-5458 match path.local_status() {
-5459 PathStatus::Available => {
-5460 let frame = frame::PathStatusAvailable {
-5461 path_id,
-5462 status_seq_no: seq,
-5463 };
-5464 builder.write_frame(frame, stats);
-5465 }
-5466 PathStatus::Backup => {
-5467 let frame = frame::PathStatusBackup {
-5468 path_id,
-5469 status_seq_no: seq,
-5470 };
-5471 builder.write_frame(frame, stats);
-5472 }
-5473 }
-5474 }
-5475
-5476 // MAX_PATH_ID
-5477 if space_id == SpaceId::Data
-5478 && space.pending.max_path_id
-5479 && frame::MaxPathId::SIZE_BOUND <= builder.frame_space_remaining()
-5480 {
-5481 let frame = frame::MaxPathId(self.local_max_path_id);
-5482 builder.write_frame(frame, stats);
-5483 space.pending.max_path_id = false;
-5484 }
-5485
-5486 // PATHS_BLOCKED
-5487 if space_id == SpaceId::Data
-5488 && space.pending.paths_blocked
-5489 && frame::PathsBlocked::SIZE_BOUND <= builder.frame_space_remaining()
-5490 {
-5491 let frame = frame::PathsBlocked(self.remote_max_path_id);
-5492 builder.write_frame(frame, stats);
-5493 space.pending.paths_blocked = false;
-5494 }
-5495
-5496 // PATH_CIDS_BLOCKED
-5497 while space_id == SpaceId::Data
-5498 && frame::PathCidsBlocked::SIZE_BOUND <= builder.frame_space_remaining()
-5499 {
-5500 let Some(path_id) = space.pending.path_cids_blocked.pop_first() else {
-5501 break;
-5502 };
-5503 let next_seq = match self.rem_cids.get(&path_id) {
-5504 Some(cid_queue) => VarInt(cid_queue.active_seq() + 1),
-5505 None => VarInt(0),
-5506 };
-5507 let frame = frame::PathCidsBlocked { path_id, next_seq };
-5508 builder.write_frame(frame, stats);
+5383 // PATH_CHALLENGE
+5384 if builder.frame_space_remaining() > frame::PathChallenge::SIZE_BOUND
+5385 && space_id == SpaceId::Data
+5386 && path.send_new_challenge
+5387 && !self.state.is_closed()
+5388 // we don't want to send new challenges if we are already closing
+5389 {
+5390 path.send_new_challenge = false;
+5391
+5392 // Generate a new challenge every time we send a new PATH_CHALLENGE
+5393 let token = self.rng.random();
+5394 let info = paths::SentChallengeInfo {
+5395 sent_instant: now,
+5396 network_path: path.network_path,
+5397 };
+5398 path.challenges_sent.insert(token, info);
+5399 let challenge = frame::PathChallenge(token);
+5400 trace!(frame = %challenge);
+5401 builder.write_frame(challenge, stats);
+5402 builder.require_padding();
+5403 let pto = self.ack_frequency.max_ack_delay_for_pto() + path.rtt.pto_base();
+5404 self.timers.set(
+5405 Timer::PerPath(path_id, PathTimer::PathChallengeLost),
+5406 now + pto,
+5407 self.qlog.with_time(now),
+5408 );
+5409
+5410 if is_multipath_negotiated && !path.validated && path.send_new_challenge {
+5411 // queue informing the path status along with the challenge
+5412 space.pending.path_status.insert(path_id);
+5413 }
+5414
+5415 // Always include an OBSERVED_ADDR frame with a PATH_CHALLENGE, regardless
+5416 // of whether one has already been sent on this path.
+5417 if space_id == SpaceId::Data
+5418 && self
+5419 .config
+5420 .address_discovery_role
+5421 .should_report(&self.peer_params.address_discovery_role)
+5422 {
+5423 let frame = frame::ObservedAddr::new(
+5424 path.network_path.remote,
+5425 self.next_observed_addr_seq_no,
+5426 );
+5427 if builder.frame_space_remaining() > frame.size() {
+5428 builder.write_frame(frame, stats);
+5429
+5430 self.next_observed_addr_seq_no =
+5431 self.next_observed_addr_seq_no.saturating_add(1u8);
+5432 path.observed_addr_sent = true;
+5433
+5434 space.pending.observed_addr = false;
+5435 }
+5436 }
+5437 }
+5438
+5439 // PATH_RESPONSE
+5440 if builder.frame_space_remaining() > frame::PathResponse::SIZE_BOUND
+5441 && space_id == SpaceId::Data
+5442 {
+5443 if let Some(token) = path.path_responses.pop_on_path(path.network_path) {
+5444 let response = frame::PathResponse(token);
+5445 trace!(frame = %response);
+5446 builder.write_frame(response, stats);
+5447 builder.require_padding();
+5448
+5449 // NOTE: this is technically not required but might be useful to ride the
+5450 // request/response nature of path challenges to refresh an observation
+5451 // Since PATH_RESPONSE is a probing frame, this is allowed by the spec.
+5452 if space_id == SpaceId::Data
+5453 && self
+5454 .config
+5455 .address_discovery_role
+5456 .should_report(&self.peer_params.address_discovery_role)
+5457 {
+5458 let frame = frame::ObservedAddr::new(
+5459 path.network_path.remote,
+5460 self.next_observed_addr_seq_no,
+5461 );
+5462 if builder.frame_space_remaining() > frame.size() {
+5463 builder.write_frame(frame, stats);
+5464
+5465 self.next_observed_addr_seq_no =
+5466 self.next_observed_addr_seq_no.saturating_add(1u8);
+5467 path.observed_addr_sent = true;
+5468
+5469 space.pending.observed_addr = false;
+5470 }
+5471 }
+5472 }
+5473 }
+5474
+5475 // CRYPTO
+5476 while !path_exclusive_only
+5477 && builder.frame_space_remaining() > frame::Crypto::SIZE_BOUND
+5478 && !is_0rtt
+5479 {
+5480 let mut frame = match space.pending.crypto.pop_front() {
+5481 Some(x) => x,
+5482 None => break,
+5483 };
+5484
+5485 // Calculate the maximum amount of crypto data we can store in the buffer.
+5486 // Since the offset is known, we can reserve the exact size required to encode it.
+5487 // For length we reserve 2bytes which allows to encode up to 2^14,
+5488 // which is more than what fits into normally sized QUIC frames.
+5489 let max_crypto_data_size = builder.frame_space_remaining()
+5490 - 1 // Frame Type
+5491 - VarInt::size(unsafe { VarInt::from_u64_unchecked(frame.offset) })
+5492 - 2; // Maximum encoded length for frame size, given we send less than 2^14 bytes
+5493
+5494 let len = frame
+5495 .data
+5496 .len()
+5497 .min(2usize.pow(14) - 1)
+5498 .min(max_crypto_data_size);
+5499
+5500 let data = frame.data.split_to(len);
+5501 let offset = frame.offset;
+5502 let truncated = frame::Crypto { offset, data };
+5503 builder.write_frame(truncated, stats);
+5504
+5505 if !frame.data.is_empty() {
+5506 frame.offset += len as u64;
+5507 space.pending.crypto.push_front(frame);
+5508 }
5509 }
5510
-5511 // RESET_STREAM, STOP_SENDING, MAX_DATA, MAX_STREAM_DATA, MAX_STREAMS
-5512 if space_id == SpaceId::Data {
-5513 self.streams
-5514 .write_control_frames(builder, &mut space.pending, stats);
-5515 }
-5516
-5517 // NEW_CONNECTION_ID
-5518 let cid_len = self
-5519 .local_cid_state
-5520 .values()
-5521 .map(|cid_state| cid_state.cid_len())
-5522 .max()
-5523 .expect("some local CID state must exist");
-5524 let new_cid_size_bound =
-5525 frame::NewConnectionId::size_bound(is_multipath_negotiated, cid_len);
-5526 while !path_exclusive_only && builder.frame_space_remaining() > new_cid_size_bound {
-5527 let issued = match space.pending.new_cids.pop() {
-5528 Some(x) => x,
-5529 None => break,
-5530 };
-5531 let retire_prior_to = self
-5532 .local_cid_state
-5533 .get(&issued.path_id)
-5534 .map(|cid_state| cid_state.retire_prior_to())
-5535 .unwrap_or_else(|| panic!("missing local CID state for path={}", issued.path_id));
-5536
-5537 let cid_path_id = match is_multipath_negotiated {
-5538 true => Some(issued.path_id),
-5539 false => {
-5540 debug_assert_eq!(issued.path_id, PathId::ZERO);
-5541 None
-5542 }
-5543 };
-5544 let frame = frame::NewConnectionId {
-5545 path_id: cid_path_id,
-5546 sequence: issued.sequence,
-5547 retire_prior_to,
-5548 id: issued.id,
-5549 reset_token: issued.reset_token,
-5550 };
-5551 builder.write_frame(frame, stats);
-5552 }
-5553
-5554 // RETIRE_CONNECTION_ID
-5555 let retire_cid_bound = frame::RetireConnectionId::size_bound(is_multipath_negotiated);
-5556 while !path_exclusive_only && builder.frame_space_remaining() > retire_cid_bound {
-5557 let (path_id, sequence) = match space.pending.retire_cids.pop() {
-5558 Some((PathId::ZERO, seq)) if !is_multipath_negotiated => (None, seq),
-5559 Some((path_id, seq)) => (Some(path_id), seq),
-5560 None => break,
-5561 };
-5562 let frame = frame::RetireConnectionId { path_id, sequence };
-5563 builder.write_frame(frame, stats);
-5564 }
-5565
-5566 // DATAGRAM
-5567 let mut sent_datagrams = false;
-5568 while !path_exclusive_only
-5569 && builder.frame_space_remaining() > Datagram::SIZE_BOUND
-5570 && space_id == SpaceId::Data
-5571 {
-5572 match self.datagrams.write(builder, stats) {
-5573 true => {
-5574 sent_datagrams = true;
-5575 }
-5576 false => break,
-5577 }
-5578 }
-5579 if self.datagrams.send_blocked && sent_datagrams {
-5580 self.events.push_back(Event::DatagramsUnblocked);
-5581 self.datagrams.send_blocked = false;
-5582 }
-5583
-5584 let path = &mut self.paths.get_mut(&path_id).expect("known path").data;
-5585
-5586 // NEW_TOKEN
-5587 if !path_exclusive_only {
-5588 while let Some(network_path) = space.pending.new_tokens.pop() {
-5589 debug_assert_eq!(space_id, SpaceId::Data);
-5590 let ConnectionSide::Server { server_config } = &self.side else {
-5591 panic!("NEW_TOKEN frames should not be enqueued by clients");
-5592 };
-5593
-5594 if !network_path.is_probably_same_path(&path.network_path) {
-5595 // NEW_TOKEN frames contain tokens bound to a client's IP address, and are only
-5596 // useful if used from the same IP address. Thus, we abandon enqueued NEW_TOKEN
-5597 // frames upon an path change. Instead, when the new path becomes validated,
-5598 // NEW_TOKEN frames may be enqueued for the new path instead.
-5599 continue;
+5511 // TODO(flub): maybe this is much higher priority?
+5512 // PATH_ABANDON
+5513 while !path_exclusive_only
+5514 && space_id == SpaceId::Data
+5515 && frame::PathAbandon::SIZE_BOUND <= builder.frame_space_remaining()
+5516 {
+5517 let Some((abandoned_path_id, error_code)) = space.pending.path_abandon.pop_first()
+5518 else {
+5519 break;
+5520 };
+5521 let frame = frame::PathAbandon {
+5522 path_id: abandoned_path_id,
+5523 error_code,
+5524 };
+5525 builder.write_frame(frame, stats);
+5526
+5527 let ack_delay = self.ack_frequency.max_ack_delay_for_pto();
+5528 // We can't access path here anymore due to borrowing issues.
+5529 let send_pto = self.paths.get(&path_id).unwrap().data.rtt.pto_base() + ack_delay;
+5530 if let Some(abandoned_path) = self.paths.get_mut(&abandoned_path_id) {
+5531 // We only want to set the deadline on the *first* PATH_ABANDON we send.
+5532 // Retransmits shouldn't run this code again
+5533 if matches!(
+5534 abandoned_path.data.abandon_state,
+5535 AbandonState::NotAbandoned
+5536 ) {
+5537 // The peer MUST respond with a corresponding PATH_ABANDON frame.
+5538 // The other peer has 3 * PTO to do that.
+5539 // This uses the PTO of the path we send on!
+5540 // If the PATH_ABANDON comes in within the deadline we're giving here, then this
+5541 // state will be set to `AbandonState::ReceivedPathAbandon`, essentially clearing
+5542 // the deadline. If we receive a frame after the deadline, we error out with a
+5543 // protocol violation.
+5544 // Receiving other frames before the deadline is fine, as those might be packets
+5545 // that were still in-flight.
+5546 abandoned_path.data.abandon_state = AbandonState::ExpectingPathAbandon {
+5547 deadline: now + 3 * send_pto,
+5548 };
+5549
+5550 // At some point, we need to forget about the path.
+5551 // If we do so too early, then we'll have discarded the CIDs of that path and won't
+5552 // handle incoming packets on that path correctly.
+5553 // To give this path enough time, we assume that the peer will have received our
+5554 // PATH_ABANDON within 3 * PTO of the path we sent the abandon on,
+5555 // and then we give the path 3 * PTO time to make it very unlikely that there will
+5556 // still be packets incoming on the path at that point.
+5557 // This timer will actually get reset to a value that's likely to be even earlier
+5558 // once we actually receive the PATH_ABANDON frame itself.
+5559 let abandoned_pto =
+5560 self.paths.get(&path_id).unwrap().data.rtt.pto_base() + ack_delay;
+5561 self.timers.set(
+5562 Timer::PerPath(abandoned_path_id, PathTimer::DiscardPath),
+5563 now + 3 * send_pto + 3 * abandoned_pto,
+5564 self.qlog.with_time(now),
+5565 );
+5566 }
+5567 } else {
+5568 warn!("sent PATH_ABANDON after path was already discarded");
+5569 }
+5570 }
+5571
+5572 // PATH_STATUS_AVAILABLE & PATH_STATUS_BACKUP
+5573 while !path_exclusive_only
+5574 && space_id == SpaceId::Data
+5575 && frame::PathStatusAvailable::SIZE_BOUND <= builder.frame_space_remaining()
+5576 {
+5577 let Some(path_id) = space.pending.path_status.pop_first() else {
+5578 break;
+5579 };
+5580 let Some(path) = self.paths.get(&path_id).map(|path_state| &path_state.data) else {
+5581 trace!(%path_id, "discarding queued path status for unknown path");
+5582 continue;
+5583 };
+5584
+5585 let seq = path.status.seq();
+5586 match path.local_status() {
+5587 PathStatus::Available => {
+5588 let frame = frame::PathStatusAvailable {
+5589 path_id,
+5590 status_seq_no: seq,
+5591 };
+5592 builder.write_frame(frame, stats);
+5593 }
+5594 PathStatus::Backup => {
+5595 let frame = frame::PathStatusBackup {
+5596 path_id,
+5597 status_seq_no: seq,
+5598 };
+5599 builder.write_frame(frame, stats);
5600 }
-5601
-5602 let token = Token::new(
-5603 TokenPayload::Validation {
-5604 ip: network_path.remote.ip(),
-5605 issued: server_config.time_source.now(),
-5606 },
-5607 &mut self.rng,
-5608 );
-5609 let new_token = NewToken {
-5610 token: token.encode(&*server_config.token_key).into(),
-5611 };
-5612
-5613 if builder.frame_space_remaining() < new_token.size() {
-5614 space.pending.new_tokens.push(network_path);
-5615 break;
-5616 }
-5617
-5618 builder.write_frame(new_token, stats);
-5619 builder.retransmits_mut().new_tokens.push(network_path);
-5620 }
-5621 }
-5622
-5623 // STREAM
-5624 if !path_exclusive_only && space_id == SpaceId::Data {
-5625 self.streams
-5626 .write_stream_frames(builder, self.config.send_fairness, stats);
-5627 }
-5628
-5629 // ADD_ADDRESS
-5630 // TODO(@divma): check if we need to do path exclusive filters
-5631 while space_id == SpaceId::Data
-5632 && frame::AddAddress::SIZE_BOUND <= builder.frame_space_remaining()
-5633 {
-5634 if let Some(added_address) = space.pending.add_address.pop_last() {
-5635 builder.write_frame(added_address, stats);
-5636 } else {
-5637 break;
-5638 }
-5639 }
-5640
-5641 // REMOVE_ADDRESS
-5642 while space_id == SpaceId::Data
-5643 && frame::RemoveAddress::SIZE_BOUND <= builder.frame_space_remaining()
-5644 {
-5645 if let Some(removed_address) = space.pending.remove_address.pop_last() {
-5646 builder.write_frame(removed_address, stats);
-5647 } else {
-5648 break;
-5649 }
-5650 }
-5651 }
-5652
-5653 /// Write pending ACKs into a buffer
-5654 fn populate_acks<'a, 'b>(
-5655 now: Instant,
-5656 receiving_ecn: bool,
-5657 path_id: PathId,
-5658 space_id: SpaceId,
-5659 space: &mut PacketSpace,
-5660 is_multipath_negotiated: bool,
-5661 builder: &mut PacketBuilder<'a, 'b>,
-5662 stats: &mut FrameStats,
-5663 ) {
-5664 // 0-RTT packets must never carry acks (which would have to be of handshake packets)
-5665 debug_assert!(space.crypto.is_some(), "tried to send ACK in 0-RTT");
-5666
-5667 debug_assert!(
-5668 is_multipath_negotiated || path_id == PathId::ZERO,
-5669 "Only PathId::ZERO allowed without multipath (have {path_id:?})"
-5670 );
-5671 if is_multipath_negotiated {
-5672 debug_assert!(
-5673 space_id == SpaceId::Data || path_id == PathId::ZERO,
-5674 "path acks must be sent in 1RTT space (have {space_id:?})"
-5675 );
-5676 }
-5677
-5678 let pns = space.for_path(path_id);
-5679 let ranges = pns.pending_acks.ranges();
-5680 debug_assert!(!ranges.is_empty(), "can not send empty ACK range");
-5681 let ecn = if receiving_ecn {
-5682 Some(&pns.ecn_counters)
-5683 } else {
-5684 None
-5685 };
-5686
-5687 let delay_micros = pns.pending_acks.ack_delay(now).as_micros() as u64;
-5688 // TODO: This should come from `TransportConfig` if that gets configurable.
-5689 let ack_delay_exp = TransportParameters::default().ack_delay_exponent;
-5690 let delay = delay_micros >> ack_delay_exp.into_inner();
-5691
-5692 if is_multipath_negotiated && space_id == SpaceId::Data {
-5693 if !ranges.is_empty() {
-5694 let frame = frame::PathAck::encoder(path_id, delay, ranges, ecn);
-5695 builder.write_frame(frame, stats);
-5696 }
-5697 } else {
-5698 builder.write_frame(frame::Ack::encoder(delay, ranges, ecn), stats);
-5699 }
-5700 }
-5701
-5702 fn close_common(&mut self) {
-5703 trace!("connection closed");
-5704 self.timers.reset();
-5705 }
-5706
-5707 fn set_close_timer(&mut self, now: Instant) {
-5708 // QUIC-MULTIPATH § 2.6 Connection Closure: draining for 3*PTO with PTO the max of
-5709 // the PTO for all paths.
-5710 let pto_max = self.pto_max_path(self.highest_space, true);
-5711 self.timers.set(
-5712 Timer::Conn(ConnTimer::Close),
-5713 now + 3 * pto_max,
-5714 self.qlog.with_time(now),
-5715 );
-5716 }
-5717
-5718 /// Handle transport parameters received from the peer
-5719 ///
-5720 /// *rem_cid* and *loc_cid* are the source and destination CIDs respectively of the
-5721 /// *packet into which the transport parameters arrived.
-5722 fn handle_peer_params(
-5723 &mut self,
-5724 params: TransportParameters,
-5725 loc_cid: ConnectionId,
-5726 rem_cid: ConnectionId,
-5727 now: Instant,
-5728 ) -> Result<(), TransportError> {
-5729 if Some(self.orig_rem_cid) != params.initial_src_cid
-5730 || (self.side.is_client()
-5731 && (Some(self.initial_dst_cid) != params.original_dst_cid
-5732 || self.retry_src_cid != params.retry_src_cid))
-5733 {
-5734 return Err(TransportError::TRANSPORT_PARAMETER_ERROR(
-5735 "CID authentication failure",
-5736 ));
-5737 }
-5738 if params.initial_max_path_id.is_some() && (loc_cid.is_empty() || rem_cid.is_empty()) {
-5739 return Err(TransportError::PROTOCOL_VIOLATION(
-5740 "multipath must not use zero-length CIDs",
-5741 ));
-5742 }
-5743
-5744 self.set_peer_params(params);
-5745 self.qlog.emit_peer_transport_params_received(self, now);
-5746
-5747 Ok(())
-5748 }
-5749
-5750 fn set_peer_params(&mut self, params: TransportParameters) {
-5751 self.streams.set_params(¶ms);
-5752 self.idle_timeout =
-5753 negotiate_max_idle_timeout(self.config.max_idle_timeout, Some(params.max_idle_timeout));
-5754 trace!("negotiated max idle timeout {:?}", self.idle_timeout);
-5755
-5756 if let Some(ref info) = params.preferred_address {
-5757 // During the handshake PathId::ZERO exists.
-5758 self.rem_cids.get_mut(&PathId::ZERO).expect("not yet abandoned").insert(frame::NewConnectionId {
-5759 path_id: None,
-5760 sequence: 1,
-5761 id: info.connection_id,
-5762 reset_token: info.stateless_reset_token,
-5763 retire_prior_to: 0,
-5764 })
-5765 .expect(
-5766 "preferred address CID is the first received, and hence is guaranteed to be legal",
-5767 );
-5768 let remote = self.path_data(PathId::ZERO).network_path.remote;
-5769 self.set_reset_token(PathId::ZERO, remote, info.stateless_reset_token);
-5770 }
-5771 self.ack_frequency.peer_max_ack_delay = get_max_ack_delay(¶ms);
-5772
-5773 let mut multipath_enabled = None;
-5774 if let (Some(local_max_path_id), Some(remote_max_path_id)) = (
-5775 self.config.get_initial_max_path_id(),
-5776 params.initial_max_path_id,
-5777 ) {
-5778 // multipath is enabled, register the local and remote maximums
-5779 self.local_max_path_id = local_max_path_id;
-5780 self.remote_max_path_id = remote_max_path_id;
-5781 let initial_max_path_id = local_max_path_id.min(remote_max_path_id);
-5782 debug!(%initial_max_path_id, "multipath negotiated");
-5783 multipath_enabled = Some(initial_max_path_id);
-5784 }
-5785
-5786 if let Some((max_locally_allowed_remote_addresses, max_remotely_allowed_remote_addresses)) =
-5787 self.config
-5788 .max_remote_nat_traversal_addresses
-5789 .zip(params.max_remote_nat_traversal_addresses)
-5790 {
-5791 if let Some(max_initial_paths) =
-5792 multipath_enabled.map(|path_id| path_id.saturating_add(1u8))
-5793 {
-5794 let max_local_addresses = max_remotely_allowed_remote_addresses.get();
-5795 let max_remote_addresses = max_locally_allowed_remote_addresses.get();
-5796 self.iroh_hp =
-5797 iroh_hp::State::new(max_remote_addresses, max_local_addresses, self.side());
-5798 debug!(
-5799 %max_remote_addresses, %max_local_addresses,
-5800 "iroh hole punching negotiated"
-5801 );
-5802
-5803 match self.side() {
-5804 Side::Client => {
-5805 if max_initial_paths.as_u32() < max_remote_addresses as u32 + 1 {
-5806 // in this case the client might try to open `max_remote_addresses` new
-5807 // paths, but the current multipath configuration will not allow it
-5808 warn!(%max_initial_paths, %max_remote_addresses, "local client configuration might cause nat traversal issues")
-5809 } else if max_local_addresses as u64
-5810 > params.active_connection_id_limit.into_inner()
-5811 {
-5812 // the server allows us to send at most `params.active_connection_id_limit`
-5813 // but they might need at least `max_local_addresses` to effectively send
-5814 // `PATH_CHALLENGE` frames to each advertised local address
-5815 warn!(%max_local_addresses, remote_cid_limit=%params.active_connection_id_limit.into_inner(), "remote server configuration might cause nat traversal issues")
-5816 }
-5817 }
-5818 Side::Server => {
-5819 if (max_initial_paths.as_u32() as u64) < crate::LOC_CID_COUNT {
-5820 warn!(%max_initial_paths, local_cid_limit=%crate::LOC_CID_COUNT, "local server configuration might cause nat traversal issues")
-5821 }
-5822 }
-5823 }
-5824 } else {
-5825 debug!("iroh nat traversal enabled for both endpoints, but multipath is missing")
-5826 }
+5601 }
+5602 }
+5603
+5604 // MAX_PATH_ID
+5605 if space_id == SpaceId::Data
+5606 && space.pending.max_path_id
+5607 && frame::MaxPathId::SIZE_BOUND <= builder.frame_space_remaining()
+5608 {
+5609 let frame = frame::MaxPathId(self.local_max_path_id);
+5610 builder.write_frame(frame, stats);
+5611 space.pending.max_path_id = false;
+5612 }
+5613
+5614 // PATHS_BLOCKED
+5615 if space_id == SpaceId::Data
+5616 && space.pending.paths_blocked
+5617 && frame::PathsBlocked::SIZE_BOUND <= builder.frame_space_remaining()
+5618 {
+5619 let frame = frame::PathsBlocked(self.remote_max_path_id);
+5620 builder.write_frame(frame, stats);
+5621 space.pending.paths_blocked = false;
+5622 }
+5623
+5624 // PATH_CIDS_BLOCKED
+5625 while space_id == SpaceId::Data
+5626 && frame::PathCidsBlocked::SIZE_BOUND <= builder.frame_space_remaining()
+5627 {
+5628 let Some(path_id) = space.pending.path_cids_blocked.pop_first() else {
+5629 break;
+5630 };
+5631 let next_seq = match self.rem_cids.get(&path_id) {
+5632 Some(cid_queue) => VarInt(cid_queue.active_seq() + 1),
+5633 None => VarInt(0),
+5634 };
+5635 let frame = frame::PathCidsBlocked { path_id, next_seq };
+5636 builder.write_frame(frame, stats);
+5637 }
+5638
+5639 // RESET_STREAM, STOP_SENDING, MAX_DATA, MAX_STREAM_DATA, MAX_STREAMS
+5640 if space_id == SpaceId::Data {
+5641 self.streams
+5642 .write_control_frames(builder, &mut space.pending, stats);
+5643 }
+5644
+5645 // NEW_CONNECTION_ID
+5646 let cid_len = self
+5647 .local_cid_state
+5648 .values()
+5649 .map(|cid_state| cid_state.cid_len())
+5650 .max()
+5651 .expect("some local CID state must exist");
+5652 let new_cid_size_bound =
+5653 frame::NewConnectionId::size_bound(is_multipath_negotiated, cid_len);
+5654 while !path_exclusive_only && builder.frame_space_remaining() > new_cid_size_bound {
+5655 let issued = match space.pending.new_cids.pop() {
+5656 Some(x) => x,
+5657 None => break,
+5658 };
+5659 let retire_prior_to = self
+5660 .local_cid_state
+5661 .get(&issued.path_id)
+5662 .map(|cid_state| cid_state.retire_prior_to())
+5663 .unwrap_or_else(|| panic!("missing local CID state for path={}", issued.path_id));
+5664
+5665 let cid_path_id = match is_multipath_negotiated {
+5666 true => Some(issued.path_id),
+5667 false => {
+5668 debug_assert_eq!(issued.path_id, PathId::ZERO);
+5669 None
+5670 }
+5671 };
+5672 let frame = frame::NewConnectionId {
+5673 path_id: cid_path_id,
+5674 sequence: issued.sequence,
+5675 retire_prior_to,
+5676 id: issued.id,
+5677 reset_token: issued.reset_token,
+5678 };
+5679 builder.write_frame(frame, stats);
+5680 }
+5681
+5682 // RETIRE_CONNECTION_ID
+5683 let retire_cid_bound = frame::RetireConnectionId::size_bound(is_multipath_negotiated);
+5684 while !path_exclusive_only && builder.frame_space_remaining() > retire_cid_bound {
+5685 let (path_id, sequence) = match space.pending.retire_cids.pop() {
+5686 Some((PathId::ZERO, seq)) if !is_multipath_negotiated => (None, seq),
+5687 Some((path_id, seq)) => (Some(path_id), seq),
+5688 None => break,
+5689 };
+5690 let frame = frame::RetireConnectionId { path_id, sequence };
+5691 builder.write_frame(frame, stats);
+5692 }
+5693
+5694 // DATAGRAM
+5695 let mut sent_datagrams = false;
+5696 while !path_exclusive_only
+5697 && builder.frame_space_remaining() > Datagram::SIZE_BOUND
+5698 && space_id == SpaceId::Data
+5699 {
+5700 match self.datagrams.write(builder, stats) {
+5701 true => {
+5702 sent_datagrams = true;
+5703 }
+5704 false => break,
+5705 }
+5706 }
+5707 if self.datagrams.send_blocked && sent_datagrams {
+5708 self.events.push_back(Event::DatagramsUnblocked);
+5709 self.datagrams.send_blocked = false;
+5710 }
+5711
+5712 let path = &mut self.paths.get_mut(&path_id).expect("known path").data;
+5713
+5714 // NEW_TOKEN
+5715 if !path_exclusive_only {
+5716 while let Some(network_path) = space.pending.new_tokens.pop() {
+5717 debug_assert_eq!(space_id, SpaceId::Data);
+5718 let ConnectionSide::Server { server_config } = &self.side else {
+5719 panic!("NEW_TOKEN frames should not be enqueued by clients");
+5720 };
+5721
+5722 if !network_path.is_probably_same_path(&path.network_path) {
+5723 // NEW_TOKEN frames contain tokens bound to a client's IP address, and are only
+5724 // useful if used from the same IP address. Thus, we abandon enqueued NEW_TOKEN
+5725 // frames upon an path change. Instead, when the new path becomes validated,
+5726 // NEW_TOKEN frames may be enqueued for the new path instead.
+5727 continue;
+5728 }
+5729
+5730 let token = Token::new(
+5731 TokenPayload::Validation {
+5732 ip: network_path.remote.ip(),
+5733 issued: server_config.time_source.now(),
+5734 },
+5735 &mut self.rng,
+5736 );
+5737 let new_token = NewToken {
+5738 token: token.encode(&*server_config.token_key).into(),
+5739 };
+5740
+5741 if builder.frame_space_remaining() < new_token.size() {
+5742 space.pending.new_tokens.push(network_path);
+5743 break;
+5744 }
+5745
+5746 builder.write_frame(new_token, stats);
+5747 builder.retransmits_mut().new_tokens.push(network_path);
+5748 }
+5749 }
+5750
+5751 // STREAM
+5752 if !path_exclusive_only && space_id == SpaceId::Data {
+5753 self.streams
+5754 .write_stream_frames(builder, self.config.send_fairness, stats);
+5755 }
+5756
+5757 // ADD_ADDRESS
+5758 // TODO(@divma): check if we need to do path exclusive filters
+5759 while space_id == SpaceId::Data
+5760 && frame::AddAddress::SIZE_BOUND <= builder.frame_space_remaining()
+5761 {
+5762 if let Some(added_address) = space.pending.add_address.pop_last() {
+5763 builder.write_frame(added_address, stats);
+5764 } else {
+5765 break;
+5766 }
+5767 }
+5768
+5769 // REMOVE_ADDRESS
+5770 while space_id == SpaceId::Data
+5771 && frame::RemoveAddress::SIZE_BOUND <= builder.frame_space_remaining()
+5772 {
+5773 if let Some(removed_address) = space.pending.remove_address.pop_last() {
+5774 builder.write_frame(removed_address, stats);
+5775 } else {
+5776 break;
+5777 }
+5778 }
+5779 }
+5780
+5781 /// Write pending ACKs into a buffer
+5782 fn populate_acks<'a, 'b>(
+5783 now: Instant,
+5784 receiving_ecn: bool,
+5785 path_id: PathId,
+5786 space_id: SpaceId,
+5787 space: &mut PacketSpace,
+5788 is_multipath_negotiated: bool,
+5789 builder: &mut PacketBuilder<'a, 'b>,
+5790 stats: &mut FrameStats,
+5791 ) {
+5792 // 0-RTT packets must never carry acks (which would have to be of handshake packets)
+5793 debug_assert!(space.crypto.is_some(), "tried to send ACK in 0-RTT");
+5794
+5795 debug_assert!(
+5796 is_multipath_negotiated || path_id == PathId::ZERO,
+5797 "Only PathId::ZERO allowed without multipath (have {path_id:?})"
+5798 );
+5799 if is_multipath_negotiated {
+5800 debug_assert!(
+5801 space_id == SpaceId::Data || path_id == PathId::ZERO,
+5802 "path acks must be sent in 1RTT space (have {space_id:?})"
+5803 );
+5804 }
+5805
+5806 let pns = space.for_path(path_id);
+5807 let ranges = pns.pending_acks.ranges();
+5808 debug_assert!(!ranges.is_empty(), "can not send empty ACK range");
+5809 let ecn = if receiving_ecn {
+5810 Some(&pns.ecn_counters)
+5811 } else {
+5812 None
+5813 };
+5814
+5815 let delay_micros = pns.pending_acks.ack_delay(now).as_micros() as u64;
+5816 // TODO: This should come from `TransportConfig` if that gets configurable.
+5817 let ack_delay_exp = TransportParameters::default().ack_delay_exponent;
+5818 let delay = delay_micros >> ack_delay_exp.into_inner();
+5819
+5820 if is_multipath_negotiated && space_id == SpaceId::Data {
+5821 if !ranges.is_empty() {
+5822 let frame = frame::PathAck::encoder(path_id, delay, ranges, ecn);
+5823 builder.write_frame(frame, stats);
+5824 }
+5825 } else {
+5826 builder.write_frame(frame::Ack::encoder(delay, ranges, ecn), stats);
5827 }
-5828
-5829 self.peer_params = params;
-5830 let peer_max_udp_payload_size =
-5831 u16::try_from(self.peer_params.max_udp_payload_size.into_inner()).unwrap_or(u16::MAX);
-5832 self.path_data_mut(PathId::ZERO)
-5833 .mtud
-5834 .on_peer_max_udp_payload_size_received(peer_max_udp_payload_size);
-5835 }
-5836
-5837 /// Decrypts a packet, returning the packet number on success
-5838 fn decrypt_packet(
-5839 &mut self,
-5840 now: Instant,
-5841 path_id: PathId,
-5842 packet: &mut Packet,
-5843 ) -> Result<Option<u64>, Option<TransportError>> {
-5844 let result = packet_crypto::decrypt_packet_body(
-5845 packet,
-5846 path_id,
-5847 &self.spaces,
-5848 self.zero_rtt_crypto.as_ref(),
-5849 self.key_phase,
-5850 self.prev_crypto.as_ref(),
-5851 self.next_crypto.as_ref(),
-5852 )?;
-5853
-5854 let result = match result {
-5855 Some(r) => r,
-5856 None => return Ok(None),
-5857 };
-5858
-5859 if result.outgoing_key_update_acked {
-5860 if let Some(prev) = self.prev_crypto.as_mut() {
-5861 prev.end_packet = Some((result.number, now));
-5862 self.set_key_discard_timer(now, packet.header.space());
-5863 }
-5864 }
-5865
-5866 if result.incoming_key_update {
-5867 trace!("key update authenticated");
-5868 self.update_keys(Some((result.number, now)), true);
-5869 self.set_key_discard_timer(now, packet.header.space());
+5828 }
+5829
+5830 fn close_common(&mut self) {
+5831 trace!("connection closed");
+5832 self.timers.reset();
+5833 }
+5834
+5835 fn set_close_timer(&mut self, now: Instant) {
+5836 // QUIC-MULTIPATH § 2.6 Connection Closure: draining for 3*PTO with PTO the max of
+5837 // the PTO for all paths.
+5838 let pto_max = self.pto_max_path(self.highest_space, true);
+5839 self.timers.set(
+5840 Timer::Conn(ConnTimer::Close),
+5841 now + 3 * pto_max,
+5842 self.qlog.with_time(now),
+5843 );
+5844 }
+5845
+5846 /// Handle transport parameters received from the peer
+5847 ///
+5848 /// *rem_cid* and *loc_cid* are the source and destination CIDs respectively of the
+5849 /// *packet into which the transport parameters arrived.
+5850 fn handle_peer_params(
+5851 &mut self,
+5852 params: TransportParameters,
+5853 loc_cid: ConnectionId,
+5854 rem_cid: ConnectionId,
+5855 now: Instant,
+5856 ) -> Result<(), TransportError> {
+5857 if Some(self.orig_rem_cid) != params.initial_src_cid
+5858 || (self.side.is_client()
+5859 && (Some(self.initial_dst_cid) != params.original_dst_cid
+5860 || self.retry_src_cid != params.retry_src_cid))
+5861 {
+5862 return Err(TransportError::TRANSPORT_PARAMETER_ERROR(
+5863 "CID authentication failure",
+5864 ));
+5865 }
+5866 if params.initial_max_path_id.is_some() && (loc_cid.is_empty() || rem_cid.is_empty()) {
+5867 return Err(TransportError::PROTOCOL_VIOLATION(
+5868 "multipath must not use zero-length CIDs",
+5869 ));
5870 }
5871
-5872 Ok(Some(result.number))
-5873 }
+5872 self.set_peer_params(params);
+5873 self.qlog.emit_peer_transport_params_received(self, now);
5874
-5875 fn update_keys(&mut self, end_packet: Option<(u64, Instant)>, remote: bool) {
-5876 trace!("executing key update");
-5877 // Generate keys for the key phase after the one we're switching to, store them in
-5878 // `next_crypto`, make the contents of `next_crypto` current, and move the current keys into
-5879 // `prev_crypto`.
-5880 let new = self
-5881 .crypto
-5882 .next_1rtt_keys()
-5883 .expect("only called for `Data` packets");
-5884 self.key_phase_size = new
-5885 .local
-5886 .confidentiality_limit()
-5887 .saturating_sub(KEY_UPDATE_MARGIN);
-5888 let old = mem::replace(
-5889 &mut self.spaces[SpaceId::Data]
-5890 .crypto
-5891 .as_mut()
-5892 .unwrap() // safe because update_keys() can only be triggered by short packets
-5893 .packet,
-5894 mem::replace(self.next_crypto.as_mut().unwrap(), new),
-5895 );
-5896 self.spaces[SpaceId::Data]
-5897 .iter_paths_mut()
-5898 .for_each(|s| s.sent_with_keys = 0);
-5899 self.prev_crypto = Some(PrevCrypto {
-5900 crypto: old,
-5901 end_packet,
-5902 update_unacked: remote,
-5903 });
-5904 self.key_phase = !self.key_phase;
-5905 }
-5906
-5907 fn peer_supports_ack_frequency(&self) -> bool {
-5908 self.peer_params.min_ack_delay.is_some()
-5909 }
-5910
-5911 /// Send an IMMEDIATE_ACK frame to the remote endpoint
-5912 ///
-5913 /// According to the spec, this will result in an error if the remote endpoint does not support
-5914 /// the Acknowledgement Frequency extension
-5915 pub(crate) fn immediate_ack(&mut self, path_id: PathId) {
-5916 debug_assert_eq!(
-5917 self.highest_space,
-5918 SpaceId::Data,
-5919 "immediate ack must be written in the data space"
-5920 );
-5921 self.spaces[self.highest_space]
-5922 .for_path(path_id)
-5923 .immediate_ack_pending = true;
-5924 }
-5925
-5926 /// Decodes a packet, returning its decrypted payload, so it can be inspected in tests
-5927 #[cfg(test)]
-5928 pub(crate) fn decode_packet(&self, event: &ConnectionEvent) -> Option<Vec<u8>> {
-5929 let (path_id, first_decode, remaining) = match &event.0 {
-5930 ConnectionEventInner::Datagram(DatagramConnectionEvent {
-5931 path_id,
-5932 first_decode,
-5933 remaining,
-5934 ..
-5935 }) => (path_id, first_decode, remaining),
-5936 _ => return None,
-5937 };
-5938
-5939 if remaining.is_some() {
-5940 panic!("Packets should never be coalesced in tests");
-5941 }
-5942
-5943 let decrypted_header = packet_crypto::unprotect_header(
-5944 first_decode.clone(),
-5945 &self.spaces,
-5946 self.zero_rtt_crypto.as_ref(),
-5947 self.peer_params.stateless_reset_token,
-5948 )?;
-5949
-5950 let mut packet = decrypted_header.packet?;
-5951 packet_crypto::decrypt_packet_body(
-5952 &mut packet,
-5953 *path_id,
-5954 &self.spaces,
-5955 self.zero_rtt_crypto.as_ref(),
-5956 self.key_phase,
-5957 self.prev_crypto.as_ref(),
-5958 self.next_crypto.as_ref(),
-5959 )
-5960 .ok()?;
-5961
-5962 Some(packet.payload.to_vec())
+5875 Ok(())
+5876 }
+5877
+5878 fn set_peer_params(&mut self, params: TransportParameters) {
+5879 self.streams.set_params(¶ms);
+5880 self.idle_timeout =
+5881 negotiate_max_idle_timeout(self.config.max_idle_timeout, Some(params.max_idle_timeout));
+5882 trace!("negotiated max idle timeout {:?}", self.idle_timeout);
+5883
+5884 if let Some(ref info) = params.preferred_address {
+5885 // During the handshake PathId::ZERO exists.
+5886 self.rem_cids.get_mut(&PathId::ZERO).expect("not yet abandoned").insert(frame::NewConnectionId {
+5887 path_id: None,
+5888 sequence: 1,
+5889 id: info.connection_id,
+5890 reset_token: info.stateless_reset_token,
+5891 retire_prior_to: 0,
+5892 })
+5893 .expect(
+5894 "preferred address CID is the first received, and hence is guaranteed to be legal",
+5895 );
+5896 let remote = self.path_data(PathId::ZERO).network_path.remote;
+5897 self.set_reset_token(PathId::ZERO, remote, info.stateless_reset_token);
+5898 }
+5899 self.ack_frequency.peer_max_ack_delay = get_max_ack_delay(¶ms);
+5900
+5901 let mut multipath_enabled = None;
+5902 if let (Some(local_max_path_id), Some(remote_max_path_id)) = (
+5903 self.config.get_initial_max_path_id(),
+5904 params.initial_max_path_id,
+5905 ) {
+5906 // multipath is enabled, register the local and remote maximums
+5907 self.local_max_path_id = local_max_path_id;
+5908 self.remote_max_path_id = remote_max_path_id;
+5909 let initial_max_path_id = local_max_path_id.min(remote_max_path_id);
+5910 debug!(%initial_max_path_id, "multipath negotiated");
+5911 multipath_enabled = Some(initial_max_path_id);
+5912 }
+5913
+5914 if let Some((max_locally_allowed_remote_addresses, max_remotely_allowed_remote_addresses)) =
+5915 self.config
+5916 .max_remote_nat_traversal_addresses
+5917 .zip(params.max_remote_nat_traversal_addresses)
+5918 {
+5919 if let Some(max_initial_paths) =
+5920 multipath_enabled.map(|path_id| path_id.saturating_add(1u8))
+5921 {
+5922 let max_local_addresses = max_remotely_allowed_remote_addresses.get();
+5923 let max_remote_addresses = max_locally_allowed_remote_addresses.get();
+5924 self.iroh_hp =
+5925 iroh_hp::State::new(max_remote_addresses, max_local_addresses, self.side());
+5926 debug!(
+5927 %max_remote_addresses, %max_local_addresses,
+5928 "iroh hole punching negotiated"
+5929 );
+5930
+5931 match self.side() {
+5932 Side::Client => {
+5933 if max_initial_paths.as_u32() < max_remote_addresses as u32 + 1 {
+5934 // in this case the client might try to open `max_remote_addresses` new
+5935 // paths, but the current multipath configuration will not allow it
+5936 warn!(%max_initial_paths, %max_remote_addresses, "local client configuration might cause nat traversal issues")
+5937 } else if max_local_addresses as u64
+5938 > params.active_connection_id_limit.into_inner()
+5939 {
+5940 // the server allows us to send at most `params.active_connection_id_limit`
+5941 // but they might need at least `max_local_addresses` to effectively send
+5942 // `PATH_CHALLENGE` frames to each advertised local address
+5943 warn!(%max_local_addresses, remote_cid_limit=%params.active_connection_id_limit.into_inner(), "remote server configuration might cause nat traversal issues")
+5944 }
+5945 }
+5946 Side::Server => {
+5947 if (max_initial_paths.as_u32() as u64) < crate::LOC_CID_COUNT {
+5948 warn!(%max_initial_paths, local_cid_limit=%crate::LOC_CID_COUNT, "local server configuration might cause nat traversal issues")
+5949 }
+5950 }
+5951 }
+5952 } else {
+5953 debug!("iroh nat traversal enabled for both endpoints, but multipath is missing")
+5954 }
+5955 }
+5956
+5957 self.peer_params = params;
+5958 let peer_max_udp_payload_size =
+5959 u16::try_from(self.peer_params.max_udp_payload_size.into_inner()).unwrap_or(u16::MAX);
+5960 self.path_data_mut(PathId::ZERO)
+5961 .mtud
+5962 .on_peer_max_udp_payload_size_received(peer_max_udp_payload_size);
5963 }
5964
-5965 /// The number of bytes of packets containing retransmittable frames that have not been
-5966 /// acknowledged or declared lost.
-5967 #[cfg(test)]
-5968 pub(crate) fn bytes_in_flight(&self) -> u64 {
-5969 // TODO(@divma): consider including for multipath?
-5970 self.path_data(PathId::ZERO).in_flight.bytes
-5971 }
-5972
-5973 /// Number of bytes worth of non-ack-only packets that may be sent
-5974 #[cfg(test)]
-5975 pub(crate) fn congestion_window(&self) -> u64 {
-5976 let path = self.path_data(PathId::ZERO);
-5977 path.congestion
-5978 .window()
-5979 .saturating_sub(path.in_flight.bytes)
-5980 }
+5965 /// Decrypts a packet, returning the packet number on success
+5966 fn decrypt_packet(
+5967 &mut self,
+5968 now: Instant,
+5969 path_id: PathId,
+5970 packet: &mut Packet,
+5971 ) -> Result<Option<u64>, Option<TransportError>> {
+5972 let result = packet_crypto::decrypt_packet_body(
+5973 packet,
+5974 path_id,
+5975 &self.spaces,
+5976 self.zero_rtt_crypto.as_ref(),
+5977 self.key_phase,
+5978 self.prev_crypto.as_ref(),
+5979 self.next_crypto.as_ref(),
+5980 )?;
5981
-5982 /// Whether no timers but keepalive, idle, rtt, pushnewcid, and key discard are running
-5983 #[cfg(test)]
-5984 pub(crate) fn is_idle(&self) -> bool {
-5985 let current_timers = self.timers.values();
-5986 current_timers
-5987 .into_iter()
-5988 .filter(|(timer, _)| {
-5989 !matches!(
-5990 timer,
-5991 Timer::Conn(ConnTimer::KeepAlive)
-5992 | Timer::PerPath(_, PathTimer::PathKeepAlive)
-5993 | Timer::Conn(ConnTimer::PushNewCid)
-5994 | Timer::Conn(ConnTimer::KeyDiscard)
-5995 )
-5996 })
-5997 .min_by_key(|(_, time)| *time)
-5998 .is_none_or(|(timer, _)| timer == Timer::Conn(ConnTimer::Idle))
-5999 }
-6000
-6001 /// Whether explicit congestion notification is in use on outgoing packets.
-6002 #[cfg(test)]
-6003 pub(crate) fn using_ecn(&self) -> bool {
-6004 self.path_data(PathId::ZERO).sending_ecn
-6005 }
-6006
-6007 /// The number of received bytes in the current path
-6008 #[cfg(test)]
-6009 pub(crate) fn total_recvd(&self) -> u64 {
-6010 self.path_data(PathId::ZERO).total_recvd
-6011 }
-6012
-6013 #[cfg(test)]
-6014 pub(crate) fn active_local_cid_seq(&self) -> (u64, u64) {
-6015 self.local_cid_state
-6016 .get(&PathId::ZERO)
-6017 .unwrap()
-6018 .active_seq()
-6019 }
-6020
-6021 #[cfg(test)]
-6022 #[track_caller]
-6023 pub(crate) fn active_local_path_cid_seq(&self, path_id: u32) -> (u64, u64) {
-6024 self.local_cid_state
-6025 .get(&PathId(path_id))
-6026 .unwrap()
-6027 .active_seq()
-6028 }
-6029
-6030 /// Instruct the peer to replace previously issued CIDs by sending a NEW_CONNECTION_ID frame
-6031 /// with updated `retire_prior_to` field set to `v`
-6032 #[cfg(test)]
-6033 pub(crate) fn rotate_local_cid(&mut self, v: u64, now: Instant) {
-6034 let n = self
-6035 .local_cid_state
-6036 .get_mut(&PathId::ZERO)
-6037 .unwrap()
-6038 .assign_retire_seq(v);
-6039 self.endpoint_events
-6040 .push_back(EndpointEventInner::NeedIdentifiers(PathId::ZERO, now, n));
-6041 }
-6042
-6043 /// Check the current active remote CID sequence for `PathId::ZERO`
-6044 #[cfg(test)]
-6045 pub(crate) fn active_rem_cid_seq(&self) -> u64 {
-6046 self.rem_cids.get(&PathId::ZERO).unwrap().active_seq()
-6047 }
-6048
-6049 /// Returns the detected maximum udp payload size for the current path
-6050 #[cfg(test)]
-6051 pub(crate) fn path_mtu(&self, path_id: PathId) -> u16 {
-6052 self.path_data(path_id).current_mtu()
-6053 }
-6054
-6055 /// Triggers path validation on all paths
-6056 #[cfg(test)]
-6057 pub(crate) fn trigger_path_validation(&mut self) {
-6058 for path in self.paths.values_mut() {
-6059 path.data.send_new_challenge = true;
-6060 }
-6061 }
-6062
-6063 /// Whether we have 1-RTT data to send
-6064 ///
-6065 /// This checks for frames that can only be sent in the data space (1-RTT):
-6066 /// - Pending PATH_CHALLENGE frames on the active and previous path if just migrated.
-6067 /// - Pending PATH_RESPONSE frames.
-6068 /// - Pending data to send in STREAM frames.
-6069 /// - Pending DATAGRAM frames to send.
-6070 ///
-6071 /// See also [`PacketSpace::can_send`] which keeps track of all other frame types that
-6072 /// may need to be sent.
-6073 fn can_send_1rtt(&self, path_id: PathId, max_size: usize) -> SendableFrames {
-6074 let path_exclusive = self.paths.get(&path_id).is_some_and(|path| {
-6075 path.data.send_new_challenge
-6076 || path
-6077 .prev
-6078 .as_ref()
-6079 .is_some_and(|(_, path)| path.send_new_challenge)
-6080 || !path.data.path_responses.is_empty()
-6081 });
-6082 let other = self.streams.can_send_stream_data()
-6083 || self
-6084 .datagrams
-6085 .outgoing
-6086 .front()
-6087 .is_some_and(|x| x.size(true) <= max_size);
-6088 SendableFrames {
-6089 acks: false,
-6090 other,
-6091 close: false,
-6092 path_exclusive,
-6093 }
-6094 }
-6095
-6096 /// Terminate the connection instantly, without sending a close packet
-6097 fn kill(&mut self, reason: ConnectionError) {
-6098 self.close_common();
-6099 self.state.move_to_drained(Some(reason));
-6100 self.endpoint_events.push_back(EndpointEventInner::Drained);
-6101 }
-6102
-6103 /// Storage size required for the largest packet that can be transmitted on all currently
-6104 /// available paths
-6105 ///
-6106 /// Buffers passed to [`Connection::poll_transmit`] should be at least this large.
-6107 ///
-6108 /// When multipath is enabled, this value is the minimum MTU across all available paths.
-6109 pub fn current_mtu(&self) -> u16 {
-6110 self.paths
-6111 .iter()
-6112 .filter(|&(path_id, _path_state)| !self.abandoned_paths.contains(path_id))
-6113 .map(|(_path_id, path_state)| path_state.data.current_mtu())
-6114 .min()
-6115 .expect("There is always at least one available path")
-6116 }
-6117
-6118 /// Size of non-frame data for a 1-RTT packet
-6119 ///
-6120 /// Quantifies space consumed by the QUIC header and AEAD tag. All other bytes in a packet are
-6121 /// frames. Changes if the length of the remote connection ID changes, which is expected to be
-6122 /// rare. If `pn` is specified, may additionally change unpredictably due to variations in
-6123 /// latency and packet loss.
-6124 fn predict_1rtt_overhead(&mut self, pn: u64, path: PathId) -> usize {
-6125 let pn_len = PacketNumber::new(
-6126 pn,
-6127 self.spaces[SpaceId::Data]
-6128 .for_path(path)
-6129 .largest_acked_packet
-6130 .unwrap_or(0),
-6131 )
-6132 .len();
-6133
-6134 // 1 byte for flags
-6135 1 + self
-6136 .rem_cids
-6137 .get(&path)
-6138 .map(|cids| cids.active().len())
-6139 .unwrap_or(20) // Max CID len in QUIC v1
-6140 + pn_len
-6141 + self.tag_len_1rtt()
-6142 }
-6143
-6144 fn predict_1rtt_overhead_no_pn(&self) -> usize {
-6145 let pn_len = 4;
-6146
-6147 let cid_len = self
-6148 .rem_cids
-6149 .values()
-6150 .map(|cids| cids.active().len())
-6151 .max()
-6152 .unwrap_or(20); // Max CID len in QUIC v1
-6153
-6154 // 1 byte for flags
-6155 1 + cid_len + pn_len + self.tag_len_1rtt()
+5982 let result = match result {
+5983 Some(r) => r,
+5984 None => return Ok(None),
+5985 };
+5986
+5987 if result.outgoing_key_update_acked {
+5988 if let Some(prev) = self.prev_crypto.as_mut() {
+5989 prev.end_packet = Some((result.number, now));
+5990 self.set_key_discard_timer(now, packet.header.space());
+5991 }
+5992 }
+5993
+5994 if result.incoming_key_update {
+5995 trace!("key update authenticated");
+5996 self.update_keys(Some((result.number, now)), true);
+5997 self.set_key_discard_timer(now, packet.header.space());
+5998 }
+5999
+6000 Ok(Some(result.number))
+6001 }
+6002
+6003 fn update_keys(&mut self, end_packet: Option<(u64, Instant)>, remote: bool) {
+6004 trace!("executing key update");
+6005 // Generate keys for the key phase after the one we're switching to, store them in
+6006 // `next_crypto`, make the contents of `next_crypto` current, and move the current keys into
+6007 // `prev_crypto`.
+6008 let new = self
+6009 .crypto
+6010 .next_1rtt_keys()
+6011 .expect("only called for `Data` packets");
+6012 self.key_phase_size = new
+6013 .local
+6014 .confidentiality_limit()
+6015 .saturating_sub(KEY_UPDATE_MARGIN);
+6016 let old = mem::replace(
+6017 &mut self.spaces[SpaceId::Data]
+6018 .crypto
+6019 .as_mut()
+6020 .unwrap() // safe because update_keys() can only be triggered by short packets
+6021 .packet,
+6022 mem::replace(self.next_crypto.as_mut().unwrap(), new),
+6023 );
+6024 self.spaces[SpaceId::Data]
+6025 .iter_paths_mut()
+6026 .for_each(|s| s.sent_with_keys = 0);
+6027 self.prev_crypto = Some(PrevCrypto {
+6028 crypto: old,
+6029 end_packet,
+6030 update_unacked: remote,
+6031 });
+6032 self.key_phase = !self.key_phase;
+6033 }
+6034
+6035 fn peer_supports_ack_frequency(&self) -> bool {
+6036 self.peer_params.min_ack_delay.is_some()
+6037 }
+6038
+6039 /// Send an IMMEDIATE_ACK frame to the remote endpoint
+6040 ///
+6041 /// According to the spec, this will result in an error if the remote endpoint does not support
+6042 /// the Acknowledgement Frequency extension
+6043 pub(crate) fn immediate_ack(&mut self, path_id: PathId) {
+6044 debug_assert_eq!(
+6045 self.highest_space,
+6046 SpaceId::Data,
+6047 "immediate ack must be written in the data space"
+6048 );
+6049 self.spaces[self.highest_space]
+6050 .for_path(path_id)
+6051 .immediate_ack_pending = true;
+6052 }
+6053
+6054 /// Decodes a packet, returning its decrypted payload, so it can be inspected in tests
+6055 #[cfg(test)]
+6056 pub(crate) fn decode_packet(&self, event: &ConnectionEvent) -> Option<Vec<u8>> {
+6057 let (path_id, first_decode, remaining) = match &event.0 {
+6058 ConnectionEventInner::Datagram(DatagramConnectionEvent {
+6059 path_id,
+6060 first_decode,
+6061 remaining,
+6062 ..
+6063 }) => (path_id, first_decode, remaining),
+6064 _ => return None,
+6065 };
+6066
+6067 if remaining.is_some() {
+6068 panic!("Packets should never be coalesced in tests");
+6069 }
+6070
+6071 let decrypted_header = packet_crypto::unprotect_header(
+6072 first_decode.clone(),
+6073 &self.spaces,
+6074 self.zero_rtt_crypto.as_ref(),
+6075 self.peer_params.stateless_reset_token,
+6076 )?;
+6077
+6078 let mut packet = decrypted_header.packet?;
+6079 packet_crypto::decrypt_packet_body(
+6080 &mut packet,
+6081 *path_id,
+6082 &self.spaces,
+6083 self.zero_rtt_crypto.as_ref(),
+6084 self.key_phase,
+6085 self.prev_crypto.as_ref(),
+6086 self.next_crypto.as_ref(),
+6087 )
+6088 .ok()?;
+6089
+6090 Some(packet.payload.to_vec())
+6091 }
+6092
+6093 /// The number of bytes of packets containing retransmittable frames that have not been
+6094 /// acknowledged or declared lost.
+6095 #[cfg(test)]
+6096 pub(crate) fn bytes_in_flight(&self) -> u64 {
+6097 // TODO(@divma): consider including for multipath?
+6098 self.path_data(PathId::ZERO).in_flight.bytes
+6099 }
+6100
+6101 /// Number of bytes worth of non-ack-only packets that may be sent
+6102 #[cfg(test)]
+6103 pub(crate) fn congestion_window(&self) -> u64 {
+6104 let path = self.path_data(PathId::ZERO);
+6105 path.congestion
+6106 .window()
+6107 .saturating_sub(path.in_flight.bytes)
+6108 }
+6109
+6110 /// Whether no timers but keepalive, idle, rtt, pushnewcid, and key discard are running
+6111 #[cfg(test)]
+6112 pub(crate) fn is_idle(&self) -> bool {
+6113 let current_timers = self.timers.values();
+6114 current_timers
+6115 .into_iter()
+6116 .filter(|(timer, _)| {
+6117 !matches!(
+6118 timer,
+6119 Timer::Conn(ConnTimer::KeepAlive)
+6120 | Timer::PerPath(_, PathTimer::PathKeepAlive)
+6121 | Timer::Conn(ConnTimer::PushNewCid)
+6122 | Timer::Conn(ConnTimer::KeyDiscard)
+6123 )
+6124 })
+6125 .min_by_key(|(_, time)| *time)
+6126 .is_none_or(|(timer, _)| timer == Timer::Conn(ConnTimer::Idle))
+6127 }
+6128
+6129 /// Whether explicit congestion notification is in use on outgoing packets.
+6130 #[cfg(test)]
+6131 pub(crate) fn using_ecn(&self) -> bool {
+6132 self.path_data(PathId::ZERO).sending_ecn
+6133 }
+6134
+6135 /// The number of received bytes in the current path
+6136 #[cfg(test)]
+6137 pub(crate) fn total_recvd(&self) -> u64 {
+6138 self.path_data(PathId::ZERO).total_recvd
+6139 }
+6140
+6141 #[cfg(test)]
+6142 pub(crate) fn active_local_cid_seq(&self) -> (u64, u64) {
+6143 self.local_cid_state
+6144 .get(&PathId::ZERO)
+6145 .unwrap()
+6146 .active_seq()
+6147 }
+6148
+6149 #[cfg(test)]
+6150 #[track_caller]
+6151 pub(crate) fn active_local_path_cid_seq(&self, path_id: u32) -> (u64, u64) {
+6152 self.local_cid_state
+6153 .get(&PathId(path_id))
+6154 .unwrap()
+6155 .active_seq()
6156 }
6157
-6158 fn tag_len_1rtt(&self) -> usize {
-6159 let key = match self.spaces[SpaceId::Data].crypto.as_ref() {
-6160 Some(crypto) => Some(&*crypto.packet.local),
-6161 None => self.zero_rtt_crypto.as_ref().map(|x| &*x.packet),
-6162 };
-6163 // If neither Data nor 0-RTT keys are available, make a reasonable tag length guess. As of
-6164 // this writing, all QUIC cipher suites use 16-byte tags. We could return `None` instead,
-6165 // but that would needlessly prevent sending datagrams during 0-RTT.
-6166 key.map_or(16, |x| x.tag_len())
-6167 }
-6168
-6169 /// Mark the path as validated, and enqueue NEW_TOKEN frames to be sent as appropriate
-6170 fn on_path_validated(&mut self, path_id: PathId) {
-6171 self.path_data_mut(path_id).validated = true;
-6172 let ConnectionSide::Server { server_config } = &self.side else {
-6173 return;
-6174 };
-6175 let network_path = self.path_data(path_id).network_path;
-6176 let new_tokens = &mut self.spaces[SpaceId::Data as usize].pending.new_tokens;
-6177 new_tokens.clear();
-6178 for _ in 0..server_config.validation_token.sent {
-6179 new_tokens.push(network_path);
-6180 }
+6158 /// Instruct the peer to replace previously issued CIDs by sending a NEW_CONNECTION_ID frame
+6159 /// with updated `retire_prior_to` field set to `v`
+6160 #[cfg(test)]
+6161 pub(crate) fn rotate_local_cid(&mut self, v: u64, now: Instant) {
+6162 let n = self
+6163 .local_cid_state
+6164 .get_mut(&PathId::ZERO)
+6165 .unwrap()
+6166 .assign_retire_seq(v);
+6167 self.endpoint_events
+6168 .push_back(EndpointEventInner::NeedIdentifiers(PathId::ZERO, now, n));
+6169 }
+6170
+6171 /// Check the current active remote CID sequence for `PathId::ZERO`
+6172 #[cfg(test)]
+6173 pub(crate) fn active_rem_cid_seq(&self) -> u64 {
+6174 self.rem_cids.get(&PathId::ZERO).unwrap().active_seq()
+6175 }
+6176
+6177 /// Returns the detected maximum udp payload size for the current path
+6178 #[cfg(test)]
+6179 pub(crate) fn path_mtu(&self, path_id: PathId) -> u16 {
+6180 self.path_data(path_id).current_mtu()
6181 }
6182
-6183 /// Handle new path status information: PATH_STATUS_AVAILABLE, PATH_STATUS_BACKUP
-6184 fn on_path_status(&mut self, path_id: PathId, status: PathStatus, status_seq_no: VarInt) {
-6185 if let Some(path) = self.paths.get_mut(&path_id) {
-6186 path.data.status.remote_update(status, status_seq_no);
-6187 } else {
-6188 debug!("PATH_STATUS_AVAILABLE received unknown path {:?}", path_id);
-6189 }
-6190 self.events.push_back(
-6191 PathEvent::RemoteStatus {
-6192 id: path_id,
-6193 status,
-6194 }
-6195 .into(),
-6196 );
-6197 }
-6198
-6199 /// Returns the maximum [`PathId`] to be used for sending in this connection.
-6200 ///
-6201 /// This is calculated as minimum between the local and remote's maximums when multipath is
-6202 /// enabled, or `None` when disabled.
-6203 ///
-6204 /// For data that's received, we should use [`Self::local_max_path_id`] instead.
-6205 /// The reasoning is that the remote might already have updated to its own newer
-6206 /// [`Self::max_path_id`] after sending out a `MAX_PATH_ID` frame, but it got re-ordered.
-6207 fn max_path_id(&self) -> Option<PathId> {
-6208 if self.is_multipath_negotiated() {
-6209 Some(self.remote_max_path_id.min(self.local_max_path_id))
-6210 } else {
-6211 None
-6212 }
-6213 }
-6214
-6215 /// Add addresses the local endpoint considers are reachable for nat traversal
-6216 pub fn add_nat_traversal_address(&mut self, address: SocketAddr) -> Result<(), iroh_hp::Error> {
-6217 if let Some(added) = self.iroh_hp.add_local_address(address)? {
-6218 self.spaces[SpaceId::Data].pending.add_address.insert(added);
-6219 };
-6220 Ok(())
-6221 }
-6222
-6223 /// Removes an address the endpoing no longer considers reachable for nat traversal
-6224 ///
-6225 /// Addresses not present in the set will be silently ignored.
-6226 pub fn remove_nat_traversal_address(
-6227 &mut self,
-6228 address: SocketAddr,
-6229 ) -> Result<(), iroh_hp::Error> {
-6230 if let Some(removed) = self.iroh_hp.remove_local_address(address)? {
-6231 self.spaces[SpaceId::Data]
-6232 .pending
-6233 .remove_address
-6234 .insert(removed);
-6235 }
-6236 Ok(())
-6237 }
-6238
-6239 /// Get the current local nat traversal addresses
-6240 pub fn get_local_nat_traversal_addresses(&self) -> Result<Vec<SocketAddr>, iroh_hp::Error> {
-6241 self.iroh_hp.get_local_nat_traversal_addresses()
-6242 }
-6243
-6244 /// Get the currently advertised nat traversal addresses by the server
-6245 pub fn get_remote_nat_traversal_addresses(&self) -> Result<Vec<SocketAddr>, iroh_hp::Error> {
-6246 Ok(self
-6247 .iroh_hp
-6248 .client_side()?
-6249 .get_remote_nat_traversal_addresses())
-6250 }
-6251
-6252 /// Attempts to open a path for nat traversal.
-6253 ///
-6254 /// `ipv6` indicates if the path should be opened using an IPV6 remote. If the address is
-6255 /// ignored, it will return `None`.
-6256 ///
-6257 /// On success returns the [`PathId`] and remote address of the path, as well as whether the path
-6258 /// existed for the adjusted remote.
-6259 fn open_nat_traversal_path(
-6260 &mut self,
-6261 now: Instant,
-6262 (ip, port): (IpAddr, u16),
-6263 ipv6: bool,
-6264 ) -> Result<Option<(PathId, SocketAddr, bool)>, PathError> {
-6265 // If this endpoint is an IPv6 endpoint we use IPv6 addresses for all remotes.
-6266 let remote = match ip {
-6267 IpAddr::V4(addr) if ipv6 => SocketAddr::new(addr.to_ipv6_mapped().into(), port),
-6268 IpAddr::V4(addr) => SocketAddr::new(addr.into(), port),
-6269 IpAddr::V6(_) if ipv6 => SocketAddr::new(ip, port),
-6270 IpAddr::V6(_) => {
-6271 trace!("not using IPv6 nat candidate for IPv4 socket");
-6272 return Ok(None);
-6273 }
-6274 };
-6275 // TODO(matheus23): Probe the correct 4-tuple, instead of only a remote address?
-6276 // By specifying None, we do two things: 1. open_path_ensure won't generate two
-6277 // paths to the same remote and 2. we let the OS choose which interface to use for
-6278 // sending on that path.
-6279 let network_path = FourTuple {
-6280 remote,
-6281 local_ip: None,
-6282 };
-6283 match self.open_path_ensure(network_path, PathStatus::Backup, now) {
-6284 Ok((path_id, path_was_known)) => {
-6285 if path_was_known {
-6286 trace!(%path_id, %remote, "nat traversal: path existed for remote");
-6287 }
-6288 Ok(Some((path_id, remote, path_was_known)))
-6289 }
-6290 Err(e) => {
-6291 debug!(%remote, %e, "nat traversal: failed to probe remote");
-6292 Err(e)
-6293 }
-6294 }
+6183 /// Triggers path validation on all paths
+6184 #[cfg(test)]
+6185 pub(crate) fn trigger_path_validation(&mut self) {
+6186 for path in self.paths.values_mut() {
+6187 path.data.send_new_challenge = true;
+6188 }
+6189 }
+6190
+6191 /// Whether we have 1-RTT data to send
+6192 ///
+6193 /// This checks for frames that can only be sent in the data space (1-RTT):
+6194 /// - Pending PATH_CHALLENGE frames on the active and previous path if just migrated.
+6195 /// - Pending PATH_RESPONSE frames.
+6196 /// - Pending data to send in STREAM frames.
+6197 /// - Pending DATAGRAM frames to send.
+6198 ///
+6199 /// See also [`PacketSpace::can_send`] which keeps track of all other frame types that
+6200 /// may need to be sent.
+6201 fn can_send_1rtt(&self, path_id: PathId, max_size: usize) -> SendableFrames {
+6202 let path_exclusive = self.paths.get(&path_id).is_some_and(|path| {
+6203 path.data.send_new_challenge
+6204 || path
+6205 .prev
+6206 .as_ref()
+6207 .is_some_and(|(_, path)| path.send_new_challenge)
+6208 || !path.data.path_responses.is_empty()
+6209 });
+6210 let other = self.streams.can_send_stream_data()
+6211 || self
+6212 .datagrams
+6213 .outgoing
+6214 .front()
+6215 .is_some_and(|x| x.size(true) <= max_size);
+6216 SendableFrames {
+6217 acks: false,
+6218 other,
+6219 close: false,
+6220 path_exclusive,
+6221 }
+6222 }
+6223
+6224 /// Terminate the connection instantly, without sending a close packet
+6225 fn kill(&mut self, reason: ConnectionError) {
+6226 self.close_common();
+6227 self.state.move_to_drained(Some(reason));
+6228 self.endpoint_events.push_back(EndpointEventInner::Drained);
+6229 }
+6230
+6231 /// Storage size required for the largest packet that can be transmitted on all currently
+6232 /// available paths
+6233 ///
+6234 /// Buffers passed to [`Connection::poll_transmit`] should be at least this large.
+6235 ///
+6236 /// When multipath is enabled, this value is the minimum MTU across all available paths.
+6237 pub fn current_mtu(&self) -> u16 {
+6238 self.paths
+6239 .iter()
+6240 .filter(|&(path_id, _path_state)| !self.abandoned_paths.contains(path_id))
+6241 .map(|(_path_id, path_state)| path_state.data.current_mtu())
+6242 .min()
+6243 .expect("There is always at least one available path")
+6244 }
+6245
+6246 /// Size of non-frame data for a 1-RTT packet
+6247 ///
+6248 /// Quantifies space consumed by the QUIC header and AEAD tag. All other bytes in a packet are
+6249 /// frames. Changes if the length of the remote connection ID changes, which is expected to be
+6250 /// rare. If `pn` is specified, may additionally change unpredictably due to variations in
+6251 /// latency and packet loss.
+6252 fn predict_1rtt_overhead(&mut self, pn: u64, path: PathId) -> usize {
+6253 let pn_len = PacketNumber::new(
+6254 pn,
+6255 self.spaces[SpaceId::Data]
+6256 .for_path(path)
+6257 .largest_acked_packet
+6258 .unwrap_or(0),
+6259 )
+6260 .len();
+6261
+6262 // 1 byte for flags
+6263 1 + self
+6264 .rem_cids
+6265 .get(&path)
+6266 .map(|cids| cids.active().len())
+6267 .unwrap_or(20) // Max CID len in QUIC v1
+6268 + pn_len
+6269 + self.tag_len_1rtt()
+6270 }
+6271
+6272 fn predict_1rtt_overhead_no_pn(&self) -> usize {
+6273 let pn_len = 4;
+6274
+6275 let cid_len = self
+6276 .rem_cids
+6277 .values()
+6278 .map(|cids| cids.active().len())
+6279 .max()
+6280 .unwrap_or(20); // Max CID len in QUIC v1
+6281
+6282 // 1 byte for flags
+6283 1 + cid_len + pn_len + self.tag_len_1rtt()
+6284 }
+6285
+6286 fn tag_len_1rtt(&self) -> usize {
+6287 let key = match self.spaces[SpaceId::Data].crypto.as_ref() {
+6288 Some(crypto) => Some(&*crypto.packet.local),
+6289 None => self.zero_rtt_crypto.as_ref().map(|x| &*x.packet),
+6290 };
+6291 // If neither Data nor 0-RTT keys are available, make a reasonable tag length guess. As of
+6292 // this writing, all QUIC cipher suites use 16-byte tags. We could return `None` instead,
+6293 // but that would needlessly prevent sending datagrams during 0-RTT.
+6294 key.map_or(16, |x| x.tag_len())
6295 }
6296
-6297 /// Initiates a new nat traversal round
-6298 ///
-6299 /// A nat traversal round involves advertising the client's local addresses in `REACH_OUT`
-6300 /// frames, and initiating probing of the known remote addresses. When a new round is
-6301 /// initiated, the previous one is cancelled, and paths that have not been opened are closed.
-6302 ///
-6303 /// Returns the server addresses that are now being probed.
-6304 /// If addresses fail due to spurious errors, these might succeed later and not be returned in
-6305 /// this set.
-6306 pub fn initiate_nat_traversal_round(
-6307 &mut self,
-6308 now: Instant,
-6309 ) -> Result<Vec<SocketAddr>, iroh_hp::Error> {
-6310 if self.state.is_closed() {
-6311 return Err(iroh_hp::Error::Closed);
-6312 }
-6313
-6314 let client_state = self.iroh_hp.client_side_mut()?;
-6315 let iroh_hp::NatTraversalRound {
-6316 new_round,
-6317 reach_out_at,
-6318 addresses_to_probe,
-6319 prev_round_path_ids,
-6320 } = client_state.initiate_nat_traversal_round()?;
-6321
-6322 self.spaces[SpaceId::Data].pending.reach_out = Some((new_round, reach_out_at));
-6323
-6324 for path_id in prev_round_path_ids {
-6325 // TODO(@divma): this sounds reasonable but we need if this actually works for the
-6326 // purposes of the protocol
-6327 let validated = self
-6328 .path(path_id)
-6329 .map(|path| path.validated)
-6330 .unwrap_or(false);
-6331
-6332 if !validated {
-6333 let _ = self.close_path(
-6334 now,
-6335 path_id,
-6336 TransportErrorCode::APPLICATION_ABANDON_PATH.into(),
-6337 );
-6338 }
-6339 }
-6340
-6341 let mut err = None;
+6297 /// Mark the path as validated, and enqueue NEW_TOKEN frames to be sent as appropriate
+6298 fn on_path_validated(&mut self, path_id: PathId) {
+6299 self.path_data_mut(path_id).validated = true;
+6300 let ConnectionSide::Server { server_config } = &self.side else {
+6301 return;
+6302 };
+6303 let network_path = self.path_data(path_id).network_path;
+6304 let new_tokens = &mut self.spaces[SpaceId::Data as usize].pending.new_tokens;
+6305 new_tokens.clear();
+6306 for _ in 0..server_config.validation_token.sent {
+6307 new_tokens.push(network_path);
+6308 }
+6309 }
+6310
+6311 /// Handle new path status information: PATH_STATUS_AVAILABLE, PATH_STATUS_BACKUP
+6312 fn on_path_status(&mut self, path_id: PathId, status: PathStatus, status_seq_no: VarInt) {
+6313 if let Some(path) = self.paths.get_mut(&path_id) {
+6314 path.data.status.remote_update(status, status_seq_no);
+6315 } else {
+6316 debug!("PATH_STATUS_AVAILABLE received unknown path {:?}", path_id);
+6317 }
+6318 self.events.push_back(
+6319 PathEvent::RemoteStatus {
+6320 id: path_id,
+6321 status,
+6322 }
+6323 .into(),
+6324 );
+6325 }
+6326
+6327 /// Returns the maximum [`PathId`] to be used for sending in this connection.
+6328 ///
+6329 /// This is calculated as minimum between the local and remote's maximums when multipath is
+6330 /// enabled, or `None` when disabled.
+6331 ///
+6332 /// For data that's received, we should use [`Self::local_max_path_id`] instead.
+6333 /// The reasoning is that the remote might already have updated to its own newer
+6334 /// [`Self::max_path_id`] after sending out a `MAX_PATH_ID` frame, but it got re-ordered.
+6335 fn max_path_id(&self) -> Option<PathId> {
+6336 if self.is_multipath_negotiated() {
+6337 Some(self.remote_max_path_id.min(self.local_max_path_id))
+6338 } else {
+6339 None
+6340 }
+6341 }
6342
-6343 let mut path_ids = Vec::with_capacity(addresses_to_probe.len());
-6344 let mut probed_addresses = Vec::with_capacity(addresses_to_probe.len());
-6345 let ipv6 = self
-6346 .paths
-6347 .values()
-6348 .any(|p| p.data.network_path.remote.is_ipv6());
-6349
-6350 for (id, address) in addresses_to_probe {
-6351 match self.open_nat_traversal_path(now, address, ipv6) {
-6352 Ok(None) => {}
-6353 Ok(Some((path_id, remote, path_was_known))) => {
-6354 if !path_was_known {
-6355 path_ids.push(path_id);
-6356 probed_addresses.push(remote);
-6357 }
-6358 }
-6359 Err(e) => {
-6360 self.iroh_hp
-6361 .client_side_mut()
-6362 .expect("validated")
-6363 .report_in_continuation(id, e);
-6364 err.get_or_insert(e);
-6365 }
-6366 }
-6367 }
-6368
-6369 if let Some(err) = err {
-6370 // We failed to probe any addresses, bail out
-6371 if probed_addresses.is_empty() {
-6372 return Err(iroh_hp::Error::Multipath(err));
-6373 }
-6374 }
-6375
-6376 self.iroh_hp
-6377 .client_side_mut()
-6378 .expect("connection side validated")
-6379 .set_round_path_ids(path_ids);
-6380
-6381 Ok(probed_addresses)
-6382 }
-6383
-6384 /// Attempts to continue a nat traversal round by trying to open paths for pending client probes.
-6385 ///
-6386 /// If there was nothing to do, it returns `None`. Otherwise it returns whether the path was
-6387 /// successfully open.
-6388 fn continue_nat_traversal_round(&mut self, now: Instant) -> Option<bool> {
-6389 let client_state = self.iroh_hp.client_side_mut().ok()?;
-6390 let (id, address) = client_state.continue_nat_traversal_round()?;
-6391 let ipv6 = self
-6392 .paths
-6393 .values()
-6394 .any(|p| p.data.network_path.remote.is_ipv6());
-6395 let open_result = self.open_nat_traversal_path(now, address, ipv6);
-6396 let client_state = self.iroh_hp.client_side_mut().expect("validated");
-6397 match open_result {
-6398 Ok(None) => Some(true),
-6399 Ok(Some((path_id, _remote, path_was_known))) => {
-6400 if !path_was_known {
-6401 client_state.add_round_path_id(path_id);
-6402 }
-6403 Some(true)
-6404 }
-6405 Err(e) => {
-6406 client_state.report_in_continuation(id, e);
-6407 Some(false)
-6408 }
-6409 }
-6410 }
-6411}
-6412
-6413impl fmt::Debug for Connection {
-6414 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
-6415 f.debug_struct("Connection")
-6416 .field("handshake_cid", &self.handshake_cid)
-6417 .finish()
-6418 }
-6419}
-6420
-6421#[derive(Debug, Copy, Clone, PartialEq, Eq)]
-6422enum PathBlocked {
-6423 No,
-6424 AntiAmplification,
-6425 Congestion,
-6426 Pacing,
-6427}
-6428
-6429/// Fields of `Connection` specific to it being client-side or server-side
-6430enum ConnectionSide {
-6431 Client {
-6432 /// Sent in every outgoing Initial packet. Always empty after Initial keys are discarded
-6433 token: Bytes,
-6434 token_store: Arc<dyn TokenStore>,
-6435 server_name: String,
-6436 },
-6437 Server {
-6438 server_config: Arc<ServerConfig>,
-6439 },
-6440}
+6343 /// Add addresses the local endpoint considers are reachable for nat traversal
+6344 pub fn add_nat_traversal_address(&mut self, address: SocketAddr) -> Result<(), iroh_hp::Error> {
+6345 if let Some(added) = self.iroh_hp.add_local_address(address)? {
+6346 self.spaces[SpaceId::Data].pending.add_address.insert(added);
+6347 };
+6348 Ok(())
+6349 }
+6350
+6351 /// Removes an address the endpoing no longer considers reachable for nat traversal
+6352 ///
+6353 /// Addresses not present in the set will be silently ignored.
+6354 pub fn remove_nat_traversal_address(
+6355 &mut self,
+6356 address: SocketAddr,
+6357 ) -> Result<(), iroh_hp::Error> {
+6358 if let Some(removed) = self.iroh_hp.remove_local_address(address)? {
+6359 self.spaces[SpaceId::Data]
+6360 .pending
+6361 .remove_address
+6362 .insert(removed);
+6363 }
+6364 Ok(())
+6365 }
+6366
+6367 /// Get the current local nat traversal addresses
+6368 pub fn get_local_nat_traversal_addresses(&self) -> Result<Vec<SocketAddr>, iroh_hp::Error> {
+6369 self.iroh_hp.get_local_nat_traversal_addresses()
+6370 }
+6371
+6372 /// Get the currently advertised nat traversal addresses by the server
+6373 pub fn get_remote_nat_traversal_addresses(&self) -> Result<Vec<SocketAddr>, iroh_hp::Error> {
+6374 Ok(self
+6375 .iroh_hp
+6376 .client_side()?
+6377 .get_remote_nat_traversal_addresses())
+6378 }
+6379
+6380 /// Attempts to open a path for nat traversal.
+6381 ///
+6382 /// `ipv6` indicates if the path should be opened using an IPV6 remote. If the address is
+6383 /// ignored, it will return `None`.
+6384 ///
+6385 /// On success returns the [`PathId`] and remote address of the path, as well as whether the path
+6386 /// existed for the adjusted remote.
+6387 fn open_nat_traversal_path(
+6388 &mut self,
+6389 now: Instant,
+6390 (ip, port): (IpAddr, u16),
+6391 ipv6: bool,
+6392 ) -> Result<Option<(PathId, SocketAddr, bool)>, PathError> {
+6393 // If this endpoint is an IPv6 endpoint we use IPv6 addresses for all remotes.
+6394 let remote = match ip {
+6395 IpAddr::V4(addr) if ipv6 => SocketAddr::new(addr.to_ipv6_mapped().into(), port),
+6396 IpAddr::V4(addr) => SocketAddr::new(addr.into(), port),
+6397 IpAddr::V6(_) if ipv6 => SocketAddr::new(ip, port),
+6398 IpAddr::V6(_) => {
+6399 trace!("not using IPv6 nat candidate for IPv4 socket");
+6400 return Ok(None);
+6401 }
+6402 };
+6403 // TODO(matheus23): Probe the correct 4-tuple, instead of only a remote address?
+6404 // By specifying None, we do two things: 1. open_path_ensure won't generate two
+6405 // paths to the same remote and 2. we let the OS choose which interface to use for
+6406 // sending on that path.
+6407 let network_path = FourTuple {
+6408 remote,
+6409 local_ip: None,
+6410 };
+6411 match self.open_path_ensure(network_path, PathStatus::Backup, now) {
+6412 Ok((path_id, path_was_known)) => {
+6413 if path_was_known {
+6414 trace!(%path_id, %remote, "nat traversal: path existed for remote");
+6415 }
+6416 Ok(Some((path_id, remote, path_was_known)))
+6417 }
+6418 Err(e) => {
+6419 debug!(%remote, %e, "nat traversal: failed to probe remote");
+6420 Err(e)
+6421 }
+6422 }
+6423 }
+6424
+6425 /// Initiates a new nat traversal round
+6426 ///
+6427 /// A nat traversal round involves advertising the client's local addresses in `REACH_OUT`
+6428 /// frames, and initiating probing of the known remote addresses. When a new round is
+6429 /// initiated, the previous one is cancelled, and paths that have not been opened are closed.
+6430 ///
+6431 /// Returns the server addresses that are now being probed.
+6432 /// If addresses fail due to spurious errors, these might succeed later and not be returned in
+6433 /// this set.
+6434 pub fn initiate_nat_traversal_round(
+6435 &mut self,
+6436 now: Instant,
+6437 ) -> Result<Vec<SocketAddr>, iroh_hp::Error> {
+6438 if self.state.is_closed() {
+6439 return Err(iroh_hp::Error::Closed);
+6440 }
6441
-6442impl ConnectionSide {
-6443 fn remote_may_migrate(&self, state: &State) -> bool {
-6444 match self {
-6445 Self::Server { server_config } => server_config.migration,
-6446 Self::Client { .. } => {
-6447 if let Some(hs) = state.as_handshake() {
-6448 hs.allow_server_migration
-6449 } else {
-6450 false
-6451 }
-6452 }
-6453 }
-6454 }
-6455
-6456 fn is_client(&self) -> bool {
-6457 self.side().is_client()
-6458 }
+6442 let client_state = self.iroh_hp.client_side_mut()?;
+6443 let iroh_hp::NatTraversalRound {
+6444 new_round,
+6445 reach_out_at,
+6446 addresses_to_probe,
+6447 prev_round_path_ids,
+6448 } = client_state.initiate_nat_traversal_round()?;
+6449
+6450 self.spaces[SpaceId::Data].pending.reach_out = Some((new_round, reach_out_at));
+6451
+6452 for path_id in prev_round_path_ids {
+6453 // TODO(@divma): this sounds reasonable but we need if this actually works for the
+6454 // purposes of the protocol
+6455 let validated = self
+6456 .path(path_id)
+6457 .map(|path| path.validated)
+6458 .unwrap_or(false);
6459
-6460 fn is_server(&self) -> bool {
-6461 self.side().is_server()
-6462 }
-6463
-6464 fn side(&self) -> Side {
-6465 match *self {
-6466 Self::Client { .. } => Side::Client,
-6467 Self::Server { .. } => Side::Server,
-6468 }
-6469 }
-6470}
-6471
-6472impl From<SideArgs> for ConnectionSide {
-6473 fn from(side: SideArgs) -> Self {
-6474 match side {
-6475 SideArgs::Client {
-6476 token_store,
-6477 server_name,
-6478 } => Self::Client {
-6479 token: token_store.take(&server_name).unwrap_or_default(),
-6480 token_store,
-6481 server_name,
-6482 },
-6483 SideArgs::Server {
-6484 server_config,
-6485 pref_addr_cid: _,
-6486 path_validated: _,
-6487 } => Self::Server { server_config },
-6488 }
-6489 }
-6490}
-6491
-6492/// Parameters to `Connection::new` specific to it being client-side or server-side
-6493pub(crate) enum SideArgs {
-6494 Client {
-6495 token_store: Arc<dyn TokenStore>,
-6496 server_name: String,
-6497 },
-6498 Server {
-6499 server_config: Arc<ServerConfig>,
-6500 pref_addr_cid: Option<ConnectionId>,
-6501 path_validated: bool,
-6502 },
-6503}
-6504
-6505impl SideArgs {
-6506 pub(crate) fn pref_addr_cid(&self) -> Option<ConnectionId> {
-6507 match *self {
-6508 Self::Client { .. } => None,
-6509 Self::Server { pref_addr_cid, .. } => pref_addr_cid,
-6510 }
-6511 }
-6512
-6513 pub(crate) fn path_validated(&self) -> bool {
-6514 match *self {
-6515 Self::Client { .. } => true,
-6516 Self::Server { path_validated, .. } => path_validated,
-6517 }
-6518 }
-6519
-6520 pub(crate) fn side(&self) -> Side {
-6521 match *self {
-6522 Self::Client { .. } => Side::Client,
-6523 Self::Server { .. } => Side::Server,
-6524 }
-6525 }
-6526}
-6527
-6528/// Reasons why a connection might be lost
-6529#[derive(Debug, Error, Clone, PartialEq, Eq)]
-6530pub enum ConnectionError {
-6531 /// The peer doesn't implement any supported version
-6532 #[error("peer doesn't implement any supported version")]
-6533 VersionMismatch,
-6534 /// The peer violated the QUIC specification as understood by this implementation
-6535 #[error(transparent)]
-6536 TransportError(#[from] TransportError),
-6537 /// The peer's QUIC stack aborted the connection automatically
-6538 #[error("aborted by peer: {0}")]
-6539 ConnectionClosed(frame::ConnectionClose),
-6540 /// The peer closed the connection
-6541 #[error("closed by peer: {0}")]
-6542 ApplicationClosed(frame::ApplicationClose),
-6543 /// The peer is unable to continue processing this connection, usually due to having restarted
-6544 #[error("reset by peer")]
-6545 Reset,
-6546 /// Communication with the peer has lapsed for longer than the negotiated idle timeout
-6547 ///
-6548 /// If neither side is sending keep-alives, a connection will time out after a long enough idle
-6549 /// period even if the peer is still reachable. See also [`TransportConfig::max_idle_timeout()`]
-6550 /// and [`TransportConfig::keep_alive_interval()`].
-6551 #[error("timed out")]
-6552 TimedOut,
-6553 /// The local application closed the connection
-6554 #[error("closed")]
-6555 LocallyClosed,
-6556 /// The connection could not be created because not enough of the CID space is available
-6557 ///
-6558 /// Try using longer connection IDs.
-6559 #[error("CIDs exhausted")]
-6560 CidsExhausted,
-6561}
-6562
-6563impl From<Close> for ConnectionError {
-6564 fn from(x: Close) -> Self {
-6565 match x {
-6566 Close::Connection(reason) => Self::ConnectionClosed(reason),
-6567 Close::Application(reason) => Self::ApplicationClosed(reason),
-6568 }
-6569 }
-6570}
-6571
-6572// For compatibility with API consumers
-6573impl From<ConnectionError> for io::Error {
-6574 fn from(x: ConnectionError) -> Self {
-6575 use ConnectionError::*;
-6576 let kind = match x {
-6577 TimedOut => io::ErrorKind::TimedOut,
-6578 Reset => io::ErrorKind::ConnectionReset,
-6579 ApplicationClosed(_) | ConnectionClosed(_) => io::ErrorKind::ConnectionAborted,
-6580 TransportError(_) | VersionMismatch | LocallyClosed | CidsExhausted => {
-6581 io::ErrorKind::Other
-6582 }
-6583 };
-6584 Self::new(kind, x)
-6585 }
-6586}
+6460 if !validated {
+6461 let _ = self.close_path(
+6462 now,
+6463 path_id,
+6464 TransportErrorCode::APPLICATION_ABANDON_PATH.into(),
+6465 );
+6466 }
+6467 }
+6468
+6469 let mut err = None;
+6470
+6471 let mut path_ids = Vec::with_capacity(addresses_to_probe.len());
+6472 let mut probed_addresses = Vec::with_capacity(addresses_to_probe.len());
+6473 let ipv6 = self
+6474 .paths
+6475 .values()
+6476 .any(|p| p.data.network_path.remote.is_ipv6());
+6477
+6478 for (id, address) in addresses_to_probe {
+6479 match self.open_nat_traversal_path(now, address, ipv6) {
+6480 Ok(None) => {}
+6481 Ok(Some((path_id, remote, path_was_known))) => {
+6482 if !path_was_known {
+6483 path_ids.push(path_id);
+6484 probed_addresses.push(remote);
+6485 }
+6486 }
+6487 Err(e) => {
+6488 self.iroh_hp
+6489 .client_side_mut()
+6490 .expect("validated")
+6491 .report_in_continuation(id, e);
+6492 err.get_or_insert(e);
+6493 }
+6494 }
+6495 }
+6496
+6497 if let Some(err) = err {
+6498 // We failed to probe any addresses, bail out
+6499 if probed_addresses.is_empty() {
+6500 return Err(iroh_hp::Error::Multipath(err));
+6501 }
+6502 }
+6503
+6504 self.iroh_hp
+6505 .client_side_mut()
+6506 .expect("connection side validated")
+6507 .set_round_path_ids(path_ids);
+6508
+6509 Ok(probed_addresses)
+6510 }
+6511
+6512 /// Attempts to continue a nat traversal round by trying to open paths for pending client probes.
+6513 ///
+6514 /// If there was nothing to do, it returns `None`. Otherwise it returns whether the path was
+6515 /// successfully open.
+6516 fn continue_nat_traversal_round(&mut self, now: Instant) -> Option<bool> {
+6517 let client_state = self.iroh_hp.client_side_mut().ok()?;
+6518 let (id, address) = client_state.continue_nat_traversal_round()?;
+6519 let ipv6 = self
+6520 .paths
+6521 .values()
+6522 .any(|p| p.data.network_path.remote.is_ipv6());
+6523 let open_result = self.open_nat_traversal_path(now, address, ipv6);
+6524 let client_state = self.iroh_hp.client_side_mut().expect("validated");
+6525 match open_result {
+6526 Ok(None) => Some(true),
+6527 Ok(Some((path_id, _remote, path_was_known))) => {
+6528 if !path_was_known {
+6529 client_state.add_round_path_id(path_id);
+6530 }
+6531 Some(true)
+6532 }
+6533 Err(e) => {
+6534 client_state.report_in_continuation(id, e);
+6535 Some(false)
+6536 }
+6537 }
+6538 }
+6539}
+6540
+6541impl fmt::Debug for Connection {
+6542 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+6543 f.debug_struct("Connection")
+6544 .field("handshake_cid", &self.handshake_cid)
+6545 .finish()
+6546 }
+6547}
+6548
+6549#[derive(Debug, Copy, Clone, PartialEq, Eq)]
+6550enum PathBlocked {
+6551 No,
+6552 AntiAmplification,
+6553 Congestion,
+6554 Pacing,
+6555}
+6556
+6557/// Fields of `Connection` specific to it being client-side or server-side
+6558enum ConnectionSide {
+6559 Client {
+6560 /// Sent in every outgoing Initial packet. Always empty after Initial keys are discarded
+6561 token: Bytes,
+6562 token_store: Arc<dyn TokenStore>,
+6563 server_name: String,
+6564 },
+6565 Server {
+6566 server_config: Arc<ServerConfig>,
+6567 },
+6568}
+6569
+6570impl ConnectionSide {
+6571 fn remote_may_migrate(&self, state: &State) -> bool {
+6572 match self {
+6573 Self::Server { server_config } => server_config.migration,
+6574 Self::Client { .. } => {
+6575 if let Some(hs) = state.as_handshake() {
+6576 hs.allow_server_migration
+6577 } else {
+6578 false
+6579 }
+6580 }
+6581 }
+6582 }
+6583
+6584 fn is_client(&self) -> bool {
+6585 self.side().is_client()
+6586 }
6587
-6588/// Errors that might trigger a path being closed
-6589// TODO(@divma): maybe needs to be reworked based on what we want to do with the public API
-6590#[derive(Debug, Error, PartialEq, Eq, Clone, Copy)]
-6591pub enum PathError {
-6592 /// The extension was not negotiated with the peer
-6593 #[error("multipath extension not negotiated")]
-6594 MultipathNotNegotiated,
-6595 /// Paths can only be opened client-side
-6596 #[error("the server side may not open a path")]
-6597 ServerSideNotAllowed,
-6598 /// Current limits do not allow us to open more paths
-6599 #[error("maximum number of concurrent paths reached")]
-6600 MaxPathIdReached,
-6601 /// No remote CIDs available to open a new path
-6602 #[error("remoted CIDs exhausted")]
-6603 RemoteCidsExhausted,
-6604 /// Path could not be validated and will be abandoned
-6605 #[error("path validation failed")]
-6606 ValidationFailed,
-6607 /// The remote address for the path is not supported by the endpoint
-6608 #[error("invalid remote address")]
-6609 InvalidRemoteAddress(SocketAddr),
-6610}
-6611
-6612/// Errors triggered when abandoning a path
-6613#[derive(Debug, Error, Clone, Eq, PartialEq)]
-6614pub enum ClosePathError {
-6615 /// The path is already closed or was never opened
-6616 #[error("closed path")]
-6617 ClosedPath,
-6618 /// This is the last path, which can not be abandoned
-6619 #[error("last open path")]
-6620 LastOpenPath,
-6621}
-6622
-6623/// Error when the multipath extension was not negotiated, but attempted to be used.
-6624#[derive(Debug, Error, Clone, Copy)]
-6625#[error("Multipath extension not negotiated")]
-6626pub struct MultipathNotNegotiated {
-6627 _private: (),
-6628}
-6629
-6630/// Events of interest to the application
-6631#[derive(Debug)]
-6632pub enum Event {
-6633 /// The connection's handshake data is ready
-6634 HandshakeDataReady,
-6635 /// The connection was successfully established
-6636 Connected,
-6637 /// The TLS handshake was confirmed
-6638 HandshakeConfirmed,
-6639 /// The connection was lost
-6640 ///
-6641 /// Emitted if the peer closes the connection or an error is encountered.
-6642 ConnectionLost {
-6643 /// Reason that the connection was closed
-6644 reason: ConnectionError,
-6645 },
-6646 /// Stream events
-6647 Stream(StreamEvent),
-6648 /// One or more application datagrams have been received
-6649 DatagramReceived,
-6650 /// One or more application datagrams have been sent after blocking
-6651 DatagramsUnblocked,
-6652 /// (Multi)Path events
-6653 Path(PathEvent),
-6654 /// Iroh's nat traversal events
-6655 NatTraversal(iroh_hp::Event),
-6656}
-6657
-6658impl From<PathEvent> for Event {
-6659 fn from(source: PathEvent) -> Self {
-6660 Self::Path(source)
-6661 }
-6662}
-6663
-6664fn get_max_ack_delay(params: &TransportParameters) -> Duration {
-6665 Duration::from_micros(params.max_ack_delay.0 * 1000)
-6666}
-6667
-6668// Prevents overflow and improves behavior in extreme circumstances
-6669const MAX_BACKOFF_EXPONENT: u32 = 16;
-6670
-6671/// Minimal remaining size to allow packet coalescing, excluding cryptographic tag
-6672///
-6673/// This must be at least as large as the header for a well-formed empty packet to be coalesced,
-6674/// plus some space for frames. We only care about handshake headers because short header packets
-6675/// necessarily have smaller headers, and initial packets are only ever the first packet in a
-6676/// datagram (because we coalesce in ascending packet space order and the only reason to split a
-6677/// packet is when packet space changes).
-6678const MIN_PACKET_SPACE: usize = MAX_HANDSHAKE_OR_0RTT_HEADER_SIZE + 32;
-6679
-6680/// Largest amount of space that could be occupied by a Handshake or 0-RTT packet's header
-6681///
-6682/// Excludes packet-type-specific fields such as packet number or Initial token
-6683// https://www.rfc-editor.org/rfc/rfc9000.html#name-0-rtt: flags + version + dcid len + dcid +
-6684// scid len + scid + length + pn
-6685const MAX_HANDSHAKE_OR_0RTT_HEADER_SIZE: usize =
-6686 1 + 4 + 1 + MAX_CID_SIZE + 1 + MAX_CID_SIZE + VarInt::from_u32(u16::MAX as u32).size() + 4;
-6687
-6688/// Perform key updates this many packets before the AEAD confidentiality limit.
-6689///
-6690/// Chosen arbitrarily, intended to be large enough to prevent spurious connection loss.
-6691const KEY_UPDATE_MARGIN: u64 = 10_000;
-6692
-6693#[derive(Default)]
-6694struct SentFrames {
-6695 retransmits: ThinRetransmits,
-6696 /// The packet number of the largest acknowledged packet for each path
-6697 largest_acked: FxHashMap<PathId, u64>,
-6698 stream_frames: StreamMetaVec,
-6699 /// Whether the packet contains non-retransmittable frames (like datagrams)
-6700 non_retransmits: bool,
-6701 /// If the datagram containing these frames should be padded to the min MTU
-6702 requires_padding: bool,
-6703}
-6704
-6705impl SentFrames {
-6706 /// Returns whether the packet contains only ACKs
-6707 fn is_ack_only(&self, streams: &StreamsState) -> bool {
-6708 !self.largest_acked.is_empty()
-6709 && !self.non_retransmits
-6710 && self.stream_frames.is_empty()
-6711 && self.retransmits.is_empty(streams)
-6712 }
-6713
-6714 fn retransmits_mut(&mut self) -> &mut Retransmits {
-6715 self.retransmits.get_or_create()
-6716 }
-6717
-6718 fn record_sent_frame(&mut self, frame: frame::EncodableFrame<'_>) {
-6719 use frame::EncodableFrame::*;
-6720 match frame {
-6721 PathAck(path_ack_encoder) => {
-6722 if let Some(max) = path_ack_encoder.ranges.max() {
-6723 self.largest_acked.insert(path_ack_encoder.path_id, max);
-6724 }
-6725 }
-6726 Ack(ack_encoder) => {
-6727 if let Some(max) = ack_encoder.ranges.max() {
-6728 self.largest_acked.insert(PathId::ZERO, max);
-6729 }
-6730 }
-6731 Close(_) => { /* non retransmittable, but after this we don't really care */ }
-6732 PathResponse(_) => self.non_retransmits = true,
-6733 HandshakeDone(_) => self.retransmits_mut().handshake_done = true,
-6734 ReachOut(frame::ReachOut { round, ip, port }) => self
-6735 .retransmits_mut()
-6736 .reach_out
-6737 .get_or_insert_with(|| (round, Vec::new()))
-6738 .1
-6739 .push((ip, port)),
-6740 ObservedAddr(_) => self.retransmits_mut().observed_addr = true,
-6741 Ping(_) => self.non_retransmits = true,
-6742 ImmediateAck(_) => self.non_retransmits = true,
-6743 AckFrequency(_) => self.retransmits_mut().ack_frequency = true,
-6744 PathChallenge(_) => self.non_retransmits = true,
-6745 Crypto(crypto) => self.retransmits_mut().crypto.push_back(crypto),
-6746 PathAbandon(path_abandon) => {
-6747 self.retransmits_mut()
-6748 .path_abandon
-6749 .entry(path_abandon.path_id)
-6750 .or_insert(path_abandon.error_code);
-6751 }
-6752 PathStatusAvailable(frame::PathStatusAvailable { path_id, .. })
-6753 | PathStatusBackup(frame::PathStatusBackup { path_id, .. }) => {
-6754 self.retransmits_mut().path_status.insert(path_id);
-6755 }
-6756 MaxPathId(_) => self.retransmits_mut().max_path_id = true,
-6757 PathsBlocked(_) => self.retransmits_mut().paths_blocked = true,
-6758 PathCidsBlocked(path_cids_blocked) => {
-6759 self.retransmits_mut()
-6760 .path_cids_blocked
-6761 .insert(path_cids_blocked.path_id);
-6762 }
-6763 ResetStream(reset) => self
-6764 .retransmits_mut()
-6765 .reset_stream
-6766 .push((reset.id, reset.error_code)),
-6767 StopSending(stop_sending) => self.retransmits_mut().stop_sending.push(stop_sending),
-6768 NewConnectionId(new_cid) => self.retransmits_mut().new_cids.push(new_cid.issued()),
-6769 RetireConnectionId(retire_cid) => self
-6770 .retransmits_mut()
-6771 .retire_cids
-6772 .push((retire_cid.path_id.unwrap_or_default(), retire_cid.sequence)),
-6773 Datagram(_) => self.non_retransmits = true,
-6774 NewToken(_) => {}
-6775 AddAddress(add_address) => {
-6776 self.retransmits_mut().add_address.insert(add_address);
-6777 }
-6778 RemoveAddress(remove_address) => {
-6779 self.retransmits_mut().remove_address.insert(remove_address);
-6780 }
-6781 StreamMeta(stream_meta_encoder) => self.stream_frames.push(stream_meta_encoder.meta),
-6782 MaxData(_) => self.retransmits_mut().max_data = true,
-6783 MaxStreamData(max) => {
-6784 self.retransmits_mut().max_stream_data.insert(max.id);
-6785 }
-6786 MaxStreams(max_streams) => {
-6787 self.retransmits_mut().max_stream_id[max_streams.dir as usize] = true
-6788 }
-6789 }
-6790 }
-6791}
-6792
-6793/// Compute the negotiated idle timeout based on local and remote max_idle_timeout transport parameters.
-6794///
-6795/// According to the definition of max_idle_timeout, a value of `0` means the timeout is disabled; see <https://www.rfc-editor.org/rfc/rfc9000#section-18.2-4.4.1.>
-6796///
-6797/// According to the negotiation procedure, either the minimum of the timeouts or one specified is used as the negotiated value; see <https://www.rfc-editor.org/rfc/rfc9000#section-10.1-2.>
-6798///
-6799/// Returns the negotiated idle timeout as a `Duration`, or `None` when both endpoints have opted out of idle timeout.
-6800fn negotiate_max_idle_timeout(x: Option<VarInt>, y: Option<VarInt>) -> Option<Duration> {
-6801 match (x, y) {
-6802 (Some(VarInt(0)) | None, Some(VarInt(0)) | None) => None,
-6803 (Some(VarInt(0)) | None, Some(y)) => Some(Duration::from_millis(y.0)),
-6804 (Some(x), Some(VarInt(0)) | None) => Some(Duration::from_millis(x.0)),
-6805 (Some(x), Some(y)) => Some(Duration::from_millis(cmp::min(x, y).0)),
-6806 }
-6807}
-6808
-6809#[cfg(test)]
-6810mod tests {
-6811 use super::*;
-6812
-6813 #[test]
-6814 fn negotiate_max_idle_timeout_commutative() {
-6815 let test_params = [
-6816 (None, None, None),
-6817 (None, Some(VarInt(0)), None),
-6818 (None, Some(VarInt(2)), Some(Duration::from_millis(2))),
-6819 (Some(VarInt(0)), Some(VarInt(0)), None),
-6820 (
-6821 Some(VarInt(2)),
-6822 Some(VarInt(0)),
-6823 Some(Duration::from_millis(2)),
-6824 ),
-6825 (
-6826 Some(VarInt(1)),
-6827 Some(VarInt(4)),
-6828 Some(Duration::from_millis(1)),
-6829 ),
-6830 ];
-6831
-6832 for (left, right, result) in test_params {
-6833 assert_eq!(negotiate_max_idle_timeout(left, right), result);
-6834 assert_eq!(negotiate_max_idle_timeout(right, left), result);
-6835 }
-6836 }
-6837}
\ No newline at end of file
+6588 fn is_server(&self) -> bool {
+6589 self.side().is_server()
+6590 }
+6591
+6592 fn side(&self) -> Side {
+6593 match *self {
+6594 Self::Client { .. } => Side::Client,
+6595 Self::Server { .. } => Side::Server,
+6596 }
+6597 }
+6598}
+6599
+6600impl From<SideArgs> for ConnectionSide {
+6601 fn from(side: SideArgs) -> Self {
+6602 match side {
+6603 SideArgs::Client {
+6604 token_store,
+6605 server_name,
+6606 } => Self::Client {
+6607 token: token_store.take(&server_name).unwrap_or_default(),
+6608 token_store,
+6609 server_name,
+6610 },
+6611 SideArgs::Server {
+6612 server_config,
+6613 pref_addr_cid: _,
+6614 path_validated: _,
+6615 } => Self::Server { server_config },
+6616 }
+6617 }
+6618}
+6619
+6620/// Parameters to `Connection::new` specific to it being client-side or server-side
+6621pub(crate) enum SideArgs {
+6622 Client {
+6623 token_store: Arc<dyn TokenStore>,
+6624 server_name: String,
+6625 },
+6626 Server {
+6627 server_config: Arc<ServerConfig>,
+6628 pref_addr_cid: Option<ConnectionId>,
+6629 path_validated: bool,
+6630 },
+6631}
+6632
+6633impl SideArgs {
+6634 pub(crate) fn pref_addr_cid(&self) -> Option<ConnectionId> {
+6635 match *self {
+6636 Self::Client { .. } => None,
+6637 Self::Server { pref_addr_cid, .. } => pref_addr_cid,
+6638 }
+6639 }
+6640
+6641 pub(crate) fn path_validated(&self) -> bool {
+6642 match *self {
+6643 Self::Client { .. } => true,
+6644 Self::Server { path_validated, .. } => path_validated,
+6645 }
+6646 }
+6647
+6648 pub(crate) fn side(&self) -> Side {
+6649 match *self {
+6650 Self::Client { .. } => Side::Client,
+6651 Self::Server { .. } => Side::Server,
+6652 }
+6653 }
+6654}
+6655
+6656/// Reasons why a connection might be lost
+6657#[derive(Debug, Error, Clone, PartialEq, Eq)]
+6658pub enum ConnectionError {
+6659 /// The peer doesn't implement any supported version
+6660 #[error("peer doesn't implement any supported version")]
+6661 VersionMismatch,
+6662 /// The peer violated the QUIC specification as understood by this implementation
+6663 #[error(transparent)]
+6664 TransportError(#[from] TransportError),
+6665 /// The peer's QUIC stack aborted the connection automatically
+6666 #[error("aborted by peer: {0}")]
+6667 ConnectionClosed(frame::ConnectionClose),
+6668 /// The peer closed the connection
+6669 #[error("closed by peer: {0}")]
+6670 ApplicationClosed(frame::ApplicationClose),
+6671 /// The peer is unable to continue processing this connection, usually due to having restarted
+6672 #[error("reset by peer")]
+6673 Reset,
+6674 /// Communication with the peer has lapsed for longer than the negotiated idle timeout
+6675 ///
+6676 /// If neither side is sending keep-alives, a connection will time out after a long enough idle
+6677 /// period even if the peer is still reachable. See also [`TransportConfig::max_idle_timeout()`]
+6678 /// and [`TransportConfig::keep_alive_interval()`].
+6679 #[error("timed out")]
+6680 TimedOut,
+6681 /// The local application closed the connection
+6682 #[error("closed")]
+6683 LocallyClosed,
+6684 /// The connection could not be created because not enough of the CID space is available
+6685 ///
+6686 /// Try using longer connection IDs.
+6687 #[error("CIDs exhausted")]
+6688 CidsExhausted,
+6689}
+6690
+6691impl From<Close> for ConnectionError {
+6692 fn from(x: Close) -> Self {
+6693 match x {
+6694 Close::Connection(reason) => Self::ConnectionClosed(reason),
+6695 Close::Application(reason) => Self::ApplicationClosed(reason),
+6696 }
+6697 }
+6698}
+6699
+6700// For compatibility with API consumers
+6701impl From<ConnectionError> for io::Error {
+6702 fn from(x: ConnectionError) -> Self {
+6703 use ConnectionError::*;
+6704 let kind = match x {
+6705 TimedOut => io::ErrorKind::TimedOut,
+6706 Reset => io::ErrorKind::ConnectionReset,
+6707 ApplicationClosed(_) | ConnectionClosed(_) => io::ErrorKind::ConnectionAborted,
+6708 TransportError(_) | VersionMismatch | LocallyClosed | CidsExhausted => {
+6709 io::ErrorKind::Other
+6710 }
+6711 };
+6712 Self::new(kind, x)
+6713 }
+6714}
+6715
+6716/// Errors that might trigger a path being closed
+6717// TODO(@divma): maybe needs to be reworked based on what we want to do with the public API
+6718#[derive(Debug, Error, PartialEq, Eq, Clone, Copy)]
+6719pub enum PathError {
+6720 /// The extension was not negotiated with the peer
+6721 #[error("multipath extension not negotiated")]
+6722 MultipathNotNegotiated,
+6723 /// Paths can only be opened client-side
+6724 #[error("the server side may not open a path")]
+6725 ServerSideNotAllowed,
+6726 /// Current limits do not allow us to open more paths
+6727 #[error("maximum number of concurrent paths reached")]
+6728 MaxPathIdReached,
+6729 /// No remote CIDs available to open a new path
+6730 #[error("remoted CIDs exhausted")]
+6731 RemoteCidsExhausted,
+6732 /// Path could not be validated and will be abandoned
+6733 #[error("path validation failed")]
+6734 ValidationFailed,
+6735 /// The remote address for the path is not supported by the endpoint
+6736 #[error("invalid remote address")]
+6737 InvalidRemoteAddress(SocketAddr),
+6738}
+6739
+6740/// Errors triggered when abandoning a path
+6741#[derive(Debug, Error, Clone, Eq, PartialEq)]
+6742pub enum ClosePathError {
+6743 /// The path is already closed or was never opened
+6744 #[error("closed path")]
+6745 ClosedPath,
+6746 /// This is the last path, which can not be abandoned
+6747 #[error("last open path")]
+6748 LastOpenPath,
+6749}
+6750
+6751/// Error when the multipath extension was not negotiated, but attempted to be used.
+6752#[derive(Debug, Error, Clone, Copy)]
+6753#[error("Multipath extension not negotiated")]
+6754pub struct MultipathNotNegotiated {
+6755 _private: (),
+6756}
+6757
+6758/// Events of interest to the application
+6759#[derive(Debug)]
+6760pub enum Event {
+6761 /// The connection's handshake data is ready
+6762 HandshakeDataReady,
+6763 /// The connection was successfully established
+6764 Connected,
+6765 /// The TLS handshake was confirmed
+6766 HandshakeConfirmed,
+6767 /// The connection was lost
+6768 ///
+6769 /// Emitted if the peer closes the connection or an error is encountered.
+6770 ConnectionLost {
+6771 /// Reason that the connection was closed
+6772 reason: ConnectionError,
+6773 },
+6774 /// Stream events
+6775 Stream(StreamEvent),
+6776 /// One or more application datagrams have been received
+6777 DatagramReceived,
+6778 /// One or more application datagrams have been sent after blocking
+6779 DatagramsUnblocked,
+6780 /// (Multi)Path events
+6781 Path(PathEvent),
+6782 /// Iroh's nat traversal events
+6783 NatTraversal(iroh_hp::Event),
+6784}
+6785
+6786impl From<PathEvent> for Event {
+6787 fn from(source: PathEvent) -> Self {
+6788 Self::Path(source)
+6789 }
+6790}
+6791
+6792fn get_max_ack_delay(params: &TransportParameters) -> Duration {
+6793 Duration::from_micros(params.max_ack_delay.0 * 1000)
+6794}
+6795
+6796// Prevents overflow and improves behavior in extreme circumstances
+6797const MAX_BACKOFF_EXPONENT: u32 = 16;
+6798
+6799/// Minimal remaining size to allow packet coalescing, excluding cryptographic tag
+6800///
+6801/// This must be at least as large as the header for a well-formed empty packet to be coalesced,
+6802/// plus some space for frames. We only care about handshake headers because short header packets
+6803/// necessarily have smaller headers, and initial packets are only ever the first packet in a
+6804/// datagram (because we coalesce in ascending packet space order and the only reason to split a
+6805/// packet is when packet space changes).
+6806const MIN_PACKET_SPACE: usize = MAX_HANDSHAKE_OR_0RTT_HEADER_SIZE + 32;
+6807
+6808/// Largest amount of space that could be occupied by a Handshake or 0-RTT packet's header
+6809///
+6810/// Excludes packet-type-specific fields such as packet number or Initial token
+6811// https://www.rfc-editor.org/rfc/rfc9000.html#name-0-rtt: flags + version + dcid len + dcid +
+6812// scid len + scid + length + pn
+6813const MAX_HANDSHAKE_OR_0RTT_HEADER_SIZE: usize =
+6814 1 + 4 + 1 + MAX_CID_SIZE + 1 + MAX_CID_SIZE + VarInt::from_u32(u16::MAX as u32).size() + 4;
+6815
+6816/// Perform key updates this many packets before the AEAD confidentiality limit.
+6817///
+6818/// Chosen arbitrarily, intended to be large enough to prevent spurious connection loss.
+6819const KEY_UPDATE_MARGIN: u64 = 10_000;
+6820
+6821#[derive(Default)]
+6822struct SentFrames {
+6823 retransmits: ThinRetransmits,
+6824 /// The packet number of the largest acknowledged packet for each path
+6825 largest_acked: FxHashMap<PathId, u64>,
+6826 stream_frames: StreamMetaVec,
+6827 /// Whether the packet contains non-retransmittable frames (like datagrams)
+6828 non_retransmits: bool,
+6829 /// If the datagram containing these frames should be padded to the min MTU
+6830 requires_padding: bool,
+6831}
+6832
+6833impl SentFrames {
+6834 /// Returns whether the packet contains only ACKs
+6835 fn is_ack_only(&self, streams: &StreamsState) -> bool {
+6836 !self.largest_acked.is_empty()
+6837 && !self.non_retransmits
+6838 && self.stream_frames.is_empty()
+6839 && self.retransmits.is_empty(streams)
+6840 }
+6841
+6842 fn retransmits_mut(&mut self) -> &mut Retransmits {
+6843 self.retransmits.get_or_create()
+6844 }
+6845
+6846 fn record_sent_frame(&mut self, frame: frame::EncodableFrame<'_>) {
+6847 use frame::EncodableFrame::*;
+6848 match frame {
+6849 PathAck(path_ack_encoder) => {
+6850 if let Some(max) = path_ack_encoder.ranges.max() {
+6851 self.largest_acked.insert(path_ack_encoder.path_id, max);
+6852 }
+6853 }
+6854 Ack(ack_encoder) => {
+6855 if let Some(max) = ack_encoder.ranges.max() {
+6856 self.largest_acked.insert(PathId::ZERO, max);
+6857 }
+6858 }
+6859 Close(_) => { /* non retransmittable, but after this we don't really care */ }
+6860 PathResponse(_) => self.non_retransmits = true,
+6861 HandshakeDone(_) => self.retransmits_mut().handshake_done = true,
+6862 ReachOut(frame::ReachOut { round, ip, port }) => self
+6863 .retransmits_mut()
+6864 .reach_out
+6865 .get_or_insert_with(|| (round, Vec::new()))
+6866 .1
+6867 .push((ip, port)),
+6868 ObservedAddr(_) => self.retransmits_mut().observed_addr = true,
+6869 Ping(_) => self.non_retransmits = true,
+6870 ImmediateAck(_) => self.non_retransmits = true,
+6871 AckFrequency(_) => self.retransmits_mut().ack_frequency = true,
+6872 PathChallenge(_) => self.non_retransmits = true,
+6873 Crypto(crypto) => self.retransmits_mut().crypto.push_back(crypto),
+6874 PathAbandon(path_abandon) => {
+6875 self.retransmits_mut()
+6876 .path_abandon
+6877 .entry(path_abandon.path_id)
+6878 .or_insert(path_abandon.error_code);
+6879 }
+6880 PathStatusAvailable(frame::PathStatusAvailable { path_id, .. })
+6881 | PathStatusBackup(frame::PathStatusBackup { path_id, .. }) => {
+6882 self.retransmits_mut().path_status.insert(path_id);
+6883 }
+6884 MaxPathId(_) => self.retransmits_mut().max_path_id = true,
+6885 PathsBlocked(_) => self.retransmits_mut().paths_blocked = true,
+6886 PathCidsBlocked(path_cids_blocked) => {
+6887 self.retransmits_mut()
+6888 .path_cids_blocked
+6889 .insert(path_cids_blocked.path_id);
+6890 }
+6891 ResetStream(reset) => self
+6892 .retransmits_mut()
+6893 .reset_stream
+6894 .push((reset.id, reset.error_code)),
+6895 StopSending(stop_sending) => self.retransmits_mut().stop_sending.push(stop_sending),
+6896 NewConnectionId(new_cid) => self.retransmits_mut().new_cids.push(new_cid.issued()),
+6897 RetireConnectionId(retire_cid) => self
+6898 .retransmits_mut()
+6899 .retire_cids
+6900 .push((retire_cid.path_id.unwrap_or_default(), retire_cid.sequence)),
+6901 Datagram(_) => self.non_retransmits = true,
+6902 NewToken(_) => {}
+6903 AddAddress(add_address) => {
+6904 self.retransmits_mut().add_address.insert(add_address);
+6905 }
+6906 RemoveAddress(remove_address) => {
+6907 self.retransmits_mut().remove_address.insert(remove_address);
+6908 }
+6909 StreamMeta(stream_meta_encoder) => self.stream_frames.push(stream_meta_encoder.meta),
+6910 MaxData(_) => self.retransmits_mut().max_data = true,
+6911 MaxStreamData(max) => {
+6912 self.retransmits_mut().max_stream_data.insert(max.id);
+6913 }
+6914 MaxStreams(max_streams) => {
+6915 self.retransmits_mut().max_stream_id[max_streams.dir as usize] = true
+6916 }
+6917 }
+6918 }
+6919}
+6920
+6921/// Compute the negotiated idle timeout based on local and remote max_idle_timeout transport parameters.
+6922///
+6923/// According to the definition of max_idle_timeout, a value of `0` means the timeout is disabled; see <https://www.rfc-editor.org/rfc/rfc9000#section-18.2-4.4.1.>
+6924///
+6925/// According to the negotiation procedure, either the minimum of the timeouts or one specified is used as the negotiated value; see <https://www.rfc-editor.org/rfc/rfc9000#section-10.1-2.>
+6926///
+6927/// Returns the negotiated idle timeout as a `Duration`, or `None` when both endpoints have opted out of idle timeout.
+6928fn negotiate_max_idle_timeout(x: Option<VarInt>, y: Option<VarInt>) -> Option<Duration> {
+6929 match (x, y) {
+6930 (Some(VarInt(0)) | None, Some(VarInt(0)) | None) => None,
+6931 (Some(VarInt(0)) | None, Some(y)) => Some(Duration::from_millis(y.0)),
+6932 (Some(x), Some(VarInt(0)) | None) => Some(Duration::from_millis(x.0)),
+6933 (Some(x), Some(y)) => Some(Duration::from_millis(cmp::min(x, y).0)),
+6934 }
+6935}
+6936
+6937#[cfg(test)]
+6938mod tests {
+6939 use super::*;
+6940
+6941 #[test]
+6942 fn negotiate_max_idle_timeout_commutative() {
+6943 let test_params = [
+6944 (None, None, None),
+6945 (None, Some(VarInt(0)), None),
+6946 (None, Some(VarInt(2)), Some(Duration::from_millis(2))),
+6947 (Some(VarInt(0)), Some(VarInt(0)), None),
+6948 (
+6949 Some(VarInt(2)),
+6950 Some(VarInt(0)),
+6951 Some(Duration::from_millis(2)),
+6952 ),
+6953 (
+6954 Some(VarInt(1)),
+6955 Some(VarInt(4)),
+6956 Some(Duration::from_millis(1)),
+6957 ),
+6958 ];
+6959
+6960 for (left, right, result) in test_params {
+6961 assert_eq!(negotiate_max_idle_timeout(left, right), result);
+6962 assert_eq!(negotiate_max_idle_timeout(right, left), result);
+6963 }
+6964 }
+6965}
\ No newline at end of file
diff --git a/pr/255/docs/src/iroh_quinn_proto/connection/packet_builder.rs.html b/pr/255/docs/src/iroh_quinn_proto/connection/packet_builder.rs.html
index cb07743e9..923cf2568 100644
--- a/pr/255/docs/src/iroh_quinn_proto/connection/packet_builder.rs.html
+++ b/pr/255/docs/src/iroh_quinn_proto/connection/packet_builder.rs.html
@@ -282,7 +282,7 @@
282 conn: &mut Connection,
283 path_id: PathId,
284 pad_datagram: PadDatagram,
-285 ) {
+285 ) -> u64 {
286 match pad_datagram {
287 PadDatagram::No => (),
288 PadDatagram::ToSize(size) => self.pad_to(size),
@@ -329,121 +329,123 @@
329 conn.set_loss_detection_timer(now, path_id);
330 conn.path_data_mut(path_id).pacing.on_transmit(size);
331 }
-332 }
-333
-334 /// Encrypt packet, returning the length of the packet and whether padding was added
-335 pub(super) fn finish(
-336 mut self,
-337 conn: &mut Connection,
-338 now: Instant,
-339 ) -> (usize, bool, SentFrames) {
-340 debug_assert!(
-341 self.buf.len() <= self.buf.datagram_max_offset() - self.tag_len,
-342 "packet exceeds maximum size"
-343 );
-344 let pad = self.buf.len() < self.min_size;
-345 if pad {
-346 let padding = self.min_size - self.buf.len();
-347 trace!("PADDING * {}", padding);
-348 self.buf.put_bytes(0, padding);
-349 self.qlog.frame_padding(padding);
-350 }
-351
-352 let space = &conn.spaces[self.space];
-353 let (header_crypto, packet_crypto) = if let Some(ref crypto) = space.crypto {
-354 (&*crypto.header.local, &*crypto.packet.local)
-355 } else if self.space == SpaceId::Data {
-356 let zero_rtt = conn.zero_rtt_crypto.as_ref().unwrap();
-357 (&*zero_rtt.header, &*zero_rtt.packet)
-358 } else {
-359 unreachable!("tried to send {:?} packet without keys", self.space);
-360 };
-361
-362 debug_assert_eq!(
-363 packet_crypto.tag_len(),
-364 self.tag_len,
-365 "Mismatching crypto tag len"
-366 );
-367
-368 self.buf.put_bytes(0, packet_crypto.tag_len());
-369 let encode_start = self.partial_encode.start;
-370 let packet_buf = &mut self.buf.as_mut_slice()[encode_start..];
-371 // for packet protection, PathId::ZERO and no path are equivalent.
-372 self.partial_encode.finish(
-373 packet_buf,
-374 header_crypto,
-375 Some((self.exact_number, self.path, packet_crypto)),
-376 );
-377
-378 let packet_len = self.buf.len() - encode_start;
-379 trace!(size = %packet_len, short_header = %self.short_header, "wrote packet");
-380 self.qlog.finalize(packet_len);
-381 conn.qlog.emit_packet_sent(self.qlog, now);
-382 (packet_len, pad, self.sent_frames)
-383 }
-384
-385 /// The number of additional bytes the current packet would take up if it was finished now
-386 ///
-387 /// This will include any padding which is required to make the size large enough to be
-388 /// encrypted correctly.
-389 pub(super) fn predict_packet_end(&self) -> usize {
-390 self.buf.len().max(self.min_size) + self.tag_len - self.buf.len()
-391 }
-392
-393 /// Returns the remaining space in the packet that can be taken up by QUIC frames
-394 ///
-395 /// This leaves space in the datagram for the cryptographic tag that needs to be written
-396 /// when the packet is finished.
-397 pub(super) fn frame_space_remaining(&self) -> usize {
-398 let max_offset = self.buf.datagram_max_offset() - self.tag_len;
-399 max_offset.saturating_sub(self.buf.len())
-400 }
-401
-402 pub(crate) fn require_padding(&mut self) {
-403 self.sent_frames.requires_padding = true;
-404 }
-405
-406 pub(crate) fn retransmits_mut(&mut self) -> &mut Retransmits {
-407 self.sent_frames.retransmits_mut()
-408 }
-409}
-410
-411#[derive(Debug, Copy, Clone)]
-412pub(super) enum PadDatagram {
-413 /// Do not pad the datagram
-414 No,
-415 /// To a specific size
-416 ToSize(u16),
-417 /// Pad to the current MTU/segment size
-418 ///
-419 /// For the first datagram in a transmit the MTU is the same as the
-420 /// [`TransmitBuf::segment_size`].
-421 ToSegmentSize,
-422 /// Pad to [`MIN_INITIAL_SIZE`], the minimal QUIC MTU of 1200 bytes
-423 ToMinMtu,
-424}
-425
-426impl std::ops::BitOrAssign for PadDatagram {
-427 fn bitor_assign(&mut self, rhs: Self) {
-428 *self = *self | rhs;
-429 }
-430}
-431
-432impl std::ops::BitOr for PadDatagram {
-433 type Output = Self;
-434
-435 fn bitor(self, rhs: Self) -> Self::Output {
-436 match (self, rhs) {
-437 (Self::No, rhs) => rhs,
-438 (Self::ToSize(size), Self::No) => Self::ToSize(size),
-439 (Self::ToSize(a), Self::ToSize(b)) => Self::ToSize(a.max(b)),
-440 (Self::ToSize(_), Self::ToSegmentSize) => Self::ToSegmentSize,
-441 (Self::ToSize(_), Self::ToMinMtu) => Self::ToMinMtu,
-442 (Self::ToSegmentSize, Self::No) => Self::ToSegmentSize,
-443 (Self::ToSegmentSize, Self::ToSize(_)) => Self::ToSegmentSize,
-444 (Self::ToSegmentSize, Self::ToSegmentSize) => Self::ToSegmentSize,
-445 (Self::ToSegmentSize, Self::ToMinMtu) => Self::ToMinMtu,
-446 (Self::ToMinMtu, _) => Self::ToMinMtu,
-447 }
-448 }
-449}
impl<T> Any for Twhere
+MultipathNotNegotiated in iroh_quinn_proto - Rust MultipathNotNegotiated
pub struct MultipathNotNegotiated { /* private fields */ }
Expand description
Error when the multipath extension was not negotiated, but attempted to be used.
+Trait Implementations§
Source§impl Clone for MultipathNotNegotiated
Source§fn clone(&self) -> MultipathNotNegotiated
Returns a duplicate of the value. Read more1.0.0 · Source§fn clone_from(&mut self, source: &Self)
Performs copy-assignment from source. Read moreSource§impl Debug for MultipathNotNegotiated
Source§impl Display for MultipathNotNegotiated
Source§impl Error for MultipathNotNegotiated
1.30.0 · Source§fn source(&self) -> Option<&(dyn Error + 'static)>
Returns the lower-level source of this error, if any. Read more1.0.0 · Source§fn description(&self) -> &str
👎Deprecated since 1.42.0: use the Display impl or to_string()Source§impl Copy for MultipathNotNegotiated
Auto Trait Implementations§
§impl Freeze for MultipathNotNegotiated
§impl RefUnwindSafe for MultipathNotNegotiated
§impl Send for MultipathNotNegotiated
§impl Sync for MultipathNotNegotiated
§impl Unpin for MultipathNotNegotiated
§impl UnwindSafe for MultipathNotNegotiated
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read moreSource§impl<T> CloneToUninit for Twhere
diff --git a/pr/255/docs/iroh_quinn_proto/struct.TransportError.html b/pr/255/docs/iroh_quinn_proto/struct.TransportError.html
index c4be43306..b6dc15d42 100644
--- a/pr/255/docs/iroh_quinn_proto/struct.TransportError.html
+++ b/pr/255/docs/iroh_quinn_proto/struct.TransportError.html
@@ -11,7 +11,7 @@
§reason: StringHuman-readable explanation of the reason
§crypto: Option<Arc<dyn Error + Send + Sync>>An underlying crypto (e.g. TLS) layer error
Implementations§
Trait Implementations§
Source§impl Error for Error
1.30.0 · Source§fn source(&self) -> Option<&(dyn Error + 'static)>
Returns the lower-level source of this error, if any. Read more1.0.0 · Source§fn description(&self) -> &str
👎Deprecated since 1.42.0: use the Display impl or to_string()Source§impl From<Error> for ConnectionClose
Source§fn from(x: TransportError) -> Self
Converts to this type from the input type.Source§impl From<Error> for ConnectionError
Source§fn from(source: TransportError) -> Self
Converts to this type from the input type.Source§impl From<Error> for TransportError
Trait Implementations§
Source§impl Error for Error
1.30.0 · Source§fn source(&self) -> Option<&(dyn Error + 'static)>
Returns the lower-level source of this error, if any. Read more1.0.0 · Source§fn description(&self) -> &str
👎Deprecated since 1.42.0: use the Display impl or to_string()Source§impl From<Error> for ConnectionClose
Source§fn from(x: TransportError) -> Self
Converts to this type from the input type.Source§impl From<Error> for ConnectionError
Source§fn from(source: TransportError) -> Self
Converts to this type from the input type.Source§impl From<Error> for TransportError
Source§impl Eq for Error
Auto Trait Implementations§
§impl Freeze for Error
§impl !RefUnwindSafe for Error
§impl Send for Error
§impl Sync for Error
§impl Unpin for Error
§impl !UnwindSafe for Error
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
diff --git a/pr/255/docs/src/iroh_quinn_proto/connection/mod.rs.html b/pr/255/docs/src/iroh_quinn_proto/connection/mod.rs.html
index 81ec7afd0..e1b5a58ea 100644
--- a/pr/255/docs/src/iroh_quinn_proto/connection/mod.rs.html
+++ b/pr/255/docs/src/iroh_quinn_proto/connection/mod.rs.html
@@ -343,826 +343,826 @@
343/// Return value for `poll_transmit_path_space`.
344#[derive(Debug)]
345enum PollPathSpaceStatus {
-346 /// Nothing to send on this space, continue to the next one.
-347 NextSpace,
-348 /// We have something to send, and data has been accumulated
-349 /// on the passed in `transmit`.
-350 Send { last_packet_number: Option<u64> },
-351 /// A transmit is ready to be sent out.
-352 SendTransmit {
-353 /// The transmit to send.
-354 transmit: Transmit,
-355 },
-356 /// Nothing to send currently.
-357 NothingToSend {
-358 /// Set to `true` if we consider the current inability to send something because of congestion control
-359 congestion_blocked: bool,
-360 },
-361}
-362
-363impl Connection {
-364 pub(crate) fn new(
-365 endpoint_config: Arc<EndpointConfig>,
-366 config: Arc<TransportConfig>,
-367 init_cid: ConnectionId,
-368 loc_cid: ConnectionId,
-369 rem_cid: ConnectionId,
-370 network_path: FourTuple,
-371 crypto: Box<dyn crypto::Session>,
-372 cid_gen: &dyn ConnectionIdGenerator,
-373 now: Instant,
-374 version: u32,
-375 allow_mtud: bool,
-376 rng_seed: [u8; 32],
-377 side_args: SideArgs,
-378 qlog: QlogSink,
-379 ) -> Self {
-380 let pref_addr_cid = side_args.pref_addr_cid();
-381 let path_validated = side_args.path_validated();
-382 let connection_side = ConnectionSide::from(side_args);
-383 let side = connection_side.side();
-384 let mut rng = StdRng::from_seed(rng_seed);
-385 let initial_space = {
-386 let mut space = PacketSpace::new(now, SpaceId::Initial, &mut rng);
-387 space.crypto = Some(crypto.initial_keys(init_cid, side));
-388 space
-389 };
-390 let handshake_space = PacketSpace::new(now, SpaceId::Handshake, &mut rng);
-391 #[cfg(test)]
-392 let data_space = match config.deterministic_packet_numbers {
-393 true => PacketSpace::new_deterministic(now, SpaceId::Data),
-394 false => PacketSpace::new(now, SpaceId::Data, &mut rng),
-395 };
-396 #[cfg(not(test))]
-397 let data_space = PacketSpace::new(now, SpaceId::Data, &mut rng);
-398 let state = State::handshake(state::Handshake {
-399 rem_cid_set: side.is_server(),
-400 expected_token: Bytes::new(),
-401 client_hello: None,
-402 allow_server_migration: side.is_client(),
-403 });
-404 let local_cid_state = FxHashMap::from_iter([(
-405 PathId::ZERO,
-406 CidState::new(
-407 cid_gen.cid_len(),
-408 cid_gen.cid_lifetime(),
-409 now,
-410 if pref_addr_cid.is_some() { 2 } else { 1 },
-411 ),
-412 )]);
-413
-414 let mut path = PathData::new(network_path, allow_mtud, None, 0, now, &config);
-415 // TODO(@divma): consider if we want to delay this until the path is validated
-416 path.open = true;
-417 let mut this = Self {
-418 endpoint_config,
-419 crypto,
-420 handshake_cid: loc_cid,
-421 rem_handshake_cid: rem_cid,
-422 local_cid_state,
-423 paths: BTreeMap::from_iter([(
-424 PathId::ZERO,
-425 PathState {
-426 data: path,
-427 prev: None,
-428 },
-429 )]),
-430 path_generation_counter: 0,
-431 allow_mtud,
-432 state,
-433 side: connection_side,
-434 zero_rtt_enabled: false,
-435 zero_rtt_crypto: None,
-436 key_phase: false,
-437 // A small initial key phase size ensures peers that don't handle key updates correctly
-438 // fail sooner rather than later. It's okay for both peers to do this, as the first one
-439 // to perform an update will reset the other's key phase size in `update_keys`, and a
-440 // simultaneous key update by both is just like a regular key update with a really fast
-441 // response. Inspired by quic-go's similar behavior of performing the first key update
-442 // at the 100th short-header packet.
-443 key_phase_size: rng.random_range(10..1000),
-444 peer_params: TransportParameters::default(),
-445 orig_rem_cid: rem_cid,
-446 initial_dst_cid: init_cid,
-447 retry_src_cid: None,
-448 events: VecDeque::new(),
-449 endpoint_events: VecDeque::new(),
-450 spin_enabled: config.allow_spin && rng.random_ratio(7, 8),
-451 spin: false,
-452 spaces: [initial_space, handshake_space, data_space],
-453 highest_space: SpaceId::Initial,
-454 prev_crypto: None,
-455 next_crypto: None,
-456 accepted_0rtt: false,
-457 permit_idle_reset: true,
-458 idle_timeout: match config.max_idle_timeout {
-459 None | Some(VarInt(0)) => None,
-460 Some(dur) => Some(Duration::from_millis(dur.0)),
-461 },
-462 timers: TimerTable::default(),
-463 authentication_failures: 0,
-464 close: false,
-465
-466 ack_frequency: AckFrequencyState::new(get_max_ack_delay(
-467 &TransportParameters::default(),
-468 )),
-469
-470 app_limited: false,
-471 receiving_ecn: false,
-472 total_authed_packets: 0,
+346 /// We have something to send, and data has been accumulated
+347 /// on the passed in `transmit`.
+348 Send { last_packet_number: Option<u64> },
+349 /// A transmit is ready to be sent out.
+350 SendTransmit {
+351 /// The transmit to send.
+352 transmit: Transmit,
+353 },
+354 /// Nothing to send currently on this space/path.
+355 NothingToSend {
+356 /// Set to `true` if we consider the current inability to send something because of congestion control
+357 congestion_blocked: bool,
+358 },
+359}
+360
+361impl Connection {
+362 pub(crate) fn new(
+363 endpoint_config: Arc<EndpointConfig>,
+364 config: Arc<TransportConfig>,
+365 init_cid: ConnectionId,
+366 loc_cid: ConnectionId,
+367 rem_cid: ConnectionId,
+368 network_path: FourTuple,
+369 crypto: Box<dyn crypto::Session>,
+370 cid_gen: &dyn ConnectionIdGenerator,
+371 now: Instant,
+372 version: u32,
+373 allow_mtud: bool,
+374 rng_seed: [u8; 32],
+375 side_args: SideArgs,
+376 qlog: QlogSink,
+377 ) -> Self {
+378 let pref_addr_cid = side_args.pref_addr_cid();
+379 let path_validated = side_args.path_validated();
+380 let connection_side = ConnectionSide::from(side_args);
+381 let side = connection_side.side();
+382 let mut rng = StdRng::from_seed(rng_seed);
+383 let initial_space = {
+384 let mut space = PacketSpace::new(now, SpaceId::Initial, &mut rng);
+385 space.crypto = Some(crypto.initial_keys(init_cid, side));
+386 space
+387 };
+388 let handshake_space = PacketSpace::new(now, SpaceId::Handshake, &mut rng);
+389 #[cfg(test)]
+390 let data_space = match config.deterministic_packet_numbers {
+391 true => PacketSpace::new_deterministic(now, SpaceId::Data),
+392 false => PacketSpace::new(now, SpaceId::Data, &mut rng),
+393 };
+394 #[cfg(not(test))]
+395 let data_space = PacketSpace::new(now, SpaceId::Data, &mut rng);
+396 let state = State::handshake(state::Handshake {
+397 rem_cid_set: side.is_server(),
+398 expected_token: Bytes::new(),
+399 client_hello: None,
+400 allow_server_migration: side.is_client(),
+401 });
+402 let local_cid_state = FxHashMap::from_iter([(
+403 PathId::ZERO,
+404 CidState::new(
+405 cid_gen.cid_len(),
+406 cid_gen.cid_lifetime(),
+407 now,
+408 if pref_addr_cid.is_some() { 2 } else { 1 },
+409 ),
+410 )]);
+411
+412 let mut path = PathData::new(network_path, allow_mtud, None, 0, now, &config);
+413 // TODO(@divma): consider if we want to delay this until the path is validated
+414 path.open = true;
+415 let mut this = Self {
+416 endpoint_config,
+417 crypto,
+418 handshake_cid: loc_cid,
+419 rem_handshake_cid: rem_cid,
+420 local_cid_state,
+421 paths: BTreeMap::from_iter([(
+422 PathId::ZERO,
+423 PathState {
+424 data: path,
+425 prev: None,
+426 },
+427 )]),
+428 path_generation_counter: 0,
+429 allow_mtud,
+430 state,
+431 side: connection_side,
+432 zero_rtt_enabled: false,
+433 zero_rtt_crypto: None,
+434 key_phase: false,
+435 // A small initial key phase size ensures peers that don't handle key updates correctly
+436 // fail sooner rather than later. It's okay for both peers to do this, as the first one
+437 // to perform an update will reset the other's key phase size in `update_keys`, and a
+438 // simultaneous key update by both is just like a regular key update with a really fast
+439 // response. Inspired by quic-go's similar behavior of performing the first key update
+440 // at the 100th short-header packet.
+441 key_phase_size: rng.random_range(10..1000),
+442 peer_params: TransportParameters::default(),
+443 orig_rem_cid: rem_cid,
+444 initial_dst_cid: init_cid,
+445 retry_src_cid: None,
+446 events: VecDeque::new(),
+447 endpoint_events: VecDeque::new(),
+448 spin_enabled: config.allow_spin && rng.random_ratio(7, 8),
+449 spin: false,
+450 spaces: [initial_space, handshake_space, data_space],
+451 highest_space: SpaceId::Initial,
+452 prev_crypto: None,
+453 next_crypto: None,
+454 accepted_0rtt: false,
+455 permit_idle_reset: true,
+456 idle_timeout: match config.max_idle_timeout {
+457 None | Some(VarInt(0)) => None,
+458 Some(dur) => Some(Duration::from_millis(dur.0)),
+459 },
+460 timers: TimerTable::default(),
+461 authentication_failures: 0,
+462 close: false,
+463
+464 ack_frequency: AckFrequencyState::new(get_max_ack_delay(
+465 &TransportParameters::default(),
+466 )),
+467
+468 app_limited: false,
+469 receiving_ecn: false,
+470 total_authed_packets: 0,
+471
+472 next_observed_addr_seq_no: 0u32.into(),
473
-474 next_observed_addr_seq_no: 0u32.into(),
-475
-476 streams: StreamsState::new(
-477 side,
-478 config.max_concurrent_uni_streams,
-479 config.max_concurrent_bidi_streams,
-480 config.send_window,
-481 config.receive_window,
-482 config.stream_receive_window,
-483 ),
-484 datagrams: DatagramState::default(),
-485 config,
-486 rem_cids: FxHashMap::from_iter([(PathId::ZERO, CidQueue::new(rem_cid))]),
-487 rng,
-488 stats: ConnectionStats::default(),
-489 path_stats: Default::default(),
-490 version,
-491
-492 // peer params are not yet known, so multipath is not enabled
-493 max_concurrent_paths: NonZeroU32::MIN,
-494 local_max_path_id: PathId::ZERO,
-495 remote_max_path_id: PathId::ZERO,
-496 max_path_id_with_cids: PathId::ZERO,
-497 abandoned_paths: Default::default(),
-498
-499 // iroh's nat traversal
-500 iroh_hp: Default::default(),
-501 qlog,
-502 };
-503 if path_validated {
-504 this.on_path_validated(PathId::ZERO);
-505 }
-506 if side.is_client() {
-507 // Kick off the connection
-508 this.write_crypto();
-509 this.init_0rtt(now);
-510 }
-511 this.qlog
-512 .emit_tuple_assigned(PathId::ZERO, network_path, now);
-513 this
-514 }
-515
-516 /// Returns the next time at which `handle_timeout` should be called
-517 ///
-518 /// The value returned may change after:
-519 /// - the application performed some I/O on the connection
-520 /// - a call was made to `handle_event`
-521 /// - a call to `poll_transmit` returned `Some`
-522 /// - a call was made to `handle_timeout`
-523 #[must_use]
-524 pub fn poll_timeout(&mut self) -> Option<Instant> {
-525 self.timers.peek()
-526 }
-527
-528 /// Returns application-facing events
-529 ///
-530 /// Connections should be polled for events after:
-531 /// - a call was made to `handle_event`
-532 /// - a call was made to `handle_timeout`
-533 #[must_use]
-534 pub fn poll(&mut self) -> Option<Event> {
-535 if let Some(x) = self.events.pop_front() {
-536 return Some(x);
-537 }
-538
-539 if let Some(event) = self.streams.poll() {
-540 return Some(Event::Stream(event));
-541 }
-542
-543 if let Some(reason) = self.state.take_error() {
-544 return Some(Event::ConnectionLost { reason });
-545 }
-546
-547 None
-548 }
-549
-550 /// Return endpoint-facing events
-551 #[must_use]
-552 pub fn poll_endpoint_events(&mut self) -> Option<EndpointEvent> {
-553 self.endpoint_events.pop_front().map(EndpointEvent)
-554 }
-555
-556 /// Provide control over streams
-557 #[must_use]
-558 pub fn streams(&mut self) -> Streams<'_> {
-559 Streams {
-560 state: &mut self.streams,
-561 conn_state: &self.state,
-562 }
-563 }
-564
-565 /// Provide control over streams
-566 #[must_use]
-567 pub fn recv_stream(&mut self, id: StreamId) -> RecvStream<'_> {
-568 assert!(id.dir() == Dir::Bi || id.initiator() != self.side.side());
-569 RecvStream {
-570 id,
-571 state: &mut self.streams,
-572 pending: &mut self.spaces[SpaceId::Data].pending,
-573 }
-574 }
-575
-576 /// Provide control over streams
-577 #[must_use]
-578 pub fn send_stream(&mut self, id: StreamId) -> SendStream<'_> {
-579 assert!(id.dir() == Dir::Bi || id.initiator() == self.side.side());
-580 SendStream {
-581 id,
-582 state: &mut self.streams,
-583 pending: &mut self.spaces[SpaceId::Data].pending,
-584 conn_state: &self.state,
-585 }
-586 }
-587
-588 /// Opens a new path only if no path on the same network path currently exists.
-589 ///
-590 /// This comparison will use [`FourTuple::is_probably_same_path`] on the given `network_path`
-591 /// and pass it existing path's network paths.
-592 ///
-593 /// This means that you can pass `local_ip: None` to make the comparison only compare
-594 /// remote addresses.
-595 ///
-596 /// This avoids having to guess which local interface will be used to communicate with the
-597 /// remote, should it not be known yet. We assume that if we already have a path to the remote,
-598 /// the OS is likely to use the same interface to talk to said remote.
-599 ///
-600 /// See also [`open_path`]. Returns `(path_id, true)` if the path already existed. `(path_id,
-601 /// false)` if was opened.
-602 ///
-603 /// [`open_path`]: Connection::open_path
-604 pub fn open_path_ensure(
-605 &mut self,
-606 network_path: FourTuple,
-607 initial_status: PathStatus,
-608 now: Instant,
-609 ) -> Result<(PathId, bool), PathError> {
-610 let existing_open_path = self.paths.iter().find(|(id, path)| {
-611 network_path.is_probably_same_path(&path.data.network_path)
-612 && !self.abandoned_paths.contains(*id)
-613 });
-614 match existing_open_path {
-615 Some((path_id, _state)) => Ok((*path_id, true)),
-616 None => Ok((self.open_path(network_path, initial_status, now)?, false)),
-617 }
-618 }
-619
-620 /// Opens a new path
-621 ///
-622 /// Further errors might occur and they will be emitted in [`PathEvent::LocallyClosed`] events.
-623 /// When the path is opened it will be reported as an [`PathEvent::Opened`].
-624 pub fn open_path(
-625 &mut self,
-626 network_path: FourTuple,
-627 initial_status: PathStatus,
-628 now: Instant,
-629 ) -> Result<PathId, PathError> {
-630 if !self.is_multipath_negotiated() {
-631 return Err(PathError::MultipathNotNegotiated);
-632 }
-633 if self.side().is_server() {
-634 return Err(PathError::ServerSideNotAllowed);
-635 }
-636
-637 let max_abandoned = self.abandoned_paths.iter().max().copied();
-638 let max_used = self.paths.keys().last().copied();
-639 let path_id = max_abandoned
-640 .max(max_used)
-641 .unwrap_or(PathId::ZERO)
-642 .saturating_add(1u8);
-643
-644 if Some(path_id) > self.max_path_id() {
-645 return Err(PathError::MaxPathIdReached);
-646 }
-647 if path_id > self.remote_max_path_id {
-648 self.spaces[SpaceId::Data].pending.paths_blocked = true;
-649 return Err(PathError::MaxPathIdReached);
-650 }
-651 if self.rem_cids.get(&path_id).map(CidQueue::active).is_none() {
-652 self.spaces[SpaceId::Data]
-653 .pending
-654 .path_cids_blocked
-655 .insert(path_id);
-656 return Err(PathError::RemoteCidsExhausted);
-657 }
-658
-659 let path = self.ensure_path(path_id, network_path, now, None);
-660 path.status.local_update(initial_status);
-661
-662 Ok(path_id)
-663 }
-664
-665 /// Closes a path by sending a PATH_ABANDON frame
-666 ///
-667 /// This will not allow closing the last path. It does allow closing paths which have
-668 /// not yet been opened, as e.g. is the case when receiving a PATH_ABANDON from the peer
-669 /// for a path that was never opened locally.
-670 pub fn close_path(
-671 &mut self,
-672 now: Instant,
-673 path_id: PathId,
-674 error_code: VarInt,
-675 ) -> Result<(), ClosePathError> {
-676 if self.abandoned_paths.contains(&path_id)
-677 || Some(path_id) > self.max_path_id()
-678 || !self.paths.contains_key(&path_id)
-679 {
-680 return Err(ClosePathError::ClosedPath);
-681 }
-682 if self
-683 .paths
-684 .iter()
-685 // Would there be any remaining, non-abandoned, validated paths
-686 .any(|(id, path)| {
-687 *id != path_id && !self.abandoned_paths.contains(id) && path.data.validated
-688 })
-689 .not()
-690 {
-691 return Err(ClosePathError::LastOpenPath);
-692 }
-693
-694 // Send PATH_ABANDON
-695 self.spaces[SpaceId::Data]
-696 .pending
-697 .path_abandon
-698 .insert(path_id, error_code.into());
-699
-700 // Remove pending NEW CIDs for this path
-701 let pending_space = &mut self.spaces[SpaceId::Data].pending;
-702 pending_space.new_cids.retain(|cid| cid.path_id != path_id);
-703 pending_space.path_cids_blocked.retain(|&id| id != path_id);
-704 pending_space.path_status.retain(|&id| id != path_id);
-705
-706 // Cleanup retransmits across ALL paths (CIDs for path_id may have been transmitted on other paths)
-707 for space in self.spaces[SpaceId::Data].iter_paths_mut() {
-708 for sent_packet in space.sent_packets.values_mut() {
-709 if let Some(retransmits) = sent_packet.retransmits.get_mut() {
-710 retransmits.new_cids.retain(|cid| cid.path_id != path_id);
-711 retransmits.path_cids_blocked.retain(|&id| id != path_id);
-712 retransmits.path_status.retain(|&id| id != path_id);
-713 }
-714 }
-715 }
-716
-717 // Consider remotely issued CIDs as retired.
-718 // Technically we don't have to do this just yet. We only need to do this *after*
-719 // the ABANDON_PATH frame is sent, allowing us to still send it on the
-720 // to-be-abandoned path. However it is recommended to send it on another path, and
-721 // we do not allow abandoning the last path anyway.
-722 self.rem_cids.remove(&path_id);
-723 self.endpoint_events
-724 .push_back(EndpointEventInner::RetireResetToken(path_id));
+474 streams: StreamsState::new(
+475 side,
+476 config.max_concurrent_uni_streams,
+477 config.max_concurrent_bidi_streams,
+478 config.send_window,
+479 config.receive_window,
+480 config.stream_receive_window,
+481 ),
+482 datagrams: DatagramState::default(),
+483 config,
+484 rem_cids: FxHashMap::from_iter([(PathId::ZERO, CidQueue::new(rem_cid))]),
+485 rng,
+486 stats: ConnectionStats::default(),
+487 path_stats: Default::default(),
+488 version,
+489
+490 // peer params are not yet known, so multipath is not enabled
+491 max_concurrent_paths: NonZeroU32::MIN,
+492 local_max_path_id: PathId::ZERO,
+493 remote_max_path_id: PathId::ZERO,
+494 max_path_id_with_cids: PathId::ZERO,
+495 abandoned_paths: Default::default(),
+496
+497 // iroh's nat traversal
+498 iroh_hp: Default::default(),
+499 qlog,
+500 };
+501 if path_validated {
+502 this.on_path_validated(PathId::ZERO);
+503 }
+504 if side.is_client() {
+505 // Kick off the connection
+506 this.write_crypto();
+507 this.init_0rtt(now);
+508 }
+509 this.qlog
+510 .emit_tuple_assigned(PathId::ZERO, network_path, now);
+511 this
+512 }
+513
+514 /// Returns the next time at which `handle_timeout` should be called
+515 ///
+516 /// The value returned may change after:
+517 /// - the application performed some I/O on the connection
+518 /// - a call was made to `handle_event`
+519 /// - a call to `poll_transmit` returned `Some`
+520 /// - a call was made to `handle_timeout`
+521 #[must_use]
+522 pub fn poll_timeout(&mut self) -> Option<Instant> {
+523 self.timers.peek()
+524 }
+525
+526 /// Returns application-facing events
+527 ///
+528 /// Connections should be polled for events after:
+529 /// - a call was made to `handle_event`
+530 /// - a call was made to `handle_timeout`
+531 #[must_use]
+532 pub fn poll(&mut self) -> Option<Event> {
+533 if let Some(x) = self.events.pop_front() {
+534 return Some(x);
+535 }
+536
+537 if let Some(event) = self.streams.poll() {
+538 return Some(Event::Stream(event));
+539 }
+540
+541 if let Some(reason) = self.state.take_error() {
+542 return Some(Event::ConnectionLost { reason });
+543 }
+544
+545 None
+546 }
+547
+548 /// Return endpoint-facing events
+549 #[must_use]
+550 pub fn poll_endpoint_events(&mut self) -> Option<EndpointEvent> {
+551 self.endpoint_events.pop_front().map(EndpointEvent)
+552 }
+553
+554 /// Provide control over streams
+555 #[must_use]
+556 pub fn streams(&mut self) -> Streams<'_> {
+557 Streams {
+558 state: &mut self.streams,
+559 conn_state: &self.state,
+560 }
+561 }
+562
+563 /// Provide control over streams
+564 #[must_use]
+565 pub fn recv_stream(&mut self, id: StreamId) -> RecvStream<'_> {
+566 assert!(id.dir() == Dir::Bi || id.initiator() != self.side.side());
+567 RecvStream {
+568 id,
+569 state: &mut self.streams,
+570 pending: &mut self.spaces[SpaceId::Data].pending,
+571 }
+572 }
+573
+574 /// Provide control over streams
+575 #[must_use]
+576 pub fn send_stream(&mut self, id: StreamId) -> SendStream<'_> {
+577 assert!(id.dir() == Dir::Bi || id.initiator() == self.side.side());
+578 SendStream {
+579 id,
+580 state: &mut self.streams,
+581 pending: &mut self.spaces[SpaceId::Data].pending,
+582 conn_state: &self.state,
+583 }
+584 }
+585
+586 /// Opens a new path only if no path on the same network path currently exists.
+587 ///
+588 /// This comparison will use [`FourTuple::is_probably_same_path`] on the given `network_path`
+589 /// and pass it existing path's network paths.
+590 ///
+591 /// This means that you can pass `local_ip: None` to make the comparison only compare
+592 /// remote addresses.
+593 ///
+594 /// This avoids having to guess which local interface will be used to communicate with the
+595 /// remote, should it not be known yet. We assume that if we already have a path to the remote,
+596 /// the OS is likely to use the same interface to talk to said remote.
+597 ///
+598 /// See also [`open_path`]. Returns `(path_id, true)` if the path already existed. `(path_id,
+599 /// false)` if was opened.
+600 ///
+601 /// [`open_path`]: Connection::open_path
+602 pub fn open_path_ensure(
+603 &mut self,
+604 network_path: FourTuple,
+605 initial_status: PathStatus,
+606 now: Instant,
+607 ) -> Result<(PathId, bool), PathError> {
+608 let existing_open_path = self.paths.iter().find(|(id, path)| {
+609 network_path.is_probably_same_path(&path.data.network_path)
+610 && !self.abandoned_paths.contains(*id)
+611 });
+612 match existing_open_path {
+613 Some((path_id, _state)) => Ok((*path_id, true)),
+614 None => Ok((self.open_path(network_path, initial_status, now)?, false)),
+615 }
+616 }
+617
+618 /// Opens a new path
+619 ///
+620 /// Further errors might occur and they will be emitted in [`PathEvent::LocallyClosed`] events.
+621 /// When the path is opened it will be reported as an [`PathEvent::Opened`].
+622 pub fn open_path(
+623 &mut self,
+624 network_path: FourTuple,
+625 initial_status: PathStatus,
+626 now: Instant,
+627 ) -> Result<PathId, PathError> {
+628 if !self.is_multipath_negotiated() {
+629 return Err(PathError::MultipathNotNegotiated);
+630 }
+631 if self.side().is_server() {
+632 return Err(PathError::ServerSideNotAllowed);
+633 }
+634
+635 let max_abandoned = self.abandoned_paths.iter().max().copied();
+636 let max_used = self.paths.keys().last().copied();
+637 let path_id = max_abandoned
+638 .max(max_used)
+639 .unwrap_or(PathId::ZERO)
+640 .saturating_add(1u8);
+641
+642 if Some(path_id) > self.max_path_id() {
+643 return Err(PathError::MaxPathIdReached);
+644 }
+645 if path_id > self.remote_max_path_id {
+646 self.spaces[SpaceId::Data].pending.paths_blocked = true;
+647 return Err(PathError::MaxPathIdReached);
+648 }
+649 if self.rem_cids.get(&path_id).map(CidQueue::active).is_none() {
+650 self.spaces[SpaceId::Data]
+651 .pending
+652 .path_cids_blocked
+653 .insert(path_id);
+654 return Err(PathError::RemoteCidsExhausted);
+655 }
+656
+657 let path = self.ensure_path(path_id, network_path, now, None);
+658 path.status.local_update(initial_status);
+659
+660 Ok(path_id)
+661 }
+662
+663 /// Closes a path by sending a PATH_ABANDON frame
+664 ///
+665 /// This will not allow closing the last path. It does allow closing paths which have
+666 /// not yet been opened, as e.g. is the case when receiving a PATH_ABANDON from the peer
+667 /// for a path that was never opened locally.
+668 pub fn close_path(
+669 &mut self,
+670 now: Instant,
+671 path_id: PathId,
+672 error_code: VarInt,
+673 ) -> Result<(), ClosePathError> {
+674 if self.abandoned_paths.contains(&path_id)
+675 || Some(path_id) > self.max_path_id()
+676 || !self.paths.contains_key(&path_id)
+677 {
+678 return Err(ClosePathError::ClosedPath);
+679 }
+680 if self
+681 .paths
+682 .iter()
+683 // Would there be any remaining, non-abandoned, validated paths
+684 .any(|(id, path)| {
+685 *id != path_id && !self.abandoned_paths.contains(id) && path.data.validated
+686 })
+687 .not()
+688 {
+689 return Err(ClosePathError::LastOpenPath);
+690 }
+691
+692 // Send PATH_ABANDON
+693 self.spaces[SpaceId::Data]
+694 .pending
+695 .path_abandon
+696 .insert(path_id, error_code.into());
+697
+698 // Remove pending NEW CIDs for this path
+699 let pending_space = &mut self.spaces[SpaceId::Data].pending;
+700 pending_space.new_cids.retain(|cid| cid.path_id != path_id);
+701 pending_space.path_cids_blocked.retain(|&id| id != path_id);
+702 pending_space.path_status.retain(|&id| id != path_id);
+703
+704 // Cleanup retransmits across ALL paths (CIDs for path_id may have been transmitted on other paths)
+705 for space in self.spaces[SpaceId::Data].iter_paths_mut() {
+706 for sent_packet in space.sent_packets.values_mut() {
+707 if let Some(retransmits) = sent_packet.retransmits.get_mut() {
+708 retransmits.new_cids.retain(|cid| cid.path_id != path_id);
+709 retransmits.path_cids_blocked.retain(|&id| id != path_id);
+710 retransmits.path_status.retain(|&id| id != path_id);
+711 }
+712 }
+713 }
+714
+715 // Consider remotely issued CIDs as retired.
+716 // Technically we don't have to do this just yet. We only need to do this *after*
+717 // the ABANDON_PATH frame is sent, allowing us to still send it on the
+718 // to-be-abandoned path. However it is recommended to send it on another path, and
+719 // we do not allow abandoning the last path anyway.
+720 self.rem_cids.remove(&path_id);
+721 self.endpoint_events
+722 .push_back(EndpointEventInner::RetireResetToken(path_id));
+723
+724 self.abandoned_paths.insert(path_id);
725
-726 self.abandoned_paths.insert(path_id);
+726 self.set_max_path_id(now, self.local_max_path_id.saturating_add(1u8));
727
-728 self.set_max_path_id(now, self.local_max_path_id.saturating_add(1u8));
-729
-730 Ok(())
-731 }
-732
-733 /// Gets the [`PathData`] for a known [`PathId`].
-734 ///
-735 /// Will panic if the path_id does not reference any known path.
-736 #[track_caller]
-737 fn path_data(&self, path_id: PathId) -> &PathData {
-738 if let Some(data) = self.paths.get(&path_id) {
-739 &data.data
-740 } else {
-741 panic!(
-742 "unknown path: {path_id}, currently known paths: {:?}",
-743 self.paths.keys().collect::<Vec<_>>()
-744 );
-745 }
-746 }
-747
-748 /// Gets a reference to the [`PathData`] for a [`PathId`]
-749 fn path(&self, path_id: PathId) -> Option<&PathData> {
-750 self.paths.get(&path_id).map(|path_state| &path_state.data)
-751 }
-752
-753 /// Gets a mutable reference to the [`PathData`] for a [`PathId`]
-754 fn path_mut(&mut self, path_id: PathId) -> Option<&mut PathData> {
-755 self.paths
-756 .get_mut(&path_id)
-757 .map(|path_state| &mut path_state.data)
-758 }
-759
-760 /// Returns all known paths.
-761 ///
-762 /// There is no guarantee any of these paths are open or usable.
-763 pub fn paths(&self) -> Vec<PathId> {
-764 self.paths.keys().copied().collect()
-765 }
-766
-767 /// Gets the local [`PathStatus`] for a known [`PathId`]
-768 pub fn path_status(&self, path_id: PathId) -> Result<PathStatus, ClosedPath> {
-769 self.path(path_id)
-770 .map(PathData::local_status)
-771 .ok_or(ClosedPath { _private: () })
-772 }
-773
-774 /// Returns the path's network path represented as a 4-tuple.
-775 pub fn network_path(&self, path_id: PathId) -> Result<FourTuple, ClosedPath> {
-776 self.path(path_id)
-777 .map(|path| path.network_path)
-778 .ok_or(ClosedPath { _private: () })
-779 }
-780
-781 /// Sets the [`PathStatus`] for a known [`PathId`]
-782 ///
-783 /// Returns the previous path status on success.
-784 pub fn set_path_status(
-785 &mut self,
-786 path_id: PathId,
-787 status: PathStatus,
-788 ) -> Result<PathStatus, SetPathStatusError> {
-789 if !self.is_multipath_negotiated() {
-790 return Err(SetPathStatusError::MultipathNotNegotiated);
-791 }
-792 let path = self
-793 .path_mut(path_id)
-794 .ok_or(SetPathStatusError::ClosedPath)?;
-795 let prev = match path.status.local_update(status) {
-796 Some(prev) => {
-797 self.spaces[SpaceId::Data]
-798 .pending
-799 .path_status
-800 .insert(path_id);
-801 prev
-802 }
-803 None => path.local_status(),
-804 };
-805 Ok(prev)
-806 }
-807
-808 /// Returns the remote path status
-809 // TODO(flub): Probably should also be some kind of path event? Not even sure if I like
-810 // this as an API, but for now it allows me to write a test easily.
-811 // TODO(flub): Technically this should be a Result<Option<PathSTatus>>?
-812 pub fn remote_path_status(&self, path_id: PathId) -> Option<PathStatus> {
-813 self.path(path_id).and_then(|path| path.remote_status())
-814 }
-815
-816 /// Sets the max_idle_timeout for a specific path
+728 Ok(())
+729 }
+730
+731 /// Gets the [`PathData`] for a known [`PathId`].
+732 ///
+733 /// Will panic if the path_id does not reference any known path.
+734 #[track_caller]
+735 fn path_data(&self, path_id: PathId) -> &PathData {
+736 if let Some(data) = self.paths.get(&path_id) {
+737 &data.data
+738 } else {
+739 panic!(
+740 "unknown path: {path_id}, currently known paths: {:?}",
+741 self.paths.keys().collect::<Vec<_>>()
+742 );
+743 }
+744 }
+745
+746 /// Gets a reference to the [`PathData`] for a [`PathId`]
+747 fn path(&self, path_id: PathId) -> Option<&PathData> {
+748 self.paths.get(&path_id).map(|path_state| &path_state.data)
+749 }
+750
+751 /// Gets a mutable reference to the [`PathData`] for a [`PathId`]
+752 fn path_mut(&mut self, path_id: PathId) -> Option<&mut PathData> {
+753 self.paths
+754 .get_mut(&path_id)
+755 .map(|path_state| &mut path_state.data)
+756 }
+757
+758 /// Returns all known paths.
+759 ///
+760 /// There is no guarantee any of these paths are open or usable.
+761 pub fn paths(&self) -> Vec<PathId> {
+762 self.paths.keys().copied().collect()
+763 }
+764
+765 /// Gets the local [`PathStatus`] for a known [`PathId`]
+766 pub fn path_status(&self, path_id: PathId) -> Result<PathStatus, ClosedPath> {
+767 self.path(path_id)
+768 .map(PathData::local_status)
+769 .ok_or(ClosedPath { _private: () })
+770 }
+771
+772 /// Returns the path's network path represented as a 4-tuple.
+773 pub fn network_path(&self, path_id: PathId) -> Result<FourTuple, ClosedPath> {
+774 self.path(path_id)
+775 .map(|path| path.network_path)
+776 .ok_or(ClosedPath { _private: () })
+777 }
+778
+779 /// Sets the [`PathStatus`] for a known [`PathId`]
+780 ///
+781 /// Returns the previous path status on success.
+782 pub fn set_path_status(
+783 &mut self,
+784 path_id: PathId,
+785 status: PathStatus,
+786 ) -> Result<PathStatus, SetPathStatusError> {
+787 if !self.is_multipath_negotiated() {
+788 return Err(SetPathStatusError::MultipathNotNegotiated);
+789 }
+790 let path = self
+791 .path_mut(path_id)
+792 .ok_or(SetPathStatusError::ClosedPath)?;
+793 let prev = match path.status.local_update(status) {
+794 Some(prev) => {
+795 self.spaces[SpaceId::Data]
+796 .pending
+797 .path_status
+798 .insert(path_id);
+799 prev
+800 }
+801 None => path.local_status(),
+802 };
+803 Ok(prev)
+804 }
+805
+806 /// Returns the remote path status
+807 // TODO(flub): Probably should also be some kind of path event? Not even sure if I like
+808 // this as an API, but for now it allows me to write a test easily.
+809 // TODO(flub): Technically this should be a Result<Option<PathSTatus>>?
+810 pub fn remote_path_status(&self, path_id: PathId) -> Option<PathStatus> {
+811 self.path(path_id).and_then(|path| path.remote_status())
+812 }
+813
+814 /// Sets the max_idle_timeout for a specific path
+815 ///
+816 /// See [`TransportConfig::default_path_max_idle_timeout`] for details.
817 ///
-818 /// See [`TransportConfig::default_path_max_idle_timeout`] for details.
-819 ///
-820 /// Returns the previous value of the setting.
-821 pub fn set_path_max_idle_timeout(
-822 &mut self,
-823 path_id: PathId,
-824 timeout: Option<Duration>,
-825 ) -> Result<Option<Duration>, ClosedPath> {
-826 let path = self
-827 .paths
-828 .get_mut(&path_id)
-829 .ok_or(ClosedPath { _private: () })?;
-830 Ok(std::mem::replace(&mut path.data.idle_timeout, timeout))
-831 }
-832
-833 /// Sets the keep_alive_interval for a specific path
+818 /// Returns the previous value of the setting.
+819 pub fn set_path_max_idle_timeout(
+820 &mut self,
+821 path_id: PathId,
+822 timeout: Option<Duration>,
+823 ) -> Result<Option<Duration>, ClosedPath> {
+824 let path = self
+825 .paths
+826 .get_mut(&path_id)
+827 .ok_or(ClosedPath { _private: () })?;
+828 Ok(std::mem::replace(&mut path.data.idle_timeout, timeout))
+829 }
+830
+831 /// Sets the keep_alive_interval for a specific path
+832 ///
+833 /// See [`TransportConfig::default_path_keep_alive_interval`] for details.
834 ///
-835 /// See [`TransportConfig::default_path_keep_alive_interval`] for details.
-836 ///
-837 /// Returns the previous value of the setting.
-838 pub fn set_path_keep_alive_interval(
-839 &mut self,
-840 path_id: PathId,
-841 interval: Option<Duration>,
-842 ) -> Result<Option<Duration>, ClosedPath> {
-843 let path = self
-844 .paths
-845 .get_mut(&path_id)
-846 .ok_or(ClosedPath { _private: () })?;
-847 Ok(std::mem::replace(&mut path.data.keep_alive, interval))
-848 }
-849
-850 /// Gets the [`PathData`] for a known [`PathId`].
-851 ///
-852 /// Will panic if the path_id does not reference any known path.
-853 #[track_caller]
-854 fn path_data_mut(&mut self, path_id: PathId) -> &mut PathData {
-855 &mut self.paths.get_mut(&path_id).expect("known path").data
-856 }
-857
-858 /// Find an open, validated path that's on the same network path as the given network path.
-859 ///
-860 /// Returns the first path matching, even if there's multiple.
-861 fn find_validated_path_on_network_path(
-862 &self,
-863 network_path: FourTuple,
-864 ) -> Option<(&PathId, &PathState)> {
-865 self.paths.iter().find(|(path_id, path_state)| {
-866 path_state.data.validated
-867 // Would this use the same network path, if network_path were used to send right now?
-868 && network_path.is_probably_same_path(&path_state.data.network_path)
-869 && !self.abandoned_paths.contains(path_id)
-870 })
-871 // TODO(@divma): we might want to ensure the path has been recently active to consider the
-872 // address validated
-873 // matheus23: Perhaps looking at !self.abandoned_paths.contains(path_id) is enough, given keep-alives?
-874 }
-875
-876 fn ensure_path(
-877 &mut self,
-878 path_id: PathId,
-879 network_path: FourTuple,
-880 now: Instant,
-881 pn: Option<u64>,
-882 ) -> &mut PathData {
-883 let valid_path = self.find_validated_path_on_network_path(network_path);
-884 let validated = valid_path.is_some();
-885 let initial_rtt = valid_path.map(|(_, path)| path.data.rtt.conservative());
-886 let vacant_entry = match self.paths.entry(path_id) {
-887 btree_map::Entry::Vacant(vacant_entry) => vacant_entry,
-888 btree_map::Entry::Occupied(occupied_entry) => {
-889 return &mut occupied_entry.into_mut().data;
-890 }
-891 };
-892
-893 debug!(%validated, %path_id, %network_path, "path added");
-894 let peer_max_udp_payload_size =
-895 u16::try_from(self.peer_params.max_udp_payload_size.into_inner()).unwrap_or(u16::MAX);
-896 self.path_generation_counter = self.path_generation_counter.wrapping_add(1);
-897 let mut data = PathData::new(
-898 network_path,
-899 self.allow_mtud,
-900 Some(peer_max_udp_payload_size),
-901 self.path_generation_counter,
-902 now,
-903 &self.config,
-904 );
-905
-906 data.validated = validated;
-907 if let Some(initial_rtt) = initial_rtt {
-908 data.rtt.reset_initial_rtt(initial_rtt);
-909 }
-910
-911 let pto = self.ack_frequency.max_ack_delay_for_pto() + data.rtt.pto_base();
-912 self.timers.set(
-913 Timer::PerPath(path_id, PathTimer::PathOpen),
-914 now + 3 * pto,
-915 self.qlog.with_time(now),
-916 );
-917
-918 // for the path to be opened we need to send a packet on the path. Sending a challenge
-919 // guarantees this
-920 data.send_new_challenge = true;
+835 /// Returns the previous value of the setting.
+836 pub fn set_path_keep_alive_interval(
+837 &mut self,
+838 path_id: PathId,
+839 interval: Option<Duration>,
+840 ) -> Result<Option<Duration>, ClosedPath> {
+841 let path = self
+842 .paths
+843 .get_mut(&path_id)
+844 .ok_or(ClosedPath { _private: () })?;
+845 Ok(std::mem::replace(&mut path.data.keep_alive, interval))
+846 }
+847
+848 /// Gets the [`PathData`] for a known [`PathId`].
+849 ///
+850 /// Will panic if the path_id does not reference any known path.
+851 #[track_caller]
+852 fn path_data_mut(&mut self, path_id: PathId) -> &mut PathData {
+853 &mut self.paths.get_mut(&path_id).expect("known path").data
+854 }
+855
+856 /// Find an open, validated path that's on the same network path as the given network path.
+857 ///
+858 /// Returns the first path matching, even if there's multiple.
+859 fn find_validated_path_on_network_path(
+860 &self,
+861 network_path: FourTuple,
+862 ) -> Option<(&PathId, &PathState)> {
+863 self.paths.iter().find(|(path_id, path_state)| {
+864 path_state.data.validated
+865 // Would this use the same network path, if network_path were used to send right now?
+866 && network_path.is_probably_same_path(&path_state.data.network_path)
+867 && !self.abandoned_paths.contains(path_id)
+868 })
+869 // TODO(@divma): we might want to ensure the path has been recently active to consider the
+870 // address validated
+871 // matheus23: Perhaps looking at !self.abandoned_paths.contains(path_id) is enough, given keep-alives?
+872 }
+873
+874 fn ensure_path(
+875 &mut self,
+876 path_id: PathId,
+877 network_path: FourTuple,
+878 now: Instant,
+879 pn: Option<u64>,
+880 ) -> &mut PathData {
+881 let valid_path = self.find_validated_path_on_network_path(network_path);
+882 let validated = valid_path.is_some();
+883 let initial_rtt = valid_path.map(|(_, path)| path.data.rtt.conservative());
+884 let vacant_entry = match self.paths.entry(path_id) {
+885 btree_map::Entry::Vacant(vacant_entry) => vacant_entry,
+886 btree_map::Entry::Occupied(occupied_entry) => {
+887 return &mut occupied_entry.into_mut().data;
+888 }
+889 };
+890
+891 debug!(%validated, %path_id, %network_path, "path added");
+892 let peer_max_udp_payload_size =
+893 u16::try_from(self.peer_params.max_udp_payload_size.into_inner()).unwrap_or(u16::MAX);
+894 self.path_generation_counter = self.path_generation_counter.wrapping_add(1);
+895 let mut data = PathData::new(
+896 network_path,
+897 self.allow_mtud,
+898 Some(peer_max_udp_payload_size),
+899 self.path_generation_counter,
+900 now,
+901 &self.config,
+902 );
+903
+904 data.validated = validated;
+905 if let Some(initial_rtt) = initial_rtt {
+906 data.rtt.reset_initial_rtt(initial_rtt);
+907 }
+908
+909 let pto = self.ack_frequency.max_ack_delay_for_pto() + data.rtt.pto_base();
+910 self.timers.set(
+911 Timer::PerPath(path_id, PathTimer::PathOpen),
+912 now + 3 * pto,
+913 self.qlog.with_time(now),
+914 );
+915
+916 // for the path to be opened we need to send a packet on the path. Sending a challenge
+917 // guarantees this
+918 data.send_new_challenge = true;
+919
+920 let path = vacant_entry.insert(PathState { data, prev: None });
921
-922 let path = vacant_entry.insert(PathState { data, prev: None });
-923
-924 let mut pn_space = spaces::PacketNumberSpace::new(now, SpaceId::Data, &mut self.rng);
-925 if let Some(pn) = pn {
-926 pn_space.dedup.insert(pn);
-927 }
-928 self.spaces[SpaceId::Data]
-929 .number_spaces
-930 .insert(path_id, pn_space);
-931 self.qlog.emit_tuple_assigned(path_id, network_path, now);
-932 &mut path.data
-933 }
-934
-935 /// Returns packets to transmit
-936 ///
-937 /// Connections should be polled for transmit after:
-938 /// - the application performed some I/O on the connection
-939 /// - a call was made to `handle_event`
-940 /// - a call was made to `handle_timeout`
-941 ///
-942 /// `max_datagrams` specifies how many datagrams can be returned inside a
-943 /// single Transmit using GSO. This must be at least 1.
-944 #[must_use]
-945 pub fn poll_transmit(
-946 &mut self,
-947 now: Instant,
-948 max_datagrams: NonZeroUsize,
-949 buf: &mut Vec<u8>,
-950 ) -> Option<Transmit> {
-951 if let Some(probing) = self
-952 .iroh_hp
-953 .server_side_mut()
-954 .ok()
-955 .and_then(iroh_hp::ServerState::next_probe)
-956 {
-957 let destination = probing.remote();
-958 trace!(%destination, "RAND_DATA packet");
-959 let token: u64 = self.rng.random();
-960 buf.put_u64(token);
-961 probing.finish(token);
-962 return Some(Transmit {
-963 destination,
-964 ecn: None,
-965 size: 8,
-966 segment_size: None,
-967 src_ip: None,
-968 });
-969 }
-970
-971 let max_datagrams = match self.config.enable_segmentation_offload {
-972 false => NonZeroUsize::MIN,
-973 true => max_datagrams,
-974 };
-975
-976 // Each call to poll_transmit can only send datagrams to one destination, because
-977 // all datagrams in a GSO batch are for the same destination. Therefore only
-978 // datagrams for one Path ID are produced for each poll_transmit call.
-979
-980 // TODO(flub): this is wishful thinking and not actually implemented, but perhaps it
-981 // should be:
-982
-983 // First, if we have to send a close, select a path for that.
-984 // Next, all paths that have a PATH_CHALLENGE or PATH_RESPONSE pending.
-985
-986 // For all, open, validated and AVAILABLE paths:
-987 // - Is the path congestion blocked or pacing blocked?
-988 // - call maybe_queue_ to ensure a tail-loss probe would be sent?
-989 // - do we need to send a close message?
-990 // - call can_send
-991 // Once there's nothing more to send on the AVAILABLE paths, do the same for BACKUP paths
-992
-993 // Check whether we need to send a close message
-994 let close = match self.state.as_type() {
-995 StateType::Drained => {
-996 self.app_limited = true;
-997 return None;
-998 }
-999 StateType::Draining | StateType::Closed => {
-1000 // self.close is only reset once the associated packet had been
-1001 // encoded successfully
-1002 if !self.close {
-1003 self.app_limited = true;
-1004 return None;
-1005 }
-1006 true
-1007 }
-1008 _ => false,
-1009 };
-1010
-1011 // Check whether we need to send an ACK_FREQUENCY frame
-1012 if let Some(config) = &self.config.ack_frequency_config {
-1013 let rtt = self
-1014 .paths
-1015 .values()
-1016 .map(|p| p.data.rtt.get())
-1017 .min()
-1018 .expect("one path exists");
-1019 self.spaces[SpaceId::Data].pending.ack_frequency = self
-1020 .ack_frequency
-1021 .should_send_ack_frequency(rtt, config, &self.peer_params)
-1022 && self.highest_space == SpaceId::Data
-1023 && self.peer_supports_ack_frequency();
-1024 }
-1025
-1026 // If there is any open, validated and available path we only want to send frames to
-1027 // any backup path that must be sent on that backup path exclusively.
-1028 let have_available_path = self.paths.iter().any(|(id, path)| {
-1029 path.data.validated
-1030 && path.data.local_status() == PathStatus::Available
-1031 && self.rem_cids.contains_key(id)
-1032 });
-1033
-1034 // TODO: how to avoid the allocation? Cannot use a for loop because of borrowing
-1035 let path_ids: Vec<_> = self.paths.keys().copied().collect();
-1036
-1037 let mut transmit = TransmitBuf::new(
-1038 buf,
-1039 max_datagrams,
-1040 self.path_data(path_ids[0]).current_mtu().into(),
-1041 );
+922 let mut pn_space = spaces::PacketNumberSpace::new(now, SpaceId::Data, &mut self.rng);
+923 if let Some(pn) = pn {
+924 pn_space.dedup.insert(pn);
+925 }
+926 self.spaces[SpaceId::Data]
+927 .number_spaces
+928 .insert(path_id, pn_space);
+929 self.qlog.emit_tuple_assigned(path_id, network_path, now);
+930 &mut path.data
+931 }
+932
+933 /// Returns packets to transmit
+934 ///
+935 /// Connections should be polled for transmit after:
+936 /// - the application performed some I/O on the connection
+937 /// - a call was made to `handle_event`
+938 /// - a call was made to `handle_timeout`
+939 ///
+940 /// `max_datagrams` specifies how many datagrams can be returned inside a
+941 /// single Transmit using GSO. This must be at least 1.
+942 #[must_use]
+943 pub fn poll_transmit(
+944 &mut self,
+945 now: Instant,
+946 max_datagrams: NonZeroUsize,
+947 buf: &mut Vec<u8>,
+948 ) -> Option<Transmit> {
+949 if let Some(probing) = self
+950 .iroh_hp
+951 .server_side_mut()
+952 .ok()
+953 .and_then(iroh_hp::ServerState::next_probe)
+954 {
+955 let destination = probing.remote();
+956 trace!(%destination, "RAND_DATA packet");
+957 let token: u64 = self.rng.random();
+958 buf.put_u64(token);
+959 probing.finish(token);
+960 return Some(Transmit {
+961 destination,
+962 ecn: None,
+963 size: 8,
+964 segment_size: None,
+965 src_ip: None,
+966 });
+967 }
+968
+969 let max_datagrams = match self.config.enable_segmentation_offload {
+970 false => NonZeroUsize::MIN,
+971 true => max_datagrams,
+972 };
+973
+974 // Each call to poll_transmit can only send datagrams to one destination, because
+975 // all datagrams in a GSO batch are for the same destination. Therefore only
+976 // datagrams for one Path ID are produced for each poll_transmit call.
+977
+978 // TODO(flub): this is wishful thinking and not actually implemented, but perhaps it
+979 // should be:
+980
+981 // First, if we have to send a close, select a path for that.
+982 // Next, all paths that have a PATH_CHALLENGE or PATH_RESPONSE pending.
+983
+984 // For all, open, validated and AVAILABLE paths:
+985 // - Is the path congestion blocked or pacing blocked?
+986 // - call maybe_queue_ to ensure a tail-loss probe would be sent?
+987 // - do we need to send a close message?
+988 // - call can_send
+989 // Once there's nothing more to send on the AVAILABLE paths, do the same for BACKUP paths
+990
+991 // Check whether we need to send a close message
+992 let close = match self.state.as_type() {
+993 StateType::Drained => {
+994 self.app_limited = true;
+995 return None;
+996 }
+997 StateType::Draining | StateType::Closed => {
+998 // self.close is only reset once the associated packet had been
+999 // encoded successfully
+1000 if !self.close {
+1001 self.app_limited = true;
+1002 return None;
+1003 }
+1004 true
+1005 }
+1006 _ => false,
+1007 };
+1008
+1009 // Check whether we need to send an ACK_FREQUENCY frame
+1010 if let Some(config) = &self.config.ack_frequency_config {
+1011 let rtt = self
+1012 .paths
+1013 .values()
+1014 .map(|p| p.data.rtt.get())
+1015 .min()
+1016 .expect("one path exists");
+1017 self.spaces[SpaceId::Data].pending.ack_frequency = self
+1018 .ack_frequency
+1019 .should_send_ack_frequency(rtt, config, &self.peer_params)
+1020 && self.highest_space == SpaceId::Data
+1021 && self.peer_supports_ack_frequency();
+1022 }
+1023
+1024 // If there is any open, validated and available path we only want to send frames to
+1025 // any backup path that must be sent on that backup path exclusively.
+1026 let have_available_path = self.paths.iter().any(|(id, path)| {
+1027 path.data.validated
+1028 && path.data.local_status() == PathStatus::Available
+1029 && self.rem_cids.contains_key(id)
+1030 });
+1031
+1032 // TODO: how to avoid the allocation? Cannot use a for loop because of borrowing
+1033 let path_ids: Vec<_> = self.paths.keys().copied().collect();
+1034
+1035 let mut transmit = TransmitBuf::new(
+1036 buf,
+1037 max_datagrams,
+1038 self.path_data(path_ids[0]).current_mtu().into(),
+1039 );
+1040
+1041 let mut congestion_blocked = false;
1042
-1043 let mut congestion_blocked = false;
-1044
-1045 for &path_id in &path_ids {
-1046 // Update per path state
-1047 transmit.set_segment_size(self.path_data(path_id).current_mtu().into());
-1048
-1049 if let Some(challenge) = self.send_prev_path_challenge(now, &mut transmit, path_id) {
-1050 return Some(challenge);
-1051 }
-1052
-1053 match self.poll_transmit_path(now, &mut transmit, path_id, have_available_path, close) {
-1054 PollPathStatus::SendTransmit { transmit } => {
-1055 return Some(transmit);
-1056 }
-1057 PollPathStatus::Send => {
-1058 let transmit = self.build_transmit(path_id, transmit);
-1059 return Some(transmit);
-1060 }
-1061 PollPathStatus::NothingToSend {
-1062 congestion_blocked: cb,
-1063 } => {
-1064 // TODO: congestion blocked should be per path? how to deal with this better
-1065
-1066 // move to next path
-1067 if cb {
-1068 // note congestion block
-1069 congestion_blocked = true;
-1070 }
-1071 }
-1072 }
-1073 // Nothing more to send.
-1074 trace!(%path_id, "nothing to send on path");
-1075 }
-1076
-1077 // We didn't produce any application data packet
-1078 debug_assert!(
-1079 transmit.is_empty(),
-1080 "there was data in the transmit, but it was not sent"
-1081 );
-1082 self.app_limited = !congestion_blocked;
-1083
-1084 if self.state.is_established() {
-1085 // Try MTU probing now
-1086 for path_id in path_ids {
-1087 // Update per path state
-1088 transmit.set_segment_size(self.path_data(path_id).current_mtu().into());
-1089 self.poll_transmit_mtu_probe(now, &mut transmit, path_id);
-1090 if !transmit.is_empty() {
-1091 let transmit = self.build_transmit(path_id, transmit);
-1092 return Some(transmit);
-1093 }
-1094 }
-1095 }
-1096
-1097 None
-1098 }
-1099
-1100 fn build_transmit(&mut self, path_id: PathId, transmit: TransmitBuf<'_>) -> Transmit {
-1101 debug_assert!(
-1102 !transmit.is_empty(),
-1103 "must not be called with an empty transmit buffer"
-1104 );
-1105
-1106 let network_path = self.path_data(path_id).network_path;
-1107 trace!(
-1108 segment_size = transmit.segment_size(),
-1109 last_datagram_len = transmit.len() % transmit.segment_size(),
-1110 %network_path,
-1111 "sending {} bytes in {} datagrams",
-1112 transmit.len(),
-1113 transmit.num_datagrams()
-1114 );
-1115 self.path_data_mut(path_id)
-1116 .inc_total_sent(transmit.len() as u64);
-1117
-1118 self.stats
-1119 .udp_tx
-1120 .on_sent(transmit.num_datagrams() as u64, transmit.len());
-1121
-1122 Transmit {
-1123 destination: network_path.remote,
-1124 size: transmit.len(),
-1125 ecn: if self.path_data(path_id).sending_ecn {
-1126 Some(EcnCodepoint::Ect0)
-1127 } else {
-1128 None
-1129 },
-1130 segment_size: match transmit.num_datagrams() {
-1131 1 => None,
-1132 _ => Some(transmit.segment_size()),
-1133 },
-1134 src_ip: network_path.local_ip,
-1135 }
-1136 }
-1137
-1138 /// poll_transmit logic for a specific path
-1139 #[must_use]
-1140 fn poll_transmit_path(
-1141 &mut self,
-1142 now: Instant,
-1143 transmit: &mut TransmitBuf<'_>,
-1144 path_id: PathId,
-1145 have_available_path: bool,
-1146 close: bool,
-1147 ) -> PollPathStatus {
-1148 // Check if there is at least one active CID to use for sending
-1149 let Some(remote_cid) = self.rem_cids.get(&path_id).map(CidQueue::active) else {
-1150 self.on_remote_cids_exhausted(now, path_id);
-1151
-1152 return PollPathStatus::NothingToSend {
-1153 congestion_blocked: false,
-1154 };
-1155 };
-1156
-1157 // Whether this packet can be coalesced with another one in the same datagram.
-1158 let mut coalesce = true;
-1159
-1160 // Whether the last packet in the datagram must be padded so the datagram takes up
-1161 // to at least MIN_INITIAL_SIZE, or to the maximum segment size if this is smaller.
-1162 let mut pad_datagram = PadDatagram::No;
-1163
-1164 // The packet number of the last built packet.
-1165 let mut last_packet_number = None;
+1043 for &path_id in &path_ids {
+1044 // Update per path state
+1045 transmit.set_segment_size(self.path_data(path_id).current_mtu().into());
+1046
+1047 if let Some(challenge) = self.send_prev_path_challenge(now, &mut transmit, path_id) {
+1048 return Some(challenge);
+1049 }
+1050
+1051 match self.poll_transmit_path(now, &mut transmit, path_id, have_available_path, close) {
+1052 PollPathStatus::SendTransmit { transmit } => {
+1053 return Some(transmit);
+1054 }
+1055 PollPathStatus::Send => {
+1056 let transmit = self.build_transmit(path_id, transmit);
+1057 return Some(transmit);
+1058 }
+1059 PollPathStatus::NothingToSend {
+1060 congestion_blocked: cb,
+1061 } => {
+1062 // TODO: congestion blocked should be per path? how to deal with this better
+1063
+1064 // move to next path
+1065 if cb {
+1066 // note congestion block
+1067 congestion_blocked = true;
+1068 }
+1069 }
+1070 }
+1071 // Nothing more to send.
+1072 trace!(%path_id, "nothing to send on path");
+1073 }
+1074
+1075 // We didn't produce any application data packet
+1076 debug_assert!(
+1077 transmit.is_empty(),
+1078 "there was data in the transmit, but it was not sent"
+1079 );
+1080 self.app_limited = !congestion_blocked;
+1081
+1082 if self.state.is_established() {
+1083 // Try MTU probing now
+1084 for path_id in path_ids {
+1085 // Update per path state
+1086 transmit.set_segment_size(self.path_data(path_id).current_mtu().into());
+1087 self.poll_transmit_mtu_probe(now, &mut transmit, path_id);
+1088 if !transmit.is_empty() {
+1089 let transmit = self.build_transmit(path_id, transmit);
+1090 return Some(transmit);
+1091 }
+1092 }
+1093 }
+1094
+1095 None
+1096 }
+1097
+1098 fn build_transmit(&mut self, path_id: PathId, transmit: TransmitBuf<'_>) -> Transmit {
+1099 debug_assert!(
+1100 !transmit.is_empty(),
+1101 "must not be called with an empty transmit buffer"
+1102 );
+1103
+1104 let network_path = self.path_data(path_id).network_path;
+1105 trace!(
+1106 segment_size = transmit.segment_size(),
+1107 last_datagram_len = transmit.len() % transmit.segment_size(),
+1108 %network_path,
+1109 "sending {} bytes in {} datagrams",
+1110 transmit.len(),
+1111 transmit.num_datagrams()
+1112 );
+1113 self.path_data_mut(path_id)
+1114 .inc_total_sent(transmit.len() as u64);
+1115
+1116 self.stats
+1117 .udp_tx
+1118 .on_sent(transmit.num_datagrams() as u64, transmit.len());
+1119
+1120 Transmit {
+1121 destination: network_path.remote,
+1122 size: transmit.len(),
+1123 ecn: if self.path_data(path_id).sending_ecn {
+1124 Some(EcnCodepoint::Ect0)
+1125 } else {
+1126 None
+1127 },
+1128 segment_size: match transmit.num_datagrams() {
+1129 1 => None,
+1130 _ => Some(transmit.segment_size()),
+1131 },
+1132 src_ip: network_path.local_ip,
+1133 }
+1134 }
+1135
+1136 /// poll_transmit logic for a specific path
+1137 #[must_use]
+1138 fn poll_transmit_path(
+1139 &mut self,
+1140 now: Instant,
+1141 transmit: &mut TransmitBuf<'_>,
+1142 path_id: PathId,
+1143 have_available_path: bool,
+1144 close: bool,
+1145 ) -> PollPathStatus {
+1146 // Check if there is at least one active CID to use for sending
+1147 let Some(remote_cid) = self.rem_cids.get(&path_id).map(CidQueue::active) else {
+1148 self.on_remote_cids_exhausted(now, path_id);
+1149
+1150 return PollPathStatus::NothingToSend {
+1151 congestion_blocked: false,
+1152 };
+1153 };
+1154
+1155 // Whether the last packet in the datagram must be padded so the datagram takes up
+1156 // to at least MIN_INITIAL_SIZE, or to the maximum segment size if this is smaller.
+1157 let mut pad_datagram = PadDatagram::No;
+1158
+1159 // The packet number of the last built packet.
+1160 let mut last_packet_number = None;
+1161
+1162 // Whether this packet can be coalesced with another one in the same datagram.
+1163 let mut coalesce = true;
+1164
+1165 let mut congestion_blocked = false;
1166
1167 // Iterate over the available spaces
1168 for space_id in SpaceId::iter() {
@@ -1170,5668 +1170,5796 @@
1170 if path_id != PathId::ZERO && space_id != SpaceId::Data {
1171 continue;
1172 }
-1173
-1174 let res = self.poll_transmit_path_space(
-1175 now,
-1176 transmit,
-1177 path_id,
-1178 space_id,
-1179 remote_cid,
-1180 have_available_path,
-1181 close,
-1182 &mut coalesce,
-1183 &mut pad_datagram,
-1184 );
-1185 match res {
-1186 PollPathSpaceStatus::Send {
-1187 last_packet_number: lp,
-1188 } => {
-1189 last_packet_number = lp;
-1190 break;
-1191 }
-1192 PollPathSpaceStatus::SendTransmit { transmit } => {
-1193 return PollPathStatus::SendTransmit { transmit };
-1194 }
-1195 PollPathSpaceStatus::NothingToSend { congestion_blocked } => {
-1196 return PollPathStatus::NothingToSend { congestion_blocked };
-1197 }
-1198 PollPathSpaceStatus::NextSpace => {
-1199 // moving onto the next space
-1200 }
-1201 }
-1202 }
-1203
-1204 if let Some(last_packet_number) = last_packet_number {
-1205 // Note that when sending in multiple packet spaces the last packet number will
-1206 // be the one from the highest packet space.
-1207 self.path_data_mut(path_id).congestion.on_sent(
-1208 now,
-1209 transmit.len() as u64,
-1210 last_packet_number,
-1211 );
-1212 }
-1213
-1214 self.qlog.emit_recovery_metrics(
-1215 path_id,
-1216 &mut self.paths.get_mut(&path_id).unwrap().data,
-1217 now,
-1218 );
-1219
-1220 if transmit.is_empty() {
-1221 PollPathStatus::NothingToSend {
-1222 congestion_blocked: false,
+1173 // We only send up to the highest space
+1174 if space_id >= self.highest_space {
+1175 break;
+1176 }
+1177
+1178 // Check if we can send in this space for lower spaces.
+1179 let sendable_frames = if space_id < SpaceId::Data {
+1180 let sendable_frames = self.space_can_send_non_data(space_id, path_id, close);
+1181 let path_should_send =
+1182 self.path_should_send(path_id, space_id, have_available_path, &sendable_frames);
+1183
+1184 if !path_should_send {
+1185 if self.spaces[space_id].crypto.is_some() {
+1186 trace!(?space_id, %path_id, "nothing to send in space");
+1187 }
+1188 continue;
+1189 }
+1190 Some(sendable_frames)
+1191 } else {
+1192 None
+1193 };
+1194
+1195 let res = self.poll_transmit_path_space(
+1196 now,
+1197 transmit,
+1198 path_id,
+1199 space_id,
+1200 remote_cid,
+1201 have_available_path,
+1202 close,
+1203 sendable_frames,
+1204 &mut coalesce,
+1205 &mut pad_datagram,
+1206 );
+1207 match res {
+1208 PollPathSpaceStatus::Send {
+1209 last_packet_number: lp,
+1210 } => {
+1211 last_packet_number = lp;
+1212 break;
+1213 }
+1214 PollPathSpaceStatus::SendTransmit { transmit } => {
+1215 return PollPathStatus::SendTransmit { transmit };
+1216 }
+1217 PollPathSpaceStatus::NothingToSend {
+1218 congestion_blocked: cb,
+1219 } => {
+1220 congestion_blocked |= cb;
+1221 continue;
+1222 }
1223 }
-1224 } else {
-1225 PollPathStatus::Send
-1226 }
-1227 }
-1228
-1229 /// poll_transmit logic for a path_id - space_id combination
-1230 #[must_use]
-1231 fn poll_transmit_path_space(
-1232 &mut self,
-1233 now: Instant,
-1234 transmit: &mut TransmitBuf<'_>,
-1235 path_id: PathId,
-1236 space_id: SpaceId,
-1237 remote_cid: ConnectionId,
-1238 have_available_path: bool,
-1239 close: bool,
-1240 coalesce: &mut bool,
-1241 pad_datagram: &mut PadDatagram,
-1242 ) -> PollPathSpaceStatus {
-1243 let mut last_packet_number = None;
-1244
-1245 loop {
-1246 // Determine if anything can be sent in this packet number space (SpaceId + PathId).
-1247 let max_packet_size = if transmit.datagram_remaining_mut() > 0 {
-1248 // We are trying to coalesce another packet into this datagram.
-1249 transmit.datagram_remaining_mut()
-1250 } else {
-1251 // A new datagram needs to be started.
-1252 transmit.segment_size()
-1253 };
-1254
-1255 let can_send = self.space_can_send(space_id, path_id, max_packet_size, close);
-1256
-1257 let path_should_send = {
-1258 let path_exclusive_only = space_id == SpaceId::Data
-1259 && have_available_path
-1260 && self.path_data(path_id).local_status() == PathStatus::Backup;
-1261 let path_should_send = if path_exclusive_only {
-1262 can_send.path_exclusive
-1263 } else {
-1264 !can_send.is_empty()
-1265 };
-1266 let needs_loss_probe = self.spaces[space_id].for_path(path_id).loss_probes > 0;
-1267 path_should_send || needs_loss_probe
-1268 };
-1269
-1270 if !path_should_send && space_id < SpaceId::Data {
-1271 if self.spaces[space_id].crypto.is_some() {
-1272 trace!(?space_id, %path_id, "nothing to send in space");
-1273 }
-1274 return PollPathSpaceStatus::NextSpace;
-1275 }
-1276
-1277 let send_blocked = if path_should_send && transmit.datagram_remaining_mut() == 0 {
-1278 // Only check congestion control if a new datagram is needed.
-1279 self.path_congestion_check(space_id, path_id, transmit, &can_send, now)
-1280 } else {
-1281 PathBlocked::No
-1282 };
-1283
-1284 // Whether congestion control stopped the next packet from being sent. Further
-1285 // packets could still be built, as e.g. tail-loss probes are not congestion
-1286 // limited.
-1287 let congestion_blocked = if send_blocked != PathBlocked::No {
-1288 trace!(?space_id, %path_id, ?send_blocked, "congestion blocked");
-1289 true
-1290 } else {
-1291 false
-1292 };
-1293 if send_blocked != PathBlocked::No && space_id < SpaceId::Data {
-1294 // Higher spaces might still have tail-loss probes to send, which are not
-1295 // congestion blocked.
-1296 return PollPathSpaceStatus::NextSpace;
-1297 }
-1298 if !path_should_send || send_blocked != PathBlocked::No {
-1299 // Nothing more to send on this path, check the next path if possible.
-1300
-1301 // If there are any datagrams in the transmit, packets for another path can
-1302 // not be built.
-1303 if transmit.num_datagrams() > 0 {
-1304 return PollPathSpaceStatus::Send { last_packet_number };
-1305 }
-1306
-1307 return PollPathSpaceStatus::NothingToSend { congestion_blocked };
-1308 }
-1309
-1310 if transmit.datagram_remaining_mut() == 0 {
-1311 if transmit.num_datagrams() >= transmit.max_datagrams().get() {
-1312 // No more datagrams allowed
-1313 return PollPathSpaceStatus::Send { last_packet_number };
-1314 }
-1315
-1316 match self.spaces[space_id].for_path(path_id).loss_probes {
-1317 0 => transmit.start_new_datagram(),
-1318 _ => {
-1319 // We need something to send for a tail-loss probe.
-1320 let request_immediate_ack =
-1321 space_id == SpaceId::Data && self.peer_supports_ack_frequency();
-1322 self.spaces[space_id].maybe_queue_probe(
-1323 path_id,
-1324 request_immediate_ack,
-1325 &self.streams,
-1326 );
-1327
-1328 self.spaces[space_id].for_path(path_id).loss_probes -= 1;
-1329
-1330 // Clamp the datagram to at most the minimum MTU to ensure that loss
-1331 // probes can get through and enable recovery even if the path MTU
-1332 // has shrank unexpectedly.
-1333 transmit.start_new_datagram_with_size(std::cmp::min(
-1334 usize::from(INITIAL_MTU),
-1335 transmit.segment_size(),
-1336 ));
-1337 }
-1338 }
-1339 trace!(count = transmit.num_datagrams(), "new datagram started");
-1340 *coalesce = true;
-1341 *pad_datagram = PadDatagram::No;
-1342 }
-1343
-1344 // If coalescing another packet into the existing datagram, there should
-1345 // still be enough space for a whole packet.
-1346 if transmit.datagram_start_offset() < transmit.len() {
-1347 debug_assert!(transmit.datagram_remaining_mut() >= MIN_PACKET_SPACE);
-1348 }
-1349
-1350 //
-1351 // From here on, we've determined that a packet will definitely be sent.
-1352 //
+1224 }
+1225
+1226 if let Some(last_packet_number) = last_packet_number {
+1227 // Note that when sending in multiple packet spaces the last packet number will
+1228 // be the one from the highest packet space.
+1229 self.path_data_mut(path_id).congestion.on_sent(
+1230 now,
+1231 transmit.len() as u64,
+1232 last_packet_number,
+1233 );
+1234 }
+1235
+1236 self.qlog.emit_recovery_metrics(
+1237 path_id,
+1238 &mut self.paths.get_mut(&path_id).unwrap().data,
+1239 now,
+1240 );
+1241
+1242 if transmit.is_empty() {
+1243 PollPathStatus::NothingToSend { congestion_blocked }
+1244 } else {
+1245 PollPathStatus::Send
+1246 }
+1247 }
+1248
+1249 /// Determine if anything can be sent in this packet number space (SpaceId + PathId).
+1250 /// poll_transmit logic for a path_id - space_id combination
+1251 #[must_use]
+1252 fn poll_transmit_path_space(
+1253 &mut self,
+1254 now: Instant,
+1255 transmit: &mut TransmitBuf<'_>,
+1256 path_id: PathId,
+1257 space_id: SpaceId,
+1258 remote_cid: ConnectionId,
+1259 have_available_path: bool,
+1260 close: bool,
+1261 sendable_frames: Option<SendableFrames>,
+1262 coalesce: &mut bool,
+1263 pad_datagram: &mut PadDatagram,
+1264 ) -> PollPathSpaceStatus {
+1265 let mut last_packet_number = None;
+1266
+1267 // Build datagrams until the transmit is full.
+1268 while transmit.num_datagrams() < transmit.max_datagrams().get() {
+1269 let max_packet_size = if transmit.datagram_remaining_mut() > 0 {
+1270 // We are trying to coalesce another packet into this datagram.
+1271 debug_assert!(*coalesce, "coalescing must be set");
+1272 transmit.datagram_remaining_mut()
+1273 } else {
+1274 // A new datagram needs to be started.
+1275 transmit.segment_size()
+1276 };
+1277
+1278 let sendable_frames = sendable_frames
+1279 .unwrap_or_else(|| self.space_can_send(space_id, path_id, max_packet_size, close));
+1280 let path_should_send =
+1281 self.path_should_send(path_id, space_id, have_available_path, &sendable_frames);
+1282
+1283 let send_blocked = if path_should_send && transmit.datagram_remaining_mut() == 0 {
+1284 // Only check congestion control if a new datagram is needed.
+1285 self.path_congestion_check(space_id, path_id, transmit, &sendable_frames, now)
+1286 } else {
+1287 PathBlocked::No
+1288 };
+1289
+1290 // Whether congestion control stopped the next packet from being sent. Further
+1291 // packets could still be built, as e.g. tail-loss probes are not congestion limited.
+1292 let congestion_blocked = if send_blocked != PathBlocked::No {
+1293 trace!(?space_id, %path_id, ?send_blocked, "congestion blocked");
+1294 true
+1295 } else {
+1296 false
+1297 };
+1298 if send_blocked != PathBlocked::No && space_id < SpaceId::Data {
+1299 // Higher spaces might still have tail-loss probes to send, which are not
+1300 // congestion blocked.
+1301 return PollPathSpaceStatus::NothingToSend { congestion_blocked };
+1302 }
+1303 if !path_should_send || send_blocked != PathBlocked::No {
+1304 // Nothing more to send on this path, check the next path if possible.
+1305
+1306 // If there are any datagrams in the transmit, packets for another path can
+1307 // not be built.
+1308 if transmit.num_datagrams() > 0 {
+1309 return PollPathSpaceStatus::Send { last_packet_number };
+1310 }
+1311
+1312 return PollPathSpaceStatus::NothingToSend { congestion_blocked };
+1313 }
+1314
+1315 // If coalescing another packet into the existing datagram, there should
+1316 // still be enough space for a whole packet.
+1317 if *coalesce && transmit.datagram_start_offset() < transmit.len() {
+1318 debug_assert!(transmit.datagram_remaining_mut() >= MIN_PACKET_SPACE);
+1319 }
+1320
+1321 if transmit.datagram_remaining_mut() == 0 {
+1322 // If the datagram is full, start a new datagram.
+1323 // Only needed if we are building a new datagram
+1324 match self.spaces[space_id].for_path(path_id).loss_probes {
+1325 0 => transmit.start_new_datagram(),
+1326 _ => {
+1327 // We need something to send for a tail-loss probe.
+1328 let request_immediate_ack =
+1329 space_id == SpaceId::Data && self.peer_supports_ack_frequency();
+1330 self.spaces[space_id].maybe_queue_probe(
+1331 path_id,
+1332 request_immediate_ack,
+1333 &self.streams,
+1334 );
+1335
+1336 self.spaces[space_id].for_path(path_id).loss_probes -= 1;
+1337
+1338 // Clamp the datagram to at most the minimum MTU to ensure that loss
+1339 // probes can get through and enable recovery even if the path MTU
+1340 // has shrank unexpectedly.
+1341 transmit.start_new_datagram_with_size(std::cmp::min(
+1342 usize::from(INITIAL_MTU),
+1343 transmit.segment_size(),
+1344 ));
+1345 }
+1346 }
+1347 trace!(count = transmit.num_datagrams(), "new datagram started");
+1348 *coalesce = true;
+1349 *pad_datagram = PadDatagram::No;
+1350 }
+1351
+1352 let is_close = sendable_frames.close;
1353
-1354 if self.spaces[SpaceId::Initial].crypto.is_some()
-1355 && space_id == SpaceId::Handshake
-1356 && self.side.is_client()
-1357 {
-1358 // A client stops both sending and processing Initial packets when it
-1359 // sends its first Handshake packet.
-1360 self.discard_space(now, SpaceId::Initial);
-1361 }
-1362 if let Some(ref mut prev) = self.prev_crypto {
-1363 prev.update_unacked = false;
-1364 }
-1365
-1366 let Some(mut builder) = PacketBuilder::new(
-1367 now,
-1368 space_id,
-1369 path_id,
-1370 remote_cid,
+1354 // Send an off-path PATH_RESPONSE. Prioritized over on-path data to ensure that
+1355 // path validation can occur while the link is saturated.
+1356 if !is_close && space_id == SpaceId::Data {
+1357 if let Some(transmit) = self.build_off_path_challenge(
+1358 now,
+1359 path_id,
+1360 space_id,
+1361 remote_cid,
+1362 sendable_frames.other,
+1363 transmit,
+1364 ) {
+1365 return PollPathSpaceStatus::SendTransmit { transmit };
+1366 }
+1367 }
+1368
+1369 let n = self.poll_transmit_path_space_datagram(
+1370 now,
1371 transmit,
-1372 can_send.other,
-1373 self,
-1374 ) else {
-1375 return PollPathSpaceStatus::NothingToSend { congestion_blocked };
-1376 };
-1377 last_packet_number = Some(builder.exact_number);
-1378 *coalesce = *coalesce && !builder.short_header;
-1379
-1380 if space_id == SpaceId::Initial && (self.side.is_client() || can_send.other) {
-1381 // https://www.rfc-editor.org/rfc/rfc9000.html#section-14.1
-1382 *pad_datagram |= PadDatagram::ToMinMtu;
-1383 }
-1384 if space_id == SpaceId::Data && self.config.pad_to_mtu {
-1385 *pad_datagram |= PadDatagram::ToSegmentSize;
+1372 path_id,
+1373 space_id,
+1374 remote_cid,
+1375 have_available_path,
+1376 close,
+1377 sendable_frames,
+1378 coalesce,
+1379 pad_datagram,
+1380 );
+1381 if let Some(n) = n {
+1382 last_packet_number.replace(n);
+1383 } else {
+1384 // error, unable to send
+1385 return PollPathSpaceStatus::NothingToSend { congestion_blocked };
1386 }
1387
-1388 if can_send.close {
-1389 trace!("sending CONNECTION_CLOSE");
-1390 // Encode ACKs before the ConnectionClose message, to give the receiver
-1391 // a better approximate on what data has been processed. This is
-1392 // especially important with ack delay, since the peer might not
-1393 // have gotten any other ACK for the data earlier on.
-1394 let is_multipath_negotiated = self.is_multipath_negotiated();
-1395 for path_id in self.spaces[space_id]
-1396 .number_spaces
-1397 .iter()
-1398 .filter(|(_, pns)| !pns.pending_acks.ranges().is_empty())
-1399 .map(|(&path_id, _)| path_id)
-1400 .collect::<Vec<_>>()
-1401 {
-1402 Self::populate_acks(
-1403 now,
-1404 self.receiving_ecn,
-1405 path_id,
-1406 space_id,
-1407 &mut self.spaces[space_id],
-1408 is_multipath_negotiated,
-1409 &mut builder,
-1410 &mut self.stats.frame_tx,
-1411 );
-1412 }
-1413
-1414 // Since there only 64 ACK frames there will always be enough space
-1415 // to encode the ConnectionClose frame too. However we still have the
-1416 // check here to prevent crashes if something changes.
-1417 debug_assert!(
-1418 builder.frame_space_remaining() > frame::ConnectionClose::SIZE_BOUND,
-1419 "ACKs should leave space for ConnectionClose"
-1420 );
-1421 let stats = &mut self.stats.frame_tx;
-1422 if frame::ConnectionClose::SIZE_BOUND < builder.frame_space_remaining() {
-1423 let max_frame_size = builder.frame_space_remaining();
-1424 let close: Close = match self.state.as_type() {
-1425 StateType::Closed => {
-1426 let reason: Close =
-1427 self.state.as_closed().expect("checked").clone().into();
-1428 if space_id == SpaceId::Data || reason.is_transport_layer() {
-1429 reason
-1430 } else {
-1431 TransportError::APPLICATION_ERROR("").into()
-1432 }
-1433 }
-1434 StateType::Draining => TransportError::NO_ERROR("").into(),
-1435 _ => unreachable!(
-1436 "tried to make a close packet when the connection wasn't closed"
-1437 ),
-1438 };
-1439 builder.write_frame(close.encoder(max_frame_size), stats);
-1440 }
-1441 builder.finish_and_track(now, self, path_id, *pad_datagram);
-1442 if space_id == self.highest_space {
-1443 // Don't send another close packet. Even with multipath we only send
-1444 // CONNECTION_CLOSE on a single path since we expect our paths to work.
-1445 self.close = false;
-1446 // `CONNECTION_CLOSE` is the final packet
-1447 return PollPathSpaceStatus::Send { last_packet_number };
-1448 } else {
-1449 // Send a close frame in every possible space for robustness, per
-1450 // RFC9000 "Immediate Close during the Handshake". Don't bother trying
-1451 // to send anything else.
-1452 return PollPathSpaceStatus::NextSpace;
-1453 }
-1454 }
-1455
-1456 // Send an off-path PATH_RESPONSE. Prioritized over on-path data to ensure that
-1457 // path validation can occur while the link is saturated.
-1458 if space_id == SpaceId::Data && builder.buf.num_datagrams() == 1 {
-1459 let path = self.path_data_mut(path_id);
-1460 if let Some((token, network_path)) =
-1461 path.path_responses.pop_off_path(path.network_path)
-1462 {
-1463 // TODO(flub): We need to use the right CID! We shouldn't use the same
-1464 // CID as the current active one for the path. Though see also
-1465 // https://github.com/quinn-rs/quinn/issues/2184
-1466 //
-1467 let stats = &mut self.stats.frame_tx;
-1468 let frame = frame::PathResponse(token);
-1469 builder.write_frame_with_log_msg(frame, stats, Some("(off-path)"));
-1470 builder.finish_and_track(now, self, path_id, PadDatagram::ToMinMtu);
-1471 self.stats.udp_tx.on_sent(1, transmit.len());
-1472 return PollPathSpaceStatus::SendTransmit {
-1473 transmit: Transmit {
-1474 destination: network_path.remote,
-1475 size: transmit.len(),
-1476 ecn: None,
-1477 segment_size: None,
-1478 src_ip: network_path.local_ip,
-1479 },
-1480 };
-1481 }
-1482 }
-1483
-1484 let path_exclusive_only =
-1485 have_available_path && self.path_data(path_id).local_status() == PathStatus::Backup;
-1486 self.populate_packet(now, space_id, path_id, path_exclusive_only, &mut builder);
-1487
-1488 // ACK-only packets should only be sent when explicitly allowed. If we write them due to
-1489 // any other reason, there is a bug which leads to one component announcing write
-1490 // readiness while not writing any data. This degrades performance. The condition is
-1491 // only checked if the full MTU is available and when potentially large fixed-size
-1492 // frames aren't queued, so that lack of space in the datagram isn't the reason for just
-1493 // writing ACKs.
-1494 debug_assert!(
-1495 !(builder.sent_frames().is_ack_only(&self.streams)
-1496 && !can_send.acks
-1497 && can_send.other
-1498 && builder.buf.segment_size()
-1499 == self.path_data(path_id).current_mtu() as usize
-1500 && self.datagrams.outgoing.is_empty()),
-1501 "SendableFrames was {can_send:?}, but only ACKs have been written"
-1502 );
-1503 if builder.sent_frames().requires_padding {
-1504 *pad_datagram |= PadDatagram::ToMinMtu;
-1505 }
-1506
-1507 for (path_id, _pn) in builder.sent_frames().largest_acked.iter() {
-1508 self.spaces[space_id]
-1509 .for_path(*path_id)
-1510 .pending_acks
-1511 .acks_sent();
-1512 self.timers.stop(
-1513 Timer::PerPath(*path_id, PathTimer::MaxAckDelay),
-1514 self.qlog.with_time(now),
-1515 );
-1516 }
-1517
-1518 // Now we need to finish the packet. Before we do so we need to know if we will
-1519 // be coalescing the next packet into this one, or will be ending the datagram
-1520 // as well. Because if this is the last packet in the datagram more padding
-1521 // might be needed because of the packet type, or to fill the GSO segment size.
-1522
-1523 // Are we allowed to coalesce AND is there enough space for another *packet* in
-1524 // this datagram AND is there another packet to send in this or the next space?
-1525 if *coalesce
-1526 && builder
-1527 .buf
-1528 .datagram_remaining_mut()
-1529 .saturating_sub(builder.predict_packet_end())
-1530 > MIN_PACKET_SPACE
-1531 && self.has_next_send_space(space_id, path_id, builder.buf, close)
-1532 {
-1533 // We can append/coalesce the next packet into the current
-1534 // datagram. Finish the current packet without adding extra padding.
-1535 builder.finish_and_track(now, self, path_id, PadDatagram::No);
-1536 } else {
-1537 // We need a new datagram for the next packet. Finish the current
-1538 // packet with padding.
-1539 if builder.buf.num_datagrams() > 1 && matches!(pad_datagram, PadDatagram::No) {
-1540 // If too many padding bytes would be required to continue the
-1541 // GSO batch after this packet, end the GSO batch here. Ensures
-1542 // that fixed-size frames with heterogeneous sizes
-1543 // (e.g. application datagrams) won't inadvertently waste large
-1544 // amounts of bandwidth. The exact threshold is a bit arbitrary
-1545 // and might benefit from further tuning, though there's no
-1546 // universally optimal value.
-1547 const MAX_PADDING: usize = 32;
-1548 if builder.buf.datagram_remaining_mut()
-1549 > builder.predict_packet_end() + MAX_PADDING
-1550 {
-1551 trace!(
-1552 "GSO truncated by demand for {} padding bytes",
-1553 builder.buf.datagram_remaining_mut() - builder.predict_packet_end()
-1554 );
-1555 builder.finish_and_track(now, self, path_id, PadDatagram::No);
-1556 return PollPathSpaceStatus::Send { last_packet_number };
-1557 }
-1558
-1559 // Pad the current datagram to GSO segment size so it can be
-1560 // included in the GSO batch.
-1561 builder.finish_and_track(now, self, path_id, PadDatagram::ToSegmentSize);
-1562 } else {
-1563 builder.finish_and_track(now, self, path_id, *pad_datagram);
-1564 }
-1565 if transmit.num_datagrams() == 1 {
-1566 transmit.clip_datagram_size();
-1567 }
-1568 }
-1569 }
-1570 }
-1571
-1572 fn on_remote_cids_exhausted(&mut self, now: Instant, path_id: PathId) {
-1573 if self.abandoned_paths.contains(&path_id) {
-1574 trace!(%path_id, "remote CIDs retired for abandoned path");
-1575 return;
-1576 }
-1577
-1578 let error = PathError::RemoteCidsExhausted;
-1579 debug!(?error, %path_id, "no active CID for path");
-1580 self.events
-1581 .push_back(Event::Path(PathEvent::LocallyClosed { id: path_id, error }));
-1582 // Locally we should have refused to open this path, the remote should
-1583 // have given us CIDs for this path before opening it. So we can always
-1584 // abandon this here.
-1585 self.close_path(
-1586 now,
-1587 path_id,
-1588 TransportErrorCode::NO_CID_AVAILABLE_FOR_PATH.into(),
-1589 )
-1590 .ok();
-1591 self.spaces[SpaceId::Data]
-1592 .pending
-1593 .path_cids_blocked
-1594 .insert(path_id);
-1595 }
-1596
-1597 fn poll_transmit_mtu_probe(
-1598 &mut self,
-1599 now: Instant,
-1600 transmit: &mut TransmitBuf<'_>,
-1601 path_id: PathId,
-1602 ) {
-1603 let Some((active_cid, probe_size)) = self.get_mtu_probe_data(now, path_id) else {
-1604 return;
-1605 };
-1606
-1607 // We are definitely sending a DPLPMTUD probe.
-1608 debug_assert_eq!(transmit.num_datagrams(), 0);
-1609 transmit.start_new_datagram_with_size(probe_size as usize);
-1610
-1611 let Some(mut builder) = PacketBuilder::new(
-1612 now,
-1613 SpaceId::Data,
-1614 path_id,
-1615 active_cid,
-1616 transmit,
-1617 true,
-1618 self,
-1619 ) else {
-1620 return;
-1621 };
-1622
-1623 // We implement MTU probes as ping packets padded up to the probe size
-1624 trace!(?probe_size, "writing MTUD probe");
-1625 builder.write_frame(frame::Ping, &mut self.stats.frame_tx);
-1626
-1627 // If supported by the peer, we want no delays to the probe's ACK
-1628 if self.peer_supports_ack_frequency() {
-1629 builder.write_frame(frame::ImmediateAck, &mut self.stats.frame_tx);
-1630 }
-1631
-1632 builder.finish_and_track(now, self, path_id, PadDatagram::ToSize(probe_size));
-1633
-1634 self.path_stats
-1635 .entry(path_id)
-1636 .or_default()
-1637 .sent_plpmtud_probes += 1;
-1638 }
-1639
-1640 fn get_mtu_probe_data(&mut self, now: Instant, path_id: PathId) -> Option<(ConnectionId, u16)> {
-1641 // We MTU probe all paths for which all of the following is true:
-1642 // - We have an active destination CID for the path.
-1643 // - The remote address *and* path are validated.
-1644 // - The path is not abandoned.
-1645 // - The MTU Discovery subsystem wants to probe the path.
-1646 let active_cid = self.rem_cids.get(&path_id).map(CidQueue::active)?;
-1647 let is_eligible = self.path_data(path_id).validated
-1648 && !self.path_data(path_id).is_validating_path()
-1649 && !self.abandoned_paths.contains(&path_id);
-1650
-1651 if !is_eligible {
-1652 return None;
+1388 if is_close {
+1389 if space_id < self.highest_space {
+1390 // Send a close frame in every possible space for robustness, per
+1391 // RFC9000 "Immediate Close during the Handshake". Don't bother trying
+1392 // to send anything else.
+1393 continue;
+1394 } else {
+1395 // Don't send another close packet, we are in the highest space
+1396
+1397 // Even with multipath we only send
+1398 // CONNECTION_CLOSE on a single path since we expect our paths to work.
+1399 self.close = false;
+1400 // `CONNECTION_CLOSE` is the final packet
+1401 break;
+1402 }
+1403 }
+1404
+1405 // Clip the first datagram
+1406 if transmit.num_datagrams() == 1 {
+1407 transmit.clip_datagram_size();
+1408 }
+1409 }
+1410
+1411 if last_packet_number.is_some() {
+1412 // No more datagrams allowed
+1413 PollPathSpaceStatus::Send { last_packet_number }
+1414 } else {
+1415 PollPathSpaceStatus::NothingToSend {
+1416 congestion_blocked: false,
+1417 }
+1418 }
+1419 }
+1420
+1421 /// Tries to build a single datagram
+1422 fn poll_transmit_path_space_datagram(
+1423 &mut self,
+1424 now: Instant,
+1425 transmit: &mut TransmitBuf<'_>,
+1426 path_id: PathId,
+1427 space_id: SpaceId,
+1428 remote_cid: ConnectionId,
+1429 have_available_path: bool,
+1430 close: bool,
+1431 sendable_frames: SendableFrames,
+1432 coalesce: &mut bool,
+1433 pad_datagram: &mut PadDatagram,
+1434 ) -> Option<u64> {
+1435 if self.spaces[SpaceId::Initial].crypto.is_some()
+1436 && space_id == SpaceId::Handshake
+1437 && self.side.is_client()
+1438 {
+1439 // A client stops both sending and processing Initial packets when it
+1440 // sends its first Handshake packet.
+1441 self.discard_space(now, SpaceId::Initial);
+1442 }
+1443 if let Some(ref mut prev) = self.prev_crypto {
+1444 prev.update_unacked = false;
+1445 }
+1446
+1447 let mut builder = PacketBuilder::new(
+1448 now,
+1449 space_id,
+1450 path_id,
+1451 remote_cid,
+1452 transmit,
+1453 sendable_frames.other,
+1454 self,
+1455 )?;
+1456 *coalesce = *coalesce && !builder.short_header;
+1457
+1458 if space_id == SpaceId::Initial && (self.side.is_client() || sendable_frames.other) {
+1459 // https://www.rfc-editor.org/rfc/rfc9000.html#section-14.1
+1460 *pad_datagram |= PadDatagram::ToMinMtu;
+1461 }
+1462 if space_id == SpaceId::Data && self.config.pad_to_mtu {
+1463 *pad_datagram |= PadDatagram::ToSegmentSize;
+1464 }
+1465
+1466 if sendable_frames.close {
+1467 trace!("sending CONNECTION_CLOSE");
+1468 // Encode ACKs before the ConnectionClose message, to give the receiver
+1469 // a better approximate on what data has been processed. This is
+1470 // especially important with ack delay, since the peer might not
+1471 // have gotten any other ACK for the data earlier on.
+1472 let is_multipath_negotiated = self.is_multipath_negotiated();
+1473 for path_id in self.spaces[space_id]
+1474 .number_spaces
+1475 .iter()
+1476 .filter(|(_, pns)| !pns.pending_acks.ranges().is_empty())
+1477 .map(|(&path_id, _)| path_id)
+1478 .collect::<Vec<_>>()
+1479 {
+1480 Self::populate_acks(
+1481 now,
+1482 self.receiving_ecn,
+1483 path_id,
+1484 space_id,
+1485 &mut self.spaces[space_id],
+1486 is_multipath_negotiated,
+1487 &mut builder,
+1488 &mut self.stats.frame_tx,
+1489 );
+1490 }
+1491
+1492 // Since there only 64 ACK frames there will always be enough space
+1493 // to encode the ConnectionClose frame too. However we still have the
+1494 // check here to prevent crashes if something changes.
+1495 debug_assert!(
+1496 builder.frame_space_remaining() > frame::ConnectionClose::SIZE_BOUND,
+1497 "ACKs should leave space for ConnectionClose"
+1498 );
+1499 let stats = &mut self.stats.frame_tx;
+1500 if frame::ConnectionClose::SIZE_BOUND < builder.frame_space_remaining() {
+1501 let max_frame_size = builder.frame_space_remaining();
+1502 let close: Close = match self.state.as_type() {
+1503 StateType::Closed => {
+1504 let reason: Close = self.state.as_closed().expect("checked").clone().into();
+1505 if space_id == SpaceId::Data || reason.is_transport_layer() {
+1506 reason
+1507 } else {
+1508 TransportError::APPLICATION_ERROR("").into()
+1509 }
+1510 }
+1511 StateType::Draining => TransportError::NO_ERROR("").into(),
+1512 _ => unreachable!(
+1513 "tried to make a close packet when the connection wasn't closed"
+1514 ),
+1515 };
+1516 builder.write_frame(close.encoder(max_frame_size), stats);
+1517 }
+1518 return Some(builder.finish_and_track(now, self, path_id, *pad_datagram));
+1519 }
+1520
+1521 let path_exclusive_only =
+1522 have_available_path && self.path_data(path_id).local_status() == PathStatus::Backup;
+1523 self.populate_packet(now, space_id, path_id, path_exclusive_only, &mut builder);
+1524
+1525 // ACK-only packets should only be sent when explicitly allowed. If we write them due to
+1526 // any other reason, there is a bug which leads to one component announcing write
+1527 // readiness while not writing any data. This degrades performance. The condition is
+1528 // only checked if the full MTU is available and when potentially large fixed-size
+1529 // frames aren't queued, so that lack of space in the datagram isn't the reason for just
+1530 // writing ACKs.
+1531 debug_assert!(
+1532 !(builder.sent_frames().is_ack_only(&self.streams)
+1533 && !sendable_frames.acks
+1534 && sendable_frames.other
+1535 && builder.buf.segment_size() == self.path_data(path_id).current_mtu() as usize
+1536 && self.datagrams.outgoing.is_empty()),
+1537 "SendableFrames was {sendable_frames:?}, but only ACKs have been written"
+1538 );
+1539 if builder.sent_frames().requires_padding {
+1540 *pad_datagram |= PadDatagram::ToMinMtu;
+1541 }
+1542
+1543 for (path_id, _pn) in builder.sent_frames().largest_acked.iter() {
+1544 self.spaces[space_id]
+1545 .for_path(*path_id)
+1546 .pending_acks
+1547 .acks_sent();
+1548 self.timers.stop(
+1549 Timer::PerPath(*path_id, PathTimer::MaxAckDelay),
+1550 self.qlog.with_time(now),
+1551 );
+1552 }
+1553
+1554 // Now we need to finish the packet. Before we do so we need to know if we will
+1555 // be coalescing the next packet into this one, or will be ending the datagram
+1556 // as well. Because if this is the last packet in the datagram more padding
+1557 // might be needed because of the packet type, or to fill the GSO segment size.
+1558 let padding =
+1559 self.calculate_padding(path_id, space_id, *coalesce, close, *pad_datagram, &builder);
+1560
+1561 Some(builder.finish_and_track(now, self, path_id, padding))
+1562 }
+1563
+1564 fn build_off_path_challenge(
+1565 &mut self,
+1566 now: Instant,
+1567 path_id: PathId,
+1568 space_id: SpaceId,
+1569 remote_cid: ConnectionId,
+1570 ack_eliciting: bool,
+1571 transmit: &mut TransmitBuf<'_>,
+1572 ) -> Option<Transmit> {
+1573 let mut builder = PacketBuilder::new(
+1574 now,
+1575 space_id,
+1576 path_id,
+1577 remote_cid,
+1578 transmit,
+1579 ack_eliciting,
+1580 self,
+1581 )?;
+1582 let path = self.path_data_mut(path_id);
+1583 if let Some((token, network_path)) = path.path_responses.pop_off_path(path.network_path) {
+1584 // TODO(flub): We need to use the right CID! We shouldn't use the same
+1585 // CID as the current active one for the path. Though see also
+1586 // https://github.com/quinn-rs/quinn/issues/2184
+1587 //
+1588 let stats = &mut self.stats.frame_tx;
+1589 let frame = frame::PathResponse(token);
+1590 builder.write_frame_with_log_msg(frame, stats, Some("(off-path)"));
+1591 builder.finish_and_track(now, self, path_id, PadDatagram::ToMinMtu);
+1592 self.stats.udp_tx.on_sent(1, transmit.len());
+1593 return Some(Transmit {
+1594 destination: network_path.remote,
+1595 size: transmit.len(),
+1596 ecn: None,
+1597 segment_size: None,
+1598 src_ip: network_path.local_ip,
+1599 });
+1600 }
+1601 None
+1602 }
+1603
+1604 fn calculate_padding(
+1605 &mut self,
+1606 path_id: PathId,
+1607 space_id: SpaceId,
+1608 coalesce: bool,
+1609 close: bool,
+1610 pad_datagram: PadDatagram,
+1611 builder: &PacketBuilder<'_, '_>,
+1612 ) -> PadDatagram {
+1613 // Are we allowed to coalesce AND is there enough space for another *packet* in
+1614 // this datagram AND is there another packet to send in this or the next space?
+1615 if coalesce
+1616 && builder
+1617 .buf
+1618 .datagram_remaining_mut()
+1619 .saturating_sub(builder.predict_packet_end())
+1620 > MIN_PACKET_SPACE
+1621 && self.has_next_send_space(space_id, path_id, builder.buf, close)
+1622 {
+1623 // We can append/coalesce the next packet into the current
+1624 // datagram. Finish the current packet without adding extra padding.
+1625 PadDatagram::No
+1626 } else {
+1627 // We need a new datagram for the next packet. Finish the current
+1628 // packet with padding.
+1629 if builder.buf.num_datagrams() > 1 && matches!(pad_datagram, PadDatagram::No) {
+1630 // If too many padding bytes would be required to continue the
+1631 // GSO batch after this packet, end the GSO batch here. Ensures
+1632 // that fixed-size frames with heterogeneous sizes
+1633 // (e.g. application datagrams) won't inadvertently waste large
+1634 // amounts of bandwidth. The exact threshold is a bit arbitrary
+1635 // and might benefit from further tuning, though there's no
+1636 // universally optimal value.
+1637 const MAX_PADDING: usize = 32;
+1638 if builder.buf.datagram_remaining_mut() > builder.predict_packet_end() + MAX_PADDING
+1639 {
+1640 trace!(
+1641 "GSO truncated by demand for {} padding bytes",
+1642 builder.buf.datagram_remaining_mut() - builder.predict_packet_end()
+1643 );
+1644 PadDatagram::No
+1645 } else {
+1646 // Pad the current datagram to GSO segment size so it can be
+1647 // included in the GSO batch.
+1648 PadDatagram::ToSegmentSize
+1649 }
+1650 } else {
+1651 pad_datagram
+1652 }
1653 }
-1654 let next_pn = self.spaces[SpaceId::Data]
-1655 .for_path(path_id)
-1656 .peek_tx_number();
-1657 let probe_size = self
-1658 .path_data_mut(path_id)
-1659 .mtud
-1660 .poll_transmit(now, next_pn)?;
-1661
-1662 Some((active_cid, probe_size))
-1663 }
-1664
-1665 /// Returns if there is anext packet space which has data to send
-1666 ///
-1667 /// This takes into account the space available to frames in the next datagram.
-1668 // TODO(flub): This duplication is not nice.
-1669 fn has_next_send_space(
-1670 &mut self,
-1671 current_space_id: SpaceId,
-1672 path_id: PathId,
-1673 buf: &TransmitBuf<'_>,
-1674 close: bool,
-1675 ) -> bool {
-1676 // Number of bytes available for frames if this is a 1-RTT packet. We're guaranteed
-1677 // to be able to send an individual frame at least this large in the next 1-RTT
-1678 // packet. This could be generalized to support every space, but it's only needed to
-1679 // handle large fixed-size frames, which only exist in 1-RTT (application
-1680 // datagrams). We don't account for coalesced packets potentially occupying space
-1681 // because frames can always spill into the next datagram.
-1682 let mut space_id = current_space_id;
-1683 loop {
-1684 let can_send = self.space_can_send(space_id, path_id, buf.segment_size(), close);
-1685 if !can_send.is_empty() || (close && self.spaces[space_id].crypto.is_some()) {
-1686 return true;
-1687 }
-1688 let Some(next_space) = space_id.next() else {
-1689 break;
-1690 };
-1691 space_id = next_space;
-1692 }
-1693 false
-1694 }
-1695
-1696 /// Checks if creating a new datagram would be blocked by congestion control
-1697 fn path_congestion_check(
-1698 &mut self,
-1699 space_id: SpaceId,
-1700 path_id: PathId,
-1701 transmit: &TransmitBuf<'_>,
-1702 can_send: &SendableFrames,
-1703 now: Instant,
-1704 ) -> PathBlocked {
-1705 // Anti-amplification is only based on `total_sent`, which gets updated after
-1706 // the transmit is sent. Therefore we pass the amount of bytes for datagrams
-1707 // that are already created, as well as 1 byte for starting another datagram. If
-1708 // there is any anti-amplification budget left, we always allow a full MTU to be
-1709 // sent (see https://github.com/quinn-rs/quinn/issues/1082).
-1710 if self.side().is_server()
-1711 && self
-1712 .path_data(path_id)
-1713 .anti_amplification_blocked(transmit.len() as u64 + 1)
-1714 {
-1715 trace!(?space_id, %path_id, "blocked by anti-amplification");
-1716 return PathBlocked::AntiAmplification;
-1717 }
-1718
-1719 // Congestion control check.
-1720 // Tail loss probes must not be blocked by congestion, or a deadlock could arise.
-1721 let bytes_to_send = transmit.segment_size() as u64;
-1722 let need_loss_probe = self.spaces[space_id].for_path(path_id).loss_probes > 0;
-1723
-1724 if can_send.other && !need_loss_probe && !can_send.close {
-1725 let path = self.path_data(path_id);
-1726 if path.in_flight.bytes + bytes_to_send >= path.congestion.window() {
-1727 trace!(?space_id, %path_id, "blocked by congestion control");
-1728 return PathBlocked::Congestion;
-1729 }
-1730 }
-1731
-1732 // Pacing check.
-1733 if let Some(delay) = self.path_data_mut(path_id).pacing_delay(bytes_to_send, now) {
-1734 self.timers.set(
-1735 Timer::PerPath(path_id, PathTimer::Pacing),
-1736 delay,
-1737 self.qlog.with_time(now),
-1738 );
-1739 // Loss probes and CONNECTION_CLOSE should be subject to pacing, even though
-1740 // they are not congestion controlled.
-1741 trace!(?space_id, %path_id, "blocked by pacing");
-1742 return PathBlocked::Pacing;
-1743 }
-1744
-1745 PathBlocked::No
-1746 }
-1747
-1748 /// Send PATH_CHALLENGE for a previous path if necessary
-1749 ///
-1750 /// QUIC-TRANSPORT section 9.3.3
-1751 /// <https://www.rfc-editor.org/rfc/rfc9000.html#name-off-path-packet-forwarding>
-1752 fn send_prev_path_challenge(
-1753 &mut self,
-1754 now: Instant,
-1755 buf: &mut TransmitBuf<'_>,
-1756 path_id: PathId,
-1757 ) -> Option<Transmit> {
-1758 let (prev_cid, prev_path) = self.paths.get_mut(&path_id)?.prev.as_mut()?;
-1759 // TODO (matheus23): We could use !prev_path.is_validating() here instead to
-1760 // (possibly) also re-send challenges when they get lost.
-1761 if !prev_path.send_new_challenge {
-1762 return None;
-1763 };
-1764 prev_path.send_new_challenge = false;
-1765 let network_path = prev_path.network_path;
-1766 let token = self.rng.random();
-1767 let info = paths::SentChallengeInfo {
-1768 sent_instant: now,
-1769 network_path,
-1770 };
-1771 prev_path.challenges_sent.insert(token, info);
-1772 debug_assert_eq!(
-1773 self.highest_space,
-1774 SpaceId::Data,
-1775 "PATH_CHALLENGE queued without 1-RTT keys"
-1776 );
-1777 buf.start_new_datagram_with_size(MIN_INITIAL_SIZE as usize);
-1778
-1779 // Use the previous CID to avoid linking the new path with the previous path. We
-1780 // don't bother accounting for possible retirement of that prev_cid because this is
-1781 // sent once, immediately after migration, when the CID is known to be valid. Even
-1782 // if a post-migration packet caused the CID to be retired, it's fair to pretend
-1783 // this is sent first.
-1784 debug_assert_eq!(buf.datagram_start_offset(), 0);
-1785 let mut builder =
-1786 PacketBuilder::new(now, SpaceId::Data, path_id, *prev_cid, buf, false, self)?;
-1787 let challenge = frame::PathChallenge(token);
-1788 let stats = &mut self.stats.frame_tx;
-1789 builder.write_frame_with_log_msg(challenge, stats, Some("validating previous path"));
-1790
-1791 // An endpoint MUST expand datagrams that contain a PATH_CHALLENGE frame
-1792 // to at least the smallest allowed maximum datagram size of 1200 bytes,
-1793 // unless the anti-amplification limit for the path does not permit
-1794 // sending a datagram of this size
-1795 builder.pad_to(MIN_INITIAL_SIZE);
-1796
-1797 builder.finish(self, now);
-1798 self.stats.udp_tx.on_sent(1, buf.len());
-1799
-1800 Some(Transmit {
-1801 destination: network_path.remote,
-1802 size: buf.len(),
-1803 ecn: None,
-1804 segment_size: None,
-1805 src_ip: network_path.local_ip,
-1806 })
-1807 }
-1808
-1809 /// Indicate what types of frames are ready to send for the given space
-1810 ///
-1811 /// *packet_size* is the number of bytes available to build the next packet. *close*
-1812 /// *indicates whether a CONNECTION_CLOSE frame needs to be sent.
-1813 fn space_can_send(
-1814 &mut self,
-1815 space_id: SpaceId,
-1816 path_id: PathId,
-1817 packet_size: usize,
-1818 close: bool,
-1819 ) -> SendableFrames {
-1820 let pn = self.spaces[SpaceId::Data]
-1821 .for_path(path_id)
-1822 .peek_tx_number();
-1823 let frame_space_1rtt = packet_size.saturating_sub(self.predict_1rtt_overhead(pn, path_id));
-1824 if self.spaces[space_id].crypto.is_none()
-1825 && (space_id != SpaceId::Data
-1826 || self.zero_rtt_crypto.is_none()
-1827 || self.side.is_server())
-1828 {
-1829 // No keys available for this space
-1830 return SendableFrames::empty();
-1831 }
-1832 let mut can_send = self.spaces[space_id].can_send(path_id, &self.streams);
-1833 if space_id == SpaceId::Data {
-1834 can_send |= self.can_send_1rtt(path_id, frame_space_1rtt);
-1835 }
-1836
-1837 can_send.close = close && self.spaces[space_id].crypto.is_some();
-1838
-1839 can_send
-1840 }
-1841
-1842 /// Process `ConnectionEvent`s generated by the associated `Endpoint`
-1843 ///
-1844 /// Will execute protocol logic upon receipt of a connection event, in turn preparing signals
-1845 /// (including application `Event`s, `EndpointEvent`s and outgoing datagrams) that should be
-1846 /// extracted through the relevant methods.
-1847 pub fn handle_event(&mut self, event: ConnectionEvent) {
-1848 use ConnectionEventInner::*;
-1849 match event.0 {
-1850 Datagram(DatagramConnectionEvent {
-1851 now,
-1852 network_path,
-1853 path_id,
-1854 ecn,
-1855 first_decode,
-1856 remaining,
-1857 }) => {
-1858 let span = trace_span!("pkt", %path_id);
-1859 let _guard = span.enter();
-1860
-1861 if self.update_network_path_or_discard(network_path, path_id) {
-1862 // A return value of true indicates we should discard this packet.
-1863 return;
-1864 }
-1865
-1866 let was_anti_amplification_blocked = self
-1867 .path(path_id)
-1868 .map(|path| path.anti_amplification_blocked(1))
-1869 .unwrap_or(true); // if we don't know about this path it's eagerly considered as unvalidated
-1870 // TODO(@divma): revisit this
-1871
-1872 self.stats.udp_rx.datagrams += 1;
-1873 self.stats.udp_rx.bytes += first_decode.len() as u64;
-1874 let data_len = first_decode.len();
-1875
-1876 self.handle_decode(now, network_path, path_id, ecn, first_decode);
-1877 // The current `path` might have changed inside `handle_decode` since the packet
-1878 // could have triggered a migration. The packet might also belong to an unknown
-1879 // path and have been rejected. Make sure the data received is accounted for the
-1880 // most recent path by accessing `path` after `handle_decode`.
-1881 if let Some(path) = self.path_mut(path_id) {
-1882 path.inc_total_recvd(data_len as u64);
-1883 }
-1884
-1885 if let Some(data) = remaining {
-1886 self.stats.udp_rx.bytes += data.len() as u64;
-1887 self.handle_coalesced(now, network_path, path_id, ecn, data);
-1888 }
-1889
-1890 if let Some(path) = self.paths.get_mut(&path_id) {
-1891 self.qlog
-1892 .emit_recovery_metrics(path_id, &mut path.data, now);
-1893 }
-1894
-1895 if was_anti_amplification_blocked {
-1896 // A prior attempt to set the loss detection timer may have failed due to
-1897 // anti-amplification, so ensure it's set now. Prevents a handshake deadlock if
-1898 // the server's first flight is lost.
-1899 self.set_loss_detection_timer(now, path_id);
-1900 }
-1901 }
-1902 NewIdentifiers(ids, now, cid_len, cid_lifetime) => {
-1903 let path_id = ids.first().map(|issued| issued.path_id).unwrap_or_default();
-1904 debug_assert!(ids.iter().all(|issued| issued.path_id == path_id));
-1905 let cid_state = self
-1906 .local_cid_state
-1907 .entry(path_id)
-1908 .or_insert_with(|| CidState::new(cid_len, cid_lifetime, now, 0));
-1909 cid_state.new_cids(&ids, now);
-1910
-1911 ids.into_iter().rev().for_each(|frame| {
-1912 self.spaces[SpaceId::Data].pending.new_cids.push(frame);
-1913 });
-1914 // Always update Timer::PushNewCid
-1915 self.reset_cid_retirement(now);
-1916 }
-1917 }
-1918 }
-1919
-1920 /// Updates the network path for `path_id`.
-1921 ///
-1922 /// Returns true if a packet coming in for this `path_id` over given `network_path` should be discarded.
-1923 /// Returns false if the path was updated and the packet doesn't need to be discarded.
-1924 fn update_network_path_or_discard(&mut self, network_path: FourTuple, path_id: PathId) -> bool {
-1925 let remote_may_migrate = self.side.remote_may_migrate(&self.state);
-1926 let local_ip_may_migrate = self.side.is_client();
-1927 // If this packet could initiate a migration and we're a client or a server that
-1928 // forbids migration, drop the datagram. This could be relaxed to heuristically
-1929 // permit NAT-rebinding-like migration.
-1930 if let Some(known_path) = self.path_mut(path_id) {
-1931 if network_path.remote != known_path.network_path.remote && !remote_may_migrate {
-1932 trace!(
-1933 %path_id,
-1934 %network_path,
-1935 %known_path.network_path,
-1936 "discarding packet from unrecognized peer"
-1937 );
-1938 return true;
-1939 }
-1940
-1941 if known_path.network_path.local_ip.is_some()
-1942 && network_path.local_ip.is_some()
-1943 && known_path.network_path.local_ip != network_path.local_ip
-1944 && !local_ip_may_migrate
-1945 {
-1946 trace!(
-1947 %path_id,
-1948 %network_path,
-1949 %known_path.network_path,
-1950 "discarding packet sent to incorrect interface"
-1951 );
-1952 return true;
-1953 }
-1954 // If the datagram indicates that we've changed our local IP, we update it.
-1955 // This is alluded to in Section 5.2 of the Multipath RFC draft 18:
-1956 // https://www.ietf.org/archive/id/draft-ietf-quic-multipath-18.html#name-using-multiple-paths-on-the
-1957 // > Client receives the packet, recognizes a path migration, updates the source address of path 2 to 192.0.2.1.
-1958 if let Some(local_ip) = network_path.local_ip {
-1959 if known_path
-1960 .network_path
-1961 .local_ip
-1962 .is_some_and(|ip| ip != local_ip)
-1963 {
-1964 debug!(
-1965 %path_id,
-1966 %network_path,
-1967 %known_path.network_path,
-1968 "path's local address seemingly migrated"
-1969 );
-1970 }
-1971 // We update the address without path validation on the client side.
-1972 // https://www.ietf.org/archive/id/draft-ietf-quic-multipath-18.html#section-5.1
-1973 // > Servers observing a 4-tuple change will perform path validation (see Section 9 of [QUIC-TRANSPORT]).
-1974 // This sounds like it's *only* the server endpoints that do this.
-1975 // TODO(matheus23): We should still consider doing a proper migration on the client side in the future.
-1976 // For now, this preserves the behavior of this code pre 4-tuple tracking.
-1977 known_path.network_path.local_ip = Some(local_ip);
-1978 }
-1979 }
-1980 false
-1981 }
-1982
-1983 /// Process timer expirations
-1984 ///
-1985 /// Executes protocol logic, potentially preparing signals (including application `Event`s,
-1986 /// `EndpointEvent`s and outgoing datagrams) that should be extracted through the relevant
-1987 /// methods.
-1988 ///
-1989 /// It is most efficient to call this immediately after the system clock reaches the latest
-1990 /// `Instant` that was output by `poll_timeout`; however spurious extra calls will simply
-1991 /// no-op and therefore are safe.
-1992 pub fn handle_timeout(&mut self, now: Instant) {
-1993 while let Some((timer, _time)) = self.timers.expire_before(now, &self.qlog) {
-1994 // TODO(@divma): remove `at` when the unicorn is born
-1995 trace!(?timer, at=?now, "timeout");
-1996 match timer {
-1997 Timer::Conn(timer) => match timer {
-1998 ConnTimer::Close => {
-1999 self.state.move_to_drained(None);
-2000 self.endpoint_events.push_back(EndpointEventInner::Drained);
-2001 }
-2002 ConnTimer::Idle => {
-2003 self.kill(ConnectionError::TimedOut);
-2004 }
-2005 ConnTimer::KeepAlive => {
-2006 trace!("sending keep-alive");
-2007 self.ping();
-2008 }
-2009 ConnTimer::KeyDiscard => {
-2010 self.zero_rtt_crypto = None;
-2011 self.prev_crypto = None;
-2012 }
-2013 ConnTimer::PushNewCid => {
-2014 while let Some((path_id, when)) = self.next_cid_retirement() {
-2015 if when > now {
-2016 break;
-2017 }
-2018 match self.local_cid_state.get_mut(&path_id) {
-2019 None => error!(%path_id, "No local CID state for path"),
-2020 Some(cid_state) => {
-2021 // Update `retire_prior_to` field in NEW_CONNECTION_ID frame
-2022 let num_new_cid = cid_state.on_cid_timeout().into();
-2023 if !self.state.is_closed() {
-2024 trace!(
-2025 "push a new CID to peer RETIRE_PRIOR_TO field {}",
-2026 cid_state.retire_prior_to()
-2027 );
-2028 self.endpoint_events.push_back(
-2029 EndpointEventInner::NeedIdentifiers(
-2030 path_id,
-2031 now,
-2032 num_new_cid,
-2033 ),
-2034 );
-2035 }
-2036 }
-2037 }
-2038 }
-2039 }
-2040 },
-2041 // TODO: add path_id as span somehow
-2042 Timer::PerPath(path_id, timer) => {
-2043 let span = trace_span!("per-path timer fired", %path_id, ?timer);
-2044 let _guard = span.enter();
-2045 match timer {
-2046 PathTimer::PathIdle => {
-2047 self.close_path(now, path_id, TransportErrorCode::NO_ERROR.into())
-2048 .ok();
-2049 }
-2050
-2051 PathTimer::PathKeepAlive => {
-2052 trace!("sending keep-alive on path");
-2053 self.ping_path(path_id).ok();
-2054 }
-2055 PathTimer::LossDetection => {
-2056 self.on_loss_detection_timeout(now, path_id);
-2057 self.qlog.emit_recovery_metrics(
-2058 path_id,
-2059 &mut self.paths.get_mut(&path_id).unwrap().data,
-2060 now,
-2061 );
-2062 }
-2063 PathTimer::PathValidation => {
-2064 let Some(path) = self.paths.get_mut(&path_id) else {
-2065 continue;
-2066 };
-2067 self.timers.stop(
-2068 Timer::PerPath(path_id, PathTimer::PathChallengeLost),
-2069 self.qlog.with_time(now),
-2070 );
-2071 debug!("path validation failed");
-2072 if let Some((_, prev)) = path.prev.take() {
-2073 path.data = prev;
-2074 }
-2075 path.data.challenges_sent.clear();
-2076 path.data.send_new_challenge = false;
-2077 }
-2078 PathTimer::PathChallengeLost => {
-2079 let Some(path) = self.paths.get_mut(&path_id) else {
-2080 continue;
-2081 };
-2082 trace!("path challenge deemed lost");
-2083 path.data.send_new_challenge = true;
-2084 }
-2085 PathTimer::PathOpen => {
-2086 let Some(path) = self.paths.get_mut(&path_id) else {
-2087 continue;
-2088 };
-2089 path.data.challenges_sent.clear();
-2090 path.data.send_new_challenge = false;
-2091 self.timers.stop(
-2092 Timer::PerPath(path_id, PathTimer::PathChallengeLost),
-2093 self.qlog.with_time(now),
-2094 );
-2095 debug!("new path validation failed");
-2096 if let Err(err) = self.close_path(
-2097 now,
-2098 path_id,
-2099 TransportErrorCode::PATH_UNSTABLE_OR_POOR.into(),
-2100 ) {
-2101 warn!(?err, "failed closing path");
-2102 }
-2103
-2104 self.events.push_back(Event::Path(PathEvent::LocallyClosed {
-2105 id: path_id,
-2106 error: PathError::ValidationFailed,
-2107 }));
-2108 }
-2109 PathTimer::Pacing => trace!("pacing timer expired"),
-2110 PathTimer::MaxAckDelay => {
-2111 trace!("max ack delay reached");
-2112 // This timer is only armed in the Data space
-2113 self.spaces[SpaceId::Data]
-2114 .for_path(path_id)
-2115 .pending_acks
-2116 .on_max_ack_delay_timeout()
-2117 }
-2118 PathTimer::DiscardPath => {
-2119 // The path was abandoned and 3*PTO has expired since. Clean up all
-2120 // remaining state and install stateless reset token.
-2121 self.timers.stop_per_path(path_id, self.qlog.with_time(now));
-2122 if let Some(loc_cid_state) = self.local_cid_state.remove(&path_id) {
-2123 let (min_seq, max_seq) = loc_cid_state.active_seq();
-2124 for seq in min_seq..=max_seq {
-2125 self.endpoint_events.push_back(
-2126 EndpointEventInner::RetireConnectionId(
-2127 now, path_id, seq, false,
-2128 ),
-2129 );
-2130 }
-2131 }
-2132 self.discard_path(path_id, now);
-2133 }
-2134 }
-2135 }
-2136 }
-2137 }
-2138 }
-2139
-2140 /// Close a connection immediately
-2141 ///
-2142 /// This does not ensure delivery of outstanding data. It is the application's responsibility to
-2143 /// call this only when all important communications have been completed, e.g. by calling
-2144 /// [`SendStream::finish`] on outstanding streams and waiting for the corresponding
-2145 /// [`StreamEvent::Finished`] event.
-2146 ///
-2147 /// If [`Streams::send_streams`] returns 0, all outstanding stream data has been
-2148 /// delivered. There may still be data from the peer that has not been received.
-2149 ///
-2150 /// [`StreamEvent::Finished`]: crate::StreamEvent::Finished
-2151 pub fn close(&mut self, now: Instant, error_code: VarInt, reason: Bytes) {
-2152 self.close_inner(
-2153 now,
-2154 Close::Application(frame::ApplicationClose { error_code, reason }),
-2155 )
-2156 }
-2157
-2158 fn close_inner(&mut self, now: Instant, reason: Close) {
-2159 let was_closed = self.state.is_closed();
-2160 if !was_closed {
-2161 self.close_common();
-2162 self.set_close_timer(now);
-2163 self.close = true;
-2164 self.state.move_to_closed_local(reason);
-2165 }
-2166 }
-2167
-2168 /// Control datagrams
-2169 pub fn datagrams(&mut self) -> Datagrams<'_> {
-2170 Datagrams { conn: self }
-2171 }
-2172
-2173 /// Returns connection statistics
-2174 pub fn stats(&mut self) -> ConnectionStats {
-2175 self.stats.clone()
-2176 }
-2177
-2178 /// Returns path statistics
-2179 pub fn path_stats(&mut self, path_id: PathId) -> Option<PathStats> {
-2180 let path = self.paths.get(&path_id)?;
-2181 let stats = self.path_stats.entry(path_id).or_default();
-2182 stats.rtt = path.data.rtt.get();
-2183 stats.cwnd = path.data.congestion.window();
-2184 stats.current_mtu = path.data.mtud.current_mtu();
-2185 Some(*stats)
-2186 }
-2187
-2188 /// Ping the remote endpoint
-2189 ///
-2190 /// Causes an ACK-eliciting packet to be transmitted on the connection.
-2191 pub fn ping(&mut self) {
-2192 // TODO(flub): This is very brute-force: it pings *all* the paths. Instead it would
-2193 // be nice if we could only send a single packet for this.
-2194 for path_data in self.spaces[self.highest_space].number_spaces.values_mut() {
-2195 path_data.ping_pending = true;
-2196 }
-2197 }
-2198
-2199 /// Ping the remote endpoint over a specific path
-2200 ///
-2201 /// Causes an ACK-eliciting packet to be transmitted on the path.
-2202 pub fn ping_path(&mut self, path: PathId) -> Result<(), ClosedPath> {
-2203 let path_data = self.spaces[self.highest_space]
-2204 .number_spaces
-2205 .get_mut(&path)
-2206 .ok_or(ClosedPath { _private: () })?;
-2207 path_data.ping_pending = true;
-2208 Ok(())
-2209 }
-2210
-2211 /// Update traffic keys spontaneously
-2212 ///
-2213 /// This can be useful for testing key updates, as they otherwise only happen infrequently.
-2214 pub fn force_key_update(&mut self) {
-2215 if !self.state.is_established() {
-2216 debug!("ignoring forced key update in illegal state");
-2217 return;
-2218 }
-2219 if self.prev_crypto.is_some() {
-2220 // We already just updated, or are currently updating, the keys. Concurrent key updates
-2221 // are illegal.
-2222 debug!("ignoring redundant forced key update");
-2223 return;
-2224 }
-2225 self.update_keys(None, false);
-2226 }
-2227
-2228 /// Get a session reference
-2229 pub fn crypto_session(&self) -> &dyn crypto::Session {
-2230 &*self.crypto
-2231 }
-2232
-2233 /// Whether the connection is in the process of being established
-2234 ///
-2235 /// If this returns `false`, the connection may be either established or closed, signaled by the
-2236 /// emission of a `Connected` or `ConnectionLost` message respectively.
-2237 pub fn is_handshaking(&self) -> bool {
-2238 self.state.is_handshake()
-2239 }
-2240
-2241 /// Whether the connection is closed
-2242 ///
-2243 /// Closed connections cannot transport any further data. A connection becomes closed when
-2244 /// either peer application intentionally closes it, or when either transport layer detects an
-2245 /// error such as a time-out or certificate validation failure.
-2246 ///
-2247 /// A `ConnectionLost` event is emitted with details when the connection becomes closed.
-2248 pub fn is_closed(&self) -> bool {
-2249 self.state.is_closed()
-2250 }
-2251
-2252 /// Whether there is no longer any need to keep the connection around
-2253 ///
-2254 /// Closed connections become drained after a brief timeout to absorb any remaining in-flight
-2255 /// packets from the peer. All drained connections have been closed.
-2256 pub fn is_drained(&self) -> bool {
-2257 self.state.is_drained()
-2258 }
-2259
-2260 /// For clients, if the peer accepted the 0-RTT data packets
-2261 ///
-2262 /// The value is meaningless until after the handshake completes.
-2263 pub fn accepted_0rtt(&self) -> bool {
-2264 self.accepted_0rtt
-2265 }
-2266
-2267 /// Whether 0-RTT is/was possible during the handshake
-2268 pub fn has_0rtt(&self) -> bool {
-2269 self.zero_rtt_enabled
-2270 }
-2271
-2272 /// Whether there are any pending retransmits
-2273 pub fn has_pending_retransmits(&self) -> bool {
-2274 !self.spaces[SpaceId::Data].pending.is_empty(&self.streams)
-2275 }
-2276
-2277 /// Look up whether we're the client or server of this Connection
-2278 pub fn side(&self) -> Side {
-2279 self.side.side()
-2280 }
-2281
-2282 /// Get the address observed by the remote over the given path
-2283 pub fn path_observed_address(&self, path_id: PathId) -> Result<Option<SocketAddr>, ClosedPath> {
-2284 self.path(path_id)
-2285 .map(|path_data| {
-2286 path_data
-2287 .last_observed_addr_report
-2288 .as_ref()
-2289 .map(|observed| observed.socket_addr())
-2290 })
-2291 .ok_or(ClosedPath { _private: () })
-2292 }
-2293
-2294 /// Current best estimate of this connection's latency (round-trip-time)
-2295 pub fn rtt(&self, path_id: PathId) -> Option<Duration> {
-2296 self.path(path_id).map(|d| d.rtt.get())
-2297 }
-2298
-2299 /// Current state of this connection's congestion controller, for debugging purposes
-2300 pub fn congestion_state(&self, path_id: PathId) -> Option<&dyn Controller> {
-2301 self.path(path_id).map(|d| d.congestion.as_ref())
-2302 }
-2303
-2304 /// Modify the number of remotely initiated streams that may be concurrently open
-2305 ///
-2306 /// No streams may be opened by the peer unless fewer than `count` are already open. Large
-2307 /// `count`s increase both minimum and worst-case memory consumption.
-2308 pub fn set_max_concurrent_streams(&mut self, dir: Dir, count: VarInt) {
-2309 self.streams.set_max_concurrent(dir, count);
-2310 // If the limit was reduced, then a flow control update previously deemed insignificant may
-2311 // now be significant.
-2312 let pending = &mut self.spaces[SpaceId::Data].pending;
-2313 self.streams.queue_max_stream_id(pending);
+1654 }
+1655
+1656 fn path_should_send(
+1657 &mut self,
+1658 path_id: PathId,
+1659 space_id: SpaceId,
+1660 have_available_path: bool,
+1661 sendable_frames: &SendableFrames,
+1662 ) -> bool {
+1663 let path_exclusive_only = space_id == SpaceId::Data
+1664 && have_available_path
+1665 && self.path_data(path_id).local_status() == PathStatus::Backup;
+1666 let path_should_send = if path_exclusive_only {
+1667 sendable_frames.path_exclusive
+1668 } else {
+1669 !sendable_frames.is_empty()
+1670 };
+1671 let needs_loss_probe = self.spaces[space_id].for_path(path_id).loss_probes > 0;
+1672 path_should_send || needs_loss_probe
+1673 }
+1674
+1675 fn on_remote_cids_exhausted(&mut self, now: Instant, path_id: PathId) {
+1676 if self.abandoned_paths.contains(&path_id) {
+1677 trace!(%path_id, "remote CIDs retired for abandoned path");
+1678 return;
+1679 }
+1680
+1681 let error = PathError::RemoteCidsExhausted;
+1682 debug!(?error, %path_id, "no active CID for path");
+1683 self.events
+1684 .push_back(Event::Path(PathEvent::LocallyClosed { id: path_id, error }));
+1685 // Locally we should have refused to open this path, the remote should
+1686 // have given us CIDs for this path before opening it. So we can always
+1687 // abandon this here.
+1688 self.close_path(
+1689 now,
+1690 path_id,
+1691 TransportErrorCode::NO_CID_AVAILABLE_FOR_PATH.into(),
+1692 )
+1693 .ok();
+1694 self.spaces[SpaceId::Data]
+1695 .pending
+1696 .path_cids_blocked
+1697 .insert(path_id);
+1698 }
+1699
+1700 fn poll_transmit_mtu_probe(
+1701 &mut self,
+1702 now: Instant,
+1703 transmit: &mut TransmitBuf<'_>,
+1704 path_id: PathId,
+1705 ) {
+1706 let Some((active_cid, probe_size)) = self.get_mtu_probe_data(now, path_id) else {
+1707 return;
+1708 };
+1709
+1710 // We are definitely sending a DPLPMTUD probe.
+1711 debug_assert_eq!(transmit.num_datagrams(), 0);
+1712 transmit.start_new_datagram_with_size(probe_size as usize);
+1713
+1714 let Some(mut builder) = PacketBuilder::new(
+1715 now,
+1716 SpaceId::Data,
+1717 path_id,
+1718 active_cid,
+1719 transmit,
+1720 true,
+1721 self,
+1722 ) else {
+1723 return;
+1724 };
+1725
+1726 // We implement MTU probes as ping packets padded up to the probe size
+1727 trace!(?probe_size, "writing MTUD probe");
+1728 builder.write_frame(frame::Ping, &mut self.stats.frame_tx);
+1729
+1730 // If supported by the peer, we want no delays to the probe's ACK
+1731 if self.peer_supports_ack_frequency() {
+1732 builder.write_frame(frame::ImmediateAck, &mut self.stats.frame_tx);
+1733 }
+1734
+1735 builder.finish_and_track(now, self, path_id, PadDatagram::ToSize(probe_size));
+1736
+1737 self.path_stats
+1738 .entry(path_id)
+1739 .or_default()
+1740 .sent_plpmtud_probes += 1;
+1741 }
+1742
+1743 fn get_mtu_probe_data(&mut self, now: Instant, path_id: PathId) -> Option<(ConnectionId, u16)> {
+1744 // We MTU probe all paths for which all of the following is true:
+1745 // - We have an active destination CID for the path.
+1746 // - The remote address *and* path are validated.
+1747 // - The path is not abandoned.
+1748 // - The MTU Discovery subsystem wants to probe the path.
+1749 let active_cid = self.rem_cids.get(&path_id).map(CidQueue::active)?;
+1750 let is_eligible = self.path_data(path_id).validated
+1751 && !self.path_data(path_id).is_validating_path()
+1752 && !self.abandoned_paths.contains(&path_id);
+1753
+1754 if !is_eligible {
+1755 return None;
+1756 }
+1757 let next_pn = self.spaces[SpaceId::Data]
+1758 .for_path(path_id)
+1759 .peek_tx_number();
+1760 let probe_size = self
+1761 .path_data_mut(path_id)
+1762 .mtud
+1763 .poll_transmit(now, next_pn)?;
+1764
+1765 Some((active_cid, probe_size))
+1766 }
+1767
+1768 /// Returns if there is anext packet space which has data to send
+1769 ///
+1770 /// This takes into account the space available to frames in the next datagram.
+1771 // TODO(flub): This duplication is not nice.
+1772 fn has_next_send_space(
+1773 &mut self,
+1774 current_space_id: SpaceId,
+1775 path_id: PathId,
+1776 buf: &TransmitBuf<'_>,
+1777 close: bool,
+1778 ) -> bool {
+1779 // Number of bytes available for frames if this is a 1-RTT packet. We're guaranteed
+1780 // to be able to send an individual frame at least this large in the next 1-RTT
+1781 // packet. This could be generalized to support every space, but it's only needed to
+1782 // handle large fixed-size frames, which only exist in 1-RTT (application
+1783 // datagrams). We don't account for coalesced packets potentially occupying space
+1784 // because frames can always spill into the next datagram.
+1785 let mut space_id = current_space_id;
+1786 loop {
+1787 let can_send = self.space_can_send(space_id, path_id, buf.segment_size(), close);
+1788 if !can_send.is_empty() || (close && self.spaces[space_id].crypto.is_some()) {
+1789 return true;
+1790 }
+1791 let Some(next_space) = space_id.next() else {
+1792 break;
+1793 };
+1794 space_id = next_space;
+1795 }
+1796 false
+1797 }
+1798
+1799 /// Checks if creating a new datagram would be blocked by congestion control
+1800 fn path_congestion_check(
+1801 &mut self,
+1802 space_id: SpaceId,
+1803 path_id: PathId,
+1804 transmit: &TransmitBuf<'_>,
+1805 can_send: &SendableFrames,
+1806 now: Instant,
+1807 ) -> PathBlocked {
+1808 // Anti-amplification is only based on `total_sent`, which gets updated after
+1809 // the transmit is sent. Therefore we pass the amount of bytes for datagrams
+1810 // that are already created, as well as 1 byte for starting another datagram. If
+1811 // there is any anti-amplification budget left, we always allow a full MTU to be
+1812 // sent (see https://github.com/quinn-rs/quinn/issues/1082).
+1813 if self.side().is_server()
+1814 && self
+1815 .path_data(path_id)
+1816 .anti_amplification_blocked(transmit.len() as u64 + 1)
+1817 {
+1818 trace!(?space_id, %path_id, "blocked by anti-amplification");
+1819 return PathBlocked::AntiAmplification;
+1820 }
+1821
+1822 // Congestion control check.
+1823 // Tail loss probes must not be blocked by congestion, or a deadlock could arise.
+1824 let bytes_to_send = transmit.segment_size() as u64;
+1825 let need_loss_probe = self.spaces[space_id].for_path(path_id).loss_probes > 0;
+1826
+1827 if can_send.other && !need_loss_probe && !can_send.close {
+1828 let path = self.path_data(path_id);
+1829 if path.in_flight.bytes + bytes_to_send >= path.congestion.window() {
+1830 trace!(?space_id, %path_id, "blocked by congestion control");
+1831 return PathBlocked::Congestion;
+1832 }
+1833 }
+1834
+1835 // Pacing check.
+1836 if let Some(delay) = self.path_data_mut(path_id).pacing_delay(bytes_to_send, now) {
+1837 self.timers.set(
+1838 Timer::PerPath(path_id, PathTimer::Pacing),
+1839 delay,
+1840 self.qlog.with_time(now),
+1841 );
+1842 // Loss probes and CONNECTION_CLOSE should be subject to pacing, even though
+1843 // they are not congestion controlled.
+1844 trace!(?space_id, %path_id, "blocked by pacing");
+1845 return PathBlocked::Pacing;
+1846 }
+1847
+1848 PathBlocked::No
+1849 }
+1850
+1851 /// Send PATH_CHALLENGE for a previous path if necessary
+1852 ///
+1853 /// QUIC-TRANSPORT section 9.3.3
+1854 /// <https://www.rfc-editor.org/rfc/rfc9000.html#name-off-path-packet-forwarding>
+1855 fn send_prev_path_challenge(
+1856 &mut self,
+1857 now: Instant,
+1858 buf: &mut TransmitBuf<'_>,
+1859 path_id: PathId,
+1860 ) -> Option<Transmit> {
+1861 let (prev_cid, prev_path) = self.paths.get_mut(&path_id)?.prev.as_mut()?;
+1862 // TODO (matheus23): We could use !prev_path.is_validating() here instead to
+1863 // (possibly) also re-send challenges when they get lost.
+1864 if !prev_path.send_new_challenge {
+1865 return None;
+1866 };
+1867 prev_path.send_new_challenge = false;
+1868 let network_path = prev_path.network_path;
+1869 let token = self.rng.random();
+1870 let info = paths::SentChallengeInfo {
+1871 sent_instant: now,
+1872 network_path,
+1873 };
+1874 prev_path.challenges_sent.insert(token, info);
+1875 debug_assert_eq!(
+1876 self.highest_space,
+1877 SpaceId::Data,
+1878 "PATH_CHALLENGE queued without 1-RTT keys"
+1879 );
+1880 buf.start_new_datagram_with_size(MIN_INITIAL_SIZE as usize);
+1881
+1882 // Use the previous CID to avoid linking the new path with the previous path. We
+1883 // don't bother accounting for possible retirement of that prev_cid because this is
+1884 // sent once, immediately after migration, when the CID is known to be valid. Even
+1885 // if a post-migration packet caused the CID to be retired, it's fair to pretend
+1886 // this is sent first.
+1887 debug_assert_eq!(buf.datagram_start_offset(), 0);
+1888 let mut builder =
+1889 PacketBuilder::new(now, SpaceId::Data, path_id, *prev_cid, buf, false, self)?;
+1890 let challenge = frame::PathChallenge(token);
+1891 let stats = &mut self.stats.frame_tx;
+1892 builder.write_frame_with_log_msg(challenge, stats, Some("validating previous path"));
+1893
+1894 // An endpoint MUST expand datagrams that contain a PATH_CHALLENGE frame
+1895 // to at least the smallest allowed maximum datagram size of 1200 bytes,
+1896 // unless the anti-amplification limit for the path does not permit
+1897 // sending a datagram of this size
+1898 builder.pad_to(MIN_INITIAL_SIZE);
+1899
+1900 builder.finish(self, now);
+1901 self.stats.udp_tx.on_sent(1, buf.len());
+1902
+1903 Some(Transmit {
+1904 destination: network_path.remote,
+1905 size: buf.len(),
+1906 ecn: None,
+1907 segment_size: None,
+1908 src_ip: network_path.local_ip,
+1909 })
+1910 }
+1911
+1912 /// Indicate what types of frames are ready to send for the given space
+1913 ///
+1914 /// *packet_size* is the number of bytes available to build the next packet.
+1915 /// *close* indicates whether a CONNECTION_CLOSE frame needs to be sent.
+1916 fn space_can_send(
+1917 &mut self,
+1918 space_id: SpaceId,
+1919 path_id: PathId,
+1920 packet_size: usize,
+1921 close: bool,
+1922 ) -> SendableFrames {
+1923 let space = &mut self.spaces[space_id];
+1924 let space_has_crypto = space.crypto.is_none();
+1925
+1926 if !space_has_crypto
+1927 && (space_id != SpaceId::Data
+1928 || self.zero_rtt_crypto.is_none()
+1929 || self.side.is_server())
+1930 {
+1931 // Nothing send in this space
+1932 return SendableFrames::empty();
+1933 }
+1934
+1935 let mut can_send = space.can_send(path_id, &self.streams);
+1936
+1937 if space_id == SpaceId::Data {
+1938 // Check for 1RTT space.
+1939 let pn = space.for_path(path_id).peek_tx_number();
+1940 let frame_space_1rtt =
+1941 packet_size.saturating_sub(self.predict_1rtt_overhead(pn, path_id));
+1942 can_send |= self.can_send_1rtt(path_id, frame_space_1rtt);
+1943 }
+1944
+1945 can_send.close = close && space_has_crypto;
+1946
+1947 can_send
+1948 }
+1949
+1950 fn space_can_send_non_data(
+1951 &mut self,
+1952 space_id: SpaceId,
+1953 path_id: PathId,
+1954 close: bool,
+1955 ) -> SendableFrames {
+1956 debug_assert!(space_id != SpaceId::Data);
+1957 let space = &mut self.spaces[space_id];
+1958 let space_has_crypto = space.crypto.is_none();
+1959
+1960 if !space_has_crypto && (self.zero_rtt_crypto.is_none() || self.side.is_server()) {
+1961 // Nothing send in this space
+1962 return SendableFrames::empty();
+1963 }
+1964
+1965 let mut can_send = space.can_send(path_id, &self.streams);
+1966 can_send.close = close && space_has_crypto;
+1967 can_send
+1968 }
+1969
+1970 /// Process `ConnectionEvent`s generated by the associated `Endpoint`
+1971 ///
+1972 /// Will execute protocol logic upon receipt of a connection event, in turn preparing signals
+1973 /// (including application `Event`s, `EndpointEvent`s and outgoing datagrams) that should be
+1974 /// extracted through the relevant methods.
+1975 pub fn handle_event(&mut self, event: ConnectionEvent) {
+1976 use ConnectionEventInner::*;
+1977 match event.0 {
+1978 Datagram(DatagramConnectionEvent {
+1979 now,
+1980 network_path,
+1981 path_id,
+1982 ecn,
+1983 first_decode,
+1984 remaining,
+1985 }) => {
+1986 let span = trace_span!("pkt", %path_id);
+1987 let _guard = span.enter();
+1988
+1989 if self.update_network_path_or_discard(network_path, path_id) {
+1990 // A return value of true indicates we should discard this packet.
+1991 return;
+1992 }
+1993
+1994 let was_anti_amplification_blocked = self
+1995 .path(path_id)
+1996 .map(|path| path.anti_amplification_blocked(1))
+1997 .unwrap_or(true); // if we don't know about this path it's eagerly considered as unvalidated
+1998 // TODO(@divma): revisit this
+1999
+2000 self.stats.udp_rx.datagrams += 1;
+2001 self.stats.udp_rx.bytes += first_decode.len() as u64;
+2002 let data_len = first_decode.len();
+2003
+2004 self.handle_decode(now, network_path, path_id, ecn, first_decode);
+2005 // The current `path` might have changed inside `handle_decode` since the packet
+2006 // could have triggered a migration. The packet might also belong to an unknown
+2007 // path and have been rejected. Make sure the data received is accounted for the
+2008 // most recent path by accessing `path` after `handle_decode`.
+2009 if let Some(path) = self.path_mut(path_id) {
+2010 path.inc_total_recvd(data_len as u64);
+2011 }
+2012
+2013 if let Some(data) = remaining {
+2014 self.stats.udp_rx.bytes += data.len() as u64;
+2015 self.handle_coalesced(now, network_path, path_id, ecn, data);
+2016 }
+2017
+2018 if let Some(path) = self.paths.get_mut(&path_id) {
+2019 self.qlog
+2020 .emit_recovery_metrics(path_id, &mut path.data, now);
+2021 }
+2022
+2023 if was_anti_amplification_blocked {
+2024 // A prior attempt to set the loss detection timer may have failed due to
+2025 // anti-amplification, so ensure it's set now. Prevents a handshake deadlock if
+2026 // the server's first flight is lost.
+2027 self.set_loss_detection_timer(now, path_id);
+2028 }
+2029 }
+2030 NewIdentifiers(ids, now, cid_len, cid_lifetime) => {
+2031 let path_id = ids.first().map(|issued| issued.path_id).unwrap_or_default();
+2032 debug_assert!(ids.iter().all(|issued| issued.path_id == path_id));
+2033 let cid_state = self
+2034 .local_cid_state
+2035 .entry(path_id)
+2036 .or_insert_with(|| CidState::new(cid_len, cid_lifetime, now, 0));
+2037 cid_state.new_cids(&ids, now);
+2038
+2039 ids.into_iter().rev().for_each(|frame| {
+2040 self.spaces[SpaceId::Data].pending.new_cids.push(frame);
+2041 });
+2042 // Always update Timer::PushNewCid
+2043 self.reset_cid_retirement(now);
+2044 }
+2045 }
+2046 }
+2047
+2048 /// Updates the network path for `path_id`.
+2049 ///
+2050 /// Returns true if a packet coming in for this `path_id` over given `network_path` should be discarded.
+2051 /// Returns false if the path was updated and the packet doesn't need to be discarded.
+2052 fn update_network_path_or_discard(&mut self, network_path: FourTuple, path_id: PathId) -> bool {
+2053 let remote_may_migrate = self.side.remote_may_migrate(&self.state);
+2054 let local_ip_may_migrate = self.side.is_client();
+2055 // If this packet could initiate a migration and we're a client or a server that
+2056 // forbids migration, drop the datagram. This could be relaxed to heuristically
+2057 // permit NAT-rebinding-like migration.
+2058 if let Some(known_path) = self.path_mut(path_id) {
+2059 if network_path.remote != known_path.network_path.remote && !remote_may_migrate {
+2060 trace!(
+2061 %path_id,
+2062 %network_path,
+2063 %known_path.network_path,
+2064 "discarding packet from unrecognized peer"
+2065 );
+2066 return true;
+2067 }
+2068
+2069 if known_path.network_path.local_ip.is_some()
+2070 && network_path.local_ip.is_some()
+2071 && known_path.network_path.local_ip != network_path.local_ip
+2072 && !local_ip_may_migrate
+2073 {
+2074 trace!(
+2075 %path_id,
+2076 %network_path,
+2077 %known_path.network_path,
+2078 "discarding packet sent to incorrect interface"
+2079 );
+2080 return true;
+2081 }
+2082 // If the datagram indicates that we've changed our local IP, we update it.
+2083 // This is alluded to in Section 5.2 of the Multipath RFC draft 18:
+2084 // https://www.ietf.org/archive/id/draft-ietf-quic-multipath-18.html#name-using-multiple-paths-on-the
+2085 // > Client receives the packet, recognizes a path migration, updates the source address of path 2 to 192.0.2.1.
+2086 if let Some(local_ip) = network_path.local_ip {
+2087 if known_path
+2088 .network_path
+2089 .local_ip
+2090 .is_some_and(|ip| ip != local_ip)
+2091 {
+2092 debug!(
+2093 %path_id,
+2094 %network_path,
+2095 %known_path.network_path,
+2096 "path's local address seemingly migrated"
+2097 );
+2098 }
+2099 // We update the address without path validation on the client side.
+2100 // https://www.ietf.org/archive/id/draft-ietf-quic-multipath-18.html#section-5.1
+2101 // > Servers observing a 4-tuple change will perform path validation (see Section 9 of [QUIC-TRANSPORT]).
+2102 // This sounds like it's *only* the server endpoints that do this.
+2103 // TODO(matheus23): We should still consider doing a proper migration on the client side in the future.
+2104 // For now, this preserves the behavior of this code pre 4-tuple tracking.
+2105 known_path.network_path.local_ip = Some(local_ip);
+2106 }
+2107 }
+2108 false
+2109 }
+2110
+2111 /// Process timer expirations
+2112 ///
+2113 /// Executes protocol logic, potentially preparing signals (including application `Event`s,
+2114 /// `EndpointEvent`s and outgoing datagrams) that should be extracted through the relevant
+2115 /// methods.
+2116 ///
+2117 /// It is most efficient to call this immediately after the system clock reaches the latest
+2118 /// `Instant` that was output by `poll_timeout`; however spurious extra calls will simply
+2119 /// no-op and therefore are safe.
+2120 pub fn handle_timeout(&mut self, now: Instant) {
+2121 while let Some((timer, _time)) = self.timers.expire_before(now, &self.qlog) {
+2122 // TODO(@divma): remove `at` when the unicorn is born
+2123 trace!(?timer, at=?now, "timeout");
+2124 match timer {
+2125 Timer::Conn(timer) => match timer {
+2126 ConnTimer::Close => {
+2127 self.state.move_to_drained(None);
+2128 self.endpoint_events.push_back(EndpointEventInner::Drained);
+2129 }
+2130 ConnTimer::Idle => {
+2131 self.kill(ConnectionError::TimedOut);
+2132 }
+2133 ConnTimer::KeepAlive => {
+2134 trace!("sending keep-alive");
+2135 self.ping();
+2136 }
+2137 ConnTimer::KeyDiscard => {
+2138 self.zero_rtt_crypto = None;
+2139 self.prev_crypto = None;
+2140 }
+2141 ConnTimer::PushNewCid => {
+2142 while let Some((path_id, when)) = self.next_cid_retirement() {
+2143 if when > now {
+2144 break;
+2145 }
+2146 match self.local_cid_state.get_mut(&path_id) {
+2147 None => error!(%path_id, "No local CID state for path"),
+2148 Some(cid_state) => {
+2149 // Update `retire_prior_to` field in NEW_CONNECTION_ID frame
+2150 let num_new_cid = cid_state.on_cid_timeout().into();
+2151 if !self.state.is_closed() {
+2152 trace!(
+2153 "push a new CID to peer RETIRE_PRIOR_TO field {}",
+2154 cid_state.retire_prior_to()
+2155 );
+2156 self.endpoint_events.push_back(
+2157 EndpointEventInner::NeedIdentifiers(
+2158 path_id,
+2159 now,
+2160 num_new_cid,
+2161 ),
+2162 );
+2163 }
+2164 }
+2165 }
+2166 }
+2167 }
+2168 },
+2169 // TODO: add path_id as span somehow
+2170 Timer::PerPath(path_id, timer) => {
+2171 let span = trace_span!("per-path timer fired", %path_id, ?timer);
+2172 let _guard = span.enter();
+2173 match timer {
+2174 PathTimer::PathIdle => {
+2175 self.close_path(now, path_id, TransportErrorCode::NO_ERROR.into())
+2176 .ok();
+2177 }
+2178
+2179 PathTimer::PathKeepAlive => {
+2180 trace!("sending keep-alive on path");
+2181 self.ping_path(path_id).ok();
+2182 }
+2183 PathTimer::LossDetection => {
+2184 self.on_loss_detection_timeout(now, path_id);
+2185 self.qlog.emit_recovery_metrics(
+2186 path_id,
+2187 &mut self.paths.get_mut(&path_id).unwrap().data,
+2188 now,
+2189 );
+2190 }
+2191 PathTimer::PathValidation => {
+2192 let Some(path) = self.paths.get_mut(&path_id) else {
+2193 continue;
+2194 };
+2195 self.timers.stop(
+2196 Timer::PerPath(path_id, PathTimer::PathChallengeLost),
+2197 self.qlog.with_time(now),
+2198 );
+2199 debug!("path validation failed");
+2200 if let Some((_, prev)) = path.prev.take() {
+2201 path.data = prev;
+2202 }
+2203 path.data.challenges_sent.clear();
+2204 path.data.send_new_challenge = false;
+2205 }
+2206 PathTimer::PathChallengeLost => {
+2207 let Some(path) = self.paths.get_mut(&path_id) else {
+2208 continue;
+2209 };
+2210 trace!("path challenge deemed lost");
+2211 path.data.send_new_challenge = true;
+2212 }
+2213 PathTimer::PathOpen => {
+2214 let Some(path) = self.paths.get_mut(&path_id) else {
+2215 continue;
+2216 };
+2217 path.data.challenges_sent.clear();
+2218 path.data.send_new_challenge = false;
+2219 self.timers.stop(
+2220 Timer::PerPath(path_id, PathTimer::PathChallengeLost),
+2221 self.qlog.with_time(now),
+2222 );
+2223 debug!("new path validation failed");
+2224 if let Err(err) = self.close_path(
+2225 now,
+2226 path_id,
+2227 TransportErrorCode::PATH_UNSTABLE_OR_POOR.into(),
+2228 ) {
+2229 warn!(?err, "failed closing path");
+2230 }
+2231
+2232 self.events.push_back(Event::Path(PathEvent::LocallyClosed {
+2233 id: path_id,
+2234 error: PathError::ValidationFailed,
+2235 }));
+2236 }
+2237 PathTimer::Pacing => trace!("pacing timer expired"),
+2238 PathTimer::MaxAckDelay => {
+2239 trace!("max ack delay reached");
+2240 // This timer is only armed in the Data space
+2241 self.spaces[SpaceId::Data]
+2242 .for_path(path_id)
+2243 .pending_acks
+2244 .on_max_ack_delay_timeout()
+2245 }
+2246 PathTimer::DiscardPath => {
+2247 // The path was abandoned and 3*PTO has expired since. Clean up all
+2248 // remaining state and install stateless reset token.
+2249 self.timers.stop_per_path(path_id, self.qlog.with_time(now));
+2250 if let Some(loc_cid_state) = self.local_cid_state.remove(&path_id) {
+2251 let (min_seq, max_seq) = loc_cid_state.active_seq();
+2252 for seq in min_seq..=max_seq {
+2253 self.endpoint_events.push_back(
+2254 EndpointEventInner::RetireConnectionId(
+2255 now, path_id, seq, false,
+2256 ),
+2257 );
+2258 }
+2259 }
+2260 self.discard_path(path_id, now);
+2261 }
+2262 }
+2263 }
+2264 }
+2265 }
+2266 }
+2267
+2268 /// Close a connection immediately
+2269 ///
+2270 /// This does not ensure delivery of outstanding data. It is the application's responsibility to
+2271 /// call this only when all important communications have been completed, e.g. by calling
+2272 /// [`SendStream::finish`] on outstanding streams and waiting for the corresponding
+2273 /// [`StreamEvent::Finished`] event.
+2274 ///
+2275 /// If [`Streams::send_streams`] returns 0, all outstanding stream data has been
+2276 /// delivered. There may still be data from the peer that has not been received.
+2277 ///
+2278 /// [`StreamEvent::Finished`]: crate::StreamEvent::Finished
+2279 pub fn close(&mut self, now: Instant, error_code: VarInt, reason: Bytes) {
+2280 self.close_inner(
+2281 now,
+2282 Close::Application(frame::ApplicationClose { error_code, reason }),
+2283 )
+2284 }
+2285
+2286 fn close_inner(&mut self, now: Instant, reason: Close) {
+2287 let was_closed = self.state.is_closed();
+2288 if !was_closed {
+2289 self.close_common();
+2290 self.set_close_timer(now);
+2291 self.close = true;
+2292 self.state.move_to_closed_local(reason);
+2293 }
+2294 }
+2295
+2296 /// Control datagrams
+2297 pub fn datagrams(&mut self) -> Datagrams<'_> {
+2298 Datagrams { conn: self }
+2299 }
+2300
+2301 /// Returns connection statistics
+2302 pub fn stats(&mut self) -> ConnectionStats {
+2303 self.stats.clone()
+2304 }
+2305
+2306 /// Returns path statistics
+2307 pub fn path_stats(&mut self, path_id: PathId) -> Option<PathStats> {
+2308 let path = self.paths.get(&path_id)?;
+2309 let stats = self.path_stats.entry(path_id).or_default();
+2310 stats.rtt = path.data.rtt.get();
+2311 stats.cwnd = path.data.congestion.window();
+2312 stats.current_mtu = path.data.mtud.current_mtu();
+2313 Some(*stats)
2314 }
2315
-2316 /// Modify the number of open paths allowed when multipath is enabled
+2316 /// Ping the remote endpoint
2317 ///
-2318 /// When reducing the number of concurrent paths this will only affect delaying sending
-2319 /// new MAX_PATH_ID frames until fewer than this number of paths are possible. To
-2320 /// actively reduce paths they must be closed using [`Connection::close_path`], which
-2321 /// can also be used to close not-yet-opened paths.
-2322 ///
-2323 /// If multipath is not negotiated (see the [`TransportConfig`]) this can not enable
-2324 /// multipath and will fail.
-2325 pub fn set_max_concurrent_paths(
-2326 &mut self,
-2327 now: Instant,
-2328 count: NonZeroU32,
-2329 ) -> Result<(), MultipathNotNegotiated> {
-2330 if !self.is_multipath_negotiated() {
-2331 return Err(MultipathNotNegotiated { _private: () });
-2332 }
-2333 self.max_concurrent_paths = count;
-2334
-2335 let in_use_count = self
-2336 .local_max_path_id
-2337 .next()
-2338 .saturating_sub(self.abandoned_paths.len() as u32)
-2339 .as_u32();
-2340 let extra_needed = count.get().saturating_sub(in_use_count);
-2341 let new_max_path_id = self.local_max_path_id.saturating_add(extra_needed);
-2342
-2343 self.set_max_path_id(now, new_max_path_id);
-2344
-2345 Ok(())
-2346 }
-2347
-2348 /// If needed, issues a new MAX_PATH_ID frame and new CIDs for any newly allowed paths
-2349 fn set_max_path_id(&mut self, now: Instant, max_path_id: PathId) {
-2350 if max_path_id <= self.local_max_path_id {
+2318 /// Causes an ACK-eliciting packet to be transmitted on the connection.
+2319 pub fn ping(&mut self) {
+2320 // TODO(flub): This is very brute-force: it pings *all* the paths. Instead it would
+2321 // be nice if we could only send a single packet for this.
+2322 for path_data in self.spaces[self.highest_space].number_spaces.values_mut() {
+2323 path_data.ping_pending = true;
+2324 }
+2325 }
+2326
+2327 /// Ping the remote endpoint over a specific path
+2328 ///
+2329 /// Causes an ACK-eliciting packet to be transmitted on the path.
+2330 pub fn ping_path(&mut self, path: PathId) -> Result<(), ClosedPath> {
+2331 let path_data = self.spaces[self.highest_space]
+2332 .number_spaces
+2333 .get_mut(&path)
+2334 .ok_or(ClosedPath { _private: () })?;
+2335 path_data.ping_pending = true;
+2336 Ok(())
+2337 }
+2338
+2339 /// Update traffic keys spontaneously
+2340 ///
+2341 /// This can be useful for testing key updates, as they otherwise only happen infrequently.
+2342 pub fn force_key_update(&mut self) {
+2343 if !self.state.is_established() {
+2344 debug!("ignoring forced key update in illegal state");
+2345 return;
+2346 }
+2347 if self.prev_crypto.is_some() {
+2348 // We already just updated, or are currently updating, the keys. Concurrent key updates
+2349 // are illegal.
+2350 debug!("ignoring redundant forced key update");
2351 return;
2352 }
-2353
-2354 self.local_max_path_id = max_path_id;
-2355 self.spaces[SpaceId::Data].pending.max_path_id = true;
-2356
-2357 self.issue_first_path_cids(now);
-2358 }
-2359
-2360 /// Current number of remotely initiated streams that may be concurrently open
-2361 ///
-2362 /// If the target for this limit is reduced using [`set_max_concurrent_streams`](Self::set_max_concurrent_streams),
-2363 /// it will not change immediately, even if fewer streams are open. Instead, it will
-2364 /// decrement by one for each time a remotely initiated stream of matching directionality is closed.
-2365 pub fn max_concurrent_streams(&self, dir: Dir) -> u64 {
-2366 self.streams.max_concurrent(dir)
+2353 self.update_keys(None, false);
+2354 }
+2355
+2356 /// Get a session reference
+2357 pub fn crypto_session(&self) -> &dyn crypto::Session {
+2358 &*self.crypto
+2359 }
+2360
+2361 /// Whether the connection is in the process of being established
+2362 ///
+2363 /// If this returns `false`, the connection may be either established or closed, signaled by the
+2364 /// emission of a `Connected` or `ConnectionLost` message respectively.
+2365 pub fn is_handshaking(&self) -> bool {
+2366 self.state.is_handshake()
2367 }
2368
-2369 /// See [`TransportConfig::send_window()`]
-2370 pub fn set_send_window(&mut self, send_window: u64) {
-2371 self.streams.set_send_window(send_window);
-2372 }
-2373
-2374 /// See [`TransportConfig::receive_window()`]
-2375 pub fn set_receive_window(&mut self, receive_window: VarInt) {
-2376 if self.streams.set_receive_window(receive_window) {
-2377 self.spaces[SpaceId::Data].pending.max_data = true;
-2378 }
-2379 }
-2380
-2381 /// Whether the Multipath for QUIC extension is enabled.
-2382 ///
-2383 /// Multipath is only enabled after the handshake is completed and if it was enabled by both
-2384 /// peers.
-2385 pub fn is_multipath_negotiated(&self) -> bool {
-2386 !self.is_handshaking()
-2387 && self.config.max_concurrent_multipath_paths.is_some()
-2388 && self.peer_params.initial_max_path_id.is_some()
-2389 }
-2390
-2391 fn on_ack_received(
-2392 &mut self,
-2393 now: Instant,
-2394 space: SpaceId,
-2395 ack: frame::Ack,
-2396 ) -> Result<(), TransportError> {
-2397 // All ACKs are referencing path 0
-2398 let path = PathId::ZERO;
-2399 self.inner_on_ack_received(now, space, path, ack)
-2400 }
-2401
-2402 fn on_path_ack_received(
-2403 &mut self,
-2404 now: Instant,
-2405 space: SpaceId,
-2406 path_ack: frame::PathAck,
-2407 ) -> Result<(), TransportError> {
-2408 let (ack, path) = path_ack.into_ack();
-2409 self.inner_on_ack_received(now, space, path, ack)
-2410 }
-2411
-2412 /// Handles an ACK frame acknowledging packets sent on *path*.
-2413 fn inner_on_ack_received(
-2414 &mut self,
-2415 now: Instant,
-2416 space: SpaceId,
-2417 path: PathId,
-2418 ack: frame::Ack,
-2419 ) -> Result<(), TransportError> {
-2420 if self.abandoned_paths.contains(&path) {
-2421 // See also https://www.ietf.org/archive/id/draft-ietf-quic-multipath-17.html#section-3.4.3-3
-2422 // > PATH_ACK frames received with an abandoned path ID are silently ignored, as specified in Section 4.
-2423 trace!("silently ignoring PATH_ACK on abandoned path");
-2424 return Ok(());
-2425 }
-2426 if ack.largest >= self.spaces[space].for_path(path).next_packet_number {
-2427 return Err(TransportError::PROTOCOL_VIOLATION("unsent packet acked"));
-2428 }
-2429 let new_largest = {
-2430 let space = &mut self.spaces[space].for_path(path);
-2431 if space.largest_acked_packet.is_none_or(|pn| ack.largest > pn) {
-2432 space.largest_acked_packet = Some(ack.largest);
-2433 if let Some(info) = space.sent_packets.get(ack.largest) {
-2434 // This should always succeed, but a misbehaving peer might ACK a packet we
-2435 // haven't sent. At worst, that will result in us spuriously reducing the
-2436 // congestion window.
-2437 space.largest_acked_packet_sent = info.time_sent;
-2438 }
-2439 true
-2440 } else {
-2441 false
-2442 }
-2443 };
-2444
-2445 if self.detect_spurious_loss(&ack, space, path) {
-2446 self.path_data_mut(path)
-2447 .congestion
-2448 .on_spurious_congestion_event();
-2449 }
-2450
-2451 // Avoid DoS from unreasonably huge ack ranges by filtering out just the new acks.
-2452 let mut newly_acked = ArrayRangeSet::new();
-2453 for range in ack.iter() {
-2454 self.spaces[space].for_path(path).check_ack(range.clone())?;
-2455 for (pn, _) in self.spaces[space]
-2456 .for_path(path)
-2457 .sent_packets
-2458 .iter_range(range)
-2459 {
-2460 newly_acked.insert_one(pn);
-2461 }
-2462 }
-2463
-2464 if newly_acked.is_empty() {
-2465 return Ok(());
-2466 }
-2467
-2468 let mut ack_eliciting_acked = false;
-2469 for packet in newly_acked.elts() {
-2470 if let Some(info) = self.spaces[space].for_path(path).take(packet) {
-2471 for (acked_path_id, acked_pn) in info.largest_acked.iter() {
-2472 // Assume ACKs for all packets below the largest acknowledged in
-2473 // `packet` have been received. This can cause the peer to spuriously
-2474 // retransmit if some of our earlier ACKs were lost, but allows for
-2475 // simpler state tracking. See discussion at
-2476 // https://www.rfc-editor.org/rfc/rfc9000.html#name-limiting-ranges-by-tracking
-2477 if let Some(pns) = self.spaces[space].path_space_mut(*acked_path_id) {
-2478 pns.pending_acks.subtract_below(*acked_pn);
-2479 }
-2480 }
-2481 ack_eliciting_acked |= info.ack_eliciting;
-2482
-2483 // Notify MTU discovery that a packet was acked, because it might be an MTU probe
-2484 let path_data = self.path_data_mut(path);
-2485 let mtu_updated = path_data.mtud.on_acked(space, packet, info.size);
-2486 if mtu_updated {
-2487 path_data
-2488 .congestion
-2489 .on_mtu_update(path_data.mtud.current_mtu());
-2490 }
-2491
-2492 // Notify ack frequency that a packet was acked, because it might contain an ACK_FREQUENCY frame
-2493 self.ack_frequency.on_acked(path, packet);
-2494
-2495 self.on_packet_acked(now, path, info);
-2496 }
-2497 }
-2498
-2499 let largest_ackd = self.spaces[space].for_path(path).largest_acked_packet;
-2500 let app_limited = self.app_limited;
-2501 let path_data = self.path_data_mut(path);
-2502 let in_flight = path_data.in_flight.bytes;
-2503
-2504 path_data
-2505 .congestion
-2506 .on_end_acks(now, in_flight, app_limited, largest_ackd);
-2507
-2508 if new_largest && ack_eliciting_acked {
-2509 let ack_delay = if space != SpaceId::Data {
-2510 Duration::from_micros(0)
-2511 } else {
-2512 cmp::min(
-2513 self.ack_frequency.peer_max_ack_delay,
-2514 Duration::from_micros(ack.delay << self.peer_params.ack_delay_exponent.0),
-2515 )
-2516 };
-2517 let rtt = now.saturating_duration_since(
-2518 self.spaces[space].for_path(path).largest_acked_packet_sent,
-2519 );
-2520
-2521 let next_pn = self.spaces[space].for_path(path).next_packet_number;
-2522 let path_data = self.path_data_mut(path);
-2523 // TODO(@divma): should be a method of path, should be contained in a single place
-2524 path_data.rtt.update(ack_delay, rtt);
-2525 if path_data.first_packet_after_rtt_sample.is_none() {
-2526 path_data.first_packet_after_rtt_sample = Some((space, next_pn));
-2527 }
-2528 }
+2369 /// Whether the connection is closed
+2370 ///
+2371 /// Closed connections cannot transport any further data. A connection becomes closed when
+2372 /// either peer application intentionally closes it, or when either transport layer detects an
+2373 /// error such as a time-out or certificate validation failure.
+2374 ///
+2375 /// A `ConnectionLost` event is emitted with details when the connection becomes closed.
+2376 pub fn is_closed(&self) -> bool {
+2377 self.state.is_closed()
+2378 }
+2379
+2380 /// Whether there is no longer any need to keep the connection around
+2381 ///
+2382 /// Closed connections become drained after a brief timeout to absorb any remaining in-flight
+2383 /// packets from the peer. All drained connections have been closed.
+2384 pub fn is_drained(&self) -> bool {
+2385 self.state.is_drained()
+2386 }
+2387
+2388 /// For clients, if the peer accepted the 0-RTT data packets
+2389 ///
+2390 /// The value is meaningless until after the handshake completes.
+2391 pub fn accepted_0rtt(&self) -> bool {
+2392 self.accepted_0rtt
+2393 }
+2394
+2395 /// Whether 0-RTT is/was possible during the handshake
+2396 pub fn has_0rtt(&self) -> bool {
+2397 self.zero_rtt_enabled
+2398 }
+2399
+2400 /// Whether there are any pending retransmits
+2401 pub fn has_pending_retransmits(&self) -> bool {
+2402 !self.spaces[SpaceId::Data].pending.is_empty(&self.streams)
+2403 }
+2404
+2405 /// Look up whether we're the client or server of this Connection
+2406 pub fn side(&self) -> Side {
+2407 self.side.side()
+2408 }
+2409
+2410 /// Get the address observed by the remote over the given path
+2411 pub fn path_observed_address(&self, path_id: PathId) -> Result<Option<SocketAddr>, ClosedPath> {
+2412 self.path(path_id)
+2413 .map(|path_data| {
+2414 path_data
+2415 .last_observed_addr_report
+2416 .as_ref()
+2417 .map(|observed| observed.socket_addr())
+2418 })
+2419 .ok_or(ClosedPath { _private: () })
+2420 }
+2421
+2422 /// Current best estimate of this connection's latency (round-trip-time)
+2423 pub fn rtt(&self, path_id: PathId) -> Option<Duration> {
+2424 self.path(path_id).map(|d| d.rtt.get())
+2425 }
+2426
+2427 /// Current state of this connection's congestion controller, for debugging purposes
+2428 pub fn congestion_state(&self, path_id: PathId) -> Option<&dyn Controller> {
+2429 self.path(path_id).map(|d| d.congestion.as_ref())
+2430 }
+2431
+2432 /// Modify the number of remotely initiated streams that may be concurrently open
+2433 ///
+2434 /// No streams may be opened by the peer unless fewer than `count` are already open. Large
+2435 /// `count`s increase both minimum and worst-case memory consumption.
+2436 pub fn set_max_concurrent_streams(&mut self, dir: Dir, count: VarInt) {
+2437 self.streams.set_max_concurrent(dir, count);
+2438 // If the limit was reduced, then a flow control update previously deemed insignificant may
+2439 // now be significant.
+2440 let pending = &mut self.spaces[SpaceId::Data].pending;
+2441 self.streams.queue_max_stream_id(pending);
+2442 }
+2443
+2444 /// Modify the number of open paths allowed when multipath is enabled
+2445 ///
+2446 /// When reducing the number of concurrent paths this will only affect delaying sending
+2447 /// new MAX_PATH_ID frames until fewer than this number of paths are possible. To
+2448 /// actively reduce paths they must be closed using [`Connection::close_path`], which
+2449 /// can also be used to close not-yet-opened paths.
+2450 ///
+2451 /// If multipath is not negotiated (see the [`TransportConfig`]) this can not enable
+2452 /// multipath and will fail.
+2453 pub fn set_max_concurrent_paths(
+2454 &mut self,
+2455 now: Instant,
+2456 count: NonZeroU32,
+2457 ) -> Result<(), MultipathNotNegotiated> {
+2458 if !self.is_multipath_negotiated() {
+2459 return Err(MultipathNotNegotiated { _private: () });
+2460 }
+2461 self.max_concurrent_paths = count;
+2462
+2463 let in_use_count = self
+2464 .local_max_path_id
+2465 .next()
+2466 .saturating_sub(self.abandoned_paths.len() as u32)
+2467 .as_u32();
+2468 let extra_needed = count.get().saturating_sub(in_use_count);
+2469 let new_max_path_id = self.local_max_path_id.saturating_add(extra_needed);
+2470
+2471 self.set_max_path_id(now, new_max_path_id);
+2472
+2473 Ok(())
+2474 }
+2475
+2476 /// If needed, issues a new MAX_PATH_ID frame and new CIDs for any newly allowed paths
+2477 fn set_max_path_id(&mut self, now: Instant, max_path_id: PathId) {
+2478 if max_path_id <= self.local_max_path_id {
+2479 return;
+2480 }
+2481
+2482 self.local_max_path_id = max_path_id;
+2483 self.spaces[SpaceId::Data].pending.max_path_id = true;
+2484
+2485 self.issue_first_path_cids(now);
+2486 }
+2487
+2488 /// Current number of remotely initiated streams that may be concurrently open
+2489 ///
+2490 /// If the target for this limit is reduced using [`set_max_concurrent_streams`](Self::set_max_concurrent_streams),
+2491 /// it will not change immediately, even if fewer streams are open. Instead, it will
+2492 /// decrement by one for each time a remotely initiated stream of matching directionality is closed.
+2493 pub fn max_concurrent_streams(&self, dir: Dir) -> u64 {
+2494 self.streams.max_concurrent(dir)
+2495 }
+2496
+2497 /// See [`TransportConfig::send_window()`]
+2498 pub fn set_send_window(&mut self, send_window: u64) {
+2499 self.streams.set_send_window(send_window);
+2500 }
+2501
+2502 /// See [`TransportConfig::receive_window()`]
+2503 pub fn set_receive_window(&mut self, receive_window: VarInt) {
+2504 if self.streams.set_receive_window(receive_window) {
+2505 self.spaces[SpaceId::Data].pending.max_data = true;
+2506 }
+2507 }
+2508
+2509 /// Whether the Multipath for QUIC extension is enabled.
+2510 ///
+2511 /// Multipath is only enabled after the handshake is completed and if it was enabled by both
+2512 /// peers.
+2513 pub fn is_multipath_negotiated(&self) -> bool {
+2514 !self.is_handshaking()
+2515 && self.config.max_concurrent_multipath_paths.is_some()
+2516 && self.peer_params.initial_max_path_id.is_some()
+2517 }
+2518
+2519 fn on_ack_received(
+2520 &mut self,
+2521 now: Instant,
+2522 space: SpaceId,
+2523 ack: frame::Ack,
+2524 ) -> Result<(), TransportError> {
+2525 // All ACKs are referencing path 0
+2526 let path = PathId::ZERO;
+2527 self.inner_on_ack_received(now, space, path, ack)
+2528 }
2529
-2530 // Must be called before crypto/pto_count are clobbered
-2531 self.detect_lost_packets(now, space, path, true);
-2532
-2533 if self.peer_completed_address_validation(path) {
-2534 self.path_data_mut(path).pto_count = 0;
-2535 }
-2536
-2537 // Explicit congestion notification
-2538 // TODO(@divma): this code is a good example of logic that should be contained in a single
-2539 // place but it's split between the path data and the packet number space data, we should
-2540 // find a way to make this work without two lookups
-2541 if self.path_data(path).sending_ecn {
-2542 if let Some(ecn) = ack.ecn {
-2543 // We only examine ECN counters from ACKs that we are certain we received in transmit
-2544 // order, allowing us to compute an increase in ECN counts to compare against the number
-2545 // of newly acked packets that remains well-defined in the presence of arbitrary packet
-2546 // reordering.
-2547 if new_largest {
-2548 let sent = self.spaces[space].for_path(path).largest_acked_packet_sent;
-2549 self.process_ecn(now, space, path, newly_acked.len() as u64, ecn, sent);
-2550 }
-2551 } else {
-2552 // We always start out sending ECN, so any ack that doesn't acknowledge it disables it.
-2553 debug!("ECN not acknowledged by peer");
-2554 self.path_data_mut(path).sending_ecn = false;
-2555 }
+2530 fn on_path_ack_received(
+2531 &mut self,
+2532 now: Instant,
+2533 space: SpaceId,
+2534 path_ack: frame::PathAck,
+2535 ) -> Result<(), TransportError> {
+2536 let (ack, path) = path_ack.into_ack();
+2537 self.inner_on_ack_received(now, space, path, ack)
+2538 }
+2539
+2540 /// Handles an ACK frame acknowledging packets sent on *path*.
+2541 fn inner_on_ack_received(
+2542 &mut self,
+2543 now: Instant,
+2544 space: SpaceId,
+2545 path: PathId,
+2546 ack: frame::Ack,
+2547 ) -> Result<(), TransportError> {
+2548 if self.abandoned_paths.contains(&path) {
+2549 // See also https://www.ietf.org/archive/id/draft-ietf-quic-multipath-17.html#section-3.4.3-3
+2550 // > PATH_ACK frames received with an abandoned path ID are silently ignored, as specified in Section 4.
+2551 trace!("silently ignoring PATH_ACK on abandoned path");
+2552 return Ok(());
+2553 }
+2554 if ack.largest >= self.spaces[space].for_path(path).next_packet_number {
+2555 return Err(TransportError::PROTOCOL_VIOLATION("unsent packet acked"));
2556 }
-2557
-2558 self.set_loss_detection_timer(now, path);
-2559 Ok(())
-2560 }
-2561
-2562 fn detect_spurious_loss(&mut self, ack: &frame::Ack, space: SpaceId, path: PathId) -> bool {
-2563 let lost_packets = &mut self.spaces[space].for_path(path).lost_packets;
-2564
-2565 if lost_packets.is_empty() {
-2566 return false;
-2567 }
-2568
-2569 for range in ack.iter() {
-2570 let spurious_losses: Vec<u64> = lost_packets
-2571 .iter_range(range.clone())
-2572 .map(|(pn, _info)| pn)
-2573 .collect();
-2574
-2575 for pn in spurious_losses {
-2576 lost_packets.remove(pn);
-2577 }
-2578 }
-2579
-2580 // If this ACK frame acknowledged all deemed lost packets,
-2581 // then we have raised a spurious congestion event in the past.
-2582 // We cannot conclude when there are remaining packets,
-2583 // but future ACK frames might indicate a spurious loss detection.
-2584 lost_packets.is_empty()
-2585 }
-2586
-2587 /// Drain lost packets that we reasonably think will never arrive
-2588 ///
-2589 /// The current criterion is copied from `msquic`:
-2590 /// discard packets that were sent earlier than 2 probe timeouts ago.
-2591 fn drain_lost_packets(&mut self, now: Instant, space: SpaceId, path: PathId) {
-2592 let two_pto = 2 * self.path_data(path).rtt.pto_base();
-2593
-2594 let lost_packets = &mut self.spaces[space].for_path(path).lost_packets;
-2595 lost_packets.retain(|_pn, info| now.saturating_duration_since(info.time_sent) <= two_pto);
-2596 }
-2597
-2598 /// Process a new ECN block from an in-order ACK
-2599 fn process_ecn(
-2600 &mut self,
-2601 now: Instant,
-2602 space: SpaceId,
-2603 path: PathId,
-2604 newly_acked: u64,
-2605 ecn: frame::EcnCounts,
-2606 largest_sent_time: Instant,
-2607 ) {
-2608 match self.spaces[space]
-2609 .for_path(path)
-2610 .detect_ecn(newly_acked, ecn)
-2611 {
-2612 Err(e) => {
-2613 debug!("halting ECN due to verification failure: {}", e);
-2614
-2615 self.path_data_mut(path).sending_ecn = false;
-2616 // Wipe out the existing value because it might be garbage and could interfere with
-2617 // future attempts to use ECN on new paths.
-2618 self.spaces[space].for_path(path).ecn_feedback = frame::EcnCounts::ZERO;
-2619 }
-2620 Ok(false) => {}
-2621 Ok(true) => {
-2622 self.path_stats.entry(path).or_default().congestion_events += 1;
-2623 self.path_data_mut(path).congestion.on_congestion_event(
-2624 now,
-2625 largest_sent_time,
-2626 false,
-2627 true,
-2628 0,
-2629 );
-2630 }
-2631 }
-2632 }
-2633
-2634 // Not timing-aware, so it's safe to call this for inferred acks, such as arise from
-2635 // high-latency handshakes
-2636 fn on_packet_acked(&mut self, now: Instant, path_id: PathId, info: SentPacket) {
-2637 self.paths
-2638 .get_mut(&path_id)
-2639 .expect("known path")
-2640 .remove_in_flight(&info);
-2641 let app_limited = self.app_limited;
-2642 let path = self.path_data_mut(path_id);
-2643 if info.ack_eliciting && !path.is_validating_path() {
-2644 // Only pass ACKs to the congestion controller if we are not validating the current
-2645 // path, so as to ignore any ACKs from older paths still coming in.
-2646 let rtt = path.rtt;
-2647 path.congestion
-2648 .on_ack(now, info.time_sent, info.size.into(), app_limited, &rtt);
-2649 }
-2650
-2651 // Update state for confirmed delivery of frames
-2652 if let Some(retransmits) = info.retransmits.get() {
-2653 for (id, _) in retransmits.reset_stream.iter() {
-2654 self.streams.reset_acked(*id);
+2557 let new_largest = {
+2558 let space = &mut self.spaces[space].for_path(path);
+2559 if space.largest_acked_packet.is_none_or(|pn| ack.largest > pn) {
+2560 space.largest_acked_packet = Some(ack.largest);
+2561 if let Some(info) = space.sent_packets.get(ack.largest) {
+2562 // This should always succeed, but a misbehaving peer might ACK a packet we
+2563 // haven't sent. At worst, that will result in us spuriously reducing the
+2564 // congestion window.
+2565 space.largest_acked_packet_sent = info.time_sent;
+2566 }
+2567 true
+2568 } else {
+2569 false
+2570 }
+2571 };
+2572
+2573 if self.detect_spurious_loss(&ack, space, path) {
+2574 self.path_data_mut(path)
+2575 .congestion
+2576 .on_spurious_congestion_event();
+2577 }
+2578
+2579 // Avoid DoS from unreasonably huge ack ranges by filtering out just the new acks.
+2580 let mut newly_acked = ArrayRangeSet::new();
+2581 for range in ack.iter() {
+2582 self.spaces[space].for_path(path).check_ack(range.clone())?;
+2583 for (pn, _) in self.spaces[space]
+2584 .for_path(path)
+2585 .sent_packets
+2586 .iter_range(range)
+2587 {
+2588 newly_acked.insert_one(pn);
+2589 }
+2590 }
+2591
+2592 if newly_acked.is_empty() {
+2593 return Ok(());
+2594 }
+2595
+2596 let mut ack_eliciting_acked = false;
+2597 for packet in newly_acked.elts() {
+2598 if let Some(info) = self.spaces[space].for_path(path).take(packet) {
+2599 for (acked_path_id, acked_pn) in info.largest_acked.iter() {
+2600 // Assume ACKs for all packets below the largest acknowledged in
+2601 // `packet` have been received. This can cause the peer to spuriously
+2602 // retransmit if some of our earlier ACKs were lost, but allows for
+2603 // simpler state tracking. See discussion at
+2604 // https://www.rfc-editor.org/rfc/rfc9000.html#name-limiting-ranges-by-tracking
+2605 if let Some(pns) = self.spaces[space].path_space_mut(*acked_path_id) {
+2606 pns.pending_acks.subtract_below(*acked_pn);
+2607 }
+2608 }
+2609 ack_eliciting_acked |= info.ack_eliciting;
+2610
+2611 // Notify MTU discovery that a packet was acked, because it might be an MTU probe
+2612 let path_data = self.path_data_mut(path);
+2613 let mtu_updated = path_data.mtud.on_acked(space, packet, info.size);
+2614 if mtu_updated {
+2615 path_data
+2616 .congestion
+2617 .on_mtu_update(path_data.mtud.current_mtu());
+2618 }
+2619
+2620 // Notify ack frequency that a packet was acked, because it might contain an ACK_FREQUENCY frame
+2621 self.ack_frequency.on_acked(path, packet);
+2622
+2623 self.on_packet_acked(now, path, info);
+2624 }
+2625 }
+2626
+2627 let largest_ackd = self.spaces[space].for_path(path).largest_acked_packet;
+2628 let app_limited = self.app_limited;
+2629 let path_data = self.path_data_mut(path);
+2630 let in_flight = path_data.in_flight.bytes;
+2631
+2632 path_data
+2633 .congestion
+2634 .on_end_acks(now, in_flight, app_limited, largest_ackd);
+2635
+2636 if new_largest && ack_eliciting_acked {
+2637 let ack_delay = if space != SpaceId::Data {
+2638 Duration::from_micros(0)
+2639 } else {
+2640 cmp::min(
+2641 self.ack_frequency.peer_max_ack_delay,
+2642 Duration::from_micros(ack.delay << self.peer_params.ack_delay_exponent.0),
+2643 )
+2644 };
+2645 let rtt = now.saturating_duration_since(
+2646 self.spaces[space].for_path(path).largest_acked_packet_sent,
+2647 );
+2648
+2649 let next_pn = self.spaces[space].for_path(path).next_packet_number;
+2650 let path_data = self.path_data_mut(path);
+2651 // TODO(@divma): should be a method of path, should be contained in a single place
+2652 path_data.rtt.update(ack_delay, rtt);
+2653 if path_data.first_packet_after_rtt_sample.is_none() {
+2654 path_data.first_packet_after_rtt_sample = Some((space, next_pn));
2655 }
2656 }
2657
-2658 for frame in info.stream_frames {
-2659 self.streams.received_ack_of(frame);
-2660 }
-2661 }
-2662
-2663 fn set_key_discard_timer(&mut self, now: Instant, space: SpaceId) {
-2664 let start = if self.zero_rtt_crypto.is_some() {
-2665 now
-2666 } else {
-2667 self.prev_crypto
-2668 .as_ref()
-2669 .expect("no previous keys")
-2670 .end_packet
-2671 .as_ref()
-2672 .expect("update not acknowledged yet")
-2673 .1
-2674 };
-2675
-2676 // QUIC-MULTIPATH § 2.5 Key Phase Update Process: use largest PTO off all paths.
-2677 self.timers.set(
-2678 Timer::Conn(ConnTimer::KeyDiscard),
-2679 start + self.pto_max_path(space, false) * 3,
-2680 self.qlog.with_time(now),
-2681 );
-2682 }
-2683
-2684 /// Handle a [`PathTimer::LossDetection`] timeout.
-2685 ///
-2686 /// This timer expires for two reasons:
-2687 /// - An ACK-eliciting packet we sent should be considered lost.
-2688 /// - The PTO may have expired and a tail-loss probe needs to be scheduled.
-2689 ///
-2690 /// The former needs us to schedule re-transmission of the lost data.
-2691 ///
-2692 /// The latter means we have not received an ACK for an ack-eliciting packet we sent
-2693 /// within the PTO time-window. We need to schedule a tail-loss probe, an ack-eliciting
-2694 /// packet, to try and elicit new acknowledgements. These new acknowledgements will
-2695 /// indicate whether the previously sent packets were lost or not.
-2696 fn on_loss_detection_timeout(&mut self, now: Instant, path_id: PathId) {
-2697 if let Some((_, pn_space)) = self.loss_time_and_space(path_id) {
-2698 // Time threshold loss Detection
-2699 self.detect_lost_packets(now, pn_space, path_id, false);
-2700 self.set_loss_detection_timer(now, path_id);
-2701 return;
-2702 }
-2703
-2704 let (_, space) = match self.pto_time_and_space(now, path_id) {
-2705 Some(x) => x,
-2706 None => {
-2707 error!(%path_id, "PTO expired while unset");
-2708 return;
-2709 }
-2710 };
-2711 trace!(
-2712 in_flight = self.path_data(path_id).in_flight.bytes,
-2713 count = self.path_data(path_id).pto_count,
-2714 ?space,
-2715 %path_id,
-2716 "PTO fired"
-2717 );
-2718
-2719 let count = match self.path_data(path_id).in_flight.ack_eliciting {
-2720 // A PTO when we're not expecting any ACKs must be due to handshake anti-amplification
-2721 // deadlock preventions
-2722 0 => {
-2723 debug_assert!(!self.peer_completed_address_validation(path_id));
-2724 1
-2725 }
-2726 // Conventional loss probe
-2727 _ => 2,
-2728 };
-2729 let pns = self.spaces[space].for_path(path_id);
-2730 pns.loss_probes = pns.loss_probes.saturating_add(count);
-2731 let path_data = self.path_data_mut(path_id);
-2732 path_data.pto_count = path_data.pto_count.saturating_add(1);
-2733 self.set_loss_detection_timer(now, path_id);
-2734 }
-2735
-2736 /// Detect any lost packets
-2737 ///
-2738 /// There are two cases in which we detects lost packets:
-2739 ///
-2740 /// - We received an ACK packet.
-2741 /// - The [`PathTimer::LossDetection`] timer expired. So there is an un-acknowledged packet
-2742 /// that was followed by an acknowledged packet. The loss timer for this
-2743 /// un-acknowledged packet expired and we need to detect that packet as lost.
-2744 ///
-2745 /// Packets are lost if they are both (See RFC9002 §6.1):
-2746 ///
-2747 /// - Unacknowledged, in flight and sent prior to an acknowledged packet.
-2748 /// - Old enough by either:
-2749 /// - Having a packet number [`TransportConfig::packet_threshold`] lower then the last
-2750 /// acknowledged packet.
-2751 /// - Being sent [`TransportConfig::time_threshold`] * RTT in the past.
-2752 fn detect_lost_packets(
-2753 &mut self,
-2754 now: Instant,
-2755 pn_space: SpaceId,
-2756 path_id: PathId,
-2757 due_to_ack: bool,
-2758 ) {
-2759 let mut lost_packets = Vec::<u64>::new();
-2760 let mut lost_mtu_probe = None;
-2761 let mut in_persistent_congestion = false;
-2762 let mut size_of_lost_packets = 0u64;
-2763 self.spaces[pn_space].for_path(path_id).loss_time = None;
-2764
-2765 // Find all the lost packets, populating all variables initialised above.
-2766
-2767 let path = self.path_data(path_id);
-2768 let in_flight_mtu_probe = path.mtud.in_flight_mtu_probe();
-2769 let loss_delay = path
-2770 .rtt
-2771 .conservative()
-2772 .mul_f32(self.config.time_threshold)
-2773 .max(TIMER_GRANULARITY);
-2774 let first_packet_after_rtt_sample = path.first_packet_after_rtt_sample;
-2775
-2776 let largest_acked_packet = self.spaces[pn_space]
-2777 .for_path(path_id)
-2778 .largest_acked_packet
-2779 .expect("detect_lost_packets only to be called if path received at least one ACK");
-2780 let packet_threshold = self.config.packet_threshold as u64;
-2781
-2782 // InPersistentCongestion: Determine if all packets in the time period before the newest
-2783 // lost packet, including the edges, are marked lost. PTO computation must always
-2784 // include max ACK delay, i.e. operate as if in Data space (see RFC9001 §7.6.1).
-2785 let congestion_period = self
-2786 .pto(SpaceId::Data, path_id)
-2787 .saturating_mul(self.config.persistent_congestion_threshold);
-2788 let mut persistent_congestion_start: Option<Instant> = None;
-2789 let mut prev_packet = None;
-2790 let space = self.spaces[pn_space].for_path(path_id);
-2791
-2792 for (packet, info) in space.sent_packets.iter_range(0..largest_acked_packet) {
-2793 if prev_packet != Some(packet.wrapping_sub(1)) {
-2794 // An intervening packet was acknowledged
-2795 persistent_congestion_start = None;
-2796 }
-2797
-2798 // Packets sent before now - loss_delay are deemed lost.
-2799 // However, we avoid subtraction as it can panic and there's no
-2800 // saturating equivalent of this subtraction operation with a Duration.
-2801 let packet_too_old = now.saturating_duration_since(info.time_sent) >= loss_delay;
-2802 if packet_too_old || largest_acked_packet >= packet + packet_threshold {
-2803 // The packet should be declared lost.
-2804 if Some(packet) == in_flight_mtu_probe {
-2805 // Lost MTU probes are not included in `lost_packets`, because they
-2806 // should not trigger a congestion control response
-2807 lost_mtu_probe = in_flight_mtu_probe;
-2808 } else {
-2809 lost_packets.push(packet);
-2810 size_of_lost_packets += info.size as u64;
-2811 if info.ack_eliciting && due_to_ack {
-2812 match persistent_congestion_start {
-2813 // Two ACK-eliciting packets lost more than
-2814 // congestion_period apart, with no ACKed packets in between
-2815 Some(start) if info.time_sent - start > congestion_period => {
-2816 in_persistent_congestion = true;
-2817 }
-2818 // Persistent congestion must start after the first RTT sample
-2819 None if first_packet_after_rtt_sample
-2820 .is_some_and(|x| x < (pn_space, packet)) =>
-2821 {
-2822 persistent_congestion_start = Some(info.time_sent);
-2823 }
-2824 _ => {}
-2825 }
-2826 }
-2827 }
-2828 } else {
-2829 // The packet should not yet be declared lost.
-2830 if space.loss_time.is_none() {
-2831 // Since we iterate in order the lowest packet number's loss time will
-2832 // always be the earliest.
-2833 space.loss_time = Some(info.time_sent + loss_delay);
-2834 }
-2835 persistent_congestion_start = None;
-2836 }
-2837
-2838 prev_packet = Some(packet);
-2839 }
-2840
-2841 self.handle_lost_packets(
-2842 pn_space,
-2843 path_id,
-2844 now,
-2845 lost_packets,
-2846 lost_mtu_probe,
-2847 loss_delay,
-2848 in_persistent_congestion,
-2849 size_of_lost_packets,
-2850 );
-2851 }
-2852
-2853 /// Drops the path state, declaring any remaining in-flight packets as lost
-2854 fn discard_path(&mut self, path_id: PathId, now: Instant) {
-2855 trace!(%path_id, "dropping path state");
-2856 let path = self.path_data(path_id);
-2857 let in_flight_mtu_probe = path.mtud.in_flight_mtu_probe();
-2858
-2859 let mut size_of_lost_packets = 0u64; // add to path_stats.lost_bytes;
-2860 let lost_pns: Vec<_> = self.spaces[SpaceId::Data]
-2861 .for_path(path_id)
-2862 .sent_packets
-2863 .iter()
-2864 .filter(|(pn, _info)| Some(*pn) != in_flight_mtu_probe)
-2865 .map(|(pn, info)| {
-2866 size_of_lost_packets += info.size as u64;
-2867 pn
-2868 })
-2869 .collect();
-2870
-2871 if !lost_pns.is_empty() {
-2872 trace!(
-2873 %path_id,
-2874 count = lost_pns.len(),
-2875 lost_bytes = size_of_lost_packets,
-2876 "packets lost on path abandon"
-2877 );
-2878 self.handle_lost_packets(
-2879 SpaceId::Data,
-2880 path_id,
-2881 now,
-2882 lost_pns,
-2883 in_flight_mtu_probe,
-2884 Duration::ZERO,
-2885 false,
-2886 size_of_lost_packets,
-2887 );
-2888 }
-2889 self.paths.remove(&path_id);
-2890 self.spaces[SpaceId::Data].number_spaces.remove(&path_id);
-2891
-2892 let path_stats = self.path_stats.remove(&path_id).unwrap_or_default();
-2893 self.events.push_back(
-2894 PathEvent::Abandoned {
-2895 id: path_id,
-2896 path_stats,
-2897 }
-2898 .into(),
-2899 );
-2900 }
-2901
-2902 fn handle_lost_packets(
-2903 &mut self,
-2904 pn_space: SpaceId,
-2905 path_id: PathId,
-2906 now: Instant,
-2907 lost_packets: Vec<u64>,
-2908 lost_mtu_probe: Option<u64>,
-2909 loss_delay: Duration,
-2910 in_persistent_congestion: bool,
-2911 size_of_lost_packets: u64,
-2912 ) {
-2913 debug_assert!(
-2914 {
-2915 let mut sorted = lost_packets.clone();
-2916 sorted.sort();
-2917 sorted == lost_packets
-2918 },
-2919 "lost_packets must be sorted"
-2920 );
-2921
-2922 self.drain_lost_packets(now, pn_space, path_id);
-2923
-2924 // OnPacketsLost
-2925 if let Some(largest_lost) = lost_packets.last().cloned() {
-2926 let old_bytes_in_flight = self.path_data_mut(path_id).in_flight.bytes;
-2927 let largest_lost_sent = self.spaces[pn_space]
-2928 .for_path(path_id)
-2929 .sent_packets
-2930 .get(largest_lost)
-2931 .unwrap()
-2932 .time_sent;
-2933 let path_stats = self.path_stats.entry(path_id).or_default();
-2934 path_stats.lost_packets += lost_packets.len() as u64;
-2935 path_stats.lost_bytes += size_of_lost_packets;
-2936 trace!(
-2937 %path_id,
-2938 count = lost_packets.len(),
-2939 lost_bytes = size_of_lost_packets,
-2940 "packets lost",
-2941 );
-2942
-2943 for &packet in &lost_packets {
-2944 let Some(info) = self.spaces[pn_space].for_path(path_id).take(packet) else {
-2945 continue;
-2946 };
-2947 self.qlog
-2948 .emit_packet_lost(packet, &info, loss_delay, pn_space, now);
-2949 self.paths
-2950 .get_mut(&path_id)
-2951 .unwrap()
-2952 .remove_in_flight(&info);
-2953
-2954 for frame in info.stream_frames {
-2955 self.streams.retransmit(frame);
-2956 }
-2957 self.spaces[pn_space].pending |= info.retransmits;
-2958 self.path_data_mut(path_id)
-2959 .mtud
-2960 .on_non_probe_lost(packet, info.size);
-2961
-2962 self.spaces[pn_space].for_path(path_id).lost_packets.insert(
-2963 packet,
-2964 LostPacket {
-2965 time_sent: info.time_sent,
-2966 },
-2967 );
-2968 }
-2969
-2970 let path = self.path_data_mut(path_id);
-2971 if path.mtud.black_hole_detected(now) {
-2972 path.congestion.on_mtu_update(path.mtud.current_mtu());
-2973 if let Some(max_datagram_size) = self.datagrams().max_size() {
-2974 if self.datagrams.drop_oversized(max_datagram_size)
-2975 && self.datagrams.send_blocked
-2976 {
-2977 self.datagrams.send_blocked = false;
-2978 self.events.push_back(Event::DatagramsUnblocked);
-2979 }
-2980 }
-2981 self.path_stats
-2982 .entry(path_id)
-2983 .or_default()
-2984 .black_holes_detected += 1;
-2985 }
+2658 // Must be called before crypto/pto_count are clobbered
+2659 self.detect_lost_packets(now, space, path, true);
+2660
+2661 if self.peer_completed_address_validation(path) {
+2662 self.path_data_mut(path).pto_count = 0;
+2663 }
+2664
+2665 // Explicit congestion notification
+2666 // TODO(@divma): this code is a good example of logic that should be contained in a single
+2667 // place but it's split between the path data and the packet number space data, we should
+2668 // find a way to make this work without two lookups
+2669 if self.path_data(path).sending_ecn {
+2670 if let Some(ecn) = ack.ecn {
+2671 // We only examine ECN counters from ACKs that we are certain we received in transmit
+2672 // order, allowing us to compute an increase in ECN counts to compare against the number
+2673 // of newly acked packets that remains well-defined in the presence of arbitrary packet
+2674 // reordering.
+2675 if new_largest {
+2676 let sent = self.spaces[space].for_path(path).largest_acked_packet_sent;
+2677 self.process_ecn(now, space, path, newly_acked.len() as u64, ecn, sent);
+2678 }
+2679 } else {
+2680 // We always start out sending ECN, so any ack that doesn't acknowledge it disables it.
+2681 debug!("ECN not acknowledged by peer");
+2682 self.path_data_mut(path).sending_ecn = false;
+2683 }
+2684 }
+2685
+2686 self.set_loss_detection_timer(now, path);
+2687 Ok(())
+2688 }
+2689
+2690 fn detect_spurious_loss(&mut self, ack: &frame::Ack, space: SpaceId, path: PathId) -> bool {
+2691 let lost_packets = &mut self.spaces[space].for_path(path).lost_packets;
+2692
+2693 if lost_packets.is_empty() {
+2694 return false;
+2695 }
+2696
+2697 for range in ack.iter() {
+2698 let spurious_losses: Vec<u64> = lost_packets
+2699 .iter_range(range.clone())
+2700 .map(|(pn, _info)| pn)
+2701 .collect();
+2702
+2703 for pn in spurious_losses {
+2704 lost_packets.remove(pn);
+2705 }
+2706 }
+2707
+2708 // If this ACK frame acknowledged all deemed lost packets,
+2709 // then we have raised a spurious congestion event in the past.
+2710 // We cannot conclude when there are remaining packets,
+2711 // but future ACK frames might indicate a spurious loss detection.
+2712 lost_packets.is_empty()
+2713 }
+2714
+2715 /// Drain lost packets that we reasonably think will never arrive
+2716 ///
+2717 /// The current criterion is copied from `msquic`:
+2718 /// discard packets that were sent earlier than 2 probe timeouts ago.
+2719 fn drain_lost_packets(&mut self, now: Instant, space: SpaceId, path: PathId) {
+2720 let two_pto = 2 * self.path_data(path).rtt.pto_base();
+2721
+2722 let lost_packets = &mut self.spaces[space].for_path(path).lost_packets;
+2723 lost_packets.retain(|_pn, info| now.saturating_duration_since(info.time_sent) <= two_pto);
+2724 }
+2725
+2726 /// Process a new ECN block from an in-order ACK
+2727 fn process_ecn(
+2728 &mut self,
+2729 now: Instant,
+2730 space: SpaceId,
+2731 path: PathId,
+2732 newly_acked: u64,
+2733 ecn: frame::EcnCounts,
+2734 largest_sent_time: Instant,
+2735 ) {
+2736 match self.spaces[space]
+2737 .for_path(path)
+2738 .detect_ecn(newly_acked, ecn)
+2739 {
+2740 Err(e) => {
+2741 debug!("halting ECN due to verification failure: {}", e);
+2742
+2743 self.path_data_mut(path).sending_ecn = false;
+2744 // Wipe out the existing value because it might be garbage and could interfere with
+2745 // future attempts to use ECN on new paths.
+2746 self.spaces[space].for_path(path).ecn_feedback = frame::EcnCounts::ZERO;
+2747 }
+2748 Ok(false) => {}
+2749 Ok(true) => {
+2750 self.path_stats.entry(path).or_default().congestion_events += 1;
+2751 self.path_data_mut(path).congestion.on_congestion_event(
+2752 now,
+2753 largest_sent_time,
+2754 false,
+2755 true,
+2756 0,
+2757 );
+2758 }
+2759 }
+2760 }
+2761
+2762 // Not timing-aware, so it's safe to call this for inferred acks, such as arise from
+2763 // high-latency handshakes
+2764 fn on_packet_acked(&mut self, now: Instant, path_id: PathId, info: SentPacket) {
+2765 self.paths
+2766 .get_mut(&path_id)
+2767 .expect("known path")
+2768 .remove_in_flight(&info);
+2769 let app_limited = self.app_limited;
+2770 let path = self.path_data_mut(path_id);
+2771 if info.ack_eliciting && !path.is_validating_path() {
+2772 // Only pass ACKs to the congestion controller if we are not validating the current
+2773 // path, so as to ignore any ACKs from older paths still coming in.
+2774 let rtt = path.rtt;
+2775 path.congestion
+2776 .on_ack(now, info.time_sent, info.size.into(), app_limited, &rtt);
+2777 }
+2778
+2779 // Update state for confirmed delivery of frames
+2780 if let Some(retransmits) = info.retransmits.get() {
+2781 for (id, _) in retransmits.reset_stream.iter() {
+2782 self.streams.reset_acked(*id);
+2783 }
+2784 }
+2785
+2786 for frame in info.stream_frames {
+2787 self.streams.received_ack_of(frame);
+2788 }
+2789 }
+2790
+2791 fn set_key_discard_timer(&mut self, now: Instant, space: SpaceId) {
+2792 let start = if self.zero_rtt_crypto.is_some() {
+2793 now
+2794 } else {
+2795 self.prev_crypto
+2796 .as_ref()
+2797 .expect("no previous keys")
+2798 .end_packet
+2799 .as_ref()
+2800 .expect("update not acknowledged yet")
+2801 .1
+2802 };
+2803
+2804 // QUIC-MULTIPATH § 2.5 Key Phase Update Process: use largest PTO off all paths.
+2805 self.timers.set(
+2806 Timer::Conn(ConnTimer::KeyDiscard),
+2807 start + self.pto_max_path(space, false) * 3,
+2808 self.qlog.with_time(now),
+2809 );
+2810 }
+2811
+2812 /// Handle a [`PathTimer::LossDetection`] timeout.
+2813 ///
+2814 /// This timer expires for two reasons:
+2815 /// - An ACK-eliciting packet we sent should be considered lost.
+2816 /// - The PTO may have expired and a tail-loss probe needs to be scheduled.
+2817 ///
+2818 /// The former needs us to schedule re-transmission of the lost data.
+2819 ///
+2820 /// The latter means we have not received an ACK for an ack-eliciting packet we sent
+2821 /// within the PTO time-window. We need to schedule a tail-loss probe, an ack-eliciting
+2822 /// packet, to try and elicit new acknowledgements. These new acknowledgements will
+2823 /// indicate whether the previously sent packets were lost or not.
+2824 fn on_loss_detection_timeout(&mut self, now: Instant, path_id: PathId) {
+2825 if let Some((_, pn_space)) = self.loss_time_and_space(path_id) {
+2826 // Time threshold loss Detection
+2827 self.detect_lost_packets(now, pn_space, path_id, false);
+2828 self.set_loss_detection_timer(now, path_id);
+2829 return;
+2830 }
+2831
+2832 let (_, space) = match self.pto_time_and_space(now, path_id) {
+2833 Some(x) => x,
+2834 None => {
+2835 error!(%path_id, "PTO expired while unset");
+2836 return;
+2837 }
+2838 };
+2839 trace!(
+2840 in_flight = self.path_data(path_id).in_flight.bytes,
+2841 count = self.path_data(path_id).pto_count,
+2842 ?space,
+2843 %path_id,
+2844 "PTO fired"
+2845 );
+2846
+2847 let count = match self.path_data(path_id).in_flight.ack_eliciting {
+2848 // A PTO when we're not expecting any ACKs must be due to handshake anti-amplification
+2849 // deadlock preventions
+2850 0 => {
+2851 debug_assert!(!self.peer_completed_address_validation(path_id));
+2852 1
+2853 }
+2854 // Conventional loss probe
+2855 _ => 2,
+2856 };
+2857 let pns = self.spaces[space].for_path(path_id);
+2858 pns.loss_probes = pns.loss_probes.saturating_add(count);
+2859 let path_data = self.path_data_mut(path_id);
+2860 path_data.pto_count = path_data.pto_count.saturating_add(1);
+2861 self.set_loss_detection_timer(now, path_id);
+2862 }
+2863
+2864 /// Detect any lost packets
+2865 ///
+2866 /// There are two cases in which we detects lost packets:
+2867 ///
+2868 /// - We received an ACK packet.
+2869 /// - The [`PathTimer::LossDetection`] timer expired. So there is an un-acknowledged packet
+2870 /// that was followed by an acknowledged packet. The loss timer for this
+2871 /// un-acknowledged packet expired and we need to detect that packet as lost.
+2872 ///
+2873 /// Packets are lost if they are both (See RFC9002 §6.1):
+2874 ///
+2875 /// - Unacknowledged, in flight and sent prior to an acknowledged packet.
+2876 /// - Old enough by either:
+2877 /// - Having a packet number [`TransportConfig::packet_threshold`] lower then the last
+2878 /// acknowledged packet.
+2879 /// - Being sent [`TransportConfig::time_threshold`] * RTT in the past.
+2880 fn detect_lost_packets(
+2881 &mut self,
+2882 now: Instant,
+2883 pn_space: SpaceId,
+2884 path_id: PathId,
+2885 due_to_ack: bool,
+2886 ) {
+2887 let mut lost_packets = Vec::<u64>::new();
+2888 let mut lost_mtu_probe = None;
+2889 let mut in_persistent_congestion = false;
+2890 let mut size_of_lost_packets = 0u64;
+2891 self.spaces[pn_space].for_path(path_id).loss_time = None;
+2892
+2893 // Find all the lost packets, populating all variables initialised above.
+2894
+2895 let path = self.path_data(path_id);
+2896 let in_flight_mtu_probe = path.mtud.in_flight_mtu_probe();
+2897 let loss_delay = path
+2898 .rtt
+2899 .conservative()
+2900 .mul_f32(self.config.time_threshold)
+2901 .max(TIMER_GRANULARITY);
+2902 let first_packet_after_rtt_sample = path.first_packet_after_rtt_sample;
+2903
+2904 let largest_acked_packet = self.spaces[pn_space]
+2905 .for_path(path_id)
+2906 .largest_acked_packet
+2907 .expect("detect_lost_packets only to be called if path received at least one ACK");
+2908 let packet_threshold = self.config.packet_threshold as u64;
+2909
+2910 // InPersistentCongestion: Determine if all packets in the time period before the newest
+2911 // lost packet, including the edges, are marked lost. PTO computation must always
+2912 // include max ACK delay, i.e. operate as if in Data space (see RFC9001 §7.6.1).
+2913 let congestion_period = self
+2914 .pto(SpaceId::Data, path_id)
+2915 .saturating_mul(self.config.persistent_congestion_threshold);
+2916 let mut persistent_congestion_start: Option<Instant> = None;
+2917 let mut prev_packet = None;
+2918 let space = self.spaces[pn_space].for_path(path_id);
+2919
+2920 for (packet, info) in space.sent_packets.iter_range(0..largest_acked_packet) {
+2921 if prev_packet != Some(packet.wrapping_sub(1)) {
+2922 // An intervening packet was acknowledged
+2923 persistent_congestion_start = None;
+2924 }
+2925
+2926 // Packets sent before now - loss_delay are deemed lost.
+2927 // However, we avoid subtraction as it can panic and there's no
+2928 // saturating equivalent of this subtraction operation with a Duration.
+2929 let packet_too_old = now.saturating_duration_since(info.time_sent) >= loss_delay;
+2930 if packet_too_old || largest_acked_packet >= packet + packet_threshold {
+2931 // The packet should be declared lost.
+2932 if Some(packet) == in_flight_mtu_probe {
+2933 // Lost MTU probes are not included in `lost_packets`, because they
+2934 // should not trigger a congestion control response
+2935 lost_mtu_probe = in_flight_mtu_probe;
+2936 } else {
+2937 lost_packets.push(packet);
+2938 size_of_lost_packets += info.size as u64;
+2939 if info.ack_eliciting && due_to_ack {
+2940 match persistent_congestion_start {
+2941 // Two ACK-eliciting packets lost more than
+2942 // congestion_period apart, with no ACKed packets in between
+2943 Some(start) if info.time_sent - start > congestion_period => {
+2944 in_persistent_congestion = true;
+2945 }
+2946 // Persistent congestion must start after the first RTT sample
+2947 None if first_packet_after_rtt_sample
+2948 .is_some_and(|x| x < (pn_space, packet)) =>
+2949 {
+2950 persistent_congestion_start = Some(info.time_sent);
+2951 }
+2952 _ => {}
+2953 }
+2954 }
+2955 }
+2956 } else {
+2957 // The packet should not yet be declared lost.
+2958 if space.loss_time.is_none() {
+2959 // Since we iterate in order the lowest packet number's loss time will
+2960 // always be the earliest.
+2961 space.loss_time = Some(info.time_sent + loss_delay);
+2962 }
+2963 persistent_congestion_start = None;
+2964 }
+2965
+2966 prev_packet = Some(packet);
+2967 }
+2968
+2969 self.handle_lost_packets(
+2970 pn_space,
+2971 path_id,
+2972 now,
+2973 lost_packets,
+2974 lost_mtu_probe,
+2975 loss_delay,
+2976 in_persistent_congestion,
+2977 size_of_lost_packets,
+2978 );
+2979 }
+2980
+2981 /// Drops the path state, declaring any remaining in-flight packets as lost
+2982 fn discard_path(&mut self, path_id: PathId, now: Instant) {
+2983 trace!(%path_id, "dropping path state");
+2984 let path = self.path_data(path_id);
+2985 let in_flight_mtu_probe = path.mtud.in_flight_mtu_probe();
2986
-2987 // Don't apply congestion penalty for lost ack-only packets
-2988 let lost_ack_eliciting =
-2989 old_bytes_in_flight != self.path_data_mut(path_id).in_flight.bytes;
-2990
-2991 if lost_ack_eliciting {
-2992 self.path_stats
-2993 .entry(path_id)
-2994 .or_default()
-2995 .congestion_events += 1;
-2996 self.path_data_mut(path_id).congestion.on_congestion_event(
-2997 now,
-2998 largest_lost_sent,
-2999 in_persistent_congestion,
-3000 false,
-3001 size_of_lost_packets,
-3002 );
-3003 }
-3004 }
-3005
-3006 // Handle a lost MTU probe
-3007 if let Some(packet) = lost_mtu_probe {
-3008 let info = self.spaces[SpaceId::Data]
-3009 .for_path(path_id)
-3010 .take(packet)
-3011 .unwrap(); // safe: lost_mtu_probe is omitted from lost_packets, and
-3012 // therefore must not have been removed yet
-3013 self.paths
-3014 .get_mut(&path_id)
-3015 .unwrap()
-3016 .remove_in_flight(&info);
-3017 self.path_data_mut(path_id).mtud.on_probe_lost();
-3018 self.path_stats
-3019 .entry(path_id)
-3020 .or_default()
-3021 .lost_plpmtud_probes += 1;
-3022 }
-3023 }
-3024
-3025 /// Returns the earliest time packets should be declared lost for all spaces on a path.
-3026 ///
-3027 /// If a path has an acknowledged packet with any prior un-acknowledged packets, the
-3028 /// earliest un-acknowledged packet can be declared lost after a timeout has elapsed.
-3029 /// The time returned is when this packet should be declared lost.
-3030 fn loss_time_and_space(&self, path_id: PathId) -> Option<(Instant, SpaceId)> {
-3031 SpaceId::iter()
-3032 .filter_map(|id| {
-3033 self.spaces[id]
-3034 .number_spaces
-3035 .get(&path_id)
-3036 .and_then(|pns| pns.loss_time)
-3037 .map(|time| (time, id))
-3038 })
-3039 .min_by_key(|&(time, _)| time)
-3040 }
-3041
-3042 /// Returns the earliest next PTO should fire for all spaces on a path.
-3043 fn pto_time_and_space(&mut self, now: Instant, path_id: PathId) -> Option<(Instant, SpaceId)> {
-3044 let path = self.path(path_id)?;
-3045 let pto_count = path.pto_count;
-3046 let backoff = 2u32.pow(pto_count.min(MAX_BACKOFF_EXPONENT));
-3047 let mut duration = path.rtt.pto_base() * backoff;
-3048
-3049 if path_id == PathId::ZERO
-3050 && path.in_flight.ack_eliciting == 0
-3051 && !self.peer_completed_address_validation(PathId::ZERO)
-3052 {
-3053 // Address Validation during Connection Establishment:
-3054 // https://www.rfc-editor.org/rfc/rfc9000.html#section-8.1. To prevent a
-3055 // deadlock if an Initial or Handshake packet from the server is lost and the
-3056 // server can not send more due to its anti-amplification limit the client must
-3057 // send another packet on PTO.
-3058 let space = match self.highest_space {
-3059 SpaceId::Handshake => SpaceId::Handshake,
-3060 _ => SpaceId::Initial,
-3061 };
-3062
-3063 return Some((now + duration, space));
-3064 }
-3065
-3066 let mut result = None;
-3067 for space in SpaceId::iter() {
-3068 let Some(pns) = self.spaces[space].number_spaces.get(&path_id) else {
-3069 continue;
-3070 };
-3071
-3072 if !pns.has_in_flight() {
-3073 continue;
-3074 }
-3075 if space == SpaceId::Data {
-3076 // Skip ApplicationData until handshake completes.
-3077 if self.is_handshaking() {
-3078 return result;
-3079 }
-3080 // Include max_ack_delay and backoff for ApplicationData.
-3081 duration += self.ack_frequency.max_ack_delay_for_pto() * backoff;
-3082 }
-3083 let Some(last_ack_eliciting) = pns.time_of_last_ack_eliciting_packet else {
-3084 continue;
-3085 };
-3086 let pto = last_ack_eliciting + duration;
-3087 if result.is_none_or(|(earliest_pto, _)| pto < earliest_pto) {
-3088 if path.anti_amplification_blocked(1) {
-3089 // Nothing would be able to be sent.
-3090 continue;
-3091 }
-3092 if path.in_flight.ack_eliciting == 0 {
-3093 // Nothing ack-eliciting, no PTO to arm/fire.
-3094 continue;
-3095 }
-3096 result = Some((pto, space));
-3097 }
-3098 }
-3099 result
-3100 }
-3101
-3102 fn peer_completed_address_validation(&self, path: PathId) -> bool {
-3103 // TODO(flub): This logic needs updating for multipath
-3104 if self.side.is_server() || self.state.is_closed() {
-3105 return true;
-3106 }
-3107 // The server is guaranteed to have validated our address if any of our handshake or 1-RTT
-3108 // packets are acknowledged or we've seen HANDSHAKE_DONE and discarded handshake keys.
-3109 self.spaces[SpaceId::Handshake]
-3110 .path_space(PathId::ZERO)
-3111 .and_then(|pns| pns.largest_acked_packet)
-3112 .is_some()
-3113 || self.spaces[SpaceId::Data]
-3114 .path_space(path)
-3115 .and_then(|pns| pns.largest_acked_packet)
-3116 .is_some()
-3117 || (self.spaces[SpaceId::Data].crypto.is_some()
-3118 && self.spaces[SpaceId::Handshake].crypto.is_none())
-3119 }
-3120
-3121 /// Resets the the [`PathTimer::LossDetection`] timer to the next instant it may be needed
-3122 ///
-3123 /// The timer must fire if either:
-3124 /// - An ack-eliciting packet we sent needs to be declared lost.
-3125 /// - A tail-loss probe needs to be sent.
-3126 ///
-3127 /// See [`Connection::on_loss_detection_timeout`] for details.
-3128 fn set_loss_detection_timer(&mut self, now: Instant, path_id: PathId) {
-3129 if self.state.is_closed() {
-3130 // No loss detection takes place on closed connections, and `close_common` already
-3131 // stopped time timer. Ensure we don't restart it inadvertently, e.g. in response to a
-3132 // reordered packet being handled by state-insensitive code.
-3133 return;
-3134 }
-3135
-3136 if let Some((loss_time, _)) = self.loss_time_and_space(path_id) {
-3137 // Time threshold loss detection.
-3138 self.timers.set(
-3139 Timer::PerPath(path_id, PathTimer::LossDetection),
-3140 loss_time,
-3141 self.qlog.with_time(now),
-3142 );
-3143 return;
-3144 }
-3145
-3146 // Determine which PN space to arm PTO for.
-3147 // Calculate PTO duration
-3148 if let Some((timeout, _)) = self.pto_time_and_space(now, path_id) {
-3149 self.timers.set(
-3150 Timer::PerPath(path_id, PathTimer::LossDetection),
-3151 timeout,
-3152 self.qlog.with_time(now),
-3153 );
-3154 } else {
-3155 self.timers.stop(
-3156 Timer::PerPath(path_id, PathTimer::LossDetection),
-3157 self.qlog.with_time(now),
-3158 );
-3159 }
-3160 }
-3161
-3162 /// The maximum probe timeout across all paths
-3163 ///
-3164 /// If `is_closing` is set to `true` it will filter out paths that have not yet been used.
-3165 ///
-3166 /// See [`Connection::pto`]
-3167 fn pto_max_path(&self, space: SpaceId, is_closing: bool) -> Duration {
-3168 match space {
-3169 SpaceId::Initial | SpaceId::Handshake => self.pto(space, PathId::ZERO),
-3170 SpaceId::Data => self
-3171 .paths
-3172 .iter()
-3173 .filter_map(|(path_id, state)| {
-3174 if is_closing && state.data.total_sent == 0 && state.data.total_recvd == 0 {
-3175 // If we are closing and haven't sent anything yet, do not include
-3176 None
-3177 } else {
-3178 let pto = self.pto(space, *path_id);
-3179 Some(pto)
-3180 }
-3181 })
-3182 .max()
-3183 .expect("there should be at least one path"),
-3184 }
-3185 }
-3186
-3187 /// Probe Timeout
-3188 ///
-3189 /// The PTO is logically the time in which you'd expect to receive an acknowledgement
-3190 /// for a packet. So approximately RTT + max_ack_delay.
-3191 fn pto(&self, space: SpaceId, path_id: PathId) -> Duration {
-3192 let max_ack_delay = match space {
-3193 SpaceId::Initial | SpaceId::Handshake => Duration::ZERO,
-3194 SpaceId::Data => self.ack_frequency.max_ack_delay_for_pto(),
-3195 };
-3196 self.path_data(path_id).rtt.pto_base() + max_ack_delay
-3197 }
-3198
-3199 fn on_packet_authenticated(
-3200 &mut self,
-3201 now: Instant,
-3202 space_id: SpaceId,
-3203 path_id: PathId,
-3204 ecn: Option<EcnCodepoint>,
-3205 packet: Option<u64>,
-3206 spin: bool,
-3207 is_1rtt: bool,
-3208 ) {
-3209 self.total_authed_packets += 1;
-3210 if let Some(AbandonState::ExpectingPathAbandon { deadline }) = self
-3211 .paths
-3212 .get(&path_id)
-3213 .map(|path| &path.data.abandon_state)
-3214 {
-3215 if now > *deadline {
-3216 warn!("received data on path which we abandoned more than 3 * PTO ago");
-3217 // The peer failed to respond with a PATH_ABANDON in time.
-3218 if !self.state.is_closed() {
-3219 self.state
-3220 .move_to_closed(TransportError::PROTOCOL_VIOLATION(
-3221 "peer failed to respond with PATH_ABANDON in time",
-3222 ));
-3223 self.close_common();
-3224 self.set_close_timer(now);
-3225 self.close = true;
-3226 }
-3227 return;
-3228 }
-3229 }
-3230
-3231 self.reset_keep_alive(path_id, now);
-3232 self.reset_idle_timeout(now, space_id, path_id);
-3233 self.permit_idle_reset = true;
-3234 self.receiving_ecn |= ecn.is_some();
-3235 if let Some(x) = ecn {
-3236 let space = &mut self.spaces[space_id];
-3237 space.for_path(path_id).ecn_counters += x;
-3238
-3239 if x.is_ce() {
-3240 space
-3241 .for_path(path_id)
-3242 .pending_acks
-3243 .set_immediate_ack_required();
-3244 }
-3245 }
-3246
-3247 let packet = match packet {
-3248 Some(x) => x,
-3249 None => return,
-3250 };
-3251 match &self.side {
-3252 ConnectionSide::Client { .. } => {
-3253 // If we received a handshake packet that authenticated, then we're talking to
-3254 // the real server. From now on we should no longer allow the server to migrate
-3255 // its address.
-3256 if space_id == SpaceId::Handshake {
-3257 if let Some(hs) = self.state.as_handshake_mut() {
-3258 hs.allow_server_migration = false;
-3259 }
-3260 }
-3261 }
-3262 ConnectionSide::Server { .. } => {
-3263 if self.spaces[SpaceId::Initial].crypto.is_some() && space_id == SpaceId::Handshake
-3264 {
-3265 // A server stops sending and processing Initial packets when it receives its first Handshake packet.
-3266 self.discard_space(now, SpaceId::Initial);
-3267 }
-3268 if self.zero_rtt_crypto.is_some() && is_1rtt {
-3269 // Discard 0-RTT keys soon after receiving a 1-RTT packet
-3270 self.set_key_discard_timer(now, space_id)
-3271 }
-3272 }
-3273 }
-3274 let space = self.spaces[space_id].for_path(path_id);
-3275 space.pending_acks.insert_one(packet, now);
-3276 if packet >= space.rx_packet.unwrap_or_default() {
-3277 space.rx_packet = Some(packet);
-3278 // Update outgoing spin bit, inverting iff we're the client
-3279 self.spin = self.side.is_client() ^ spin;
-3280 }
-3281 }
-3282
-3283 /// Resets the idle timeout timers
-3284 ///
-3285 /// Without multipath there is only the connection-wide idle timeout. When multipath is
-3286 /// enabled there is an additional per-path idle timeout.
-3287 fn reset_idle_timeout(&mut self, now: Instant, space: SpaceId, path_id: PathId) {
-3288 // First reset the global idle timeout.
-3289 if let Some(timeout) = self.idle_timeout {
-3290 if self.state.is_closed() {
-3291 self.timers
-3292 .stop(Timer::Conn(ConnTimer::Idle), self.qlog.with_time(now));
-3293 } else {
-3294 let dt = cmp::max(timeout, 3 * self.pto_max_path(space, false));
-3295 self.timers.set(
-3296 Timer::Conn(ConnTimer::Idle),
-3297 now + dt,
-3298 self.qlog.with_time(now),
-3299 );
-3300 }
-3301 }
-3302
-3303 // Now handle the per-path state
-3304 if let Some(timeout) = self.path_data(path_id).idle_timeout {
-3305 if self.state.is_closed() {
-3306 self.timers.stop(
-3307 Timer::PerPath(path_id, PathTimer::PathIdle),
-3308 self.qlog.with_time(now),
-3309 );
-3310 } else {
-3311 let dt = cmp::max(timeout, 3 * self.pto(space, path_id));
-3312 self.timers.set(
-3313 Timer::PerPath(path_id, PathTimer::PathIdle),
-3314 now + dt,
-3315 self.qlog.with_time(now),
-3316 );
-3317 }
-3318 }
-3319 }
-3320
-3321 /// Resets both the [`ConnTimer::KeepAlive`] and [`PathTimer::PathKeepAlive`] timers
-3322 fn reset_keep_alive(&mut self, path_id: PathId, now: Instant) {
-3323 if !self.state.is_established() {
-3324 return;
-3325 }
+2987 let mut size_of_lost_packets = 0u64; // add to path_stats.lost_bytes;
+2988 let lost_pns: Vec<_> = self.spaces[SpaceId::Data]
+2989 .for_path(path_id)
+2990 .sent_packets
+2991 .iter()
+2992 .filter(|(pn, _info)| Some(*pn) != in_flight_mtu_probe)
+2993 .map(|(pn, info)| {
+2994 size_of_lost_packets += info.size as u64;
+2995 pn
+2996 })
+2997 .collect();
+2998
+2999 if !lost_pns.is_empty() {
+3000 trace!(
+3001 %path_id,
+3002 count = lost_pns.len(),
+3003 lost_bytes = size_of_lost_packets,
+3004 "packets lost on path abandon"
+3005 );
+3006 self.handle_lost_packets(
+3007 SpaceId::Data,
+3008 path_id,
+3009 now,
+3010 lost_pns,
+3011 in_flight_mtu_probe,
+3012 Duration::ZERO,
+3013 false,
+3014 size_of_lost_packets,
+3015 );
+3016 }
+3017 self.paths.remove(&path_id);
+3018 self.spaces[SpaceId::Data].number_spaces.remove(&path_id);
+3019
+3020 let path_stats = self.path_stats.remove(&path_id).unwrap_or_default();
+3021 self.events.push_back(
+3022 PathEvent::Abandoned {
+3023 id: path_id,
+3024 path_stats,
+3025 }
+3026 .into(),
+3027 );
+3028 }
+3029
+3030 fn handle_lost_packets(
+3031 &mut self,
+3032 pn_space: SpaceId,
+3033 path_id: PathId,
+3034 now: Instant,
+3035 lost_packets: Vec<u64>,
+3036 lost_mtu_probe: Option<u64>,
+3037 loss_delay: Duration,
+3038 in_persistent_congestion: bool,
+3039 size_of_lost_packets: u64,
+3040 ) {
+3041 debug_assert!(
+3042 {
+3043 let mut sorted = lost_packets.clone();
+3044 sorted.sort();
+3045 sorted == lost_packets
+3046 },
+3047 "lost_packets must be sorted"
+3048 );
+3049
+3050 self.drain_lost_packets(now, pn_space, path_id);
+3051
+3052 // OnPacketsLost
+3053 if let Some(largest_lost) = lost_packets.last().cloned() {
+3054 let old_bytes_in_flight = self.path_data_mut(path_id).in_flight.bytes;
+3055 let largest_lost_sent = self.spaces[pn_space]
+3056 .for_path(path_id)
+3057 .sent_packets
+3058 .get(largest_lost)
+3059 .unwrap()
+3060 .time_sent;
+3061 let path_stats = self.path_stats.entry(path_id).or_default();
+3062 path_stats.lost_packets += lost_packets.len() as u64;
+3063 path_stats.lost_bytes += size_of_lost_packets;
+3064 trace!(
+3065 %path_id,
+3066 count = lost_packets.len(),
+3067 lost_bytes = size_of_lost_packets,
+3068 "packets lost",
+3069 );
+3070
+3071 for &packet in &lost_packets {
+3072 let Some(info) = self.spaces[pn_space].for_path(path_id).take(packet) else {
+3073 continue;
+3074 };
+3075 self.qlog
+3076 .emit_packet_lost(packet, &info, loss_delay, pn_space, now);
+3077 self.paths
+3078 .get_mut(&path_id)
+3079 .unwrap()
+3080 .remove_in_flight(&info);
+3081
+3082 for frame in info.stream_frames {
+3083 self.streams.retransmit(frame);
+3084 }
+3085 self.spaces[pn_space].pending |= info.retransmits;
+3086 self.path_data_mut(path_id)
+3087 .mtud
+3088 .on_non_probe_lost(packet, info.size);
+3089
+3090 self.spaces[pn_space].for_path(path_id).lost_packets.insert(
+3091 packet,
+3092 LostPacket {
+3093 time_sent: info.time_sent,
+3094 },
+3095 );
+3096 }
+3097
+3098 let path = self.path_data_mut(path_id);
+3099 if path.mtud.black_hole_detected(now) {
+3100 path.congestion.on_mtu_update(path.mtud.current_mtu());
+3101 if let Some(max_datagram_size) = self.datagrams().max_size() {
+3102 if self.datagrams.drop_oversized(max_datagram_size)
+3103 && self.datagrams.send_blocked
+3104 {
+3105 self.datagrams.send_blocked = false;
+3106 self.events.push_back(Event::DatagramsUnblocked);
+3107 }
+3108 }
+3109 self.path_stats
+3110 .entry(path_id)
+3111 .or_default()
+3112 .black_holes_detected += 1;
+3113 }
+3114
+3115 // Don't apply congestion penalty for lost ack-only packets
+3116 let lost_ack_eliciting =
+3117 old_bytes_in_flight != self.path_data_mut(path_id).in_flight.bytes;
+3118
+3119 if lost_ack_eliciting {
+3120 self.path_stats
+3121 .entry(path_id)
+3122 .or_default()
+3123 .congestion_events += 1;
+3124 self.path_data_mut(path_id).congestion.on_congestion_event(
+3125 now,
+3126 largest_lost_sent,
+3127 in_persistent_congestion,
+3128 false,
+3129 size_of_lost_packets,
+3130 );
+3131 }
+3132 }
+3133
+3134 // Handle a lost MTU probe
+3135 if let Some(packet) = lost_mtu_probe {
+3136 let info = self.spaces[SpaceId::Data]
+3137 .for_path(path_id)
+3138 .take(packet)
+3139 .unwrap(); // safe: lost_mtu_probe is omitted from lost_packets, and
+3140 // therefore must not have been removed yet
+3141 self.paths
+3142 .get_mut(&path_id)
+3143 .unwrap()
+3144 .remove_in_flight(&info);
+3145 self.path_data_mut(path_id).mtud.on_probe_lost();
+3146 self.path_stats
+3147 .entry(path_id)
+3148 .or_default()
+3149 .lost_plpmtud_probes += 1;
+3150 }
+3151 }
+3152
+3153 /// Returns the earliest time packets should be declared lost for all spaces on a path.
+3154 ///
+3155 /// If a path has an acknowledged packet with any prior un-acknowledged packets, the
+3156 /// earliest un-acknowledged packet can be declared lost after a timeout has elapsed.
+3157 /// The time returned is when this packet should be declared lost.
+3158 fn loss_time_and_space(&self, path_id: PathId) -> Option<(Instant, SpaceId)> {
+3159 SpaceId::iter()
+3160 .filter_map(|id| {
+3161 self.spaces[id]
+3162 .number_spaces
+3163 .get(&path_id)
+3164 .and_then(|pns| pns.loss_time)
+3165 .map(|time| (time, id))
+3166 })
+3167 .min_by_key(|&(time, _)| time)
+3168 }
+3169
+3170 /// Returns the earliest next PTO should fire for all spaces on a path.
+3171 fn pto_time_and_space(&mut self, now: Instant, path_id: PathId) -> Option<(Instant, SpaceId)> {
+3172 let path = self.path(path_id)?;
+3173 let pto_count = path.pto_count;
+3174 let backoff = 2u32.pow(pto_count.min(MAX_BACKOFF_EXPONENT));
+3175 let mut duration = path.rtt.pto_base() * backoff;
+3176
+3177 if path_id == PathId::ZERO
+3178 && path.in_flight.ack_eliciting == 0
+3179 && !self.peer_completed_address_validation(PathId::ZERO)
+3180 {
+3181 // Address Validation during Connection Establishment:
+3182 // https://www.rfc-editor.org/rfc/rfc9000.html#section-8.1. To prevent a
+3183 // deadlock if an Initial or Handshake packet from the server is lost and the
+3184 // server can not send more due to its anti-amplification limit the client must
+3185 // send another packet on PTO.
+3186 let space = match self.highest_space {
+3187 SpaceId::Handshake => SpaceId::Handshake,
+3188 _ => SpaceId::Initial,
+3189 };
+3190
+3191 return Some((now + duration, space));
+3192 }
+3193
+3194 let mut result = None;
+3195 for space in SpaceId::iter() {
+3196 let Some(pns) = self.spaces[space].number_spaces.get(&path_id) else {
+3197 continue;
+3198 };
+3199
+3200 if !pns.has_in_flight() {
+3201 continue;
+3202 }
+3203 if space == SpaceId::Data {
+3204 // Skip ApplicationData until handshake completes.
+3205 if self.is_handshaking() {
+3206 return result;
+3207 }
+3208 // Include max_ack_delay and backoff for ApplicationData.
+3209 duration += self.ack_frequency.max_ack_delay_for_pto() * backoff;
+3210 }
+3211 let Some(last_ack_eliciting) = pns.time_of_last_ack_eliciting_packet else {
+3212 continue;
+3213 };
+3214 let pto = last_ack_eliciting + duration;
+3215 if result.is_none_or(|(earliest_pto, _)| pto < earliest_pto) {
+3216 if path.anti_amplification_blocked(1) {
+3217 // Nothing would be able to be sent.
+3218 continue;
+3219 }
+3220 if path.in_flight.ack_eliciting == 0 {
+3221 // Nothing ack-eliciting, no PTO to arm/fire.
+3222 continue;
+3223 }
+3224 result = Some((pto, space));
+3225 }
+3226 }
+3227 result
+3228 }
+3229
+3230 fn peer_completed_address_validation(&self, path: PathId) -> bool {
+3231 // TODO(flub): This logic needs updating for multipath
+3232 if self.side.is_server() || self.state.is_closed() {
+3233 return true;
+3234 }
+3235 // The server is guaranteed to have validated our address if any of our handshake or 1-RTT
+3236 // packets are acknowledged or we've seen HANDSHAKE_DONE and discarded handshake keys.
+3237 self.spaces[SpaceId::Handshake]
+3238 .path_space(PathId::ZERO)
+3239 .and_then(|pns| pns.largest_acked_packet)
+3240 .is_some()
+3241 || self.spaces[SpaceId::Data]
+3242 .path_space(path)
+3243 .and_then(|pns| pns.largest_acked_packet)
+3244 .is_some()
+3245 || (self.spaces[SpaceId::Data].crypto.is_some()
+3246 && self.spaces[SpaceId::Handshake].crypto.is_none())
+3247 }
+3248
+3249 /// Resets the the [`PathTimer::LossDetection`] timer to the next instant it may be needed
+3250 ///
+3251 /// The timer must fire if either:
+3252 /// - An ack-eliciting packet we sent needs to be declared lost.
+3253 /// - A tail-loss probe needs to be sent.
+3254 ///
+3255 /// See [`Connection::on_loss_detection_timeout`] for details.
+3256 fn set_loss_detection_timer(&mut self, now: Instant, path_id: PathId) {
+3257 if self.state.is_closed() {
+3258 // No loss detection takes place on closed connections, and `close_common` already
+3259 // stopped time timer. Ensure we don't restart it inadvertently, e.g. in response to a
+3260 // reordered packet being handled by state-insensitive code.
+3261 return;
+3262 }
+3263
+3264 if let Some((loss_time, _)) = self.loss_time_and_space(path_id) {
+3265 // Time threshold loss detection.
+3266 self.timers.set(
+3267 Timer::PerPath(path_id, PathTimer::LossDetection),
+3268 loss_time,
+3269 self.qlog.with_time(now),
+3270 );
+3271 return;
+3272 }
+3273
+3274 // Determine which PN space to arm PTO for.
+3275 // Calculate PTO duration
+3276 if let Some((timeout, _)) = self.pto_time_and_space(now, path_id) {
+3277 self.timers.set(
+3278 Timer::PerPath(path_id, PathTimer::LossDetection),
+3279 timeout,
+3280 self.qlog.with_time(now),
+3281 );
+3282 } else {
+3283 self.timers.stop(
+3284 Timer::PerPath(path_id, PathTimer::LossDetection),
+3285 self.qlog.with_time(now),
+3286 );
+3287 }
+3288 }
+3289
+3290 /// The maximum probe timeout across all paths
+3291 ///
+3292 /// If `is_closing` is set to `true` it will filter out paths that have not yet been used.
+3293 ///
+3294 /// See [`Connection::pto`]
+3295 fn pto_max_path(&self, space: SpaceId, is_closing: bool) -> Duration {
+3296 match space {
+3297 SpaceId::Initial | SpaceId::Handshake => self.pto(space, PathId::ZERO),
+3298 SpaceId::Data => self
+3299 .paths
+3300 .iter()
+3301 .filter_map(|(path_id, state)| {
+3302 if is_closing && state.data.total_sent == 0 && state.data.total_recvd == 0 {
+3303 // If we are closing and haven't sent anything yet, do not include
+3304 None
+3305 } else {
+3306 let pto = self.pto(space, *path_id);
+3307 Some(pto)
+3308 }
+3309 })
+3310 .max()
+3311 .expect("there should be at least one path"),
+3312 }
+3313 }
+3314
+3315 /// Probe Timeout
+3316 ///
+3317 /// The PTO is logically the time in which you'd expect to receive an acknowledgement
+3318 /// for a packet. So approximately RTT + max_ack_delay.
+3319 fn pto(&self, space: SpaceId, path_id: PathId) -> Duration {
+3320 let max_ack_delay = match space {
+3321 SpaceId::Initial | SpaceId::Handshake => Duration::ZERO,
+3322 SpaceId::Data => self.ack_frequency.max_ack_delay_for_pto(),
+3323 };
+3324 self.path_data(path_id).rtt.pto_base() + max_ack_delay
+3325 }
3326
-3327 if let Some(interval) = self.config.keep_alive_interval {
-3328 self.timers.set(
-3329 Timer::Conn(ConnTimer::KeepAlive),
-3330 now + interval,
-3331 self.qlog.with_time(now),
-3332 );
-3333 }
-3334
-3335 if let Some(interval) = self.path_data(path_id).keep_alive {
-3336 self.timers.set(
-3337 Timer::PerPath(path_id, PathTimer::PathKeepAlive),
-3338 now + interval,
-3339 self.qlog.with_time(now),
-3340 );
-3341 }
-3342 }
-3343
-3344 /// Sets the timer for when a previously issued CID should be retired next
-3345 fn reset_cid_retirement(&mut self, now: Instant) {
-3346 if let Some((_path, t)) = self.next_cid_retirement() {
-3347 self.timers.set(
-3348 Timer::Conn(ConnTimer::PushNewCid),
-3349 t,
-3350 self.qlog.with_time(now),
-3351 );
-3352 }
-3353 }
-3354
-3355 /// The next time when a previously issued CID should be retired
-3356 fn next_cid_retirement(&self) -> Option<(PathId, Instant)> {
-3357 self.local_cid_state
-3358 .iter()
-3359 .filter_map(|(path_id, cid_state)| cid_state.next_timeout().map(|t| (*path_id, t)))
-3360 .min_by_key(|(_path_id, timeout)| *timeout)
-3361 }
-3362
-3363 /// Handle the already-decrypted first packet from the client
-3364 ///
-3365 /// Decrypting the first packet in the `Endpoint` allows stateless packet handling to be more
-3366 /// efficient.
-3367 pub(crate) fn handle_first_packet(
-3368 &mut self,
-3369 now: Instant,
-3370 network_path: FourTuple,
-3371 ecn: Option<EcnCodepoint>,
-3372 packet_number: u64,
-3373 packet: InitialPacket,
-3374 remaining: Option<BytesMut>,
-3375 ) -> Result<(), ConnectionError> {
-3376 let span = trace_span!("first recv");
-3377 let _guard = span.enter();
-3378 debug_assert!(self.side.is_server());
-3379 let len = packet.header_data.len() + packet.payload.len();
-3380 let path_id = PathId::ZERO;
-3381 self.path_data_mut(path_id).total_recvd = len as u64;
-3382
-3383 if let Some(hs) = self.state.as_handshake_mut() {
-3384 hs.expected_token = packet.header.token.clone();
-3385 } else {
-3386 unreachable!("first packet must be delivered in Handshake state");
-3387 }
-3388
-3389 // The first packet is always on PathId::ZERO
-3390 self.on_packet_authenticated(
-3391 now,
-3392 SpaceId::Initial,
-3393 path_id,
-3394 ecn,
-3395 Some(packet_number),
-3396 false,
-3397 false,
-3398 );
-3399
-3400 let packet: Packet = packet.into();
-3401
-3402 let mut qlog = QlogRecvPacket::new(len);
-3403 qlog.header(&packet.header, Some(packet_number), path_id);
-3404
-3405 self.process_decrypted_packet(
-3406 now,
-3407 network_path,
-3408 path_id,
-3409 Some(packet_number),
-3410 packet,
-3411 &mut qlog,
-3412 )?;
-3413 self.qlog.emit_packet_received(qlog, now);
-3414 if let Some(data) = remaining {
-3415 self.handle_coalesced(now, network_path, path_id, ecn, data);
-3416 }
-3417
-3418 self.qlog.emit_recovery_metrics(
-3419 path_id,
-3420 &mut self.paths.get_mut(&path_id).unwrap().data,
-3421 now,
-3422 );
-3423
-3424 Ok(())
-3425 }
-3426
-3427 fn init_0rtt(&mut self, now: Instant) {
-3428 let (header, packet) = match self.crypto.early_crypto() {
-3429 Some(x) => x,
-3430 None => return,
-3431 };
-3432 if self.side.is_client() {
-3433 match self.crypto.transport_parameters() {
-3434 Ok(params) => {
-3435 let params = params
-3436 .expect("crypto layer didn't supply transport parameters with ticket");
-3437 // Certain values must not be cached
-3438 let params = TransportParameters {
-3439 initial_src_cid: None,
-3440 original_dst_cid: None,
-3441 preferred_address: None,
-3442 retry_src_cid: None,
-3443 stateless_reset_token: None,
-3444 min_ack_delay: None,
-3445 ack_delay_exponent: TransportParameters::default().ack_delay_exponent,
-3446 max_ack_delay: TransportParameters::default().max_ack_delay,
-3447 initial_max_path_id: None,
-3448 ..params
-3449 };
-3450 self.set_peer_params(params);
-3451 self.qlog.emit_peer_transport_params_restored(self, now);
-3452 }
-3453 Err(e) => {
-3454 error!("session ticket has malformed transport parameters: {}", e);
-3455 return;
-3456 }
-3457 }
-3458 }
-3459 trace!("0-RTT enabled");
-3460 self.zero_rtt_enabled = true;
-3461 self.zero_rtt_crypto = Some(ZeroRttCrypto { header, packet });
-3462 }
-3463
-3464 fn read_crypto(
-3465 &mut self,
-3466 space: SpaceId,
-3467 crypto: &frame::Crypto,
-3468 payload_len: usize,
-3469 ) -> Result<(), TransportError> {
-3470 let expected = if !self.state.is_handshake() {
-3471 SpaceId::Data
-3472 } else if self.highest_space == SpaceId::Initial {
-3473 SpaceId::Initial
-3474 } else {
-3475 // On the server, self.highest_space can be Data after receiving the client's first
-3476 // flight, but we expect Handshake CRYPTO until the handshake is complete.
-3477 SpaceId::Handshake
-3478 };
-3479 // We can't decrypt Handshake packets when highest_space is Initial, CRYPTO frames in 0-RTT
-3480 // packets are illegal, and we don't process 1-RTT packets until the handshake is
-3481 // complete. Therefore, we will never see CRYPTO data from a later-than-expected space.
-3482 debug_assert!(space <= expected, "received out-of-order CRYPTO data");
-3483
-3484 let end = crypto.offset + crypto.data.len() as u64;
-3485 if space < expected && end > self.spaces[space].crypto_stream.bytes_read() {
-3486 warn!(
-3487 "received new {:?} CRYPTO data when expecting {:?}",
-3488 space, expected
-3489 );
-3490 return Err(TransportError::PROTOCOL_VIOLATION(
-3491 "new data at unexpected encryption level",
-3492 ));
-3493 }
-3494
-3495 let space = &mut self.spaces[space];
-3496 let max = end.saturating_sub(space.crypto_stream.bytes_read());
-3497 if max > self.config.crypto_buffer_size as u64 {
-3498 return Err(TransportError::CRYPTO_BUFFER_EXCEEDED(""));
-3499 }
-3500
-3501 space
-3502 .crypto_stream
-3503 .insert(crypto.offset, crypto.data.clone(), payload_len);
-3504 while let Some(chunk) = space.crypto_stream.read(usize::MAX, true) {
-3505 trace!("consumed {} CRYPTO bytes", chunk.bytes.len());
-3506 if self.crypto.read_handshake(&chunk.bytes)? {
-3507 self.events.push_back(Event::HandshakeDataReady);
-3508 }
-3509 }
+3327 fn on_packet_authenticated(
+3328 &mut self,
+3329 now: Instant,
+3330 space_id: SpaceId,
+3331 path_id: PathId,
+3332 ecn: Option<EcnCodepoint>,
+3333 packet: Option<u64>,
+3334 spin: bool,
+3335 is_1rtt: bool,
+3336 ) {
+3337 self.total_authed_packets += 1;
+3338 if let Some(AbandonState::ExpectingPathAbandon { deadline }) = self
+3339 .paths
+3340 .get(&path_id)
+3341 .map(|path| &path.data.abandon_state)
+3342 {
+3343 if now > *deadline {
+3344 warn!("received data on path which we abandoned more than 3 * PTO ago");
+3345 // The peer failed to respond with a PATH_ABANDON in time.
+3346 if !self.state.is_closed() {
+3347 self.state
+3348 .move_to_closed(TransportError::PROTOCOL_VIOLATION(
+3349 "peer failed to respond with PATH_ABANDON in time",
+3350 ));
+3351 self.close_common();
+3352 self.set_close_timer(now);
+3353 self.close = true;
+3354 }
+3355 return;
+3356 }
+3357 }
+3358
+3359 self.reset_keep_alive(path_id, now);
+3360 self.reset_idle_timeout(now, space_id, path_id);
+3361 self.permit_idle_reset = true;
+3362 self.receiving_ecn |= ecn.is_some();
+3363 if let Some(x) = ecn {
+3364 let space = &mut self.spaces[space_id];
+3365 space.for_path(path_id).ecn_counters += x;
+3366
+3367 if x.is_ce() {
+3368 space
+3369 .for_path(path_id)
+3370 .pending_acks
+3371 .set_immediate_ack_required();
+3372 }
+3373 }
+3374
+3375 let packet = match packet {
+3376 Some(x) => x,
+3377 None => return,
+3378 };
+3379 match &self.side {
+3380 ConnectionSide::Client { .. } => {
+3381 // If we received a handshake packet that authenticated, then we're talking to
+3382 // the real server. From now on we should no longer allow the server to migrate
+3383 // its address.
+3384 if space_id == SpaceId::Handshake {
+3385 if let Some(hs) = self.state.as_handshake_mut() {
+3386 hs.allow_server_migration = false;
+3387 }
+3388 }
+3389 }
+3390 ConnectionSide::Server { .. } => {
+3391 if self.spaces[SpaceId::Initial].crypto.is_some() && space_id == SpaceId::Handshake
+3392 {
+3393 // A server stops sending and processing Initial packets when it receives its first Handshake packet.
+3394 self.discard_space(now, SpaceId::Initial);
+3395 }
+3396 if self.zero_rtt_crypto.is_some() && is_1rtt {
+3397 // Discard 0-RTT keys soon after receiving a 1-RTT packet
+3398 self.set_key_discard_timer(now, space_id)
+3399 }
+3400 }
+3401 }
+3402 let space = self.spaces[space_id].for_path(path_id);
+3403 space.pending_acks.insert_one(packet, now);
+3404 if packet >= space.rx_packet.unwrap_or_default() {
+3405 space.rx_packet = Some(packet);
+3406 // Update outgoing spin bit, inverting iff we're the client
+3407 self.spin = self.side.is_client() ^ spin;
+3408 }
+3409 }
+3410
+3411 /// Resets the idle timeout timers
+3412 ///
+3413 /// Without multipath there is only the connection-wide idle timeout. When multipath is
+3414 /// enabled there is an additional per-path idle timeout.
+3415 fn reset_idle_timeout(&mut self, now: Instant, space: SpaceId, path_id: PathId) {
+3416 // First reset the global idle timeout.
+3417 if let Some(timeout) = self.idle_timeout {
+3418 if self.state.is_closed() {
+3419 self.timers
+3420 .stop(Timer::Conn(ConnTimer::Idle), self.qlog.with_time(now));
+3421 } else {
+3422 let dt = cmp::max(timeout, 3 * self.pto_max_path(space, false));
+3423 self.timers.set(
+3424 Timer::Conn(ConnTimer::Idle),
+3425 now + dt,
+3426 self.qlog.with_time(now),
+3427 );
+3428 }
+3429 }
+3430
+3431 // Now handle the per-path state
+3432 if let Some(timeout) = self.path_data(path_id).idle_timeout {
+3433 if self.state.is_closed() {
+3434 self.timers.stop(
+3435 Timer::PerPath(path_id, PathTimer::PathIdle),
+3436 self.qlog.with_time(now),
+3437 );
+3438 } else {
+3439 let dt = cmp::max(timeout, 3 * self.pto(space, path_id));
+3440 self.timers.set(
+3441 Timer::PerPath(path_id, PathTimer::PathIdle),
+3442 now + dt,
+3443 self.qlog.with_time(now),
+3444 );
+3445 }
+3446 }
+3447 }
+3448
+3449 /// Resets both the [`ConnTimer::KeepAlive`] and [`PathTimer::PathKeepAlive`] timers
+3450 fn reset_keep_alive(&mut self, path_id: PathId, now: Instant) {
+3451 if !self.state.is_established() {
+3452 return;
+3453 }
+3454
+3455 if let Some(interval) = self.config.keep_alive_interval {
+3456 self.timers.set(
+3457 Timer::Conn(ConnTimer::KeepAlive),
+3458 now + interval,
+3459 self.qlog.with_time(now),
+3460 );
+3461 }
+3462
+3463 if let Some(interval) = self.path_data(path_id).keep_alive {
+3464 self.timers.set(
+3465 Timer::PerPath(path_id, PathTimer::PathKeepAlive),
+3466 now + interval,
+3467 self.qlog.with_time(now),
+3468 );
+3469 }
+3470 }
+3471
+3472 /// Sets the timer for when a previously issued CID should be retired next
+3473 fn reset_cid_retirement(&mut self, now: Instant) {
+3474 if let Some((_path, t)) = self.next_cid_retirement() {
+3475 self.timers.set(
+3476 Timer::Conn(ConnTimer::PushNewCid),
+3477 t,
+3478 self.qlog.with_time(now),
+3479 );
+3480 }
+3481 }
+3482
+3483 /// The next time when a previously issued CID should be retired
+3484 fn next_cid_retirement(&self) -> Option<(PathId, Instant)> {
+3485 self.local_cid_state
+3486 .iter()
+3487 .filter_map(|(path_id, cid_state)| cid_state.next_timeout().map(|t| (*path_id, t)))
+3488 .min_by_key(|(_path_id, timeout)| *timeout)
+3489 }
+3490
+3491 /// Handle the already-decrypted first packet from the client
+3492 ///
+3493 /// Decrypting the first packet in the `Endpoint` allows stateless packet handling to be more
+3494 /// efficient.
+3495 pub(crate) fn handle_first_packet(
+3496 &mut self,
+3497 now: Instant,
+3498 network_path: FourTuple,
+3499 ecn: Option<EcnCodepoint>,
+3500 packet_number: u64,
+3501 packet: InitialPacket,
+3502 remaining: Option<BytesMut>,
+3503 ) -> Result<(), ConnectionError> {
+3504 let span = trace_span!("first recv");
+3505 let _guard = span.enter();
+3506 debug_assert!(self.side.is_server());
+3507 let len = packet.header_data.len() + packet.payload.len();
+3508 let path_id = PathId::ZERO;
+3509 self.path_data_mut(path_id).total_recvd = len as u64;
3510
-3511 Ok(())
-3512 }
-3513
-3514 fn write_crypto(&mut self) {
-3515 loop {
-3516 let space = self.highest_space;
-3517 let mut outgoing = Vec::new();
-3518 if let Some(crypto) = self.crypto.write_handshake(&mut outgoing) {
-3519 match space {
-3520 SpaceId::Initial => {
-3521 self.upgrade_crypto(SpaceId::Handshake, crypto);
-3522 }
-3523 SpaceId::Handshake => {
-3524 self.upgrade_crypto(SpaceId::Data, crypto);
-3525 }
-3526 _ => unreachable!("got updated secrets during 1-RTT"),
-3527 }
-3528 }
-3529 if outgoing.is_empty() {
-3530 if space == self.highest_space {
-3531 break;
-3532 } else {
-3533 // Keys updated, check for more data to send
-3534 continue;
-3535 }
-3536 }
-3537 let offset = self.spaces[space].crypto_offset;
-3538 let outgoing = Bytes::from(outgoing);
-3539 if let Some(hs) = self.state.as_handshake_mut() {
-3540 if space == SpaceId::Initial && offset == 0 && self.side.is_client() {
-3541 hs.client_hello = Some(outgoing.clone());
-3542 }
-3543 }
-3544 self.spaces[space].crypto_offset += outgoing.len() as u64;
-3545 trace!("wrote {} {:?} CRYPTO bytes", outgoing.len(), space);
-3546 self.spaces[space].pending.crypto.push_back(frame::Crypto {
-3547 offset,
-3548 data: outgoing,
-3549 });
-3550 }
-3551 }
-3552
-3553 /// Switch to stronger cryptography during handshake
-3554 fn upgrade_crypto(&mut self, space: SpaceId, crypto: Keys) {
-3555 debug_assert!(
-3556 self.spaces[space].crypto.is_none(),
-3557 "already reached packet space {space:?}"
-3558 );
-3559 trace!("{:?} keys ready", space);
-3560 if space == SpaceId::Data {
-3561 // Precompute the first key update
-3562 self.next_crypto = Some(
-3563 self.crypto
-3564 .next_1rtt_keys()
-3565 .expect("handshake should be complete"),
-3566 );
-3567 }
-3568
-3569 self.spaces[space].crypto = Some(crypto);
-3570 debug_assert!(space as usize > self.highest_space as usize);
-3571 self.highest_space = space;
-3572 if space == SpaceId::Data && self.side.is_client() {
-3573 // Discard 0-RTT keys because 1-RTT keys are available.
-3574 self.zero_rtt_crypto = None;
-3575 }
-3576 }
-3577
-3578 fn discard_space(&mut self, now: Instant, space_id: SpaceId) {
-3579 debug_assert!(space_id != SpaceId::Data);
-3580 trace!("discarding {:?} keys", space_id);
-3581 if space_id == SpaceId::Initial {
-3582 // No longer needed
-3583 if let ConnectionSide::Client { token, .. } = &mut self.side {
-3584 *token = Bytes::new();
+3511 if let Some(hs) = self.state.as_handshake_mut() {
+3512 hs.expected_token = packet.header.token.clone();
+3513 } else {
+3514 unreachable!("first packet must be delivered in Handshake state");
+3515 }
+3516
+3517 // The first packet is always on PathId::ZERO
+3518 self.on_packet_authenticated(
+3519 now,
+3520 SpaceId::Initial,
+3521 path_id,
+3522 ecn,
+3523 Some(packet_number),
+3524 false,
+3525 false,
+3526 );
+3527
+3528 let packet: Packet = packet.into();
+3529
+3530 let mut qlog = QlogRecvPacket::new(len);
+3531 qlog.header(&packet.header, Some(packet_number), path_id);
+3532
+3533 self.process_decrypted_packet(
+3534 now,
+3535 network_path,
+3536 path_id,
+3537 Some(packet_number),
+3538 packet,
+3539 &mut qlog,
+3540 )?;
+3541 self.qlog.emit_packet_received(qlog, now);
+3542 if let Some(data) = remaining {
+3543 self.handle_coalesced(now, network_path, path_id, ecn, data);
+3544 }
+3545
+3546 self.qlog.emit_recovery_metrics(
+3547 path_id,
+3548 &mut self.paths.get_mut(&path_id).unwrap().data,
+3549 now,
+3550 );
+3551
+3552 Ok(())
+3553 }
+3554
+3555 fn init_0rtt(&mut self, now: Instant) {
+3556 let (header, packet) = match self.crypto.early_crypto() {
+3557 Some(x) => x,
+3558 None => return,
+3559 };
+3560 if self.side.is_client() {
+3561 match self.crypto.transport_parameters() {
+3562 Ok(params) => {
+3563 let params = params
+3564 .expect("crypto layer didn't supply transport parameters with ticket");
+3565 // Certain values must not be cached
+3566 let params = TransportParameters {
+3567 initial_src_cid: None,
+3568 original_dst_cid: None,
+3569 preferred_address: None,
+3570 retry_src_cid: None,
+3571 stateless_reset_token: None,
+3572 min_ack_delay: None,
+3573 ack_delay_exponent: TransportParameters::default().ack_delay_exponent,
+3574 max_ack_delay: TransportParameters::default().max_ack_delay,
+3575 initial_max_path_id: None,
+3576 ..params
+3577 };
+3578 self.set_peer_params(params);
+3579 self.qlog.emit_peer_transport_params_restored(self, now);
+3580 }
+3581 Err(e) => {
+3582 error!("session ticket has malformed transport parameters: {}", e);
+3583 return;
+3584 }
3585 }
3586 }
-3587 let space = &mut self.spaces[space_id];
-3588 space.crypto = None;
-3589 let pns = space.for_path(PathId::ZERO);
-3590 pns.time_of_last_ack_eliciting_packet = None;
-3591 pns.loss_time = None;
-3592 pns.loss_probes = 0;
-3593 let sent_packets = mem::take(&mut pns.sent_packets);
-3594 let path = self.paths.get_mut(&PathId::ZERO).unwrap();
-3595 for (_, packet) in sent_packets.into_iter() {
-3596 path.data.remove_in_flight(&packet);
-3597 }
-3598
-3599 self.set_loss_detection_timer(now, PathId::ZERO)
-3600 }
-3601
-3602 fn handle_coalesced(
-3603 &mut self,
-3604 now: Instant,
-3605 network_path: FourTuple,
-3606 path_id: PathId,
-3607 ecn: Option<EcnCodepoint>,
-3608 data: BytesMut,
-3609 ) {
-3610 self.path_data_mut(path_id)
-3611 .inc_total_recvd(data.len() as u64);
-3612 let mut remaining = Some(data);
-3613 let cid_len = self
-3614 .local_cid_state
-3615 .values()
-3616 .map(|cid_state| cid_state.cid_len())
-3617 .next()
-3618 .expect("one cid_state must exist");
-3619 while let Some(data) = remaining {
-3620 match PartialDecode::new(
-3621 data,
-3622 &FixedLengthConnectionIdParser::new(cid_len),
-3623 &[self.version],
-3624 self.endpoint_config.grease_quic_bit,
-3625 ) {
-3626 Ok((partial_decode, rest)) => {
-3627 remaining = rest;
-3628 self.handle_decode(now, network_path, path_id, ecn, partial_decode);
-3629 }
-3630 Err(e) => {
-3631 trace!("malformed header: {}", e);
-3632 return;
-3633 }
-3634 }
-3635 }
-3636 }
-3637
-3638 fn handle_decode(
-3639 &mut self,
-3640 now: Instant,
-3641 network_path: FourTuple,
-3642 path_id: PathId,
-3643 ecn: Option<EcnCodepoint>,
-3644 partial_decode: PartialDecode,
-3645 ) {
-3646 let qlog = QlogRecvPacket::new(partial_decode.len());
-3647 if let Some(decoded) = packet_crypto::unprotect_header(
-3648 partial_decode,
-3649 &self.spaces,
-3650 self.zero_rtt_crypto.as_ref(),
-3651 self.peer_params.stateless_reset_token,
-3652 ) {
-3653 self.handle_packet(
-3654 now,
-3655 network_path,
-3656 path_id,
-3657 ecn,
-3658 decoded.packet,
-3659 decoded.stateless_reset,
-3660 qlog,
-3661 );
-3662 }
-3663 }
-3664
-3665 fn handle_packet(
-3666 &mut self,
-3667 now: Instant,
-3668 network_path: FourTuple,
-3669 path_id: PathId,
-3670 ecn: Option<EcnCodepoint>,
-3671 packet: Option<Packet>,
-3672 stateless_reset: bool,
-3673 mut qlog: QlogRecvPacket,
-3674 ) {
-3675 self.stats.udp_rx.ios += 1;
-3676 if let Some(ref packet) = packet {
-3677 trace!(
-3678 "got {:?} packet ({} bytes) from {} using id {}",
-3679 packet.header.space(),
-3680 packet.payload.len() + packet.header_data.len(),
-3681 network_path,
-3682 packet.header.dst_cid(),
-3683 );
-3684 }
-3685
-3686 if self.is_handshaking() {
-3687 if path_id != PathId::ZERO {
-3688 debug!(%network_path, %path_id, "discarding multipath packet during handshake");
-3689 return;
-3690 }
-3691 if network_path != self.path_data_mut(path_id).network_path {
-3692 if let Some(hs) = self.state.as_handshake() {
-3693 if hs.allow_server_migration {
-3694 trace!(%network_path, prev = %self.path_data(path_id).network_path, "server migrated to new remote");
-3695 self.path_data_mut(path_id).network_path = network_path;
-3696 self.qlog.emit_tuple_assigned(path_id, network_path, now);
-3697 } else {
-3698 debug!("discarding packet with unexpected remote during handshake");
-3699 return;
-3700 }
-3701 } else {
-3702 debug!("discarding packet with unexpected remote during handshake");
-3703 return;
-3704 }
-3705 }
-3706 }
-3707
-3708 let was_closed = self.state.is_closed();
-3709 let was_drained = self.state.is_drained();
-3710
-3711 let decrypted = match packet {
-3712 None => Err(None),
-3713 Some(mut packet) => self
-3714 .decrypt_packet(now, path_id, &mut packet)
-3715 .map(move |number| (packet, number)),
-3716 };
-3717 let result = match decrypted {
-3718 _ if stateless_reset => {
-3719 debug!("got stateless reset");
-3720 Err(ConnectionError::Reset)
-3721 }
-3722 Err(Some(e)) => {
-3723 warn!("illegal packet: {}", e);
-3724 Err(e.into())
-3725 }
-3726 Err(None) => {
-3727 debug!("failed to authenticate packet");
-3728 self.authentication_failures += 1;
-3729 let integrity_limit = self.spaces[self.highest_space]
-3730 .crypto
-3731 .as_ref()
-3732 .unwrap()
-3733 .packet
-3734 .local
-3735 .integrity_limit();
-3736 if self.authentication_failures > integrity_limit {
-3737 Err(TransportError::AEAD_LIMIT_REACHED("integrity limit violated").into())
-3738 } else {
-3739 return;
-3740 }
-3741 }
-3742 Ok((packet, number)) => {
-3743 qlog.header(&packet.header, number, path_id);
-3744 let span = match number {
-3745 Some(pn) => trace_span!("recv", space = ?packet.header.space(), pn),
-3746 None => trace_span!("recv", space = ?packet.header.space()),
-3747 };
-3748 let _guard = span.enter();
-3749
-3750 let dedup = self.spaces[packet.header.space()]
-3751 .path_space_mut(path_id)
-3752 .map(|pns| &mut pns.dedup);
-3753 if number.zip(dedup).is_some_and(|(n, d)| d.insert(n)) {
-3754 debug!("discarding possible duplicate packet");
-3755 self.qlog.emit_packet_received(qlog, now);
-3756 return;
-3757 } else if self.state.is_handshake() && packet.header.is_short() {
-3758 // TODO: SHOULD buffer these to improve reordering tolerance.
-3759 trace!("dropping short packet during handshake");
-3760 self.qlog.emit_packet_received(qlog, now);
-3761 return;
-3762 } else {
-3763 if let Header::Initial(InitialHeader { ref token, .. }) = packet.header {
-3764 if let Some(hs) = self.state.as_handshake() {
-3765 if self.side.is_server() && token != &hs.expected_token {
-3766 // Clients must send the same retry token in every Initial. Initial
-3767 // packets can be spoofed, so we discard rather than killing the
-3768 // connection.
-3769 warn!("discarding Initial with invalid retry token");
-3770 self.qlog.emit_packet_received(qlog, now);
-3771 return;
-3772 }
-3773 }
-3774 }
-3775
-3776 if !self.state.is_closed() {
-3777 let spin = match packet.header {
-3778 Header::Short { spin, .. } => spin,
-3779 _ => false,
-3780 };
-3781
-3782 if self.side().is_server() && !self.abandoned_paths.contains(&path_id) {
-3783 // Only the client is allowed to open paths
-3784 self.ensure_path(path_id, network_path, now, number);
-3785 }
-3786 if self.paths.contains_key(&path_id) {
-3787 self.on_packet_authenticated(
-3788 now,
-3789 packet.header.space(),
-3790 path_id,
-3791 ecn,
-3792 number,
-3793 spin,
-3794 packet.header.is_1rtt(),
-3795 );
-3796 }
-3797 }
-3798
-3799 let res = self.process_decrypted_packet(
-3800 now,
-3801 network_path,
-3802 path_id,
-3803 number,
-3804 packet,
-3805 &mut qlog,
-3806 );
-3807
-3808 self.qlog.emit_packet_received(qlog, now);
-3809 res
-3810 }
-3811 }
-3812 };
+3587 trace!("0-RTT enabled");
+3588 self.zero_rtt_enabled = true;
+3589 self.zero_rtt_crypto = Some(ZeroRttCrypto { header, packet });
+3590 }
+3591
+3592 fn read_crypto(
+3593 &mut self,
+3594 space: SpaceId,
+3595 crypto: &frame::Crypto,
+3596 payload_len: usize,
+3597 ) -> Result<(), TransportError> {
+3598 let expected = if !self.state.is_handshake() {
+3599 SpaceId::Data
+3600 } else if self.highest_space == SpaceId::Initial {
+3601 SpaceId::Initial
+3602 } else {
+3603 // On the server, self.highest_space can be Data after receiving the client's first
+3604 // flight, but we expect Handshake CRYPTO until the handshake is complete.
+3605 SpaceId::Handshake
+3606 };
+3607 // We can't decrypt Handshake packets when highest_space is Initial, CRYPTO frames in 0-RTT
+3608 // packets are illegal, and we don't process 1-RTT packets until the handshake is
+3609 // complete. Therefore, we will never see CRYPTO data from a later-than-expected space.
+3610 debug_assert!(space <= expected, "received out-of-order CRYPTO data");
+3611
+3612 let end = crypto.offset + crypto.data.len() as u64;
+3613 if space < expected && end > self.spaces[space].crypto_stream.bytes_read() {
+3614 warn!(
+3615 "received new {:?} CRYPTO data when expecting {:?}",
+3616 space, expected
+3617 );
+3618 return Err(TransportError::PROTOCOL_VIOLATION(
+3619 "new data at unexpected encryption level",
+3620 ));
+3621 }
+3622
+3623 let space = &mut self.spaces[space];
+3624 let max = end.saturating_sub(space.crypto_stream.bytes_read());
+3625 if max > self.config.crypto_buffer_size as u64 {
+3626 return Err(TransportError::CRYPTO_BUFFER_EXCEEDED(""));
+3627 }
+3628
+3629 space
+3630 .crypto_stream
+3631 .insert(crypto.offset, crypto.data.clone(), payload_len);
+3632 while let Some(chunk) = space.crypto_stream.read(usize::MAX, true) {
+3633 trace!("consumed {} CRYPTO bytes", chunk.bytes.len());
+3634 if self.crypto.read_handshake(&chunk.bytes)? {
+3635 self.events.push_back(Event::HandshakeDataReady);
+3636 }
+3637 }
+3638
+3639 Ok(())
+3640 }
+3641
+3642 fn write_crypto(&mut self) {
+3643 loop {
+3644 let space = self.highest_space;
+3645 let mut outgoing = Vec::new();
+3646 if let Some(crypto) = self.crypto.write_handshake(&mut outgoing) {
+3647 match space {
+3648 SpaceId::Initial => {
+3649 self.upgrade_crypto(SpaceId::Handshake, crypto);
+3650 }
+3651 SpaceId::Handshake => {
+3652 self.upgrade_crypto(SpaceId::Data, crypto);
+3653 }
+3654 _ => unreachable!("got updated secrets during 1-RTT"),
+3655 }
+3656 }
+3657 if outgoing.is_empty() {
+3658 if space == self.highest_space {
+3659 break;
+3660 } else {
+3661 // Keys updated, check for more data to send
+3662 continue;
+3663 }
+3664 }
+3665 let offset = self.spaces[space].crypto_offset;
+3666 let outgoing = Bytes::from(outgoing);
+3667 if let Some(hs) = self.state.as_handshake_mut() {
+3668 if space == SpaceId::Initial && offset == 0 && self.side.is_client() {
+3669 hs.client_hello = Some(outgoing.clone());
+3670 }
+3671 }
+3672 self.spaces[space].crypto_offset += outgoing.len() as u64;
+3673 trace!("wrote {} {:?} CRYPTO bytes", outgoing.len(), space);
+3674 self.spaces[space].pending.crypto.push_back(frame::Crypto {
+3675 offset,
+3676 data: outgoing,
+3677 });
+3678 }
+3679 }
+3680
+3681 /// Switch to stronger cryptography during handshake
+3682 fn upgrade_crypto(&mut self, space: SpaceId, crypto: Keys) {
+3683 debug_assert!(
+3684 self.spaces[space].crypto.is_none(),
+3685 "already reached packet space {space:?}"
+3686 );
+3687 trace!("{:?} keys ready", space);
+3688 if space == SpaceId::Data {
+3689 // Precompute the first key update
+3690 self.next_crypto = Some(
+3691 self.crypto
+3692 .next_1rtt_keys()
+3693 .expect("handshake should be complete"),
+3694 );
+3695 }
+3696
+3697 self.spaces[space].crypto = Some(crypto);
+3698 debug_assert!(space as usize > self.highest_space as usize);
+3699 self.highest_space = space;
+3700 if space == SpaceId::Data && self.side.is_client() {
+3701 // Discard 0-RTT keys because 1-RTT keys are available.
+3702 self.zero_rtt_crypto = None;
+3703 }
+3704 }
+3705
+3706 fn discard_space(&mut self, now: Instant, space_id: SpaceId) {
+3707 debug_assert!(space_id != SpaceId::Data);
+3708 trace!("discarding {:?} keys", space_id);
+3709 if space_id == SpaceId::Initial {
+3710 // No longer needed
+3711 if let ConnectionSide::Client { token, .. } = &mut self.side {
+3712 *token = Bytes::new();
+3713 }
+3714 }
+3715 let space = &mut self.spaces[space_id];
+3716 space.crypto = None;
+3717 let pns = space.for_path(PathId::ZERO);
+3718 pns.time_of_last_ack_eliciting_packet = None;
+3719 pns.loss_time = None;
+3720 pns.loss_probes = 0;
+3721 let sent_packets = mem::take(&mut pns.sent_packets);
+3722 let path = self.paths.get_mut(&PathId::ZERO).unwrap();
+3723 for (_, packet) in sent_packets.into_iter() {
+3724 path.data.remove_in_flight(&packet);
+3725 }
+3726
+3727 self.set_loss_detection_timer(now, PathId::ZERO)
+3728 }
+3729
+3730 fn handle_coalesced(
+3731 &mut self,
+3732 now: Instant,
+3733 network_path: FourTuple,
+3734 path_id: PathId,
+3735 ecn: Option<EcnCodepoint>,
+3736 data: BytesMut,
+3737 ) {
+3738 self.path_data_mut(path_id)
+3739 .inc_total_recvd(data.len() as u64);
+3740 let mut remaining = Some(data);
+3741 let cid_len = self
+3742 .local_cid_state
+3743 .values()
+3744 .map(|cid_state| cid_state.cid_len())
+3745 .next()
+3746 .expect("one cid_state must exist");
+3747 while let Some(data) = remaining {
+3748 match PartialDecode::new(
+3749 data,
+3750 &FixedLengthConnectionIdParser::new(cid_len),
+3751 &[self.version],
+3752 self.endpoint_config.grease_quic_bit,
+3753 ) {
+3754 Ok((partial_decode, rest)) => {
+3755 remaining = rest;
+3756 self.handle_decode(now, network_path, path_id, ecn, partial_decode);
+3757 }
+3758 Err(e) => {
+3759 trace!("malformed header: {}", e);
+3760 return;
+3761 }
+3762 }
+3763 }
+3764 }
+3765
+3766 fn handle_decode(
+3767 &mut self,
+3768 now: Instant,
+3769 network_path: FourTuple,
+3770 path_id: PathId,
+3771 ecn: Option<EcnCodepoint>,
+3772 partial_decode: PartialDecode,
+3773 ) {
+3774 let qlog = QlogRecvPacket::new(partial_decode.len());
+3775 if let Some(decoded) = packet_crypto::unprotect_header(
+3776 partial_decode,
+3777 &self.spaces,
+3778 self.zero_rtt_crypto.as_ref(),
+3779 self.peer_params.stateless_reset_token,
+3780 ) {
+3781 self.handle_packet(
+3782 now,
+3783 network_path,
+3784 path_id,
+3785 ecn,
+3786 decoded.packet,
+3787 decoded.stateless_reset,
+3788 qlog,
+3789 );
+3790 }
+3791 }
+3792
+3793 fn handle_packet(
+3794 &mut self,
+3795 now: Instant,
+3796 network_path: FourTuple,
+3797 path_id: PathId,
+3798 ecn: Option<EcnCodepoint>,
+3799 packet: Option<Packet>,
+3800 stateless_reset: bool,
+3801 mut qlog: QlogRecvPacket,
+3802 ) {
+3803 self.stats.udp_rx.ios += 1;
+3804 if let Some(ref packet) = packet {
+3805 trace!(
+3806 "got {:?} packet ({} bytes) from {} using id {}",
+3807 packet.header.space(),
+3808 packet.payload.len() + packet.header_data.len(),
+3809 network_path,
+3810 packet.header.dst_cid(),
+3811 );
+3812 }
3813
-3814 // State transitions for error cases
-3815 if let Err(conn_err) = result {
-3816 match conn_err {
-3817 ConnectionError::ApplicationClosed(reason) => self.state.move_to_closed(reason),
-3818 ConnectionError::ConnectionClosed(reason) => self.state.move_to_closed(reason),
-3819 ConnectionError::Reset
-3820 | ConnectionError::TransportError(TransportError {
-3821 code: TransportErrorCode::AEAD_LIMIT_REACHED,
-3822 ..
-3823 }) => {
-3824 self.state.move_to_drained(Some(conn_err));
-3825 }
-3826 ConnectionError::TimedOut => {
-3827 unreachable!("timeouts aren't generated by packet processing");
-3828 }
-3829 ConnectionError::TransportError(err) => {
-3830 debug!("closing connection due to transport error: {}", err);
-3831 self.state.move_to_closed(err);
+3814 if self.is_handshaking() {
+3815 if path_id != PathId::ZERO {
+3816 debug!(%network_path, %path_id, "discarding multipath packet during handshake");
+3817 return;
+3818 }
+3819 if network_path != self.path_data_mut(path_id).network_path {
+3820 if let Some(hs) = self.state.as_handshake() {
+3821 if hs.allow_server_migration {
+3822 trace!(%network_path, prev = %self.path_data(path_id).network_path, "server migrated to new remote");
+3823 self.path_data_mut(path_id).network_path = network_path;
+3824 self.qlog.emit_tuple_assigned(path_id, network_path, now);
+3825 } else {
+3826 debug!("discarding packet with unexpected remote during handshake");
+3827 return;
+3828 }
+3829 } else {
+3830 debug!("discarding packet with unexpected remote during handshake");
+3831 return;
3832 }
-3833 ConnectionError::VersionMismatch => {
-3834 self.state.move_to_draining(Some(conn_err));
-3835 }
-3836 ConnectionError::LocallyClosed => {
-3837 unreachable!("LocallyClosed isn't generated by packet processing");
-3838 }
-3839 ConnectionError::CidsExhausted => {
-3840 unreachable!("CidsExhausted isn't generated by packet processing");
-3841 }
-3842 };
-3843 }
-3844
-3845 if !was_closed && self.state.is_closed() {
-3846 self.close_common();
-3847 if !self.state.is_drained() {
-3848 self.set_close_timer(now);
+3833 }
+3834 }
+3835
+3836 let was_closed = self.state.is_closed();
+3837 let was_drained = self.state.is_drained();
+3838
+3839 let decrypted = match packet {
+3840 None => Err(None),
+3841 Some(mut packet) => self
+3842 .decrypt_packet(now, path_id, &mut packet)
+3843 .map(move |number| (packet, number)),
+3844 };
+3845 let result = match decrypted {
+3846 _ if stateless_reset => {
+3847 debug!("got stateless reset");
+3848 Err(ConnectionError::Reset)
3849 }
-3850 }
-3851 if !was_drained && self.state.is_drained() {
-3852 self.endpoint_events.push_back(EndpointEventInner::Drained);
-3853 // Close timer may have been started previously, e.g. if we sent a close and got a
-3854 // stateless reset in response
-3855 self.timers
-3856 .stop(Timer::Conn(ConnTimer::Close), self.qlog.with_time(now));
-3857 }
-3858
-3859 // Transmit CONNECTION_CLOSE if necessary
-3860 if matches!(self.state.as_type(), StateType::Closed) {
-3861 // If there is no PathData for this PathId the packet was for a brand new
-3862 // path. It was a valid packet however, so the remote is valid and we want to
-3863 // send CONNECTION_CLOSE.
-3864 let path_remote = self
-3865 .paths
-3866 .get(&path_id)
-3867 .map(|p| p.data.network_path)
-3868 .unwrap_or(network_path);
-3869 self.close = network_path == path_remote;
-3870 }
-3871 }
-3872
-3873 fn process_decrypted_packet(
-3874 &mut self,
-3875 now: Instant,
-3876 network_path: FourTuple,
-3877 path_id: PathId,
-3878 number: Option<u64>,
-3879 packet: Packet,
-3880 qlog: &mut QlogRecvPacket,
-3881 ) -> Result<(), ConnectionError> {
-3882 if !self.paths.contains_key(&path_id) {
-3883 // There is a chance this is a server side, first (for this path) packet, which would
-3884 // be a protocol violation. It's more likely, however, that this is a packet of a
-3885 // pruned path
-3886 trace!(%path_id, ?number, "discarding packet for unknown path");
-3887 return Ok(());
-3888 }
-3889 let state = match self.state.as_type() {
-3890 StateType::Established => {
-3891 match packet.header.space() {
-3892 SpaceId::Data => self.process_payload(
-3893 now,
-3894 network_path,
-3895 path_id,
-3896 number.unwrap(),
-3897 packet,
-3898 qlog,
-3899 )?,
-3900 _ if packet.header.has_frames() => {
-3901 self.process_early_payload(now, path_id, packet, qlog)?
-3902 }
-3903 _ => {
-3904 trace!("discarding unexpected pre-handshake packet");
-3905 }
-3906 }
-3907 return Ok(());
-3908 }
-3909 StateType::Closed => {
-3910 for result in frame::Iter::new(packet.payload.freeze())? {
-3911 let frame = match result {
-3912 Ok(frame) => frame,
-3913 Err(err) => {
-3914 debug!("frame decoding error: {err:?}");
-3915 continue;
-3916 }
-3917 };
-3918 qlog.frame(&frame);
-3919
-3920 if let Frame::Padding = frame {
-3921 continue;
-3922 };
-3923
-3924 self.stats.frame_rx.record(frame.ty());
-3925
-3926 if let Frame::Close(_error) = frame {
-3927 self.state.move_to_draining(None);
-3928 break;
-3929 }
-3930 }
-3931 return Ok(());
-3932 }
-3933 StateType::Draining | StateType::Drained => return Ok(()),
-3934 StateType::Handshake => self.state.as_handshake_mut().expect("checked"),
-3935 };
-3936
-3937 match packet.header {
-3938 Header::Retry {
-3939 src_cid: rem_cid, ..
-3940 } => {
-3941 debug_assert_eq!(path_id, PathId::ZERO);
-3942 if self.side.is_server() {
-3943 return Err(TransportError::PROTOCOL_VIOLATION("client sent Retry").into());
-3944 }
-3945
-3946 let is_valid_retry = self
-3947 .rem_cids
-3948 .get(&path_id)
-3949 .map(|cids| cids.active())
-3950 .map(|orig_dst_cid| {
-3951 self.crypto.is_valid_retry(
-3952 orig_dst_cid,
-3953 &packet.header_data,
-3954 &packet.payload,
-3955 )
-3956 })
-3957 .unwrap_or_default();
-3958 if self.total_authed_packets > 1
-3959 || packet.payload.len() <= 16 // token + 16 byte tag
-3960 || !is_valid_retry
-3961 {
-3962 trace!("discarding invalid Retry");
-3963 // - After the client has received and processed an Initial or Retry
-3964 // packet from the server, it MUST discard any subsequent Retry
-3965 // packets that it receives.
-3966 // - A client MUST discard a Retry packet with a zero-length Retry Token
-3967 // field.
-3968 // - Clients MUST discard Retry packets that have a Retry Integrity Tag
-3969 // that cannot be validated
-3970 return Ok(());
-3971 }
+3850 Err(Some(e)) => {
+3851 warn!("illegal packet: {}", e);
+3852 Err(e.into())
+3853 }
+3854 Err(None) => {
+3855 debug!("failed to authenticate packet");
+3856 self.authentication_failures += 1;
+3857 let integrity_limit = self.spaces[self.highest_space]
+3858 .crypto
+3859 .as_ref()
+3860 .unwrap()
+3861 .packet
+3862 .local
+3863 .integrity_limit();
+3864 if self.authentication_failures > integrity_limit {
+3865 Err(TransportError::AEAD_LIMIT_REACHED("integrity limit violated").into())
+3866 } else {
+3867 return;
+3868 }
+3869 }
+3870 Ok((packet, number)) => {
+3871 qlog.header(&packet.header, number, path_id);
+3872 let span = match number {
+3873 Some(pn) => trace_span!("recv", space = ?packet.header.space(), pn),
+3874 None => trace_span!("recv", space = ?packet.header.space()),
+3875 };
+3876 let _guard = span.enter();
+3877
+3878 let dedup = self.spaces[packet.header.space()]
+3879 .path_space_mut(path_id)
+3880 .map(|pns| &mut pns.dedup);
+3881 if number.zip(dedup).is_some_and(|(n, d)| d.insert(n)) {
+3882 debug!("discarding possible duplicate packet");
+3883 self.qlog.emit_packet_received(qlog, now);
+3884 return;
+3885 } else if self.state.is_handshake() && packet.header.is_short() {
+3886 // TODO: SHOULD buffer these to improve reordering tolerance.
+3887 trace!("dropping short packet during handshake");
+3888 self.qlog.emit_packet_received(qlog, now);
+3889 return;
+3890 } else {
+3891 if let Header::Initial(InitialHeader { ref token, .. }) = packet.header {
+3892 if let Some(hs) = self.state.as_handshake() {
+3893 if self.side.is_server() && token != &hs.expected_token {
+3894 // Clients must send the same retry token in every Initial. Initial
+3895 // packets can be spoofed, so we discard rather than killing the
+3896 // connection.
+3897 warn!("discarding Initial with invalid retry token");
+3898 self.qlog.emit_packet_received(qlog, now);
+3899 return;
+3900 }
+3901 }
+3902 }
+3903
+3904 if !self.state.is_closed() {
+3905 let spin = match packet.header {
+3906 Header::Short { spin, .. } => spin,
+3907 _ => false,
+3908 };
+3909
+3910 if self.side().is_server() && !self.abandoned_paths.contains(&path_id) {
+3911 // Only the client is allowed to open paths
+3912 self.ensure_path(path_id, network_path, now, number);
+3913 }
+3914 if self.paths.contains_key(&path_id) {
+3915 self.on_packet_authenticated(
+3916 now,
+3917 packet.header.space(),
+3918 path_id,
+3919 ecn,
+3920 number,
+3921 spin,
+3922 packet.header.is_1rtt(),
+3923 );
+3924 }
+3925 }
+3926
+3927 let res = self.process_decrypted_packet(
+3928 now,
+3929 network_path,
+3930 path_id,
+3931 number,
+3932 packet,
+3933 &mut qlog,
+3934 );
+3935
+3936 self.qlog.emit_packet_received(qlog, now);
+3937 res
+3938 }
+3939 }
+3940 };
+3941
+3942 // State transitions for error cases
+3943 if let Err(conn_err) = result {
+3944 match conn_err {
+3945 ConnectionError::ApplicationClosed(reason) => self.state.move_to_closed(reason),
+3946 ConnectionError::ConnectionClosed(reason) => self.state.move_to_closed(reason),
+3947 ConnectionError::Reset
+3948 | ConnectionError::TransportError(TransportError {
+3949 code: TransportErrorCode::AEAD_LIMIT_REACHED,
+3950 ..
+3951 }) => {
+3952 self.state.move_to_drained(Some(conn_err));
+3953 }
+3954 ConnectionError::TimedOut => {
+3955 unreachable!("timeouts aren't generated by packet processing");
+3956 }
+3957 ConnectionError::TransportError(err) => {
+3958 debug!("closing connection due to transport error: {}", err);
+3959 self.state.move_to_closed(err);
+3960 }
+3961 ConnectionError::VersionMismatch => {
+3962 self.state.move_to_draining(Some(conn_err));
+3963 }
+3964 ConnectionError::LocallyClosed => {
+3965 unreachable!("LocallyClosed isn't generated by packet processing");
+3966 }
+3967 ConnectionError::CidsExhausted => {
+3968 unreachable!("CidsExhausted isn't generated by packet processing");
+3969 }
+3970 };
+3971 }
3972
-3973 trace!("retrying with CID {}", rem_cid);
-3974 let client_hello = state.client_hello.take().unwrap();
-3975 self.retry_src_cid = Some(rem_cid);
-3976 self.rem_cids
-3977 .get_mut(&path_id)
-3978 .expect("PathId::ZERO not yet abandoned, is_valid_retry would have been false")
-3979 .update_initial_cid(rem_cid);
-3980 self.rem_handshake_cid = rem_cid;
-3981
-3982 let space = &mut self.spaces[SpaceId::Initial];
-3983 if let Some(info) = space.for_path(PathId::ZERO).take(0) {
-3984 self.on_packet_acked(now, PathId::ZERO, info);
-3985 };
+3973 if !was_closed && self.state.is_closed() {
+3974 self.close_common();
+3975 if !self.state.is_drained() {
+3976 self.set_close_timer(now);
+3977 }
+3978 }
+3979 if !was_drained && self.state.is_drained() {
+3980 self.endpoint_events.push_back(EndpointEventInner::Drained);
+3981 // Close timer may have been started previously, e.g. if we sent a close and got a
+3982 // stateless reset in response
+3983 self.timers
+3984 .stop(Timer::Conn(ConnTimer::Close), self.qlog.with_time(now));
+3985 }
3986
-3987 self.discard_space(now, SpaceId::Initial); // Make sure we clean up after
-3988 // any retransmitted Initials
-3989 self.spaces[SpaceId::Initial] = {
-3990 let mut space = PacketSpace::new(now, SpaceId::Initial, &mut self.rng);
-3991 space.crypto = Some(self.crypto.initial_keys(rem_cid, self.side.side()));
-3992 space.crypto_offset = client_hello.len() as u64;
-3993 space.for_path(path_id).next_packet_number = self.spaces[SpaceId::Initial]
-3994 .for_path(path_id)
-3995 .next_packet_number;
-3996 space.pending.crypto.push_back(frame::Crypto {
-3997 offset: 0,
-3998 data: client_hello,
-3999 });
-4000 space
-4001 };
-4002
-4003 // Retransmit all 0-RTT data
-4004 let zero_rtt = mem::take(
-4005 &mut self.spaces[SpaceId::Data]
-4006 .for_path(PathId::ZERO)
-4007 .sent_packets,
-4008 );
-4009 for (_, info) in zero_rtt.into_iter() {
-4010 self.paths
-4011 .get_mut(&PathId::ZERO)
-4012 .unwrap()
-4013 .remove_in_flight(&info);
-4014 self.spaces[SpaceId::Data].pending |= info.retransmits;
-4015 }
-4016 self.streams.retransmit_all_for_0rtt();
-4017
-4018 let token_len = packet.payload.len() - 16;
-4019 let ConnectionSide::Client { ref mut token, .. } = self.side else {
-4020 unreachable!("we already short-circuited if we're server");
-4021 };
-4022 *token = packet.payload.freeze().split_to(token_len);
-4023
-4024 self.state = State::handshake(state::Handshake {
-4025 expected_token: Bytes::new(),
-4026 rem_cid_set: false,
-4027 client_hello: None,
-4028 allow_server_migration: true,
-4029 });
-4030 Ok(())
-4031 }
-4032 Header::Long {
-4033 ty: LongType::Handshake,
-4034 src_cid: rem_cid,
-4035 dst_cid: loc_cid,
-4036 ..
-4037 } => {
-4038 debug_assert_eq!(path_id, PathId::ZERO);
-4039 if rem_cid != self.rem_handshake_cid {
-4040 debug!(
-4041 "discarding packet with mismatched remote CID: {} != {}",
-4042 self.rem_handshake_cid, rem_cid
-4043 );
-4044 return Ok(());
-4045 }
-4046 self.on_path_validated(path_id);
+3987 // Transmit CONNECTION_CLOSE if necessary
+3988 if matches!(self.state.as_type(), StateType::Closed) {
+3989 // If there is no PathData for this PathId the packet was for a brand new
+3990 // path. It was a valid packet however, so the remote is valid and we want to
+3991 // send CONNECTION_CLOSE.
+3992 let path_remote = self
+3993 .paths
+3994 .get(&path_id)
+3995 .map(|p| p.data.network_path)
+3996 .unwrap_or(network_path);
+3997 self.close = network_path == path_remote;
+3998 }
+3999 }
+4000
+4001 fn process_decrypted_packet(
+4002 &mut self,
+4003 now: Instant,
+4004 network_path: FourTuple,
+4005 path_id: PathId,
+4006 number: Option<u64>,
+4007 packet: Packet,
+4008 qlog: &mut QlogRecvPacket,
+4009 ) -> Result<(), ConnectionError> {
+4010 if !self.paths.contains_key(&path_id) {
+4011 // There is a chance this is a server side, first (for this path) packet, which would
+4012 // be a protocol violation. It's more likely, however, that this is a packet of a
+4013 // pruned path
+4014 trace!(%path_id, ?number, "discarding packet for unknown path");
+4015 return Ok(());
+4016 }
+4017 let state = match self.state.as_type() {
+4018 StateType::Established => {
+4019 match packet.header.space() {
+4020 SpaceId::Data => self.process_payload(
+4021 now,
+4022 network_path,
+4023 path_id,
+4024 number.unwrap(),
+4025 packet,
+4026 qlog,
+4027 )?,
+4028 _ if packet.header.has_frames() => {
+4029 self.process_early_payload(now, path_id, packet, qlog)?
+4030 }
+4031 _ => {
+4032 trace!("discarding unexpected pre-handshake packet");
+4033 }
+4034 }
+4035 return Ok(());
+4036 }
+4037 StateType::Closed => {
+4038 for result in frame::Iter::new(packet.payload.freeze())? {
+4039 let frame = match result {
+4040 Ok(frame) => frame,
+4041 Err(err) => {
+4042 debug!("frame decoding error: {err:?}");
+4043 continue;
+4044 }
+4045 };
+4046 qlog.frame(&frame);
4047
-4048 self.process_early_payload(now, path_id, packet, qlog)?;
-4049 if self.state.is_closed() {
-4050 return Ok(());
-4051 }
-4052
-4053 if self.crypto.is_handshaking() {
-4054 trace!("handshake ongoing");
-4055 return Ok(());
-4056 }
-4057
-4058 if self.side.is_client() {
-4059 // Client-only because server params were set from the client's Initial
-4060 let params = self.crypto.transport_parameters()?.ok_or_else(|| {
-4061 TransportError::new(
-4062 TransportErrorCode::crypto(0x6d),
-4063 "transport parameters missing".to_owned(),
-4064 )
-4065 })?;
-4066
-4067 if self.has_0rtt() {
-4068 if !self.crypto.early_data_accepted().unwrap() {
-4069 debug_assert!(self.side.is_client());
-4070 debug!("0-RTT rejected");
-4071 self.accepted_0rtt = false;
-4072 self.streams.zero_rtt_rejected();
+4048 if let Frame::Padding = frame {
+4049 continue;
+4050 };
+4051
+4052 self.stats.frame_rx.record(frame.ty());
+4053
+4054 if let Frame::Close(_error) = frame {
+4055 self.state.move_to_draining(None);
+4056 break;
+4057 }
+4058 }
+4059 return Ok(());
+4060 }
+4061 StateType::Draining | StateType::Drained => return Ok(()),
+4062 StateType::Handshake => self.state.as_handshake_mut().expect("checked"),
+4063 };
+4064
+4065 match packet.header {
+4066 Header::Retry {
+4067 src_cid: rem_cid, ..
+4068 } => {
+4069 debug_assert_eq!(path_id, PathId::ZERO);
+4070 if self.side.is_server() {
+4071 return Err(TransportError::PROTOCOL_VIOLATION("client sent Retry").into());
+4072 }
4073
-4074 // Discard already-queued frames
-4075 self.spaces[SpaceId::Data].pending = Retransmits::default();
-4076
-4077 // Discard 0-RTT packets
-4078 let sent_packets = mem::take(
-4079 &mut self.spaces[SpaceId::Data].for_path(path_id).sent_packets,
-4080 );
-4081 for (_, packet) in sent_packets.into_iter() {
-4082 self.paths
-4083 .get_mut(&path_id)
-4084 .unwrap()
-4085 .remove_in_flight(&packet);
-4086 }
-4087 } else {
-4088 self.accepted_0rtt = true;
-4089 params.validate_resumption_from(&self.peer_params)?;
-4090 }
-4091 }
-4092 if let Some(token) = params.stateless_reset_token {
-4093 // TODO(matheus23): Reset token for a remote, or for a 4-tuple?
-4094 let remote = self.path_data(path_id).network_path.remote;
-4095 self.endpoint_events
-4096 .push_back(EndpointEventInner::ResetToken(path_id, remote, token));
-4097 }
-4098 self.handle_peer_params(params, loc_cid, rem_cid, now)?;
-4099 self.issue_first_cids(now);
-4100 } else {
-4101 // Server-only
-4102 self.spaces[SpaceId::Data].pending.handshake_done = true;
-4103 self.discard_space(now, SpaceId::Handshake);
-4104 self.events.push_back(Event::HandshakeConfirmed);
-4105 trace!("handshake confirmed");
-4106 }
-4107
-4108 self.events.push_back(Event::Connected);
-4109 self.state.move_to_established();
-4110 trace!("established");
-4111
-4112 // Multipath can only be enabled after the state has reached Established.
-4113 // So this can not happen any earlier.
-4114 self.issue_first_path_cids(now);
-4115 Ok(())
-4116 }
-4117 Header::Initial(InitialHeader {
-4118 src_cid: rem_cid,
-4119 dst_cid: loc_cid,
-4120 ..
-4121 }) => {
-4122 debug_assert_eq!(path_id, PathId::ZERO);
-4123 if !state.rem_cid_set {
-4124 trace!("switching remote CID to {}", rem_cid);
-4125 let mut state = state.clone();
-4126 self.rem_cids
-4127 .get_mut(&path_id)
-4128 .expect("PathId::ZERO not yet abandoned")
-4129 .update_initial_cid(rem_cid);
-4130 self.rem_handshake_cid = rem_cid;
-4131 self.orig_rem_cid = rem_cid;
-4132 state.rem_cid_set = true;
-4133 self.state.move_to_handshake(state);
-4134 } else if rem_cid != self.rem_handshake_cid {
-4135 debug!(
-4136 "discarding packet with mismatched remote CID: {} != {}",
-4137 self.rem_handshake_cid, rem_cid
-4138 );
-4139 return Ok(());
-4140 }
-4141
-4142 let starting_space = self.highest_space;
-4143 self.process_early_payload(now, path_id, packet, qlog)?;
-4144
-4145 if self.side.is_server()
-4146 && starting_space == SpaceId::Initial
-4147 && self.highest_space != SpaceId::Initial
-4148 {
-4149 let params = self.crypto.transport_parameters()?.ok_or_else(|| {
-4150 TransportError::new(
-4151 TransportErrorCode::crypto(0x6d),
-4152 "transport parameters missing".to_owned(),
-4153 )
-4154 })?;
-4155 self.handle_peer_params(params, loc_cid, rem_cid, now)?;
-4156 self.issue_first_cids(now);
-4157 self.init_0rtt(now);
-4158 }
-4159 Ok(())
-4160 }
-4161 Header::Long {
-4162 ty: LongType::ZeroRtt,
-4163 ..
-4164 } => {
-4165 self.process_payload(now, network_path, path_id, number.unwrap(), packet, qlog)?;
-4166 Ok(())
-4167 }
-4168 Header::VersionNegotiate { .. } => {
-4169 if self.total_authed_packets > 1 {
-4170 return Ok(());
-4171 }
-4172 let supported = packet
-4173 .payload
-4174 .chunks(4)
-4175 .any(|x| match <[u8; 4]>::try_from(x) {
-4176 Ok(version) => self.version == u32::from_be_bytes(version),
-4177 Err(_) => false,
-4178 });
-4179 if supported {
-4180 return Ok(());
-4181 }
-4182 debug!("remote doesn't support our version");
-4183 Err(ConnectionError::VersionMismatch)
-4184 }
-4185 Header::Short { .. } => unreachable!(
-4186 "short packets received during handshake are discarded in handle_packet"
-4187 ),
-4188 }
-4189 }
-4190
-4191 /// Process an Initial or Handshake packet payload
-4192 fn process_early_payload(
-4193 &mut self,
-4194 now: Instant,
-4195 path_id: PathId,
-4196 packet: Packet,
-4197 #[allow(unused)] qlog: &mut QlogRecvPacket,
-4198 ) -> Result<(), TransportError> {
-4199 debug_assert_ne!(packet.header.space(), SpaceId::Data);
-4200 debug_assert_eq!(path_id, PathId::ZERO);
-4201 let payload_len = packet.payload.len();
-4202 let mut ack_eliciting = false;
-4203 for result in frame::Iter::new(packet.payload.freeze())? {
-4204 let frame = result?;
-4205 qlog.frame(&frame);
-4206 let span = match frame {
-4207 Frame::Padding => continue,
-4208 _ => Some(trace_span!("frame", ty = %frame.ty(), path = tracing::field::Empty)),
-4209 };
-4210
-4211 self.stats.frame_rx.record(frame.ty());
-4212
-4213 let _guard = span.as_ref().map(|x| x.enter());
-4214 ack_eliciting |= frame.is_ack_eliciting();
-4215
-4216 // Process frames
-4217 if frame.is_1rtt() && packet.header.space() != SpaceId::Data {
-4218 return Err(TransportError::PROTOCOL_VIOLATION(
-4219 "illegal frame type in handshake",
-4220 ));
-4221 }
-4222
-4223 match frame {
-4224 Frame::Padding | Frame::Ping => {}
-4225 Frame::Crypto(frame) => {
-4226 self.read_crypto(packet.header.space(), &frame, payload_len)?;
-4227 }
-4228 Frame::Ack(ack) => {
-4229 self.on_ack_received(now, packet.header.space(), ack)?;
-4230 }
-4231 Frame::PathAck(ack) => {
-4232 span.as_ref()
-4233 .map(|span| span.record("path", tracing::field::debug(&ack.path_id)));
-4234 self.on_path_ack_received(now, packet.header.space(), ack)?;
-4235 }
-4236 Frame::Close(reason) => {
-4237 self.state.move_to_draining(Some(reason.into()));
-4238 return Ok(());
-4239 }
-4240 _ => {
-4241 let mut err =
-4242 TransportError::PROTOCOL_VIOLATION("illegal frame type in handshake");
-4243 err.frame = frame::MaybeFrame::Known(frame.ty());
-4244 return Err(err);
-4245 }
-4246 }
-4247 }
-4248
-4249 if ack_eliciting {
-4250 // In the initial and handshake spaces, ACKs must be sent immediately
-4251 self.spaces[packet.header.space()]
-4252 .for_path(path_id)
-4253 .pending_acks
-4254 .set_immediate_ack_required();
-4255 }
-4256
-4257 self.write_crypto();
-4258 Ok(())
-4259 }
-4260
-4261 /// Processes the packet payload, always in the data space.
-4262 fn process_payload(
-4263 &mut self,
-4264 now: Instant,
-4265 network_path: FourTuple,
-4266 path_id: PathId,
-4267 number: u64,
-4268 packet: Packet,
-4269 #[allow(unused)] qlog: &mut QlogRecvPacket,
-4270 ) -> Result<(), TransportError> {
-4271 let payload = packet.payload.freeze();
-4272 let mut is_probing_packet = true;
-4273 let mut close = None;
-4274 let payload_len = payload.len();
-4275 let mut ack_eliciting = false;
-4276 // if this packet triggers a path migration and includes a observed address frame, it's
-4277 // stored here
-4278 let mut migration_observed_addr = None;
-4279 for result in frame::Iter::new(payload)? {
-4280 let frame = result?;
-4281 qlog.frame(&frame);
-4282 let span = match frame {
-4283 Frame::Padding => continue,
-4284 _ => trace_span!("frame", ty = %frame.ty(), path = tracing::field::Empty),
-4285 };
-4286
-4287 self.stats.frame_rx.record(frame.ty());
-4288 // Crypto, Stream and Datagram frames are special cased in order no pollute
-4289 // the log with payload data
-4290 match &frame {
-4291 Frame::Crypto(f) => {
-4292 trace!(offset = f.offset, len = f.data.len(), "got crypto frame");
-4293 }
-4294 Frame::Stream(f) => {
-4295 trace!(id = %f.id, offset = f.offset, len = f.data.len(), fin = f.fin, "got stream frame");
-4296 }
-4297 Frame::Datagram(f) => {
-4298 trace!(len = f.data.len(), "got datagram frame");
+4074 let is_valid_retry = self
+4075 .rem_cids
+4076 .get(&path_id)
+4077 .map(|cids| cids.active())
+4078 .map(|orig_dst_cid| {
+4079 self.crypto.is_valid_retry(
+4080 orig_dst_cid,
+4081 &packet.header_data,
+4082 &packet.payload,
+4083 )
+4084 })
+4085 .unwrap_or_default();
+4086 if self.total_authed_packets > 1
+4087 || packet.payload.len() <= 16 // token + 16 byte tag
+4088 || !is_valid_retry
+4089 {
+4090 trace!("discarding invalid Retry");
+4091 // - After the client has received and processed an Initial or Retry
+4092 // packet from the server, it MUST discard any subsequent Retry
+4093 // packets that it receives.
+4094 // - A client MUST discard a Retry packet with a zero-length Retry Token
+4095 // field.
+4096 // - Clients MUST discard Retry packets that have a Retry Integrity Tag
+4097 // that cannot be validated
+4098 return Ok(());
+4099 }
+4100
+4101 trace!("retrying with CID {}", rem_cid);
+4102 let client_hello = state.client_hello.take().unwrap();
+4103 self.retry_src_cid = Some(rem_cid);
+4104 self.rem_cids
+4105 .get_mut(&path_id)
+4106 .expect("PathId::ZERO not yet abandoned, is_valid_retry would have been false")
+4107 .update_initial_cid(rem_cid);
+4108 self.rem_handshake_cid = rem_cid;
+4109
+4110 let space = &mut self.spaces[SpaceId::Initial];
+4111 if let Some(info) = space.for_path(PathId::ZERO).take(0) {
+4112 self.on_packet_acked(now, PathId::ZERO, info);
+4113 };
+4114
+4115 self.discard_space(now, SpaceId::Initial); // Make sure we clean up after
+4116 // any retransmitted Initials
+4117 self.spaces[SpaceId::Initial] = {
+4118 let mut space = PacketSpace::new(now, SpaceId::Initial, &mut self.rng);
+4119 space.crypto = Some(self.crypto.initial_keys(rem_cid, self.side.side()));
+4120 space.crypto_offset = client_hello.len() as u64;
+4121 space.for_path(path_id).next_packet_number = self.spaces[SpaceId::Initial]
+4122 .for_path(path_id)
+4123 .next_packet_number;
+4124 space.pending.crypto.push_back(frame::Crypto {
+4125 offset: 0,
+4126 data: client_hello,
+4127 });
+4128 space
+4129 };
+4130
+4131 // Retransmit all 0-RTT data
+4132 let zero_rtt = mem::take(
+4133 &mut self.spaces[SpaceId::Data]
+4134 .for_path(PathId::ZERO)
+4135 .sent_packets,
+4136 );
+4137 for (_, info) in zero_rtt.into_iter() {
+4138 self.paths
+4139 .get_mut(&PathId::ZERO)
+4140 .unwrap()
+4141 .remove_in_flight(&info);
+4142 self.spaces[SpaceId::Data].pending |= info.retransmits;
+4143 }
+4144 self.streams.retransmit_all_for_0rtt();
+4145
+4146 let token_len = packet.payload.len() - 16;
+4147 let ConnectionSide::Client { ref mut token, .. } = self.side else {
+4148 unreachable!("we already short-circuited if we're server");
+4149 };
+4150 *token = packet.payload.freeze().split_to(token_len);
+4151
+4152 self.state = State::handshake(state::Handshake {
+4153 expected_token: Bytes::new(),
+4154 rem_cid_set: false,
+4155 client_hello: None,
+4156 allow_server_migration: true,
+4157 });
+4158 Ok(())
+4159 }
+4160 Header::Long {
+4161 ty: LongType::Handshake,
+4162 src_cid: rem_cid,
+4163 dst_cid: loc_cid,
+4164 ..
+4165 } => {
+4166 debug_assert_eq!(path_id, PathId::ZERO);
+4167 if rem_cid != self.rem_handshake_cid {
+4168 debug!(
+4169 "discarding packet with mismatched remote CID: {} != {}",
+4170 self.rem_handshake_cid, rem_cid
+4171 );
+4172 return Ok(());
+4173 }
+4174 self.on_path_validated(path_id);
+4175
+4176 self.process_early_payload(now, path_id, packet, qlog)?;
+4177 if self.state.is_closed() {
+4178 return Ok(());
+4179 }
+4180
+4181 if self.crypto.is_handshaking() {
+4182 trace!("handshake ongoing");
+4183 return Ok(());
+4184 }
+4185
+4186 if self.side.is_client() {
+4187 // Client-only because server params were set from the client's Initial
+4188 let params = self.crypto.transport_parameters()?.ok_or_else(|| {
+4189 TransportError::new(
+4190 TransportErrorCode::crypto(0x6d),
+4191 "transport parameters missing".to_owned(),
+4192 )
+4193 })?;
+4194
+4195 if self.has_0rtt() {
+4196 if !self.crypto.early_data_accepted().unwrap() {
+4197 debug_assert!(self.side.is_client());
+4198 debug!("0-RTT rejected");
+4199 self.accepted_0rtt = false;
+4200 self.streams.zero_rtt_rejected();
+4201
+4202 // Discard already-queued frames
+4203 self.spaces[SpaceId::Data].pending = Retransmits::default();
+4204
+4205 // Discard 0-RTT packets
+4206 let sent_packets = mem::take(
+4207 &mut self.spaces[SpaceId::Data].for_path(path_id).sent_packets,
+4208 );
+4209 for (_, packet) in sent_packets.into_iter() {
+4210 self.paths
+4211 .get_mut(&path_id)
+4212 .unwrap()
+4213 .remove_in_flight(&packet);
+4214 }
+4215 } else {
+4216 self.accepted_0rtt = true;
+4217 params.validate_resumption_from(&self.peer_params)?;
+4218 }
+4219 }
+4220 if let Some(token) = params.stateless_reset_token {
+4221 // TODO(matheus23): Reset token for a remote, or for a 4-tuple?
+4222 let remote = self.path_data(path_id).network_path.remote;
+4223 self.endpoint_events
+4224 .push_back(EndpointEventInner::ResetToken(path_id, remote, token));
+4225 }
+4226 self.handle_peer_params(params, loc_cid, rem_cid, now)?;
+4227 self.issue_first_cids(now);
+4228 } else {
+4229 // Server-only
+4230 self.spaces[SpaceId::Data].pending.handshake_done = true;
+4231 self.discard_space(now, SpaceId::Handshake);
+4232 self.events.push_back(Event::HandshakeConfirmed);
+4233 trace!("handshake confirmed");
+4234 }
+4235
+4236 self.events.push_back(Event::Connected);
+4237 self.state.move_to_established();
+4238 trace!("established");
+4239
+4240 // Multipath can only be enabled after the state has reached Established.
+4241 // So this can not happen any earlier.
+4242 self.issue_first_path_cids(now);
+4243 Ok(())
+4244 }
+4245 Header::Initial(InitialHeader {
+4246 src_cid: rem_cid,
+4247 dst_cid: loc_cid,
+4248 ..
+4249 }) => {
+4250 debug_assert_eq!(path_id, PathId::ZERO);
+4251 if !state.rem_cid_set {
+4252 trace!("switching remote CID to {}", rem_cid);
+4253 let mut state = state.clone();
+4254 self.rem_cids
+4255 .get_mut(&path_id)
+4256 .expect("PathId::ZERO not yet abandoned")
+4257 .update_initial_cid(rem_cid);
+4258 self.rem_handshake_cid = rem_cid;
+4259 self.orig_rem_cid = rem_cid;
+4260 state.rem_cid_set = true;
+4261 self.state.move_to_handshake(state);
+4262 } else if rem_cid != self.rem_handshake_cid {
+4263 debug!(
+4264 "discarding packet with mismatched remote CID: {} != {}",
+4265 self.rem_handshake_cid, rem_cid
+4266 );
+4267 return Ok(());
+4268 }
+4269
+4270 let starting_space = self.highest_space;
+4271 self.process_early_payload(now, path_id, packet, qlog)?;
+4272
+4273 if self.side.is_server()
+4274 && starting_space == SpaceId::Initial
+4275 && self.highest_space != SpaceId::Initial
+4276 {
+4277 let params = self.crypto.transport_parameters()?.ok_or_else(|| {
+4278 TransportError::new(
+4279 TransportErrorCode::crypto(0x6d),
+4280 "transport parameters missing".to_owned(),
+4281 )
+4282 })?;
+4283 self.handle_peer_params(params, loc_cid, rem_cid, now)?;
+4284 self.issue_first_cids(now);
+4285 self.init_0rtt(now);
+4286 }
+4287 Ok(())
+4288 }
+4289 Header::Long {
+4290 ty: LongType::ZeroRtt,
+4291 ..
+4292 } => {
+4293 self.process_payload(now, network_path, path_id, number.unwrap(), packet, qlog)?;
+4294 Ok(())
+4295 }
+4296 Header::VersionNegotiate { .. } => {
+4297 if self.total_authed_packets > 1 {
+4298 return Ok(());
4299 }
-4300 f => {
-4301 trace!("got frame {f}");
-4302 }
-4303 }
-4304
-4305 let _guard = span.enter();
-4306 if packet.header.is_0rtt() {
-4307 match frame {
-4308 Frame::Crypto(_) | Frame::Close(Close::Application(_)) => {
-4309 return Err(TransportError::PROTOCOL_VIOLATION(
-4310 "illegal frame type in 0-RTT",
-4311 ));
-4312 }
-4313 _ => {
-4314 if frame.is_1rtt() {
-4315 return Err(TransportError::PROTOCOL_VIOLATION(
-4316 "illegal frame type in 0-RTT",
-4317 ));
-4318 }
-4319 }
-4320 }
-4321 }
-4322 ack_eliciting |= frame.is_ack_eliciting();
-4323
-4324 // Check whether this could be a probing packet
-4325 match frame {
-4326 Frame::Padding
-4327 | Frame::PathChallenge(_)
-4328 | Frame::PathResponse(_)
-4329 | Frame::NewConnectionId(_)
-4330 | Frame::ObservedAddr(_) => {}
-4331 _ => {
-4332 is_probing_packet = false;
-4333 }
-4334 }
-4335
-4336 match frame {
-4337 Frame::Crypto(frame) => {
-4338 self.read_crypto(SpaceId::Data, &frame, payload_len)?;
-4339 }
-4340 Frame::Stream(frame) => {
-4341 if self.streams.received(frame, payload_len)?.should_transmit() {
-4342 self.spaces[SpaceId::Data].pending.max_data = true;
-4343 }
-4344 }
-4345 Frame::Ack(ack) => {
-4346 self.on_ack_received(now, SpaceId::Data, ack)?;
-4347 }
-4348 Frame::PathAck(ack) => {
-4349 span.record("path", tracing::field::debug(&ack.path_id));
-4350 self.on_path_ack_received(now, SpaceId::Data, ack)?;
-4351 }
+4300 let supported = packet
+4301 .payload
+4302 .chunks(4)
+4303 .any(|x| match <[u8; 4]>::try_from(x) {
+4304 Ok(version) => self.version == u32::from_be_bytes(version),
+4305 Err(_) => false,
+4306 });
+4307 if supported {
+4308 return Ok(());
+4309 }
+4310 debug!("remote doesn't support our version");
+4311 Err(ConnectionError::VersionMismatch)
+4312 }
+4313 Header::Short { .. } => unreachable!(
+4314 "short packets received during handshake are discarded in handle_packet"
+4315 ),
+4316 }
+4317 }
+4318
+4319 /// Process an Initial or Handshake packet payload
+4320 fn process_early_payload(
+4321 &mut self,
+4322 now: Instant,
+4323 path_id: PathId,
+4324 packet: Packet,
+4325 #[allow(unused)] qlog: &mut QlogRecvPacket,
+4326 ) -> Result<(), TransportError> {
+4327 debug_assert_ne!(packet.header.space(), SpaceId::Data);
+4328 debug_assert_eq!(path_id, PathId::ZERO);
+4329 let payload_len = packet.payload.len();
+4330 let mut ack_eliciting = false;
+4331 for result in frame::Iter::new(packet.payload.freeze())? {
+4332 let frame = result?;
+4333 qlog.frame(&frame);
+4334 let span = match frame {
+4335 Frame::Padding => continue,
+4336 _ => Some(trace_span!("frame", ty = %frame.ty(), path = tracing::field::Empty)),
+4337 };
+4338
+4339 self.stats.frame_rx.record(frame.ty());
+4340
+4341 let _guard = span.as_ref().map(|x| x.enter());
+4342 ack_eliciting |= frame.is_ack_eliciting();
+4343
+4344 // Process frames
+4345 if frame.is_1rtt() && packet.header.space() != SpaceId::Data {
+4346 return Err(TransportError::PROTOCOL_VIOLATION(
+4347 "illegal frame type in handshake",
+4348 ));
+4349 }
+4350
+4351 match frame {
4352 Frame::Padding | Frame::Ping => {}
-4353 Frame::Close(reason) => {
-4354 close = Some(reason);
+4353 Frame::Crypto(frame) => {
+4354 self.read_crypto(packet.header.space(), &frame, payload_len)?;
4355 }
-4356 Frame::PathChallenge(challenge) => {
-4357 let path = &mut self
-4358 .path_mut(path_id)
-4359 .expect("payload is processed only after the path becomes known");
-4360 path.path_responses.push(number, challenge.0, network_path);
-4361 // At this point, update_network_path_or_discard was already called, so
-4362 // we don't need to be lenient about `local_ip` possibly mis-matching.
-4363 if network_path == path.network_path {
-4364 // PATH_CHALLENGE on active path, possible off-path packet forwarding
-4365 // attack. Send a non-probing packet to recover the active path.
-4366 // TODO(flub): No longer true! We now path_challege also to validate
-4367 // the path if the path is new, without an RFC9000-style
-4368 // migration involved. This means we add in an extra
-4369 // IMMEDIATE_ACK on some challenges. It isn't really wrong to do
-4370 // so, but it still is something untidy. We should instead
-4371 // suppress this when we know the remote is still validating the
-4372 // path.
-4373 match self.peer_supports_ack_frequency() {
-4374 true => self.immediate_ack(path_id),
-4375 false => {
-4376 self.ping_path(path_id).ok();
-4377 }
-4378 }
-4379 }
-4380 }
-4381 Frame::PathResponse(response) => {
-4382 let path = self
-4383 .paths
-4384 .get_mut(&path_id)
-4385 .expect("payload is processed only after the path becomes known");
-4386
-4387 use PathTimer::*;
-4388 use paths::OnPathResponseReceived::*;
-4389 match path
-4390 .data
-4391 .on_path_response_received(now, response.0, network_path)
-4392 {
-4393 OnPath { was_open } => {
-4394 let qlog = self.qlog.with_time(now);
-4395
-4396 self.timers
-4397 .stop(Timer::PerPath(path_id, PathValidation), qlog.clone());
-4398 self.timers
-4399 .stop(Timer::PerPath(path_id, PathOpen), qlog.clone());
-4400
-4401 let next_challenge = path
-4402 .data
-4403 .earliest_expiring_challenge()
-4404 .map(|time| time + self.ack_frequency.max_ack_delay_for_pto());
-4405 self.timers.set_or_stop(
-4406 Timer::PerPath(path_id, PathChallengeLost),
-4407 next_challenge,
-4408 qlog,
-4409 );
-4410
-4411 if !was_open {
-4412 self.events
-4413 .push_back(Event::Path(PathEvent::Opened { id: path_id }));
-4414 if let Some(observed) = path.data.last_observed_addr_report.as_ref()
-4415 {
-4416 self.events.push_back(Event::Path(PathEvent::ObservedAddr {
-4417 id: path_id,
-4418 addr: observed.socket_addr(),
-4419 }));
-4420 }
-4421 }
-4422 if let Some((_, ref mut prev)) = path.prev {
-4423 prev.challenges_sent.clear();
-4424 prev.send_new_challenge = false;
-4425 }
-4426 }
-4427 OffPath => {
-4428 debug!("Response to off-path PathChallenge!");
-4429 let next_challenge = path
-4430 .data
-4431 .earliest_expiring_challenge()
-4432 .map(|time| time + self.ack_frequency.max_ack_delay_for_pto());
-4433 self.timers.set_or_stop(
-4434 Timer::PerPath(path_id, PathChallengeLost),
-4435 next_challenge,
-4436 self.qlog.with_time(now),
-4437 );
-4438 }
-4439 Invalid { expected } => {
-4440 debug!(%response, %network_path, %expected, "ignoring invalid PATH_RESPONSE")
-4441 }
-4442 Unknown => debug!(%response, "ignoring invalid PATH_RESPONSE"),
-4443 }
-4444 }
-4445 Frame::MaxData(frame::MaxData(bytes)) => {
-4446 self.streams.received_max_data(bytes);
-4447 }
-4448 Frame::MaxStreamData(frame::MaxStreamData { id, offset }) => {
-4449 self.streams.received_max_stream_data(id, offset)?;
-4450 }
-4451 Frame::MaxStreams(frame::MaxStreams { dir, count }) => {
-4452 self.streams.received_max_streams(dir, count)?;
-4453 }
-4454 Frame::ResetStream(frame) => {
-4455 if self.streams.received_reset(frame)?.should_transmit() {
-4456 self.spaces[SpaceId::Data].pending.max_data = true;
-4457 }
-4458 }
-4459 Frame::DataBlocked { offset } => {
-4460 debug!(offset, "peer claims to be blocked at connection level");
+4356 Frame::Ack(ack) => {
+4357 self.on_ack_received(now, packet.header.space(), ack)?;
+4358 }
+4359 Frame::PathAck(ack) => {
+4360 span.as_ref()
+4361 .map(|span| span.record("path", tracing::field::debug(&ack.path_id)));
+4362 self.on_path_ack_received(now, packet.header.space(), ack)?;
+4363 }
+4364 Frame::Close(reason) => {
+4365 self.state.move_to_draining(Some(reason.into()));
+4366 return Ok(());
+4367 }
+4368 _ => {
+4369 let mut err =
+4370 TransportError::PROTOCOL_VIOLATION("illegal frame type in handshake");
+4371 err.frame = frame::MaybeFrame::Known(frame.ty());
+4372 return Err(err);
+4373 }
+4374 }
+4375 }
+4376
+4377 if ack_eliciting {
+4378 // In the initial and handshake spaces, ACKs must be sent immediately
+4379 self.spaces[packet.header.space()]
+4380 .for_path(path_id)
+4381 .pending_acks
+4382 .set_immediate_ack_required();
+4383 }
+4384
+4385 self.write_crypto();
+4386 Ok(())
+4387 }
+4388
+4389 /// Processes the packet payload, always in the data space.
+4390 fn process_payload(
+4391 &mut self,
+4392 now: Instant,
+4393 network_path: FourTuple,
+4394 path_id: PathId,
+4395 number: u64,
+4396 packet: Packet,
+4397 #[allow(unused)] qlog: &mut QlogRecvPacket,
+4398 ) -> Result<(), TransportError> {
+4399 let payload = packet.payload.freeze();
+4400 let mut is_probing_packet = true;
+4401 let mut close = None;
+4402 let payload_len = payload.len();
+4403 let mut ack_eliciting = false;
+4404 // if this packet triggers a path migration and includes a observed address frame, it's
+4405 // stored here
+4406 let mut migration_observed_addr = None;
+4407 for result in frame::Iter::new(payload)? {
+4408 let frame = result?;
+4409 qlog.frame(&frame);
+4410 let span = match frame {
+4411 Frame::Padding => continue,
+4412 _ => trace_span!("frame", ty = %frame.ty(), path = tracing::field::Empty),
+4413 };
+4414
+4415 self.stats.frame_rx.record(frame.ty());
+4416 // Crypto, Stream and Datagram frames are special cased in order no pollute
+4417 // the log with payload data
+4418 match &frame {
+4419 Frame::Crypto(f) => {
+4420 trace!(offset = f.offset, len = f.data.len(), "got crypto frame");
+4421 }
+4422 Frame::Stream(f) => {
+4423 trace!(id = %f.id, offset = f.offset, len = f.data.len(), fin = f.fin, "got stream frame");
+4424 }
+4425 Frame::Datagram(f) => {
+4426 trace!(len = f.data.len(), "got datagram frame");
+4427 }
+4428 f => {
+4429 trace!("got frame {f}");
+4430 }
+4431 }
+4432
+4433 let _guard = span.enter();
+4434 if packet.header.is_0rtt() {
+4435 match frame {
+4436 Frame::Crypto(_) | Frame::Close(Close::Application(_)) => {
+4437 return Err(TransportError::PROTOCOL_VIOLATION(
+4438 "illegal frame type in 0-RTT",
+4439 ));
+4440 }
+4441 _ => {
+4442 if frame.is_1rtt() {
+4443 return Err(TransportError::PROTOCOL_VIOLATION(
+4444 "illegal frame type in 0-RTT",
+4445 ));
+4446 }
+4447 }
+4448 }
+4449 }
+4450 ack_eliciting |= frame.is_ack_eliciting();
+4451
+4452 // Check whether this could be a probing packet
+4453 match frame {
+4454 Frame::Padding
+4455 | Frame::PathChallenge(_)
+4456 | Frame::PathResponse(_)
+4457 | Frame::NewConnectionId(_)
+4458 | Frame::ObservedAddr(_) => {}
+4459 _ => {
+4460 is_probing_packet = false;
4461 }
-4462 Frame::StreamDataBlocked { id, offset } => {
-4463 if id.initiator() == self.side.side() && id.dir() == Dir::Uni {
-4464 debug!("got STREAM_DATA_BLOCKED on send-only {}", id);
-4465 return Err(TransportError::STREAM_STATE_ERROR(
-4466 "STREAM_DATA_BLOCKED on send-only stream",
-4467 ));
-4468 }
-4469 debug!(
-4470 stream = %id,
-4471 offset, "peer claims to be blocked at stream level"
-4472 );
-4473 }
-4474 Frame::StreamsBlocked { dir, limit } => {
-4475 if limit > MAX_STREAM_COUNT {
-4476 return Err(TransportError::FRAME_ENCODING_ERROR(
-4477 "unrepresentable stream limit",
-4478 ));
-4479 }
-4480 debug!(
-4481 "peer claims to be blocked opening more than {} {} streams",
-4482 limit, dir
-4483 );
-4484 }
-4485 Frame::StopSending(frame::StopSending { id, error_code }) => {
-4486 if id.initiator() != self.side.side() {
-4487 if id.dir() == Dir::Uni {
-4488 debug!("got STOP_SENDING on recv-only {}", id);
-4489 return Err(TransportError::STREAM_STATE_ERROR(
-4490 "STOP_SENDING on recv-only stream",
-4491 ));
-4492 }
-4493 } else if self.streams.is_local_unopened(id) {
-4494 return Err(TransportError::STREAM_STATE_ERROR(
-4495 "STOP_SENDING on unopened stream",
-4496 ));
-4497 }
-4498 self.streams.received_stop_sending(id, error_code);
-4499 }
-4500 Frame::RetireConnectionId(frame::RetireConnectionId { path_id, sequence }) => {
-4501 if let Some(ref path_id) = path_id {
-4502 span.record("path", tracing::field::debug(&path_id));
-4503 }
-4504 let path_id = path_id.unwrap_or_default();
-4505 match self.local_cid_state.get_mut(&path_id) {
-4506 None => error!(?path_id, "RETIRE_CONNECTION_ID for unknown path"),
-4507 Some(cid_state) => {
-4508 let allow_more_cids = cid_state
-4509 .on_cid_retirement(sequence, self.peer_params.issue_cids_limit())?;
-4510
-4511 // If the path has closed, we do not issue more CIDs for this path
-4512 // For details see https://www.ietf.org/archive/id/draft-ietf-quic-multipath-17.html#section-3.2.2
-4513 // > an endpoint SHOULD provide new connection IDs for that path, if still open, using PATH_NEW_CONNECTION_ID frames.
-4514 let has_path = !self.abandoned_paths.contains(&path_id);
-4515 let allow_more_cids = allow_more_cids && has_path;
-4516
-4517 self.endpoint_events
-4518 .push_back(EndpointEventInner::RetireConnectionId(
-4519 now,
-4520 path_id,
-4521 sequence,
-4522 allow_more_cids,
-4523 ));
-4524 }
-4525 }
-4526 }
-4527 Frame::NewConnectionId(frame) => {
-4528 let path_id = if let Some(path_id) = frame.path_id {
-4529 if !self.is_multipath_negotiated() {
-4530 return Err(TransportError::PROTOCOL_VIOLATION(
-4531 "received PATH_NEW_CONNECTION_ID frame when multipath was not negotiated",
-4532 ));
-4533 }
-4534 if path_id > self.local_max_path_id {
-4535 return Err(TransportError::PROTOCOL_VIOLATION(
-4536 "PATH_NEW_CONNECTION_ID contains path_id exceeding current max",
-4537 ));
-4538 }
-4539 path_id
-4540 } else {
-4541 PathId::ZERO
-4542 };
-4543
-4544 if self.abandoned_paths.contains(&path_id) {
-4545 trace!("ignoring issued CID for abandoned path");
-4546 continue;
-4547 }
-4548 if let Some(ref path_id) = frame.path_id {
-4549 span.record("path", tracing::field::debug(&path_id));
-4550 }
-4551 let rem_cids = self
-4552 .rem_cids
-4553 .entry(path_id)
-4554 .or_insert_with(|| CidQueue::new(frame.id));
-4555 if rem_cids.active().is_empty() {
-4556 return Err(TransportError::PROTOCOL_VIOLATION(
-4557 "NEW_CONNECTION_ID when CIDs aren't in use",
-4558 ));
-4559 }
-4560 if frame.retire_prior_to > frame.sequence {
-4561 return Err(TransportError::PROTOCOL_VIOLATION(
-4562 "NEW_CONNECTION_ID retiring unissued CIDs",
-4563 ));
-4564 }
-4565
-4566 use crate::cid_queue::InsertError;
-4567 match rem_cids.insert(frame) {
-4568 Ok(None) if self.path(path_id).is_none() => {
-4569 // if this gives us CIDs to open a new path and a nat traversal attempt
-4570 // is underway we could try to probe a pending remote
-4571 self.continue_nat_traversal_round(now);
-4572 }
-4573 Ok(None) => {}
-4574 Ok(Some((retired, reset_token))) => {
-4575 let pending_retired =
-4576 &mut self.spaces[SpaceId::Data].pending.retire_cids;
-4577 /// Ensure `pending_retired` cannot grow without bound. Limit is
-4578 /// somewhat arbitrary but very permissive.
-4579 const MAX_PENDING_RETIRED_CIDS: u64 = CidQueue::LEN as u64 * 10;
-4580 // We don't bother counting in-flight frames because those are bounded
-4581 // by congestion control.
-4582 if (pending_retired.len() as u64)
-4583 .saturating_add(retired.end.saturating_sub(retired.start))
-4584 > MAX_PENDING_RETIRED_CIDS
-4585 {
-4586 return Err(TransportError::CONNECTION_ID_LIMIT_ERROR(
-4587 "queued too many retired CIDs",
-4588 ));
-4589 }
-4590 pending_retired.extend(retired.map(|seq| (path_id, seq)));
-4591 // TODO(matheus23): Reset token for a remote or a full 4-tuple?
-4592 self.set_reset_token(path_id, network_path.remote, reset_token);
-4593 }
-4594 Err(InsertError::ExceedsLimit) => {
-4595 return Err(TransportError::CONNECTION_ID_LIMIT_ERROR(""));
-4596 }
-4597 Err(InsertError::Retired) => {
-4598 trace!("discarding already-retired");
-4599 // RETIRE_CONNECTION_ID might not have been previously sent if e.g. a
-4600 // range of connection IDs larger than the active connection ID limit
-4601 // was retired all at once via retire_prior_to.
-4602 self.spaces[SpaceId::Data]
-4603 .pending
-4604 .retire_cids
-4605 .push((path_id, frame.sequence));
-4606 continue;
-4607 }
-4608 };
-4609
-4610 if self.side.is_server()
-4611 && path_id == PathId::ZERO
-4612 && self
-4613 .rem_cids
-4614 .get(&PathId::ZERO)
-4615 .map(|cids| cids.active_seq() == 0)
-4616 .unwrap_or_default()
-4617 {
-4618 // We're a server still using the initial remote CID for the client, so
-4619 // let's switch immediately to enable clientside stateless resets.
-4620 self.update_rem_cid(PathId::ZERO);
-4621 }
-4622 }
-4623 Frame::NewToken(NewToken { token }) => {
-4624 let ConnectionSide::Client {
-4625 token_store,
-4626 server_name,
-4627 ..
-4628 } = &self.side
-4629 else {
-4630 return Err(TransportError::PROTOCOL_VIOLATION("client sent NEW_TOKEN"));
-4631 };
-4632 if token.is_empty() {
-4633 return Err(TransportError::FRAME_ENCODING_ERROR("empty token"));
-4634 }
-4635 trace!("got new token");
-4636 token_store.insert(server_name, token);
-4637 }
-4638 Frame::Datagram(datagram) => {
-4639 if self
-4640 .datagrams
-4641 .received(datagram, &self.config.datagram_receive_buffer_size)?
-4642 {
-4643 self.events.push_back(Event::DatagramReceived);
-4644 }
-4645 }
-4646 Frame::AckFrequency(ack_frequency) => {
-4647 // This frame can only be sent in the Data space
-4648
-4649 if !self.ack_frequency.ack_frequency_received(&ack_frequency)? {
-4650 // The AckFrequency frame is stale (we have already received a more
-4651 // recent one)
-4652 continue;
+4462 }
+4463
+4464 match frame {
+4465 Frame::Crypto(frame) => {
+4466 self.read_crypto(SpaceId::Data, &frame, payload_len)?;
+4467 }
+4468 Frame::Stream(frame) => {
+4469 if self.streams.received(frame, payload_len)?.should_transmit() {
+4470 self.spaces[SpaceId::Data].pending.max_data = true;
+4471 }
+4472 }
+4473 Frame::Ack(ack) => {
+4474 self.on_ack_received(now, SpaceId::Data, ack)?;
+4475 }
+4476 Frame::PathAck(ack) => {
+4477 span.record("path", tracing::field::debug(&ack.path_id));
+4478 self.on_path_ack_received(now, SpaceId::Data, ack)?;
+4479 }
+4480 Frame::Padding | Frame::Ping => {}
+4481 Frame::Close(reason) => {
+4482 close = Some(reason);
+4483 }
+4484 Frame::PathChallenge(challenge) => {
+4485 let path = &mut self
+4486 .path_mut(path_id)
+4487 .expect("payload is processed only after the path becomes known");
+4488 path.path_responses.push(number, challenge.0, network_path);
+4489 // At this point, update_network_path_or_discard was already called, so
+4490 // we don't need to be lenient about `local_ip` possibly mis-matching.
+4491 if network_path == path.network_path {
+4492 // PATH_CHALLENGE on active path, possible off-path packet forwarding
+4493 // attack. Send a non-probing packet to recover the active path.
+4494 // TODO(flub): No longer true! We now path_challege also to validate
+4495 // the path if the path is new, without an RFC9000-style
+4496 // migration involved. This means we add in an extra
+4497 // IMMEDIATE_ACK on some challenges. It isn't really wrong to do
+4498 // so, but it still is something untidy. We should instead
+4499 // suppress this when we know the remote is still validating the
+4500 // path.
+4501 match self.peer_supports_ack_frequency() {
+4502 true => self.immediate_ack(path_id),
+4503 false => {
+4504 self.ping_path(path_id).ok();
+4505 }
+4506 }
+4507 }
+4508 }
+4509 Frame::PathResponse(response) => {
+4510 let path = self
+4511 .paths
+4512 .get_mut(&path_id)
+4513 .expect("payload is processed only after the path becomes known");
+4514
+4515 use PathTimer::*;
+4516 use paths::OnPathResponseReceived::*;
+4517 match path
+4518 .data
+4519 .on_path_response_received(now, response.0, network_path)
+4520 {
+4521 OnPath { was_open } => {
+4522 let qlog = self.qlog.with_time(now);
+4523
+4524 self.timers
+4525 .stop(Timer::PerPath(path_id, PathValidation), qlog.clone());
+4526 self.timers
+4527 .stop(Timer::PerPath(path_id, PathOpen), qlog.clone());
+4528
+4529 let next_challenge = path
+4530 .data
+4531 .earliest_expiring_challenge()
+4532 .map(|time| time + self.ack_frequency.max_ack_delay_for_pto());
+4533 self.timers.set_or_stop(
+4534 Timer::PerPath(path_id, PathChallengeLost),
+4535 next_challenge,
+4536 qlog,
+4537 );
+4538
+4539 if !was_open {
+4540 self.events
+4541 .push_back(Event::Path(PathEvent::Opened { id: path_id }));
+4542 if let Some(observed) = path.data.last_observed_addr_report.as_ref()
+4543 {
+4544 self.events.push_back(Event::Path(PathEvent::ObservedAddr {
+4545 id: path_id,
+4546 addr: observed.socket_addr(),
+4547 }));
+4548 }
+4549 }
+4550 if let Some((_, ref mut prev)) = path.prev {
+4551 prev.challenges_sent.clear();
+4552 prev.send_new_challenge = false;
+4553 }
+4554 }
+4555 OffPath => {
+4556 debug!("Response to off-path PathChallenge!");
+4557 let next_challenge = path
+4558 .data
+4559 .earliest_expiring_challenge()
+4560 .map(|time| time + self.ack_frequency.max_ack_delay_for_pto());
+4561 self.timers.set_or_stop(
+4562 Timer::PerPath(path_id, PathChallengeLost),
+4563 next_challenge,
+4564 self.qlog.with_time(now),
+4565 );
+4566 }
+4567 Invalid { expected } => {
+4568 debug!(%response, %network_path, %expected, "ignoring invalid PATH_RESPONSE")
+4569 }
+4570 Unknown => debug!(%response, "ignoring invalid PATH_RESPONSE"),
+4571 }
+4572 }
+4573 Frame::MaxData(frame::MaxData(bytes)) => {
+4574 self.streams.received_max_data(bytes);
+4575 }
+4576 Frame::MaxStreamData(frame::MaxStreamData { id, offset }) => {
+4577 self.streams.received_max_stream_data(id, offset)?;
+4578 }
+4579 Frame::MaxStreams(frame::MaxStreams { dir, count }) => {
+4580 self.streams.received_max_streams(dir, count)?;
+4581 }
+4582 Frame::ResetStream(frame) => {
+4583 if self.streams.received_reset(frame)?.should_transmit() {
+4584 self.spaces[SpaceId::Data].pending.max_data = true;
+4585 }
+4586 }
+4587 Frame::DataBlocked { offset } => {
+4588 debug!(offset, "peer claims to be blocked at connection level");
+4589 }
+4590 Frame::StreamDataBlocked { id, offset } => {
+4591 if id.initiator() == self.side.side() && id.dir() == Dir::Uni {
+4592 debug!("got STREAM_DATA_BLOCKED on send-only {}", id);
+4593 return Err(TransportError::STREAM_STATE_ERROR(
+4594 "STREAM_DATA_BLOCKED on send-only stream",
+4595 ));
+4596 }
+4597 debug!(
+4598 stream = %id,
+4599 offset, "peer claims to be blocked at stream level"
+4600 );
+4601 }
+4602 Frame::StreamsBlocked { dir, limit } => {
+4603 if limit > MAX_STREAM_COUNT {
+4604 return Err(TransportError::FRAME_ENCODING_ERROR(
+4605 "unrepresentable stream limit",
+4606 ));
+4607 }
+4608 debug!(
+4609 "peer claims to be blocked opening more than {} {} streams",
+4610 limit, dir
+4611 );
+4612 }
+4613 Frame::StopSending(frame::StopSending { id, error_code }) => {
+4614 if id.initiator() != self.side.side() {
+4615 if id.dir() == Dir::Uni {
+4616 debug!("got STOP_SENDING on recv-only {}", id);
+4617 return Err(TransportError::STREAM_STATE_ERROR(
+4618 "STOP_SENDING on recv-only stream",
+4619 ));
+4620 }
+4621 } else if self.streams.is_local_unopened(id) {
+4622 return Err(TransportError::STREAM_STATE_ERROR(
+4623 "STOP_SENDING on unopened stream",
+4624 ));
+4625 }
+4626 self.streams.received_stop_sending(id, error_code);
+4627 }
+4628 Frame::RetireConnectionId(frame::RetireConnectionId { path_id, sequence }) => {
+4629 if let Some(ref path_id) = path_id {
+4630 span.record("path", tracing::field::debug(&path_id));
+4631 }
+4632 let path_id = path_id.unwrap_or_default();
+4633 match self.local_cid_state.get_mut(&path_id) {
+4634 None => error!(?path_id, "RETIRE_CONNECTION_ID for unknown path"),
+4635 Some(cid_state) => {
+4636 let allow_more_cids = cid_state
+4637 .on_cid_retirement(sequence, self.peer_params.issue_cids_limit())?;
+4638
+4639 // If the path has closed, we do not issue more CIDs for this path
+4640 // For details see https://www.ietf.org/archive/id/draft-ietf-quic-multipath-17.html#section-3.2.2
+4641 // > an endpoint SHOULD provide new connection IDs for that path, if still open, using PATH_NEW_CONNECTION_ID frames.
+4642 let has_path = !self.abandoned_paths.contains(&path_id);
+4643 let allow_more_cids = allow_more_cids && has_path;
+4644
+4645 self.endpoint_events
+4646 .push_back(EndpointEventInner::RetireConnectionId(
+4647 now,
+4648 path_id,
+4649 sequence,
+4650 allow_more_cids,
+4651 ));
+4652 }
4653 }
-4654
-4655 // Update the params for all of our paths
-4656 for (path_id, space) in self.spaces[SpaceId::Data].number_spaces.iter_mut() {
-4657 space.pending_acks.set_ack_frequency_params(&ack_frequency);
-4658
-4659 // Our `max_ack_delay` has been updated, so we may need to adjust
-4660 // its associated timeout
-4661 if let Some(timeout) = space
-4662 .pending_acks
-4663 .max_ack_delay_timeout(self.ack_frequency.max_ack_delay)
-4664 {
-4665 self.timers.set(
-4666 Timer::PerPath(*path_id, PathTimer::MaxAckDelay),
-4667 timeout,
-4668 self.qlog.with_time(now),
-4669 );
-4670 }
-4671 }
-4672 }
-4673 Frame::ImmediateAck => {
-4674 // This frame can only be sent in the Data space
-4675 for pns in self.spaces[SpaceId::Data].iter_paths_mut() {
-4676 pns.pending_acks.set_immediate_ack_required();
-4677 }
-4678 }
-4679 Frame::HandshakeDone => {
-4680 if self.side.is_server() {
-4681 return Err(TransportError::PROTOCOL_VIOLATION(
-4682 "client sent HANDSHAKE_DONE",
-4683 ));
-4684 }
-4685 if self.spaces[SpaceId::Handshake].crypto.is_some() {
-4686 self.discard_space(now, SpaceId::Handshake);
+4654 }
+4655 Frame::NewConnectionId(frame) => {
+4656 let path_id = if let Some(path_id) = frame.path_id {
+4657 if !self.is_multipath_negotiated() {
+4658 return Err(TransportError::PROTOCOL_VIOLATION(
+4659 "received PATH_NEW_CONNECTION_ID frame when multipath was not negotiated",
+4660 ));
+4661 }
+4662 if path_id > self.local_max_path_id {
+4663 return Err(TransportError::PROTOCOL_VIOLATION(
+4664 "PATH_NEW_CONNECTION_ID contains path_id exceeding current max",
+4665 ));
+4666 }
+4667 path_id
+4668 } else {
+4669 PathId::ZERO
+4670 };
+4671
+4672 if self.abandoned_paths.contains(&path_id) {
+4673 trace!("ignoring issued CID for abandoned path");
+4674 continue;
+4675 }
+4676 if let Some(ref path_id) = frame.path_id {
+4677 span.record("path", tracing::field::debug(&path_id));
+4678 }
+4679 let rem_cids = self
+4680 .rem_cids
+4681 .entry(path_id)
+4682 .or_insert_with(|| CidQueue::new(frame.id));
+4683 if rem_cids.active().is_empty() {
+4684 return Err(TransportError::PROTOCOL_VIOLATION(
+4685 "NEW_CONNECTION_ID when CIDs aren't in use",
+4686 ));
4687 }
-4688 self.events.push_back(Event::HandshakeConfirmed);
-4689 trace!("handshake confirmed");
-4690 }
-4691 Frame::ObservedAddr(observed) => {
-4692 // check if params allows the peer to send report and this node to receive it
-4693 trace!(seq_no = %observed.seq_no, ip = %observed.ip, port = observed.port);
-4694 if !self
-4695 .peer_params
-4696 .address_discovery_role
-4697 .should_report(&self.config.address_discovery_role)
-4698 {
-4699 return Err(TransportError::PROTOCOL_VIOLATION(
-4700 "received OBSERVED_ADDRESS frame when not negotiated",
-4701 ));
-4702 }
-4703 // must only be sent in data space
-4704 if packet.header.space() != SpaceId::Data {
-4705 return Err(TransportError::PROTOCOL_VIOLATION(
-4706 "OBSERVED_ADDRESS frame outside data space",
-4707 ));
-4708 }
-4709
-4710 let path = self.path_data_mut(path_id);
-4711 if network_path == path.network_path {
-4712 if let Some(updated) = path.update_observed_addr_report(observed) {
-4713 if path.open {
-4714 self.events.push_back(Event::Path(PathEvent::ObservedAddr {
-4715 id: path_id,
-4716 addr: updated,
-4717 }));
-4718 }
-4719 // otherwise the event is reported when the path is deemed open
-4720 }
-4721 } else {
-4722 // include in migration
-4723 migration_observed_addr = Some(observed)
-4724 }
-4725 }
-4726 Frame::PathAbandon(frame::PathAbandon {
-4727 path_id,
-4728 error_code,
-4729 }) => {
-4730 span.record("path", tracing::field::debug(&path_id));
-4731 // TODO(flub): don't really know which error code to use here.
-4732 match self.close_path(now, path_id, error_code.into()) {
-4733 Ok(()) => {
-4734 trace!("peer abandoned path");
+4688 if frame.retire_prior_to > frame.sequence {
+4689 return Err(TransportError::PROTOCOL_VIOLATION(
+4690 "NEW_CONNECTION_ID retiring unissued CIDs",
+4691 ));
+4692 }
+4693
+4694 use crate::cid_queue::InsertError;
+4695 match rem_cids.insert(frame) {
+4696 Ok(None) if self.path(path_id).is_none() => {
+4697 // if this gives us CIDs to open a new path and a nat traversal attempt
+4698 // is underway we could try to probe a pending remote
+4699 self.continue_nat_traversal_round(now);
+4700 }
+4701 Ok(None) => {}
+4702 Ok(Some((retired, reset_token))) => {
+4703 let pending_retired =
+4704 &mut self.spaces[SpaceId::Data].pending.retire_cids;
+4705 /// Ensure `pending_retired` cannot grow without bound. Limit is
+4706 /// somewhat arbitrary but very permissive.
+4707 const MAX_PENDING_RETIRED_CIDS: u64 = CidQueue::LEN as u64 * 10;
+4708 // We don't bother counting in-flight frames because those are bounded
+4709 // by congestion control.
+4710 if (pending_retired.len() as u64)
+4711 .saturating_add(retired.end.saturating_sub(retired.start))
+4712 > MAX_PENDING_RETIRED_CIDS
+4713 {
+4714 return Err(TransportError::CONNECTION_ID_LIMIT_ERROR(
+4715 "queued too many retired CIDs",
+4716 ));
+4717 }
+4718 pending_retired.extend(retired.map(|seq| (path_id, seq)));
+4719 // TODO(matheus23): Reset token for a remote or a full 4-tuple?
+4720 self.set_reset_token(path_id, network_path.remote, reset_token);
+4721 }
+4722 Err(InsertError::ExceedsLimit) => {
+4723 return Err(TransportError::CONNECTION_ID_LIMIT_ERROR(""));
+4724 }
+4725 Err(InsertError::Retired) => {
+4726 trace!("discarding already-retired");
+4727 // RETIRE_CONNECTION_ID might not have been previously sent if e.g. a
+4728 // range of connection IDs larger than the active connection ID limit
+4729 // was retired all at once via retire_prior_to.
+4730 self.spaces[SpaceId::Data]
+4731 .pending
+4732 .retire_cids
+4733 .push((path_id, frame.sequence));
+4734 continue;
4735 }
-4736 Err(ClosePathError::LastOpenPath) => {
-4737 trace!("peer abandoned last path, closing connection");
-4738 return Err(TransportError::NO_VIABLE_PATH(
-4739 "last path abandoned by peer",
-4740 ));
-4741 }
-4742 Err(ClosePathError::ClosedPath) => {
-4743 trace!("peer abandoned already closed path");
-4744 }
-4745 };
-4746 // If we receive a retransmit of PATH_ABANDON then we may already have
-4747 // abandoned this path locally. In that case the DiscardPath timer
-4748 // may already have fired and we no longer have any state for this path.
-4749 // Only set this timer if we still have path state.
-4750 if let Some(path) = self.paths.get_mut(&path_id) {
-4751 if !matches!(path.data.abandon_state, AbandonState::ReceivedPathAbandon) {
-4752 let ack_delay = self.ack_frequency.max_ack_delay_for_pto();
-4753 let pto = path.data.rtt.pto_base() + ack_delay;
-4754 self.timers.set(
-4755 Timer::PerPath(path_id, PathTimer::DiscardPath),
-4756 now + 3 * pto,
-4757 self.qlog.with_time(now),
-4758 );
-4759 // We received a PATH_ABANDON, we don't expect another one by a certain time.
-4760 path.data.abandon_state = AbandonState::ReceivedPathAbandon;
-4761 }
+4736 };
+4737
+4738 if self.side.is_server()
+4739 && path_id == PathId::ZERO
+4740 && self
+4741 .rem_cids
+4742 .get(&PathId::ZERO)
+4743 .map(|cids| cids.active_seq() == 0)
+4744 .unwrap_or_default()
+4745 {
+4746 // We're a server still using the initial remote CID for the client, so
+4747 // let's switch immediately to enable clientside stateless resets.
+4748 self.update_rem_cid(PathId::ZERO);
+4749 }
+4750 }
+4751 Frame::NewToken(NewToken { token }) => {
+4752 let ConnectionSide::Client {
+4753 token_store,
+4754 server_name,
+4755 ..
+4756 } = &self.side
+4757 else {
+4758 return Err(TransportError::PROTOCOL_VIOLATION("client sent NEW_TOKEN"));
+4759 };
+4760 if token.is_empty() {
+4761 return Err(TransportError::FRAME_ENCODING_ERROR("empty token"));
4762 }
-4763 }
-4764 Frame::PathStatusAvailable(info) => {
-4765 span.record("path", tracing::field::debug(&info.path_id));
-4766 if self.is_multipath_negotiated() {
-4767 self.on_path_status(
-4768 info.path_id,
-4769 PathStatus::Available,
-4770 info.status_seq_no,
-4771 );
-4772 } else {
-4773 return Err(TransportError::PROTOCOL_VIOLATION(
-4774 "received PATH_STATUS_AVAILABLE frame when multipath was not negotiated",
-4775 ));
-4776 }
-4777 }
-4778 Frame::PathStatusBackup(info) => {
-4779 span.record("path", tracing::field::debug(&info.path_id));
-4780 if self.is_multipath_negotiated() {
-4781 self.on_path_status(info.path_id, PathStatus::Backup, info.status_seq_no);
-4782 } else {
-4783 return Err(TransportError::PROTOCOL_VIOLATION(
-4784 "received PATH_STATUS_BACKUP frame when multipath was not negotiated",
-4785 ));
-4786 }
-4787 }
-4788 Frame::MaxPathId(frame::MaxPathId(path_id)) => {
-4789 span.record("path", tracing::field::debug(&path_id));
-4790 if !self.is_multipath_negotiated() {
-4791 return Err(TransportError::PROTOCOL_VIOLATION(
-4792 "received MAX_PATH_ID frame when multipath was not negotiated",
-4793 ));
-4794 }
-4795 // frames that do not increase the path id are ignored
-4796 if path_id > self.remote_max_path_id {
-4797 self.remote_max_path_id = path_id;
-4798 self.issue_first_path_cids(now);
-4799 while let Some(true) = self.continue_nat_traversal_round(now) {}
-4800 }
-4801 }
-4802 Frame::PathsBlocked(frame::PathsBlocked(max_path_id)) => {
-4803 // Receipt of a value of Maximum Path Identifier or Path Identifier that is higher than the local maximum value MUST
-4804 // be treated as a connection error of type PROTOCOL_VIOLATION.
-4805 // Ref <https://www.ietf.org/archive/id/draft-ietf-quic-multipath-14.html#name-paths_blocked-and-path_cids>
-4806 if self.is_multipath_negotiated() {
-4807 if max_path_id > self.local_max_path_id {
-4808 return Err(TransportError::PROTOCOL_VIOLATION(
-4809 "PATHS_BLOCKED maximum path identifier was larger than local maximum",
-4810 ));
-4811 }
-4812 debug!("received PATHS_BLOCKED({:?})", max_path_id);
-4813 // TODO(@divma): ensure max concurrent paths
-4814 } else {
-4815 return Err(TransportError::PROTOCOL_VIOLATION(
-4816 "received PATHS_BLOCKED frame when not multipath was not negotiated",
-4817 ));
-4818 }
-4819 }
-4820 Frame::PathCidsBlocked(frame::PathCidsBlocked { path_id, next_seq }) => {
-4821 // Nothing to do. This is recorded in the frame stats, but otherwise we
-4822 // always issue all CIDs we're allowed to issue, so either this is an
-4823 // impatient peer or a bug on our side.
-4824
-4825 // Receipt of a value of Maximum Path Identifier or Path Identifier that is higher than the local maximum value MUST
-4826 // be treated as a connection error of type PROTOCOL_VIOLATION.
-4827 // Ref <https://www.ietf.org/archive/id/draft-ietf-quic-multipath-14.html#name-paths_blocked-and-path_cids>
-4828 if self.is_multipath_negotiated() {
-4829 if path_id > self.local_max_path_id {
-4830 return Err(TransportError::PROTOCOL_VIOLATION(
-4831 "PATH_CIDS_BLOCKED path identifier was larger than local maximum",
-4832 ));
-4833 }
-4834 if next_seq.0
-4835 > self
-4836 .local_cid_state
-4837 .get(&path_id)
-4838 .map(|cid_state| cid_state.active_seq().1 + 1)
-4839 .unwrap_or_default()
-4840 {
-4841 return Err(TransportError::PROTOCOL_VIOLATION(
-4842 "PATH_CIDS_BLOCKED next sequence number larger than in local state",
-4843 ));
-4844 }
-4845 debug!(%path_id, %next_seq, "received PATH_CIDS_BLOCKED");
-4846 } else {
-4847 return Err(TransportError::PROTOCOL_VIOLATION(
-4848 "received PATH_CIDS_BLOCKED frame when not multipath was not negotiated",
-4849 ));
-4850 }
-4851 }
-4852 Frame::AddAddress(addr) => {
-4853 let client_state = match self.iroh_hp.client_side_mut() {
-4854 Ok(state) => state,
-4855 Err(err) => {
-4856 return Err(TransportError::PROTOCOL_VIOLATION(format!(
-4857 "Nat traversal(ADD_ADDRESS): {err}"
-4858 )));
-4859 }
-4860 };
-4861
-4862 if !client_state.check_remote_address(&addr) {
-4863 // if the address is not valid we flag it, but update anyway
-4864 warn!(?addr, "server sent illegal ADD_ADDRESS frame");
-4865 }
-4866
-4867 match client_state.add_remote_address(addr) {
-4868 Ok(maybe_added) => {
-4869 if let Some(added) = maybe_added {
-4870 self.events.push_back(Event::NatTraversal(
-4871 iroh_hp::Event::AddressAdded(added),
-4872 ));
-4873 }
-4874 }
-4875 Err(e) => {
-4876 warn!(%e, "failed to add remote address")
-4877 }
-4878 }
-4879 }
-4880 Frame::RemoveAddress(addr) => {
-4881 let client_state = match self.iroh_hp.client_side_mut() {
-4882 Ok(state) => state,
-4883 Err(err) => {
-4884 return Err(TransportError::PROTOCOL_VIOLATION(format!(
-4885 "Nat traversal(REMOVE_ADDRESS): {err}"
-4886 )));
-4887 }
-4888 };
-4889 if let Some(removed_addr) = client_state.remove_remote_address(addr) {
-4890 self.events
-4891 .push_back(Event::NatTraversal(iroh_hp::Event::AddressRemoved(
-4892 removed_addr,
-4893 )));
-4894 }
-4895 }
-4896 Frame::ReachOut(reach_out) => {
-4897 let server_state = match self.iroh_hp.server_side_mut() {
-4898 Ok(state) => state,
-4899 Err(err) => {
-4900 return Err(TransportError::PROTOCOL_VIOLATION(format!(
-4901 "Nat traversal(REACH_OUT): {err}"
-4902 )));
-4903 }
-4904 };
-4905
-4906 if let Err(err) = server_state.handle_reach_out(reach_out) {
-4907 return Err(TransportError::PROTOCOL_VIOLATION(format!(
-4908 "Nat traversal(REACH_OUT): {err}"
-4909 )));
-4910 }
-4911 }
-4912 }
-4913 }
-4914
-4915 let space = self.spaces[SpaceId::Data].for_path(path_id);
-4916 if space
-4917 .pending_acks
-4918 .packet_received(now, number, ack_eliciting, &space.dedup)
-4919 {
-4920 if self.abandoned_paths.contains(&path_id) {
-4921 // § 3.4.3 QUIC-MULTIPATH: promptly send ACKs for packets received from
-4922 // abandoned paths.
-4923 space.pending_acks.set_immediate_ack_required();
-4924 } else {
-4925 self.timers.set(
-4926 Timer::PerPath(path_id, PathTimer::MaxAckDelay),
-4927 now + self.ack_frequency.max_ack_delay,
-4928 self.qlog.with_time(now),
-4929 );
-4930 }
-4931 }
-4932
-4933 // Issue stream ID credit due to ACKs of outgoing finish/resets and incoming finish/resets
-4934 // on stopped streams. Incoming finishes/resets on open streams are not handled here as they
-4935 // are only freed, and hence only issue credit, once the application has been notified
-4936 // during a read on the stream.
-4937 let pending = &mut self.spaces[SpaceId::Data].pending;
-4938 self.streams.queue_max_stream_id(pending);
-4939
-4940 if let Some(reason) = close {
-4941 self.state.move_to_draining(Some(reason.into()));
-4942 self.close = true;
-4943 }
-4944
-4945 if Some(number) == self.spaces[SpaceId::Data].for_path(path_id).rx_packet
-4946 && !is_probing_packet
-4947 && network_path != self.path_data(path_id).network_path
-4948 {
-4949 let ConnectionSide::Server { ref server_config } = self.side else {
-4950 panic!("packets from unknown remote should be dropped by clients");
-4951 };
-4952 debug_assert!(
-4953 server_config.migration,
-4954 "migration-initiating packets should have been dropped immediately"
-4955 );
-4956 self.migrate(path_id, now, network_path, migration_observed_addr);
-4957 // Break linkability, if possible
-4958 self.update_rem_cid(path_id);
-4959 self.spin = false;
-4960 }
-4961
-4962 Ok(())
-4963 }
-4964
-4965 fn migrate(
-4966 &mut self,
-4967 path_id: PathId,
-4968 now: Instant,
-4969 network_path: FourTuple,
-4970 observed_addr: Option<ObservedAddr>,
-4971 ) {
-4972 trace!(%network_path, %path_id, "migration initiated");
-4973 self.path_generation_counter = self.path_generation_counter.wrapping_add(1);
-4974 // TODO(@divma): conditions for path migration in multipath are very specific, check them
-4975 // again to prevent path migrations that should actually create a new path
-4976
-4977 // Reset rtt/congestion state for new path unless it looks like a NAT rebinding.
-4978 // Note that the congestion window will not grow until validation terminates. Helps mitigate
-4979 // amplification attacks performed by spoofing source addresses.
-4980 let prev_pto = self.pto(SpaceId::Data, path_id);
-4981 let known_path = self.paths.get_mut(&path_id).expect("known path");
-4982 let path = &mut known_path.data;
-4983 let mut new_path = if network_path.remote.is_ipv4()
-4984 && network_path.remote.ip() == path.network_path.remote.ip()
-4985 {
-4986 PathData::from_previous(network_path, path, self.path_generation_counter, now)
-4987 } else {
-4988 let peer_max_udp_payload_size =
-4989 u16::try_from(self.peer_params.max_udp_payload_size.into_inner())
-4990 .unwrap_or(u16::MAX);
-4991 PathData::new(
-4992 network_path,
-4993 self.allow_mtud,
-4994 Some(peer_max_udp_payload_size),
-4995 self.path_generation_counter,
-4996 now,
-4997 &self.config,
-4998 )
-4999 };
-5000 new_path.last_observed_addr_report = path.last_observed_addr_report.clone();
-5001 if let Some(report) = observed_addr {
-5002 if let Some(updated) = new_path.update_observed_addr_report(report) {
-5003 tracing::info!("adding observed addr event from migration");
-5004 self.events.push_back(Event::Path(PathEvent::ObservedAddr {
-5005 id: path_id,
-5006 addr: updated,
-5007 }));
-5008 }
-5009 }
-5010 new_path.send_new_challenge = true;
-5011
-5012 let mut prev = mem::replace(path, new_path);
-5013 // Don't clobber the original path if the previous one hasn't been validated yet
-5014 if !prev.is_validating_path() {
-5015 prev.send_new_challenge = true;
-5016 // We haven't updated the remote CID yet, this captures the remote CID we were using on
-5017 // the previous path.
-5018
-5019 known_path.prev = Some((self.rem_cids.get(&path_id).unwrap().active(), prev));
-5020 }
-5021
-5022 // We need to re-assign the correct remote to this path in qlog
-5023 self.qlog.emit_tuple_assigned(path_id, network_path, now);
-5024
-5025 self.timers.set(
-5026 Timer::PerPath(path_id, PathTimer::PathValidation),
-5027 now + 3 * cmp::max(self.pto(SpaceId::Data, path_id), prev_pto),
-5028 self.qlog.with_time(now),
-5029 );
-5030 }
-5031
-5032 /// Handle a change in the local address, i.e. an active migration
-5033 pub fn local_address_changed(&mut self) {
-5034 // TODO(flub): if multipath is enabled this needs to create a new path entirely.
-5035 self.update_rem_cid(PathId::ZERO);
-5036 self.ping();
-5037 }
-5038
-5039 /// Switch to a previously unused remote connection ID, if possible
-5040 fn update_rem_cid(&mut self, path_id: PathId) {
-5041 let Some((reset_token, retired)) =
-5042 self.rem_cids.get_mut(&path_id).and_then(|cids| cids.next())
-5043 else {
-5044 return;
-5045 };
-5046
-5047 // Retire the current remote CID and any CIDs we had to skip.
-5048 self.spaces[SpaceId::Data]
-5049 .pending
-5050 .retire_cids
-5051 .extend(retired.map(|seq| (path_id, seq)));
-5052 let remote = self.path_data(path_id).network_path.remote;
-5053 self.set_reset_token(path_id, remote, reset_token);
-5054 }
-5055
-5056 /// Sends this reset token to the endpoint
-5057 ///
-5058 /// The endpoint needs to know the reset tokens issued by the peer, so that if the peer
-5059 /// sends a reset token it knows to route it to this connection. See RFC 9000 section
-5060 /// 10.3. Stateless Reset.
-5061 ///
-5062 /// Reset tokens are different for each path, the endpoint identifies paths by peer
-5063 /// socket address however, not by path ID.
-5064 fn set_reset_token(&mut self, path_id: PathId, remote: SocketAddr, reset_token: ResetToken) {
-5065 self.endpoint_events
-5066 .push_back(EndpointEventInner::ResetToken(path_id, remote, reset_token));
+4763 trace!("got new token");
+4764 token_store.insert(server_name, token);
+4765 }
+4766 Frame::Datagram(datagram) => {
+4767 if self
+4768 .datagrams
+4769 .received(datagram, &self.config.datagram_receive_buffer_size)?
+4770 {
+4771 self.events.push_back(Event::DatagramReceived);
+4772 }
+4773 }
+4774 Frame::AckFrequency(ack_frequency) => {
+4775 // This frame can only be sent in the Data space
+4776
+4777 if !self.ack_frequency.ack_frequency_received(&ack_frequency)? {
+4778 // The AckFrequency frame is stale (we have already received a more
+4779 // recent one)
+4780 continue;
+4781 }
+4782
+4783 // Update the params for all of our paths
+4784 for (path_id, space) in self.spaces[SpaceId::Data].number_spaces.iter_mut() {
+4785 space.pending_acks.set_ack_frequency_params(&ack_frequency);
+4786
+4787 // Our `max_ack_delay` has been updated, so we may need to adjust
+4788 // its associated timeout
+4789 if let Some(timeout) = space
+4790 .pending_acks
+4791 .max_ack_delay_timeout(self.ack_frequency.max_ack_delay)
+4792 {
+4793 self.timers.set(
+4794 Timer::PerPath(*path_id, PathTimer::MaxAckDelay),
+4795 timeout,
+4796 self.qlog.with_time(now),
+4797 );
+4798 }
+4799 }
+4800 }
+4801 Frame::ImmediateAck => {
+4802 // This frame can only be sent in the Data space
+4803 for pns in self.spaces[SpaceId::Data].iter_paths_mut() {
+4804 pns.pending_acks.set_immediate_ack_required();
+4805 }
+4806 }
+4807 Frame::HandshakeDone => {
+4808 if self.side.is_server() {
+4809 return Err(TransportError::PROTOCOL_VIOLATION(
+4810 "client sent HANDSHAKE_DONE",
+4811 ));
+4812 }
+4813 if self.spaces[SpaceId::Handshake].crypto.is_some() {
+4814 self.discard_space(now, SpaceId::Handshake);
+4815 }
+4816 self.events.push_back(Event::HandshakeConfirmed);
+4817 trace!("handshake confirmed");
+4818 }
+4819 Frame::ObservedAddr(observed) => {
+4820 // check if params allows the peer to send report and this node to receive it
+4821 trace!(seq_no = %observed.seq_no, ip = %observed.ip, port = observed.port);
+4822 if !self
+4823 .peer_params
+4824 .address_discovery_role
+4825 .should_report(&self.config.address_discovery_role)
+4826 {
+4827 return Err(TransportError::PROTOCOL_VIOLATION(
+4828 "received OBSERVED_ADDRESS frame when not negotiated",
+4829 ));
+4830 }
+4831 // must only be sent in data space
+4832 if packet.header.space() != SpaceId::Data {
+4833 return Err(TransportError::PROTOCOL_VIOLATION(
+4834 "OBSERVED_ADDRESS frame outside data space",
+4835 ));
+4836 }
+4837
+4838 let path = self.path_data_mut(path_id);
+4839 if network_path == path.network_path {
+4840 if let Some(updated) = path.update_observed_addr_report(observed) {
+4841 if path.open {
+4842 self.events.push_back(Event::Path(PathEvent::ObservedAddr {
+4843 id: path_id,
+4844 addr: updated,
+4845 }));
+4846 }
+4847 // otherwise the event is reported when the path is deemed open
+4848 }
+4849 } else {
+4850 // include in migration
+4851 migration_observed_addr = Some(observed)
+4852 }
+4853 }
+4854 Frame::PathAbandon(frame::PathAbandon {
+4855 path_id,
+4856 error_code,
+4857 }) => {
+4858 span.record("path", tracing::field::debug(&path_id));
+4859 // TODO(flub): don't really know which error code to use here.
+4860 match self.close_path(now, path_id, error_code.into()) {
+4861 Ok(()) => {
+4862 trace!("peer abandoned path");
+4863 }
+4864 Err(ClosePathError::LastOpenPath) => {
+4865 trace!("peer abandoned last path, closing connection");
+4866 return Err(TransportError::NO_VIABLE_PATH(
+4867 "last path abandoned by peer",
+4868 ));
+4869 }
+4870 Err(ClosePathError::ClosedPath) => {
+4871 trace!("peer abandoned already closed path");
+4872 }
+4873 };
+4874 // If we receive a retransmit of PATH_ABANDON then we may already have
+4875 // abandoned this path locally. In that case the DiscardPath timer
+4876 // may already have fired and we no longer have any state for this path.
+4877 // Only set this timer if we still have path state.
+4878 if let Some(path) = self.paths.get_mut(&path_id) {
+4879 if !matches!(path.data.abandon_state, AbandonState::ReceivedPathAbandon) {
+4880 let ack_delay = self.ack_frequency.max_ack_delay_for_pto();
+4881 let pto = path.data.rtt.pto_base() + ack_delay;
+4882 self.timers.set(
+4883 Timer::PerPath(path_id, PathTimer::DiscardPath),
+4884 now + 3 * pto,
+4885 self.qlog.with_time(now),
+4886 );
+4887 // We received a PATH_ABANDON, we don't expect another one by a certain time.
+4888 path.data.abandon_state = AbandonState::ReceivedPathAbandon;
+4889 }
+4890 }
+4891 }
+4892 Frame::PathStatusAvailable(info) => {
+4893 span.record("path", tracing::field::debug(&info.path_id));
+4894 if self.is_multipath_negotiated() {
+4895 self.on_path_status(
+4896 info.path_id,
+4897 PathStatus::Available,
+4898 info.status_seq_no,
+4899 );
+4900 } else {
+4901 return Err(TransportError::PROTOCOL_VIOLATION(
+4902 "received PATH_STATUS_AVAILABLE frame when multipath was not negotiated",
+4903 ));
+4904 }
+4905 }
+4906 Frame::PathStatusBackup(info) => {
+4907 span.record("path", tracing::field::debug(&info.path_id));
+4908 if self.is_multipath_negotiated() {
+4909 self.on_path_status(info.path_id, PathStatus::Backup, info.status_seq_no);
+4910 } else {
+4911 return Err(TransportError::PROTOCOL_VIOLATION(
+4912 "received PATH_STATUS_BACKUP frame when multipath was not negotiated",
+4913 ));
+4914 }
+4915 }
+4916 Frame::MaxPathId(frame::MaxPathId(path_id)) => {
+4917 span.record("path", tracing::field::debug(&path_id));
+4918 if !self.is_multipath_negotiated() {
+4919 return Err(TransportError::PROTOCOL_VIOLATION(
+4920 "received MAX_PATH_ID frame when multipath was not negotiated",
+4921 ));
+4922 }
+4923 // frames that do not increase the path id are ignored
+4924 if path_id > self.remote_max_path_id {
+4925 self.remote_max_path_id = path_id;
+4926 self.issue_first_path_cids(now);
+4927 while let Some(true) = self.continue_nat_traversal_round(now) {}
+4928 }
+4929 }
+4930 Frame::PathsBlocked(frame::PathsBlocked(max_path_id)) => {
+4931 // Receipt of a value of Maximum Path Identifier or Path Identifier that is higher than the local maximum value MUST
+4932 // be treated as a connection error of type PROTOCOL_VIOLATION.
+4933 // Ref <https://www.ietf.org/archive/id/draft-ietf-quic-multipath-14.html#name-paths_blocked-and-path_cids>
+4934 if self.is_multipath_negotiated() {
+4935 if max_path_id > self.local_max_path_id {
+4936 return Err(TransportError::PROTOCOL_VIOLATION(
+4937 "PATHS_BLOCKED maximum path identifier was larger than local maximum",
+4938 ));
+4939 }
+4940 debug!("received PATHS_BLOCKED({:?})", max_path_id);
+4941 // TODO(@divma): ensure max concurrent paths
+4942 } else {
+4943 return Err(TransportError::PROTOCOL_VIOLATION(
+4944 "received PATHS_BLOCKED frame when not multipath was not negotiated",
+4945 ));
+4946 }
+4947 }
+4948 Frame::PathCidsBlocked(frame::PathCidsBlocked { path_id, next_seq }) => {
+4949 // Nothing to do. This is recorded in the frame stats, but otherwise we
+4950 // always issue all CIDs we're allowed to issue, so either this is an
+4951 // impatient peer or a bug on our side.
+4952
+4953 // Receipt of a value of Maximum Path Identifier or Path Identifier that is higher than the local maximum value MUST
+4954 // be treated as a connection error of type PROTOCOL_VIOLATION.
+4955 // Ref <https://www.ietf.org/archive/id/draft-ietf-quic-multipath-14.html#name-paths_blocked-and-path_cids>
+4956 if self.is_multipath_negotiated() {
+4957 if path_id > self.local_max_path_id {
+4958 return Err(TransportError::PROTOCOL_VIOLATION(
+4959 "PATH_CIDS_BLOCKED path identifier was larger than local maximum",
+4960 ));
+4961 }
+4962 if next_seq.0
+4963 > self
+4964 .local_cid_state
+4965 .get(&path_id)
+4966 .map(|cid_state| cid_state.active_seq().1 + 1)
+4967 .unwrap_or_default()
+4968 {
+4969 return Err(TransportError::PROTOCOL_VIOLATION(
+4970 "PATH_CIDS_BLOCKED next sequence number larger than in local state",
+4971 ));
+4972 }
+4973 debug!(%path_id, %next_seq, "received PATH_CIDS_BLOCKED");
+4974 } else {
+4975 return Err(TransportError::PROTOCOL_VIOLATION(
+4976 "received PATH_CIDS_BLOCKED frame when not multipath was not negotiated",
+4977 ));
+4978 }
+4979 }
+4980 Frame::AddAddress(addr) => {
+4981 let client_state = match self.iroh_hp.client_side_mut() {
+4982 Ok(state) => state,
+4983 Err(err) => {
+4984 return Err(TransportError::PROTOCOL_VIOLATION(format!(
+4985 "Nat traversal(ADD_ADDRESS): {err}"
+4986 )));
+4987 }
+4988 };
+4989
+4990 if !client_state.check_remote_address(&addr) {
+4991 // if the address is not valid we flag it, but update anyway
+4992 warn!(?addr, "server sent illegal ADD_ADDRESS frame");
+4993 }
+4994
+4995 match client_state.add_remote_address(addr) {
+4996 Ok(maybe_added) => {
+4997 if let Some(added) = maybe_added {
+4998 self.events.push_back(Event::NatTraversal(
+4999 iroh_hp::Event::AddressAdded(added),
+5000 ));
+5001 }
+5002 }
+5003 Err(e) => {
+5004 warn!(%e, "failed to add remote address")
+5005 }
+5006 }
+5007 }
+5008 Frame::RemoveAddress(addr) => {
+5009 let client_state = match self.iroh_hp.client_side_mut() {
+5010 Ok(state) => state,
+5011 Err(err) => {
+5012 return Err(TransportError::PROTOCOL_VIOLATION(format!(
+5013 "Nat traversal(REMOVE_ADDRESS): {err}"
+5014 )));
+5015 }
+5016 };
+5017 if let Some(removed_addr) = client_state.remove_remote_address(addr) {
+5018 self.events
+5019 .push_back(Event::NatTraversal(iroh_hp::Event::AddressRemoved(
+5020 removed_addr,
+5021 )));
+5022 }
+5023 }
+5024 Frame::ReachOut(reach_out) => {
+5025 let server_state = match self.iroh_hp.server_side_mut() {
+5026 Ok(state) => state,
+5027 Err(err) => {
+5028 return Err(TransportError::PROTOCOL_VIOLATION(format!(
+5029 "Nat traversal(REACH_OUT): {err}"
+5030 )));
+5031 }
+5032 };
+5033
+5034 if let Err(err) = server_state.handle_reach_out(reach_out) {
+5035 return Err(TransportError::PROTOCOL_VIOLATION(format!(
+5036 "Nat traversal(REACH_OUT): {err}"
+5037 )));
+5038 }
+5039 }
+5040 }
+5041 }
+5042
+5043 let space = self.spaces[SpaceId::Data].for_path(path_id);
+5044 if space
+5045 .pending_acks
+5046 .packet_received(now, number, ack_eliciting, &space.dedup)
+5047 {
+5048 if self.abandoned_paths.contains(&path_id) {
+5049 // § 3.4.3 QUIC-MULTIPATH: promptly send ACKs for packets received from
+5050 // abandoned paths.
+5051 space.pending_acks.set_immediate_ack_required();
+5052 } else {
+5053 self.timers.set(
+5054 Timer::PerPath(path_id, PathTimer::MaxAckDelay),
+5055 now + self.ack_frequency.max_ack_delay,
+5056 self.qlog.with_time(now),
+5057 );
+5058 }
+5059 }
+5060
+5061 // Issue stream ID credit due to ACKs of outgoing finish/resets and incoming finish/resets
+5062 // on stopped streams. Incoming finishes/resets on open streams are not handled here as they
+5063 // are only freed, and hence only issue credit, once the application has been notified
+5064 // during a read on the stream.
+5065 let pending = &mut self.spaces[SpaceId::Data].pending;
+5066 self.streams.queue_max_stream_id(pending);
5067
-5068 // During the handshake the server sends a reset token in the transport
-5069 // parameters. When we are the client and we receive the reset token during the
-5070 // handshake we want this to affect our peer transport parameters.
-5071 // TODO(flub): Pretty sure this is pointless, the entire params is overwritten
-5072 // shortly after this was called. And then the params don't have this anymore.
-5073 if path_id == PathId::ZERO {
-5074 self.peer_params.stateless_reset_token = Some(reset_token);
-5075 }
-5076 }
-5077
-5078 /// Issue an initial set of connection IDs to the peer upon connection
-5079 fn issue_first_cids(&mut self, now: Instant) {
-5080 if self
-5081 .local_cid_state
-5082 .get(&PathId::ZERO)
-5083 .expect("PathId::ZERO exists when the connection is created")
-5084 .cid_len()
-5085 == 0
-5086 {
-5087 return;
+5068 if let Some(reason) = close {
+5069 self.state.move_to_draining(Some(reason.into()));
+5070 self.close = true;
+5071 }
+5072
+5073 if Some(number) == self.spaces[SpaceId::Data].for_path(path_id).rx_packet
+5074 && !is_probing_packet
+5075 && network_path != self.path_data(path_id).network_path
+5076 {
+5077 let ConnectionSide::Server { ref server_config } = self.side else {
+5078 panic!("packets from unknown remote should be dropped by clients");
+5079 };
+5080 debug_assert!(
+5081 server_config.migration,
+5082 "migration-initiating packets should have been dropped immediately"
+5083 );
+5084 self.migrate(path_id, now, network_path, migration_observed_addr);
+5085 // Break linkability, if possible
+5086 self.update_rem_cid(path_id);
+5087 self.spin = false;
5088 }
5089
-5090 // Subtract 1 to account for the CID we supplied while handshaking
-5091 let mut n = self.peer_params.issue_cids_limit() - 1;
-5092 if let ConnectionSide::Server { server_config } = &self.side {
-5093 if server_config.has_preferred_address() {
-5094 // We also sent a CID in the transport parameters
-5095 n -= 1;
-5096 }
-5097 }
-5098 self.endpoint_events
-5099 .push_back(EndpointEventInner::NeedIdentifiers(PathId::ZERO, now, n));
-5100 }
-5101
-5102 /// Issues an initial set of CIDs for paths that have not yet had any CIDs issued
-5103 ///
-5104 /// Later CIDs are issued when CIDs expire or are retired by the peer.
-5105 fn issue_first_path_cids(&mut self, now: Instant) {
-5106 if let Some(max_path_id) = self.max_path_id() {
-5107 let mut path_id = self.max_path_id_with_cids.next();
-5108 while path_id <= max_path_id {
-5109 self.endpoint_events
-5110 .push_back(EndpointEventInner::NeedIdentifiers(
-5111 path_id,
-5112 now,
-5113 self.peer_params.issue_cids_limit(),
-5114 ));
-5115 path_id = path_id.next();
-5116 }
-5117 self.max_path_id_with_cids = max_path_id;
-5118 }
-5119 }
-5120
-5121 /// Populates a packet with frames
-5122 ///
-5123 /// This tries to fit as many frames as possible into the packet.
-5124 ///
-5125 /// *path_exclusive_only* means to only build frames which can only be sent on this
-5126 /// *path. This is used in multipath for backup paths while there is still an active
-5127 /// *path.
-5128 fn populate_packet<'a, 'b>(
-5129 &mut self,
-5130 now: Instant,
-5131 space_id: SpaceId,
-5132 path_id: PathId,
-5133 path_exclusive_only: bool,
-5134 builder: &mut PacketBuilder<'a, 'b>,
-5135 ) {
-5136 let pn = builder.exact_number;
-5137 let is_multipath_negotiated = self.is_multipath_negotiated();
-5138 let stats = &mut self.stats.frame_tx;
-5139 let space = &mut self.spaces[space_id];
-5140 let path = &mut self.paths.get_mut(&path_id).expect("known path").data;
-5141 let is_0rtt = space_id == SpaceId::Data && space.crypto.is_none();
-5142 space
-5143 .for_path(path_id)
-5144 .pending_acks
-5145 .maybe_ack_non_eliciting();
+5090 Ok(())
+5091 }
+5092
+5093 fn migrate(
+5094 &mut self,
+5095 path_id: PathId,
+5096 now: Instant,
+5097 network_path: FourTuple,
+5098 observed_addr: Option<ObservedAddr>,
+5099 ) {
+5100 trace!(%network_path, %path_id, "migration initiated");
+5101 self.path_generation_counter = self.path_generation_counter.wrapping_add(1);
+5102 // TODO(@divma): conditions for path migration in multipath are very specific, check them
+5103 // again to prevent path migrations that should actually create a new path
+5104
+5105 // Reset rtt/congestion state for new path unless it looks like a NAT rebinding.
+5106 // Note that the congestion window will not grow until validation terminates. Helps mitigate
+5107 // amplification attacks performed by spoofing source addresses.
+5108 let prev_pto = self.pto(SpaceId::Data, path_id);
+5109 let known_path = self.paths.get_mut(&path_id).expect("known path");
+5110 let path = &mut known_path.data;
+5111 let mut new_path = if network_path.remote.is_ipv4()
+5112 && network_path.remote.ip() == path.network_path.remote.ip()
+5113 {
+5114 PathData::from_previous(network_path, path, self.path_generation_counter, now)
+5115 } else {
+5116 let peer_max_udp_payload_size =
+5117 u16::try_from(self.peer_params.max_udp_payload_size.into_inner())
+5118 .unwrap_or(u16::MAX);
+5119 PathData::new(
+5120 network_path,
+5121 self.allow_mtud,
+5122 Some(peer_max_udp_payload_size),
+5123 self.path_generation_counter,
+5124 now,
+5125 &self.config,
+5126 )
+5127 };
+5128 new_path.last_observed_addr_report = path.last_observed_addr_report.clone();
+5129 if let Some(report) = observed_addr {
+5130 if let Some(updated) = new_path.update_observed_addr_report(report) {
+5131 tracing::info!("adding observed addr event from migration");
+5132 self.events.push_back(Event::Path(PathEvent::ObservedAddr {
+5133 id: path_id,
+5134 addr: updated,
+5135 }));
+5136 }
+5137 }
+5138 new_path.send_new_challenge = true;
+5139
+5140 let mut prev = mem::replace(path, new_path);
+5141 // Don't clobber the original path if the previous one hasn't been validated yet
+5142 if !prev.is_validating_path() {
+5143 prev.send_new_challenge = true;
+5144 // We haven't updated the remote CID yet, this captures the remote CID we were using on
+5145 // the previous path.
5146
-5147 // HANDSHAKE_DONE
-5148 if !is_0rtt && mem::replace(&mut space.pending.handshake_done, false) {
-5149 builder.write_frame(frame::HandshakeDone, stats);
-5150 }
-5151
-5152 // REACH_OUT
-5153 // TODO(@divma): path explusive considerations
-5154 if let Some((round, addresses)) = space.pending.reach_out.as_mut() {
-5155 while let Some(local_addr) = addresses.pop() {
-5156 let reach_out = frame::ReachOut::new(*round, local_addr);
-5157 if builder.frame_space_remaining() > reach_out.size() {
-5158 builder.write_frame(reach_out, stats);
-5159 } else {
-5160 addresses.push(local_addr);
-5161 break;
-5162 }
-5163 }
-5164 if addresses.is_empty() {
-5165 space.pending.reach_out = None;
-5166 }
-5167 }
-5168
-5169 // OBSERVED_ADDR
-5170 if !path_exclusive_only
-5171 && space_id == SpaceId::Data
-5172 && self
-5173 .config
-5174 .address_discovery_role
-5175 .should_report(&self.peer_params.address_discovery_role)
-5176 && (!path.observed_addr_sent || space.pending.observed_addr)
-5177 {
-5178 let frame =
-5179 frame::ObservedAddr::new(path.network_path.remote, self.next_observed_addr_seq_no);
-5180 if builder.frame_space_remaining() > frame.size() {
-5181 builder.write_frame(frame, stats);
-5182
-5183 self.next_observed_addr_seq_no = self.next_observed_addr_seq_no.saturating_add(1u8);
-5184 path.observed_addr_sent = true;
-5185
-5186 space.pending.observed_addr = false;
-5187 }
-5188 }
-5189
-5190 // PING
-5191 if mem::replace(&mut space.for_path(path_id).ping_pending, false) {
-5192 builder.write_frame(frame::Ping, stats);
-5193 }
-5194
-5195 // IMMEDIATE_ACK
-5196 if mem::replace(&mut space.for_path(path_id).immediate_ack_pending, false) {
-5197 debug_assert_eq!(
-5198 space_id,
-5199 SpaceId::Data,
-5200 "immediate acks must be sent in the data space"
-5201 );
-5202 builder.write_frame(frame::ImmediateAck, stats);
+5147 known_path.prev = Some((self.rem_cids.get(&path_id).unwrap().active(), prev));
+5148 }
+5149
+5150 // We need to re-assign the correct remote to this path in qlog
+5151 self.qlog.emit_tuple_assigned(path_id, network_path, now);
+5152
+5153 self.timers.set(
+5154 Timer::PerPath(path_id, PathTimer::PathValidation),
+5155 now + 3 * cmp::max(self.pto(SpaceId::Data, path_id), prev_pto),
+5156 self.qlog.with_time(now),
+5157 );
+5158 }
+5159
+5160 /// Handle a change in the local address, i.e. an active migration
+5161 pub fn local_address_changed(&mut self) {
+5162 // TODO(flub): if multipath is enabled this needs to create a new path entirely.
+5163 self.update_rem_cid(PathId::ZERO);
+5164 self.ping();
+5165 }
+5166
+5167 /// Switch to a previously unused remote connection ID, if possible
+5168 fn update_rem_cid(&mut self, path_id: PathId) {
+5169 let Some((reset_token, retired)) =
+5170 self.rem_cids.get_mut(&path_id).and_then(|cids| cids.next())
+5171 else {
+5172 return;
+5173 };
+5174
+5175 // Retire the current remote CID and any CIDs we had to skip.
+5176 self.spaces[SpaceId::Data]
+5177 .pending
+5178 .retire_cids
+5179 .extend(retired.map(|seq| (path_id, seq)));
+5180 let remote = self.path_data(path_id).network_path.remote;
+5181 self.set_reset_token(path_id, remote, reset_token);
+5182 }
+5183
+5184 /// Sends this reset token to the endpoint
+5185 ///
+5186 /// The endpoint needs to know the reset tokens issued by the peer, so that if the peer
+5187 /// sends a reset token it knows to route it to this connection. See RFC 9000 section
+5188 /// 10.3. Stateless Reset.
+5189 ///
+5190 /// Reset tokens are different for each path, the endpoint identifies paths by peer
+5191 /// socket address however, not by path ID.
+5192 fn set_reset_token(&mut self, path_id: PathId, remote: SocketAddr, reset_token: ResetToken) {
+5193 self.endpoint_events
+5194 .push_back(EndpointEventInner::ResetToken(path_id, remote, reset_token));
+5195
+5196 // During the handshake the server sends a reset token in the transport
+5197 // parameters. When we are the client and we receive the reset token during the
+5198 // handshake we want this to affect our peer transport parameters.
+5199 // TODO(flub): Pretty sure this is pointless, the entire params is overwritten
+5200 // shortly after this was called. And then the params don't have this anymore.
+5201 if path_id == PathId::ZERO {
+5202 self.peer_params.stateless_reset_token = Some(reset_token);
5203 }
-5204
-5205 // ACK
-5206 // TODO(flub): Should this send acks for this path anyway?
-5207
-5208 if !path_exclusive_only {
-5209 for path_id in space
-5210 .number_spaces
-5211 .iter_mut()
-5212 .filter(|(_, pns)| pns.pending_acks.can_send())
-5213 .map(|(&path_id, _)| path_id)
-5214 .collect::<Vec<_>>()
-5215 {
-5216 Self::populate_acks(
-5217 now,
-5218 self.receiving_ecn,
-5219 path_id,
-5220 space_id,
-5221 space,
-5222 is_multipath_negotiated,
-5223 builder,
-5224 stats,
-5225 );
-5226 }
-5227 }
-5228
-5229 // ACK_FREQUENCY
-5230 if !path_exclusive_only && mem::replace(&mut space.pending.ack_frequency, false) {
-5231 let sequence_number = self.ack_frequency.next_sequence_number();
-5232
-5233 // Safe to unwrap because this is always provided when ACK frequency is enabled
-5234 let config = self.config.ack_frequency_config.as_ref().unwrap();
-5235
-5236 // Ensure the delay is within bounds to avoid a PROTOCOL_VIOLATION error
-5237 let max_ack_delay = self.ack_frequency.candidate_max_ack_delay(
-5238 path.rtt.get(),
-5239 config,
-5240 &self.peer_params,
-5241 );
-5242
-5243 let frame = frame::AckFrequency {
-5244 sequence: sequence_number,
-5245 ack_eliciting_threshold: config.ack_eliciting_threshold,
-5246 request_max_ack_delay: max_ack_delay.as_micros().try_into().unwrap_or(VarInt::MAX),
-5247 reordering_threshold: config.reordering_threshold,
-5248 };
-5249 builder.write_frame(frame, stats);
-5250
-5251 self.ack_frequency
-5252 .ack_frequency_sent(path_id, pn, max_ack_delay);
-5253 }
-5254
-5255 // PATH_CHALLENGE
-5256 if builder.frame_space_remaining() > frame::PathChallenge::SIZE_BOUND
-5257 && space_id == SpaceId::Data
-5258 && path.send_new_challenge
-5259 && !self.state.is_closed()
-5260 // we don't want to send new challenges if we are already closing
-5261 {
-5262 path.send_new_challenge = false;
-5263
-5264 // Generate a new challenge every time we send a new PATH_CHALLENGE
-5265 let token = self.rng.random();
-5266 let info = paths::SentChallengeInfo {
-5267 sent_instant: now,
-5268 network_path: path.network_path,
-5269 };
-5270 path.challenges_sent.insert(token, info);
-5271 let challenge = frame::PathChallenge(token);
-5272 trace!(frame = %challenge);
-5273 builder.write_frame(challenge, stats);
-5274 builder.require_padding();
-5275 let pto = self.ack_frequency.max_ack_delay_for_pto() + path.rtt.pto_base();
-5276 self.timers.set(
-5277 Timer::PerPath(path_id, PathTimer::PathChallengeLost),
-5278 now + pto,
-5279 self.qlog.with_time(now),
-5280 );
-5281
-5282 if is_multipath_negotiated && !path.validated && path.send_new_challenge {
-5283 // queue informing the path status along with the challenge
-5284 space.pending.path_status.insert(path_id);
-5285 }
-5286
-5287 // Always include an OBSERVED_ADDR frame with a PATH_CHALLENGE, regardless
-5288 // of whether one has already been sent on this path.
-5289 if space_id == SpaceId::Data
-5290 && self
-5291 .config
-5292 .address_discovery_role
-5293 .should_report(&self.peer_params.address_discovery_role)
-5294 {
-5295 let frame = frame::ObservedAddr::new(
-5296 path.network_path.remote,
-5297 self.next_observed_addr_seq_no,
-5298 );
-5299 if builder.frame_space_remaining() > frame.size() {
-5300 builder.write_frame(frame, stats);
-5301
-5302 self.next_observed_addr_seq_no =
-5303 self.next_observed_addr_seq_no.saturating_add(1u8);
-5304 path.observed_addr_sent = true;
-5305
-5306 space.pending.observed_addr = false;
-5307 }
-5308 }
-5309 }
+5204 }
+5205
+5206 /// Issue an initial set of connection IDs to the peer upon connection
+5207 fn issue_first_cids(&mut self, now: Instant) {
+5208 if self
+5209 .local_cid_state
+5210 .get(&PathId::ZERO)
+5211 .expect("PathId::ZERO exists when the connection is created")
+5212 .cid_len()
+5213 == 0
+5214 {
+5215 return;
+5216 }
+5217
+5218 // Subtract 1 to account for the CID we supplied while handshaking
+5219 let mut n = self.peer_params.issue_cids_limit() - 1;
+5220 if let ConnectionSide::Server { server_config } = &self.side {
+5221 if server_config.has_preferred_address() {
+5222 // We also sent a CID in the transport parameters
+5223 n -= 1;
+5224 }
+5225 }
+5226 self.endpoint_events
+5227 .push_back(EndpointEventInner::NeedIdentifiers(PathId::ZERO, now, n));
+5228 }
+5229
+5230 /// Issues an initial set of CIDs for paths that have not yet had any CIDs issued
+5231 ///
+5232 /// Later CIDs are issued when CIDs expire or are retired by the peer.
+5233 fn issue_first_path_cids(&mut self, now: Instant) {
+5234 if let Some(max_path_id) = self.max_path_id() {
+5235 let mut path_id = self.max_path_id_with_cids.next();
+5236 while path_id <= max_path_id {
+5237 self.endpoint_events
+5238 .push_back(EndpointEventInner::NeedIdentifiers(
+5239 path_id,
+5240 now,
+5241 self.peer_params.issue_cids_limit(),
+5242 ));
+5243 path_id = path_id.next();
+5244 }
+5245 self.max_path_id_with_cids = max_path_id;
+5246 }
+5247 }
+5248
+5249 /// Populates a packet with frames
+5250 ///
+5251 /// This tries to fit as many frames as possible into the packet.
+5252 ///
+5253 /// *path_exclusive_only* means to only build frames which can only be sent on this
+5254 /// *path. This is used in multipath for backup paths while there is still an active
+5255 /// *path.
+5256 fn populate_packet<'a, 'b>(
+5257 &mut self,
+5258 now: Instant,
+5259 space_id: SpaceId,
+5260 path_id: PathId,
+5261 path_exclusive_only: bool,
+5262 builder: &mut PacketBuilder<'a, 'b>,
+5263 ) {
+5264 let pn = builder.exact_number;
+5265 let is_multipath_negotiated = self.is_multipath_negotiated();
+5266 let stats = &mut self.stats.frame_tx;
+5267 let space = &mut self.spaces[space_id];
+5268 let path = &mut self.paths.get_mut(&path_id).expect("known path").data;
+5269 let is_0rtt = space_id == SpaceId::Data && space.crypto.is_none();
+5270 space
+5271 .for_path(path_id)
+5272 .pending_acks
+5273 .maybe_ack_non_eliciting();
+5274
+5275 // HANDSHAKE_DONE
+5276 if !is_0rtt && mem::replace(&mut space.pending.handshake_done, false) {
+5277 builder.write_frame(frame::HandshakeDone, stats);
+5278 }
+5279
+5280 // REACH_OUT
+5281 // TODO(@divma): path explusive considerations
+5282 if let Some((round, addresses)) = space.pending.reach_out.as_mut() {
+5283 while let Some(local_addr) = addresses.pop() {
+5284 let reach_out = frame::ReachOut::new(*round, local_addr);
+5285 if builder.frame_space_remaining() > reach_out.size() {
+5286 builder.write_frame(reach_out, stats);
+5287 } else {
+5288 addresses.push(local_addr);
+5289 break;
+5290 }
+5291 }
+5292 if addresses.is_empty() {
+5293 space.pending.reach_out = None;
+5294 }
+5295 }
+5296
+5297 // OBSERVED_ADDR
+5298 if !path_exclusive_only
+5299 && space_id == SpaceId::Data
+5300 && self
+5301 .config
+5302 .address_discovery_role
+5303 .should_report(&self.peer_params.address_discovery_role)
+5304 && (!path.observed_addr_sent || space.pending.observed_addr)
+5305 {
+5306 let frame =
+5307 frame::ObservedAddr::new(path.network_path.remote, self.next_observed_addr_seq_no);
+5308 if builder.frame_space_remaining() > frame.size() {
+5309 builder.write_frame(frame, stats);
5310
-5311 // PATH_RESPONSE
-5312 if builder.frame_space_remaining() > frame::PathResponse::SIZE_BOUND
-5313 && space_id == SpaceId::Data
-5314 {
-5315 if let Some(token) = path.path_responses.pop_on_path(path.network_path) {
-5316 let response = frame::PathResponse(token);
-5317 trace!(frame = %response);
-5318 builder.write_frame(response, stats);
-5319 builder.require_padding();
-5320
-5321 // NOTE: this is technically not required but might be useful to ride the
-5322 // request/response nature of path challenges to refresh an observation
-5323 // Since PATH_RESPONSE is a probing frame, this is allowed by the spec.
-5324 if space_id == SpaceId::Data
-5325 && self
-5326 .config
-5327 .address_discovery_role
-5328 .should_report(&self.peer_params.address_discovery_role)
-5329 {
-5330 let frame = frame::ObservedAddr::new(
-5331 path.network_path.remote,
-5332 self.next_observed_addr_seq_no,
-5333 );
-5334 if builder.frame_space_remaining() > frame.size() {
-5335 builder.write_frame(frame, stats);
-5336
-5337 self.next_observed_addr_seq_no =
-5338 self.next_observed_addr_seq_no.saturating_add(1u8);
-5339 path.observed_addr_sent = true;
-5340
-5341 space.pending.observed_addr = false;
-5342 }
-5343 }
-5344 }
-5345 }
-5346
-5347 // CRYPTO
-5348 while !path_exclusive_only
-5349 && builder.frame_space_remaining() > frame::Crypto::SIZE_BOUND
-5350 && !is_0rtt
-5351 {
-5352 let mut frame = match space.pending.crypto.pop_front() {
-5353 Some(x) => x,
-5354 None => break,
-5355 };
+5311 self.next_observed_addr_seq_no = self.next_observed_addr_seq_no.saturating_add(1u8);
+5312 path.observed_addr_sent = true;
+5313
+5314 space.pending.observed_addr = false;
+5315 }
+5316 }
+5317
+5318 // PING
+5319 if mem::replace(&mut space.for_path(path_id).ping_pending, false) {
+5320 builder.write_frame(frame::Ping, stats);
+5321 }
+5322
+5323 // IMMEDIATE_ACK
+5324 if mem::replace(&mut space.for_path(path_id).immediate_ack_pending, false) {
+5325 debug_assert_eq!(
+5326 space_id,
+5327 SpaceId::Data,
+5328 "immediate acks must be sent in the data space"
+5329 );
+5330 builder.write_frame(frame::ImmediateAck, stats);
+5331 }
+5332
+5333 // ACK
+5334 // TODO(flub): Should this send acks for this path anyway?
+5335
+5336 if !path_exclusive_only {
+5337 for path_id in space
+5338 .number_spaces
+5339 .iter_mut()
+5340 .filter(|(_, pns)| pns.pending_acks.can_send())
+5341 .map(|(&path_id, _)| path_id)
+5342 .collect::<Vec<_>>()
+5343 {
+5344 Self::populate_acks(
+5345 now,
+5346 self.receiving_ecn,
+5347 path_id,
+5348 space_id,
+5349 space,
+5350 is_multipath_negotiated,
+5351 builder,
+5352 stats,
+5353 );
+5354 }
+5355 }
5356
-5357 // Calculate the maximum amount of crypto data we can store in the buffer.
-5358 // Since the offset is known, we can reserve the exact size required to encode it.
-5359 // For length we reserve 2bytes which allows to encode up to 2^14,
-5360 // which is more than what fits into normally sized QUIC frames.
-5361 let max_crypto_data_size = builder.frame_space_remaining()
-5362 - 1 // Frame Type
-5363 - VarInt::size(unsafe { VarInt::from_u64_unchecked(frame.offset) })
-5364 - 2; // Maximum encoded length for frame size, given we send less than 2^14 bytes
-5365
-5366 let len = frame
-5367 .data
-5368 .len()
-5369 .min(2usize.pow(14) - 1)
-5370 .min(max_crypto_data_size);
-5371
-5372 let data = frame.data.split_to(len);
-5373 let offset = frame.offset;
-5374 let truncated = frame::Crypto { offset, data };
-5375 builder.write_frame(truncated, stats);
-5376
-5377 if !frame.data.is_empty() {
-5378 frame.offset += len as u64;
-5379 space.pending.crypto.push_front(frame);
-5380 }
+5357 // ACK_FREQUENCY
+5358 if !path_exclusive_only && mem::replace(&mut space.pending.ack_frequency, false) {
+5359 let sequence_number = self.ack_frequency.next_sequence_number();
+5360
+5361 // Safe to unwrap because this is always provided when ACK frequency is enabled
+5362 let config = self.config.ack_frequency_config.as_ref().unwrap();
+5363
+5364 // Ensure the delay is within bounds to avoid a PROTOCOL_VIOLATION error
+5365 let max_ack_delay = self.ack_frequency.candidate_max_ack_delay(
+5366 path.rtt.get(),
+5367 config,
+5368 &self.peer_params,
+5369 );
+5370
+5371 let frame = frame::AckFrequency {
+5372 sequence: sequence_number,
+5373 ack_eliciting_threshold: config.ack_eliciting_threshold,
+5374 request_max_ack_delay: max_ack_delay.as_micros().try_into().unwrap_or(VarInt::MAX),
+5375 reordering_threshold: config.reordering_threshold,
+5376 };
+5377 builder.write_frame(frame, stats);
+5378
+5379 self.ack_frequency
+5380 .ack_frequency_sent(path_id, pn, max_ack_delay);
5381 }
5382
-5383 // TODO(flub): maybe this is much higher priority?
-5384 // PATH_ABANDON
-5385 while !path_exclusive_only
-5386 && space_id == SpaceId::Data
-5387 && frame::PathAbandon::SIZE_BOUND <= builder.frame_space_remaining()
-5388 {
-5389 let Some((abandoned_path_id, error_code)) = space.pending.path_abandon.pop_first()
-5390 else {
-5391 break;
-5392 };
-5393 let frame = frame::PathAbandon {
-5394 path_id: abandoned_path_id,
-5395 error_code,
-5396 };
-5397 builder.write_frame(frame, stats);
-5398
-5399 let ack_delay = self.ack_frequency.max_ack_delay_for_pto();
-5400 // We can't access path here anymore due to borrowing issues.
-5401 let send_pto = self.paths.get(&path_id).unwrap().data.rtt.pto_base() + ack_delay;
-5402 if let Some(abandoned_path) = self.paths.get_mut(&abandoned_path_id) {
-5403 // We only want to set the deadline on the *first* PATH_ABANDON we send.
-5404 // Retransmits shouldn't run this code again
-5405 if matches!(
-5406 abandoned_path.data.abandon_state,
-5407 AbandonState::NotAbandoned
-5408 ) {
-5409 // The peer MUST respond with a corresponding PATH_ABANDON frame.
-5410 // The other peer has 3 * PTO to do that.
-5411 // This uses the PTO of the path we send on!
-5412 // If the PATH_ABANDON comes in within the deadline we're giving here, then this
-5413 // state will be set to `AbandonState::ReceivedPathAbandon`, essentially clearing
-5414 // the deadline. If we receive a frame after the deadline, we error out with a
-5415 // protocol violation.
-5416 // Receiving other frames before the deadline is fine, as those might be packets
-5417 // that were still in-flight.
-5418 abandoned_path.data.abandon_state = AbandonState::ExpectingPathAbandon {
-5419 deadline: now + 3 * send_pto,
-5420 };
-5421
-5422 // At some point, we need to forget about the path.
-5423 // If we do so too early, then we'll have discarded the CIDs of that path and won't
-5424 // handle incoming packets on that path correctly.
-5425 // To give this path enough time, we assume that the peer will have received our
-5426 // PATH_ABANDON within 3 * PTO of the path we sent the abandon on,
-5427 // and then we give the path 3 * PTO time to make it very unlikely that there will
-5428 // still be packets incoming on the path at that point.
-5429 // This timer will actually get reset to a value that's likely to be even earlier
-5430 // once we actually receive the PATH_ABANDON frame itself.
-5431 let abandoned_pto =
-5432 self.paths.get(&path_id).unwrap().data.rtt.pto_base() + ack_delay;
-5433 self.timers.set(
-5434 Timer::PerPath(abandoned_path_id, PathTimer::DiscardPath),
-5435 now + 3 * send_pto + 3 * abandoned_pto,
-5436 self.qlog.with_time(now),
-5437 );
-5438 }
-5439 } else {
-5440 warn!("sent PATH_ABANDON after path was already discarded");
-5441 }
-5442 }
-5443
-5444 // PATH_STATUS_AVAILABLE & PATH_STATUS_BACKUP
-5445 while !path_exclusive_only
-5446 && space_id == SpaceId::Data
-5447 && frame::PathStatusAvailable::SIZE_BOUND <= builder.frame_space_remaining()
-5448 {
-5449 let Some(path_id) = space.pending.path_status.pop_first() else {
-5450 break;
-5451 };
-5452 let Some(path) = self.paths.get(&path_id).map(|path_state| &path_state.data) else {
-5453 trace!(%path_id, "discarding queued path status for unknown path");
-5454 continue;
-5455 };
-5456
-5457 let seq = path.status.seq();
-5458 match path.local_status() {
-5459 PathStatus::Available => {
-5460 let frame = frame::PathStatusAvailable {
-5461 path_id,
-5462 status_seq_no: seq,
-5463 };
-5464 builder.write_frame(frame, stats);
-5465 }
-5466 PathStatus::Backup => {
-5467 let frame = frame::PathStatusBackup {
-5468 path_id,
-5469 status_seq_no: seq,
-5470 };
-5471 builder.write_frame(frame, stats);
-5472 }
-5473 }
-5474 }
-5475
-5476 // MAX_PATH_ID
-5477 if space_id == SpaceId::Data
-5478 && space.pending.max_path_id
-5479 && frame::MaxPathId::SIZE_BOUND <= builder.frame_space_remaining()
-5480 {
-5481 let frame = frame::MaxPathId(self.local_max_path_id);
-5482 builder.write_frame(frame, stats);
-5483 space.pending.max_path_id = false;
-5484 }
-5485
-5486 // PATHS_BLOCKED
-5487 if space_id == SpaceId::Data
-5488 && space.pending.paths_blocked
-5489 && frame::PathsBlocked::SIZE_BOUND <= builder.frame_space_remaining()
-5490 {
-5491 let frame = frame::PathsBlocked(self.remote_max_path_id);
-5492 builder.write_frame(frame, stats);
-5493 space.pending.paths_blocked = false;
-5494 }
-5495
-5496 // PATH_CIDS_BLOCKED
-5497 while space_id == SpaceId::Data
-5498 && frame::PathCidsBlocked::SIZE_BOUND <= builder.frame_space_remaining()
-5499 {
-5500 let Some(path_id) = space.pending.path_cids_blocked.pop_first() else {
-5501 break;
-5502 };
-5503 let next_seq = match self.rem_cids.get(&path_id) {
-5504 Some(cid_queue) => VarInt(cid_queue.active_seq() + 1),
-5505 None => VarInt(0),
-5506 };
-5507 let frame = frame::PathCidsBlocked { path_id, next_seq };
-5508 builder.write_frame(frame, stats);
+5383 // PATH_CHALLENGE
+5384 if builder.frame_space_remaining() > frame::PathChallenge::SIZE_BOUND
+5385 && space_id == SpaceId::Data
+5386 && path.send_new_challenge
+5387 && !self.state.is_closed()
+5388 // we don't want to send new challenges if we are already closing
+5389 {
+5390 path.send_new_challenge = false;
+5391
+5392 // Generate a new challenge every time we send a new PATH_CHALLENGE
+5393 let token = self.rng.random();
+5394 let info = paths::SentChallengeInfo {
+5395 sent_instant: now,
+5396 network_path: path.network_path,
+5397 };
+5398 path.challenges_sent.insert(token, info);
+5399 let challenge = frame::PathChallenge(token);
+5400 trace!(frame = %challenge);
+5401 builder.write_frame(challenge, stats);
+5402 builder.require_padding();
+5403 let pto = self.ack_frequency.max_ack_delay_for_pto() + path.rtt.pto_base();
+5404 self.timers.set(
+5405 Timer::PerPath(path_id, PathTimer::PathChallengeLost),
+5406 now + pto,
+5407 self.qlog.with_time(now),
+5408 );
+5409
+5410 if is_multipath_negotiated && !path.validated && path.send_new_challenge {
+5411 // queue informing the path status along with the challenge
+5412 space.pending.path_status.insert(path_id);
+5413 }
+5414
+5415 // Always include an OBSERVED_ADDR frame with a PATH_CHALLENGE, regardless
+5416 // of whether one has already been sent on this path.
+5417 if space_id == SpaceId::Data
+5418 && self
+5419 .config
+5420 .address_discovery_role
+5421 .should_report(&self.peer_params.address_discovery_role)
+5422 {
+5423 let frame = frame::ObservedAddr::new(
+5424 path.network_path.remote,
+5425 self.next_observed_addr_seq_no,
+5426 );
+5427 if builder.frame_space_remaining() > frame.size() {
+5428 builder.write_frame(frame, stats);
+5429
+5430 self.next_observed_addr_seq_no =
+5431 self.next_observed_addr_seq_no.saturating_add(1u8);
+5432 path.observed_addr_sent = true;
+5433
+5434 space.pending.observed_addr = false;
+5435 }
+5436 }
+5437 }
+5438
+5439 // PATH_RESPONSE
+5440 if builder.frame_space_remaining() > frame::PathResponse::SIZE_BOUND
+5441 && space_id == SpaceId::Data
+5442 {
+5443 if let Some(token) = path.path_responses.pop_on_path(path.network_path) {
+5444 let response = frame::PathResponse(token);
+5445 trace!(frame = %response);
+5446 builder.write_frame(response, stats);
+5447 builder.require_padding();
+5448
+5449 // NOTE: this is technically not required but might be useful to ride the
+5450 // request/response nature of path challenges to refresh an observation
+5451 // Since PATH_RESPONSE is a probing frame, this is allowed by the spec.
+5452 if space_id == SpaceId::Data
+5453 && self
+5454 .config
+5455 .address_discovery_role
+5456 .should_report(&self.peer_params.address_discovery_role)
+5457 {
+5458 let frame = frame::ObservedAddr::new(
+5459 path.network_path.remote,
+5460 self.next_observed_addr_seq_no,
+5461 );
+5462 if builder.frame_space_remaining() > frame.size() {
+5463 builder.write_frame(frame, stats);
+5464
+5465 self.next_observed_addr_seq_no =
+5466 self.next_observed_addr_seq_no.saturating_add(1u8);
+5467 path.observed_addr_sent = true;
+5468
+5469 space.pending.observed_addr = false;
+5470 }
+5471 }
+5472 }
+5473 }
+5474
+5475 // CRYPTO
+5476 while !path_exclusive_only
+5477 && builder.frame_space_remaining() > frame::Crypto::SIZE_BOUND
+5478 && !is_0rtt
+5479 {
+5480 let mut frame = match space.pending.crypto.pop_front() {
+5481 Some(x) => x,
+5482 None => break,
+5483 };
+5484
+5485 // Calculate the maximum amount of crypto data we can store in the buffer.
+5486 // Since the offset is known, we can reserve the exact size required to encode it.
+5487 // For length we reserve 2bytes which allows to encode up to 2^14,
+5488 // which is more than what fits into normally sized QUIC frames.
+5489 let max_crypto_data_size = builder.frame_space_remaining()
+5490 - 1 // Frame Type
+5491 - VarInt::size(unsafe { VarInt::from_u64_unchecked(frame.offset) })
+5492 - 2; // Maximum encoded length for frame size, given we send less than 2^14 bytes
+5493
+5494 let len = frame
+5495 .data
+5496 .len()
+5497 .min(2usize.pow(14) - 1)
+5498 .min(max_crypto_data_size);
+5499
+5500 let data = frame.data.split_to(len);
+5501 let offset = frame.offset;
+5502 let truncated = frame::Crypto { offset, data };
+5503 builder.write_frame(truncated, stats);
+5504
+5505 if !frame.data.is_empty() {
+5506 frame.offset += len as u64;
+5507 space.pending.crypto.push_front(frame);
+5508 }
5509 }
5510
-5511 // RESET_STREAM, STOP_SENDING, MAX_DATA, MAX_STREAM_DATA, MAX_STREAMS
-5512 if space_id == SpaceId::Data {
-5513 self.streams
-5514 .write_control_frames(builder, &mut space.pending, stats);
-5515 }
-5516
-5517 // NEW_CONNECTION_ID
-5518 let cid_len = self
-5519 .local_cid_state
-5520 .values()
-5521 .map(|cid_state| cid_state.cid_len())
-5522 .max()
-5523 .expect("some local CID state must exist");
-5524 let new_cid_size_bound =
-5525 frame::NewConnectionId::size_bound(is_multipath_negotiated, cid_len);
-5526 while !path_exclusive_only && builder.frame_space_remaining() > new_cid_size_bound {
-5527 let issued = match space.pending.new_cids.pop() {
-5528 Some(x) => x,
-5529 None => break,
-5530 };
-5531 let retire_prior_to = self
-5532 .local_cid_state
-5533 .get(&issued.path_id)
-5534 .map(|cid_state| cid_state.retire_prior_to())
-5535 .unwrap_or_else(|| panic!("missing local CID state for path={}", issued.path_id));
-5536
-5537 let cid_path_id = match is_multipath_negotiated {
-5538 true => Some(issued.path_id),
-5539 false => {
-5540 debug_assert_eq!(issued.path_id, PathId::ZERO);
-5541 None
-5542 }
-5543 };
-5544 let frame = frame::NewConnectionId {
-5545 path_id: cid_path_id,
-5546 sequence: issued.sequence,
-5547 retire_prior_to,
-5548 id: issued.id,
-5549 reset_token: issued.reset_token,
-5550 };
-5551 builder.write_frame(frame, stats);
-5552 }
-5553
-5554 // RETIRE_CONNECTION_ID
-5555 let retire_cid_bound = frame::RetireConnectionId::size_bound(is_multipath_negotiated);
-5556 while !path_exclusive_only && builder.frame_space_remaining() > retire_cid_bound {
-5557 let (path_id, sequence) = match space.pending.retire_cids.pop() {
-5558 Some((PathId::ZERO, seq)) if !is_multipath_negotiated => (None, seq),
-5559 Some((path_id, seq)) => (Some(path_id), seq),
-5560 None => break,
-5561 };
-5562 let frame = frame::RetireConnectionId { path_id, sequence };
-5563 builder.write_frame(frame, stats);
-5564 }
-5565
-5566 // DATAGRAM
-5567 let mut sent_datagrams = false;
-5568 while !path_exclusive_only
-5569 && builder.frame_space_remaining() > Datagram::SIZE_BOUND
-5570 && space_id == SpaceId::Data
-5571 {
-5572 match self.datagrams.write(builder, stats) {
-5573 true => {
-5574 sent_datagrams = true;
-5575 }
-5576 false => break,
-5577 }
-5578 }
-5579 if self.datagrams.send_blocked && sent_datagrams {
-5580 self.events.push_back(Event::DatagramsUnblocked);
-5581 self.datagrams.send_blocked = false;
-5582 }
-5583
-5584 let path = &mut self.paths.get_mut(&path_id).expect("known path").data;
-5585
-5586 // NEW_TOKEN
-5587 if !path_exclusive_only {
-5588 while let Some(network_path) = space.pending.new_tokens.pop() {
-5589 debug_assert_eq!(space_id, SpaceId::Data);
-5590 let ConnectionSide::Server { server_config } = &self.side else {
-5591 panic!("NEW_TOKEN frames should not be enqueued by clients");
-5592 };
-5593
-5594 if !network_path.is_probably_same_path(&path.network_path) {
-5595 // NEW_TOKEN frames contain tokens bound to a client's IP address, and are only
-5596 // useful if used from the same IP address. Thus, we abandon enqueued NEW_TOKEN
-5597 // frames upon an path change. Instead, when the new path becomes validated,
-5598 // NEW_TOKEN frames may be enqueued for the new path instead.
-5599 continue;
+5511 // TODO(flub): maybe this is much higher priority?
+5512 // PATH_ABANDON
+5513 while !path_exclusive_only
+5514 && space_id == SpaceId::Data
+5515 && frame::PathAbandon::SIZE_BOUND <= builder.frame_space_remaining()
+5516 {
+5517 let Some((abandoned_path_id, error_code)) = space.pending.path_abandon.pop_first()
+5518 else {
+5519 break;
+5520 };
+5521 let frame = frame::PathAbandon {
+5522 path_id: abandoned_path_id,
+5523 error_code,
+5524 };
+5525 builder.write_frame(frame, stats);
+5526
+5527 let ack_delay = self.ack_frequency.max_ack_delay_for_pto();
+5528 // We can't access path here anymore due to borrowing issues.
+5529 let send_pto = self.paths.get(&path_id).unwrap().data.rtt.pto_base() + ack_delay;
+5530 if let Some(abandoned_path) = self.paths.get_mut(&abandoned_path_id) {
+5531 // We only want to set the deadline on the *first* PATH_ABANDON we send.
+5532 // Retransmits shouldn't run this code again
+5533 if matches!(
+5534 abandoned_path.data.abandon_state,
+5535 AbandonState::NotAbandoned
+5536 ) {
+5537 // The peer MUST respond with a corresponding PATH_ABANDON frame.
+5538 // The other peer has 3 * PTO to do that.
+5539 // This uses the PTO of the path we send on!
+5540 // If the PATH_ABANDON comes in within the deadline we're giving here, then this
+5541 // state will be set to `AbandonState::ReceivedPathAbandon`, essentially clearing
+5542 // the deadline. If we receive a frame after the deadline, we error out with a
+5543 // protocol violation.
+5544 // Receiving other frames before the deadline is fine, as those might be packets
+5545 // that were still in-flight.
+5546 abandoned_path.data.abandon_state = AbandonState::ExpectingPathAbandon {
+5547 deadline: now + 3 * send_pto,
+5548 };
+5549
+5550 // At some point, we need to forget about the path.
+5551 // If we do so too early, then we'll have discarded the CIDs of that path and won't
+5552 // handle incoming packets on that path correctly.
+5553 // To give this path enough time, we assume that the peer will have received our
+5554 // PATH_ABANDON within 3 * PTO of the path we sent the abandon on,
+5555 // and then we give the path 3 * PTO time to make it very unlikely that there will
+5556 // still be packets incoming on the path at that point.
+5557 // This timer will actually get reset to a value that's likely to be even earlier
+5558 // once we actually receive the PATH_ABANDON frame itself.
+5559 let abandoned_pto =
+5560 self.paths.get(&path_id).unwrap().data.rtt.pto_base() + ack_delay;
+5561 self.timers.set(
+5562 Timer::PerPath(abandoned_path_id, PathTimer::DiscardPath),
+5563 now + 3 * send_pto + 3 * abandoned_pto,
+5564 self.qlog.with_time(now),
+5565 );
+5566 }
+5567 } else {
+5568 warn!("sent PATH_ABANDON after path was already discarded");
+5569 }
+5570 }
+5571
+5572 // PATH_STATUS_AVAILABLE & PATH_STATUS_BACKUP
+5573 while !path_exclusive_only
+5574 && space_id == SpaceId::Data
+5575 && frame::PathStatusAvailable::SIZE_BOUND <= builder.frame_space_remaining()
+5576 {
+5577 let Some(path_id) = space.pending.path_status.pop_first() else {
+5578 break;
+5579 };
+5580 let Some(path) = self.paths.get(&path_id).map(|path_state| &path_state.data) else {
+5581 trace!(%path_id, "discarding queued path status for unknown path");
+5582 continue;
+5583 };
+5584
+5585 let seq = path.status.seq();
+5586 match path.local_status() {
+5587 PathStatus::Available => {
+5588 let frame = frame::PathStatusAvailable {
+5589 path_id,
+5590 status_seq_no: seq,
+5591 };
+5592 builder.write_frame(frame, stats);
+5593 }
+5594 PathStatus::Backup => {
+5595 let frame = frame::PathStatusBackup {
+5596 path_id,
+5597 status_seq_no: seq,
+5598 };
+5599 builder.write_frame(frame, stats);
5600 }
-5601
-5602 let token = Token::new(
-5603 TokenPayload::Validation {
-5604 ip: network_path.remote.ip(),
-5605 issued: server_config.time_source.now(),
-5606 },
-5607 &mut self.rng,
-5608 );
-5609 let new_token = NewToken {
-5610 token: token.encode(&*server_config.token_key).into(),
-5611 };
-5612
-5613 if builder.frame_space_remaining() < new_token.size() {
-5614 space.pending.new_tokens.push(network_path);
-5615 break;
-5616 }
-5617
-5618 builder.write_frame(new_token, stats);
-5619 builder.retransmits_mut().new_tokens.push(network_path);
-5620 }
-5621 }
-5622
-5623 // STREAM
-5624 if !path_exclusive_only && space_id == SpaceId::Data {
-5625 self.streams
-5626 .write_stream_frames(builder, self.config.send_fairness, stats);
-5627 }
-5628
-5629 // ADD_ADDRESS
-5630 // TODO(@divma): check if we need to do path exclusive filters
-5631 while space_id == SpaceId::Data
-5632 && frame::AddAddress::SIZE_BOUND <= builder.frame_space_remaining()
-5633 {
-5634 if let Some(added_address) = space.pending.add_address.pop_last() {
-5635 builder.write_frame(added_address, stats);
-5636 } else {
-5637 break;
-5638 }
-5639 }
-5640
-5641 // REMOVE_ADDRESS
-5642 while space_id == SpaceId::Data
-5643 && frame::RemoveAddress::SIZE_BOUND <= builder.frame_space_remaining()
-5644 {
-5645 if let Some(removed_address) = space.pending.remove_address.pop_last() {
-5646 builder.write_frame(removed_address, stats);
-5647 } else {
-5648 break;
-5649 }
-5650 }
-5651 }
-5652
-5653 /// Write pending ACKs into a buffer
-5654 fn populate_acks<'a, 'b>(
-5655 now: Instant,
-5656 receiving_ecn: bool,
-5657 path_id: PathId,
-5658 space_id: SpaceId,
-5659 space: &mut PacketSpace,
-5660 is_multipath_negotiated: bool,
-5661 builder: &mut PacketBuilder<'a, 'b>,
-5662 stats: &mut FrameStats,
-5663 ) {
-5664 // 0-RTT packets must never carry acks (which would have to be of handshake packets)
-5665 debug_assert!(space.crypto.is_some(), "tried to send ACK in 0-RTT");
-5666
-5667 debug_assert!(
-5668 is_multipath_negotiated || path_id == PathId::ZERO,
-5669 "Only PathId::ZERO allowed without multipath (have {path_id:?})"
-5670 );
-5671 if is_multipath_negotiated {
-5672 debug_assert!(
-5673 space_id == SpaceId::Data || path_id == PathId::ZERO,
-5674 "path acks must be sent in 1RTT space (have {space_id:?})"
-5675 );
-5676 }
-5677
-5678 let pns = space.for_path(path_id);
-5679 let ranges = pns.pending_acks.ranges();
-5680 debug_assert!(!ranges.is_empty(), "can not send empty ACK range");
-5681 let ecn = if receiving_ecn {
-5682 Some(&pns.ecn_counters)
-5683 } else {
-5684 None
-5685 };
-5686
-5687 let delay_micros = pns.pending_acks.ack_delay(now).as_micros() as u64;
-5688 // TODO: This should come from `TransportConfig` if that gets configurable.
-5689 let ack_delay_exp = TransportParameters::default().ack_delay_exponent;
-5690 let delay = delay_micros >> ack_delay_exp.into_inner();
-5691
-5692 if is_multipath_negotiated && space_id == SpaceId::Data {
-5693 if !ranges.is_empty() {
-5694 let frame = frame::PathAck::encoder(path_id, delay, ranges, ecn);
-5695 builder.write_frame(frame, stats);
-5696 }
-5697 } else {
-5698 builder.write_frame(frame::Ack::encoder(delay, ranges, ecn), stats);
-5699 }
-5700 }
-5701
-5702 fn close_common(&mut self) {
-5703 trace!("connection closed");
-5704 self.timers.reset();
-5705 }
-5706
-5707 fn set_close_timer(&mut self, now: Instant) {
-5708 // QUIC-MULTIPATH § 2.6 Connection Closure: draining for 3*PTO with PTO the max of
-5709 // the PTO for all paths.
-5710 let pto_max = self.pto_max_path(self.highest_space, true);
-5711 self.timers.set(
-5712 Timer::Conn(ConnTimer::Close),
-5713 now + 3 * pto_max,
-5714 self.qlog.with_time(now),
-5715 );
-5716 }
-5717
-5718 /// Handle transport parameters received from the peer
-5719 ///
-5720 /// *rem_cid* and *loc_cid* are the source and destination CIDs respectively of the
-5721 /// *packet into which the transport parameters arrived.
-5722 fn handle_peer_params(
-5723 &mut self,
-5724 params: TransportParameters,
-5725 loc_cid: ConnectionId,
-5726 rem_cid: ConnectionId,
-5727 now: Instant,
-5728 ) -> Result<(), TransportError> {
-5729 if Some(self.orig_rem_cid) != params.initial_src_cid
-5730 || (self.side.is_client()
-5731 && (Some(self.initial_dst_cid) != params.original_dst_cid
-5732 || self.retry_src_cid != params.retry_src_cid))
-5733 {
-5734 return Err(TransportError::TRANSPORT_PARAMETER_ERROR(
-5735 "CID authentication failure",
-5736 ));
-5737 }
-5738 if params.initial_max_path_id.is_some() && (loc_cid.is_empty() || rem_cid.is_empty()) {
-5739 return Err(TransportError::PROTOCOL_VIOLATION(
-5740 "multipath must not use zero-length CIDs",
-5741 ));
-5742 }
-5743
-5744 self.set_peer_params(params);
-5745 self.qlog.emit_peer_transport_params_received(self, now);
-5746
-5747 Ok(())
-5748 }
-5749
-5750 fn set_peer_params(&mut self, params: TransportParameters) {
-5751 self.streams.set_params(¶ms);
-5752 self.idle_timeout =
-5753 negotiate_max_idle_timeout(self.config.max_idle_timeout, Some(params.max_idle_timeout));
-5754 trace!("negotiated max idle timeout {:?}", self.idle_timeout);
-5755
-5756 if let Some(ref info) = params.preferred_address {
-5757 // During the handshake PathId::ZERO exists.
-5758 self.rem_cids.get_mut(&PathId::ZERO).expect("not yet abandoned").insert(frame::NewConnectionId {
-5759 path_id: None,
-5760 sequence: 1,
-5761 id: info.connection_id,
-5762 reset_token: info.stateless_reset_token,
-5763 retire_prior_to: 0,
-5764 })
-5765 .expect(
-5766 "preferred address CID is the first received, and hence is guaranteed to be legal",
-5767 );
-5768 let remote = self.path_data(PathId::ZERO).network_path.remote;
-5769 self.set_reset_token(PathId::ZERO, remote, info.stateless_reset_token);
-5770 }
-5771 self.ack_frequency.peer_max_ack_delay = get_max_ack_delay(¶ms);
-5772
-5773 let mut multipath_enabled = None;
-5774 if let (Some(local_max_path_id), Some(remote_max_path_id)) = (
-5775 self.config.get_initial_max_path_id(),
-5776 params.initial_max_path_id,
-5777 ) {
-5778 // multipath is enabled, register the local and remote maximums
-5779 self.local_max_path_id = local_max_path_id;
-5780 self.remote_max_path_id = remote_max_path_id;
-5781 let initial_max_path_id = local_max_path_id.min(remote_max_path_id);
-5782 debug!(%initial_max_path_id, "multipath negotiated");
-5783 multipath_enabled = Some(initial_max_path_id);
-5784 }
-5785
-5786 if let Some((max_locally_allowed_remote_addresses, max_remotely_allowed_remote_addresses)) =
-5787 self.config
-5788 .max_remote_nat_traversal_addresses
-5789 .zip(params.max_remote_nat_traversal_addresses)
-5790 {
-5791 if let Some(max_initial_paths) =
-5792 multipath_enabled.map(|path_id| path_id.saturating_add(1u8))
-5793 {
-5794 let max_local_addresses = max_remotely_allowed_remote_addresses.get();
-5795 let max_remote_addresses = max_locally_allowed_remote_addresses.get();
-5796 self.iroh_hp =
-5797 iroh_hp::State::new(max_remote_addresses, max_local_addresses, self.side());
-5798 debug!(
-5799 %max_remote_addresses, %max_local_addresses,
-5800 "iroh hole punching negotiated"
-5801 );
-5802
-5803 match self.side() {
-5804 Side::Client => {
-5805 if max_initial_paths.as_u32() < max_remote_addresses as u32 + 1 {
-5806 // in this case the client might try to open `max_remote_addresses` new
-5807 // paths, but the current multipath configuration will not allow it
-5808 warn!(%max_initial_paths, %max_remote_addresses, "local client configuration might cause nat traversal issues")
-5809 } else if max_local_addresses as u64
-5810 > params.active_connection_id_limit.into_inner()
-5811 {
-5812 // the server allows us to send at most `params.active_connection_id_limit`
-5813 // but they might need at least `max_local_addresses` to effectively send
-5814 // `PATH_CHALLENGE` frames to each advertised local address
-5815 warn!(%max_local_addresses, remote_cid_limit=%params.active_connection_id_limit.into_inner(), "remote server configuration might cause nat traversal issues")
-5816 }
-5817 }
-5818 Side::Server => {
-5819 if (max_initial_paths.as_u32() as u64) < crate::LOC_CID_COUNT {
-5820 warn!(%max_initial_paths, local_cid_limit=%crate::LOC_CID_COUNT, "local server configuration might cause nat traversal issues")
-5821 }
-5822 }
-5823 }
-5824 } else {
-5825 debug!("iroh nat traversal enabled for both endpoints, but multipath is missing")
-5826 }
+5601 }
+5602 }
+5603
+5604 // MAX_PATH_ID
+5605 if space_id == SpaceId::Data
+5606 && space.pending.max_path_id
+5607 && frame::MaxPathId::SIZE_BOUND <= builder.frame_space_remaining()
+5608 {
+5609 let frame = frame::MaxPathId(self.local_max_path_id);
+5610 builder.write_frame(frame, stats);
+5611 space.pending.max_path_id = false;
+5612 }
+5613
+5614 // PATHS_BLOCKED
+5615 if space_id == SpaceId::Data
+5616 && space.pending.paths_blocked
+5617 && frame::PathsBlocked::SIZE_BOUND <= builder.frame_space_remaining()
+5618 {
+5619 let frame = frame::PathsBlocked(self.remote_max_path_id);
+5620 builder.write_frame(frame, stats);
+5621 space.pending.paths_blocked = false;
+5622 }
+5623
+5624 // PATH_CIDS_BLOCKED
+5625 while space_id == SpaceId::Data
+5626 && frame::PathCidsBlocked::SIZE_BOUND <= builder.frame_space_remaining()
+5627 {
+5628 let Some(path_id) = space.pending.path_cids_blocked.pop_first() else {
+5629 break;
+5630 };
+5631 let next_seq = match self.rem_cids.get(&path_id) {
+5632 Some(cid_queue) => VarInt(cid_queue.active_seq() + 1),
+5633 None => VarInt(0),
+5634 };
+5635 let frame = frame::PathCidsBlocked { path_id, next_seq };
+5636 builder.write_frame(frame, stats);
+5637 }
+5638
+5639 // RESET_STREAM, STOP_SENDING, MAX_DATA, MAX_STREAM_DATA, MAX_STREAMS
+5640 if space_id == SpaceId::Data {
+5641 self.streams
+5642 .write_control_frames(builder, &mut space.pending, stats);
+5643 }
+5644
+5645 // NEW_CONNECTION_ID
+5646 let cid_len = self
+5647 .local_cid_state
+5648 .values()
+5649 .map(|cid_state| cid_state.cid_len())
+5650 .max()
+5651 .expect("some local CID state must exist");
+5652 let new_cid_size_bound =
+5653 frame::NewConnectionId::size_bound(is_multipath_negotiated, cid_len);
+5654 while !path_exclusive_only && builder.frame_space_remaining() > new_cid_size_bound {
+5655 let issued = match space.pending.new_cids.pop() {
+5656 Some(x) => x,
+5657 None => break,
+5658 };
+5659 let retire_prior_to = self
+5660 .local_cid_state
+5661 .get(&issued.path_id)
+5662 .map(|cid_state| cid_state.retire_prior_to())
+5663 .unwrap_or_else(|| panic!("missing local CID state for path={}", issued.path_id));
+5664
+5665 let cid_path_id = match is_multipath_negotiated {
+5666 true => Some(issued.path_id),
+5667 false => {
+5668 debug_assert_eq!(issued.path_id, PathId::ZERO);
+5669 None
+5670 }
+5671 };
+5672 let frame = frame::NewConnectionId {
+5673 path_id: cid_path_id,
+5674 sequence: issued.sequence,
+5675 retire_prior_to,
+5676 id: issued.id,
+5677 reset_token: issued.reset_token,
+5678 };
+5679 builder.write_frame(frame, stats);
+5680 }
+5681
+5682 // RETIRE_CONNECTION_ID
+5683 let retire_cid_bound = frame::RetireConnectionId::size_bound(is_multipath_negotiated);
+5684 while !path_exclusive_only && builder.frame_space_remaining() > retire_cid_bound {
+5685 let (path_id, sequence) = match space.pending.retire_cids.pop() {
+5686 Some((PathId::ZERO, seq)) if !is_multipath_negotiated => (None, seq),
+5687 Some((path_id, seq)) => (Some(path_id), seq),
+5688 None => break,
+5689 };
+5690 let frame = frame::RetireConnectionId { path_id, sequence };
+5691 builder.write_frame(frame, stats);
+5692 }
+5693
+5694 // DATAGRAM
+5695 let mut sent_datagrams = false;
+5696 while !path_exclusive_only
+5697 && builder.frame_space_remaining() > Datagram::SIZE_BOUND
+5698 && space_id == SpaceId::Data
+5699 {
+5700 match self.datagrams.write(builder, stats) {
+5701 true => {
+5702 sent_datagrams = true;
+5703 }
+5704 false => break,
+5705 }
+5706 }
+5707 if self.datagrams.send_blocked && sent_datagrams {
+5708 self.events.push_back(Event::DatagramsUnblocked);
+5709 self.datagrams.send_blocked = false;
+5710 }
+5711
+5712 let path = &mut self.paths.get_mut(&path_id).expect("known path").data;
+5713
+5714 // NEW_TOKEN
+5715 if !path_exclusive_only {
+5716 while let Some(network_path) = space.pending.new_tokens.pop() {
+5717 debug_assert_eq!(space_id, SpaceId::Data);
+5718 let ConnectionSide::Server { server_config } = &self.side else {
+5719 panic!("NEW_TOKEN frames should not be enqueued by clients");
+5720 };
+5721
+5722 if !network_path.is_probably_same_path(&path.network_path) {
+5723 // NEW_TOKEN frames contain tokens bound to a client's IP address, and are only
+5724 // useful if used from the same IP address. Thus, we abandon enqueued NEW_TOKEN
+5725 // frames upon an path change. Instead, when the new path becomes validated,
+5726 // NEW_TOKEN frames may be enqueued for the new path instead.
+5727 continue;
+5728 }
+5729
+5730 let token = Token::new(
+5731 TokenPayload::Validation {
+5732 ip: network_path.remote.ip(),
+5733 issued: server_config.time_source.now(),
+5734 },
+5735 &mut self.rng,
+5736 );
+5737 let new_token = NewToken {
+5738 token: token.encode(&*server_config.token_key).into(),
+5739 };
+5740
+5741 if builder.frame_space_remaining() < new_token.size() {
+5742 space.pending.new_tokens.push(network_path);
+5743 break;
+5744 }
+5745
+5746 builder.write_frame(new_token, stats);
+5747 builder.retransmits_mut().new_tokens.push(network_path);
+5748 }
+5749 }
+5750
+5751 // STREAM
+5752 if !path_exclusive_only && space_id == SpaceId::Data {
+5753 self.streams
+5754 .write_stream_frames(builder, self.config.send_fairness, stats);
+5755 }
+5756
+5757 // ADD_ADDRESS
+5758 // TODO(@divma): check if we need to do path exclusive filters
+5759 while space_id == SpaceId::Data
+5760 && frame::AddAddress::SIZE_BOUND <= builder.frame_space_remaining()
+5761 {
+5762 if let Some(added_address) = space.pending.add_address.pop_last() {
+5763 builder.write_frame(added_address, stats);
+5764 } else {
+5765 break;
+5766 }
+5767 }
+5768
+5769 // REMOVE_ADDRESS
+5770 while space_id == SpaceId::Data
+5771 && frame::RemoveAddress::SIZE_BOUND <= builder.frame_space_remaining()
+5772 {
+5773 if let Some(removed_address) = space.pending.remove_address.pop_last() {
+5774 builder.write_frame(removed_address, stats);
+5775 } else {
+5776 break;
+5777 }
+5778 }
+5779 }
+5780
+5781 /// Write pending ACKs into a buffer
+5782 fn populate_acks<'a, 'b>(
+5783 now: Instant,
+5784 receiving_ecn: bool,
+5785 path_id: PathId,
+5786 space_id: SpaceId,
+5787 space: &mut PacketSpace,
+5788 is_multipath_negotiated: bool,
+5789 builder: &mut PacketBuilder<'a, 'b>,
+5790 stats: &mut FrameStats,
+5791 ) {
+5792 // 0-RTT packets must never carry acks (which would have to be of handshake packets)
+5793 debug_assert!(space.crypto.is_some(), "tried to send ACK in 0-RTT");
+5794
+5795 debug_assert!(
+5796 is_multipath_negotiated || path_id == PathId::ZERO,
+5797 "Only PathId::ZERO allowed without multipath (have {path_id:?})"
+5798 );
+5799 if is_multipath_negotiated {
+5800 debug_assert!(
+5801 space_id == SpaceId::Data || path_id == PathId::ZERO,
+5802 "path acks must be sent in 1RTT space (have {space_id:?})"
+5803 );
+5804 }
+5805
+5806 let pns = space.for_path(path_id);
+5807 let ranges = pns.pending_acks.ranges();
+5808 debug_assert!(!ranges.is_empty(), "can not send empty ACK range");
+5809 let ecn = if receiving_ecn {
+5810 Some(&pns.ecn_counters)
+5811 } else {
+5812 None
+5813 };
+5814
+5815 let delay_micros = pns.pending_acks.ack_delay(now).as_micros() as u64;
+5816 // TODO: This should come from `TransportConfig` if that gets configurable.
+5817 let ack_delay_exp = TransportParameters::default().ack_delay_exponent;
+5818 let delay = delay_micros >> ack_delay_exp.into_inner();
+5819
+5820 if is_multipath_negotiated && space_id == SpaceId::Data {
+5821 if !ranges.is_empty() {
+5822 let frame = frame::PathAck::encoder(path_id, delay, ranges, ecn);
+5823 builder.write_frame(frame, stats);
+5824 }
+5825 } else {
+5826 builder.write_frame(frame::Ack::encoder(delay, ranges, ecn), stats);
5827 }
-5828
-5829 self.peer_params = params;
-5830 let peer_max_udp_payload_size =
-5831 u16::try_from(self.peer_params.max_udp_payload_size.into_inner()).unwrap_or(u16::MAX);
-5832 self.path_data_mut(PathId::ZERO)
-5833 .mtud
-5834 .on_peer_max_udp_payload_size_received(peer_max_udp_payload_size);
-5835 }
-5836
-5837 /// Decrypts a packet, returning the packet number on success
-5838 fn decrypt_packet(
-5839 &mut self,
-5840 now: Instant,
-5841 path_id: PathId,
-5842 packet: &mut Packet,
-5843 ) -> Result<Option<u64>, Option<TransportError>> {
-5844 let result = packet_crypto::decrypt_packet_body(
-5845 packet,
-5846 path_id,
-5847 &self.spaces,
-5848 self.zero_rtt_crypto.as_ref(),
-5849 self.key_phase,
-5850 self.prev_crypto.as_ref(),
-5851 self.next_crypto.as_ref(),
-5852 )?;
-5853
-5854 let result = match result {
-5855 Some(r) => r,
-5856 None => return Ok(None),
-5857 };
-5858
-5859 if result.outgoing_key_update_acked {
-5860 if let Some(prev) = self.prev_crypto.as_mut() {
-5861 prev.end_packet = Some((result.number, now));
-5862 self.set_key_discard_timer(now, packet.header.space());
-5863 }
-5864 }
-5865
-5866 if result.incoming_key_update {
-5867 trace!("key update authenticated");
-5868 self.update_keys(Some((result.number, now)), true);
-5869 self.set_key_discard_timer(now, packet.header.space());
+5828 }
+5829
+5830 fn close_common(&mut self) {
+5831 trace!("connection closed");
+5832 self.timers.reset();
+5833 }
+5834
+5835 fn set_close_timer(&mut self, now: Instant) {
+5836 // QUIC-MULTIPATH § 2.6 Connection Closure: draining for 3*PTO with PTO the max of
+5837 // the PTO for all paths.
+5838 let pto_max = self.pto_max_path(self.highest_space, true);
+5839 self.timers.set(
+5840 Timer::Conn(ConnTimer::Close),
+5841 now + 3 * pto_max,
+5842 self.qlog.with_time(now),
+5843 );
+5844 }
+5845
+5846 /// Handle transport parameters received from the peer
+5847 ///
+5848 /// *rem_cid* and *loc_cid* are the source and destination CIDs respectively of the
+5849 /// *packet into which the transport parameters arrived.
+5850 fn handle_peer_params(
+5851 &mut self,
+5852 params: TransportParameters,
+5853 loc_cid: ConnectionId,
+5854 rem_cid: ConnectionId,
+5855 now: Instant,
+5856 ) -> Result<(), TransportError> {
+5857 if Some(self.orig_rem_cid) != params.initial_src_cid
+5858 || (self.side.is_client()
+5859 && (Some(self.initial_dst_cid) != params.original_dst_cid
+5860 || self.retry_src_cid != params.retry_src_cid))
+5861 {
+5862 return Err(TransportError::TRANSPORT_PARAMETER_ERROR(
+5863 "CID authentication failure",
+5864 ));
+5865 }
+5866 if params.initial_max_path_id.is_some() && (loc_cid.is_empty() || rem_cid.is_empty()) {
+5867 return Err(TransportError::PROTOCOL_VIOLATION(
+5868 "multipath must not use zero-length CIDs",
+5869 ));
5870 }
5871
-5872 Ok(Some(result.number))
-5873 }
+5872 self.set_peer_params(params);
+5873 self.qlog.emit_peer_transport_params_received(self, now);
5874
-5875 fn update_keys(&mut self, end_packet: Option<(u64, Instant)>, remote: bool) {
-5876 trace!("executing key update");
-5877 // Generate keys for the key phase after the one we're switching to, store them in
-5878 // `next_crypto`, make the contents of `next_crypto` current, and move the current keys into
-5879 // `prev_crypto`.
-5880 let new = self
-5881 .crypto
-5882 .next_1rtt_keys()
-5883 .expect("only called for `Data` packets");
-5884 self.key_phase_size = new
-5885 .local
-5886 .confidentiality_limit()
-5887 .saturating_sub(KEY_UPDATE_MARGIN);
-5888 let old = mem::replace(
-5889 &mut self.spaces[SpaceId::Data]
-5890 .crypto
-5891 .as_mut()
-5892 .unwrap() // safe because update_keys() can only be triggered by short packets
-5893 .packet,
-5894 mem::replace(self.next_crypto.as_mut().unwrap(), new),
-5895 );
-5896 self.spaces[SpaceId::Data]
-5897 .iter_paths_mut()
-5898 .for_each(|s| s.sent_with_keys = 0);
-5899 self.prev_crypto = Some(PrevCrypto {
-5900 crypto: old,
-5901 end_packet,
-5902 update_unacked: remote,
-5903 });
-5904 self.key_phase = !self.key_phase;
-5905 }
-5906
-5907 fn peer_supports_ack_frequency(&self) -> bool {
-5908 self.peer_params.min_ack_delay.is_some()
-5909 }
-5910
-5911 /// Send an IMMEDIATE_ACK frame to the remote endpoint
-5912 ///
-5913 /// According to the spec, this will result in an error if the remote endpoint does not support
-5914 /// the Acknowledgement Frequency extension
-5915 pub(crate) fn immediate_ack(&mut self, path_id: PathId) {
-5916 debug_assert_eq!(
-5917 self.highest_space,
-5918 SpaceId::Data,
-5919 "immediate ack must be written in the data space"
-5920 );
-5921 self.spaces[self.highest_space]
-5922 .for_path(path_id)
-5923 .immediate_ack_pending = true;
-5924 }
-5925
-5926 /// Decodes a packet, returning its decrypted payload, so it can be inspected in tests
-5927 #[cfg(test)]
-5928 pub(crate) fn decode_packet(&self, event: &ConnectionEvent) -> Option<Vec<u8>> {
-5929 let (path_id, first_decode, remaining) = match &event.0 {
-5930 ConnectionEventInner::Datagram(DatagramConnectionEvent {
-5931 path_id,
-5932 first_decode,
-5933 remaining,
-5934 ..
-5935 }) => (path_id, first_decode, remaining),
-5936 _ => return None,
-5937 };
-5938
-5939 if remaining.is_some() {
-5940 panic!("Packets should never be coalesced in tests");
-5941 }
-5942
-5943 let decrypted_header = packet_crypto::unprotect_header(
-5944 first_decode.clone(),
-5945 &self.spaces,
-5946 self.zero_rtt_crypto.as_ref(),
-5947 self.peer_params.stateless_reset_token,
-5948 )?;
-5949
-5950 let mut packet = decrypted_header.packet?;
-5951 packet_crypto::decrypt_packet_body(
-5952 &mut packet,
-5953 *path_id,
-5954 &self.spaces,
-5955 self.zero_rtt_crypto.as_ref(),
-5956 self.key_phase,
-5957 self.prev_crypto.as_ref(),
-5958 self.next_crypto.as_ref(),
-5959 )
-5960 .ok()?;
-5961
-5962 Some(packet.payload.to_vec())
+5875 Ok(())
+5876 }
+5877
+5878 fn set_peer_params(&mut self, params: TransportParameters) {
+5879 self.streams.set_params(¶ms);
+5880 self.idle_timeout =
+5881 negotiate_max_idle_timeout(self.config.max_idle_timeout, Some(params.max_idle_timeout));
+5882 trace!("negotiated max idle timeout {:?}", self.idle_timeout);
+5883
+5884 if let Some(ref info) = params.preferred_address {
+5885 // During the handshake PathId::ZERO exists.
+5886 self.rem_cids.get_mut(&PathId::ZERO).expect("not yet abandoned").insert(frame::NewConnectionId {
+5887 path_id: None,
+5888 sequence: 1,
+5889 id: info.connection_id,
+5890 reset_token: info.stateless_reset_token,
+5891 retire_prior_to: 0,
+5892 })
+5893 .expect(
+5894 "preferred address CID is the first received, and hence is guaranteed to be legal",
+5895 );
+5896 let remote = self.path_data(PathId::ZERO).network_path.remote;
+5897 self.set_reset_token(PathId::ZERO, remote, info.stateless_reset_token);
+5898 }
+5899 self.ack_frequency.peer_max_ack_delay = get_max_ack_delay(¶ms);
+5900
+5901 let mut multipath_enabled = None;
+5902 if let (Some(local_max_path_id), Some(remote_max_path_id)) = (
+5903 self.config.get_initial_max_path_id(),
+5904 params.initial_max_path_id,
+5905 ) {
+5906 // multipath is enabled, register the local and remote maximums
+5907 self.local_max_path_id = local_max_path_id;
+5908 self.remote_max_path_id = remote_max_path_id;
+5909 let initial_max_path_id = local_max_path_id.min(remote_max_path_id);
+5910 debug!(%initial_max_path_id, "multipath negotiated");
+5911 multipath_enabled = Some(initial_max_path_id);
+5912 }
+5913
+5914 if let Some((max_locally_allowed_remote_addresses, max_remotely_allowed_remote_addresses)) =
+5915 self.config
+5916 .max_remote_nat_traversal_addresses
+5917 .zip(params.max_remote_nat_traversal_addresses)
+5918 {
+5919 if let Some(max_initial_paths) =
+5920 multipath_enabled.map(|path_id| path_id.saturating_add(1u8))
+5921 {
+5922 let max_local_addresses = max_remotely_allowed_remote_addresses.get();
+5923 let max_remote_addresses = max_locally_allowed_remote_addresses.get();
+5924 self.iroh_hp =
+5925 iroh_hp::State::new(max_remote_addresses, max_local_addresses, self.side());
+5926 debug!(
+5927 %max_remote_addresses, %max_local_addresses,
+5928 "iroh hole punching negotiated"
+5929 );
+5930
+5931 match self.side() {
+5932 Side::Client => {
+5933 if max_initial_paths.as_u32() < max_remote_addresses as u32 + 1 {
+5934 // in this case the client might try to open `max_remote_addresses` new
+5935 // paths, but the current multipath configuration will not allow it
+5936 warn!(%max_initial_paths, %max_remote_addresses, "local client configuration might cause nat traversal issues")
+5937 } else if max_local_addresses as u64
+5938 > params.active_connection_id_limit.into_inner()
+5939 {
+5940 // the server allows us to send at most `params.active_connection_id_limit`
+5941 // but they might need at least `max_local_addresses` to effectively send
+5942 // `PATH_CHALLENGE` frames to each advertised local address
+5943 warn!(%max_local_addresses, remote_cid_limit=%params.active_connection_id_limit.into_inner(), "remote server configuration might cause nat traversal issues")
+5944 }
+5945 }
+5946 Side::Server => {
+5947 if (max_initial_paths.as_u32() as u64) < crate::LOC_CID_COUNT {
+5948 warn!(%max_initial_paths, local_cid_limit=%crate::LOC_CID_COUNT, "local server configuration might cause nat traversal issues")
+5949 }
+5950 }
+5951 }
+5952 } else {
+5953 debug!("iroh nat traversal enabled for both endpoints, but multipath is missing")
+5954 }
+5955 }
+5956
+5957 self.peer_params = params;
+5958 let peer_max_udp_payload_size =
+5959 u16::try_from(self.peer_params.max_udp_payload_size.into_inner()).unwrap_or(u16::MAX);
+5960 self.path_data_mut(PathId::ZERO)
+5961 .mtud
+5962 .on_peer_max_udp_payload_size_received(peer_max_udp_payload_size);
5963 }
5964
-5965 /// The number of bytes of packets containing retransmittable frames that have not been
-5966 /// acknowledged or declared lost.
-5967 #[cfg(test)]
-5968 pub(crate) fn bytes_in_flight(&self) -> u64 {
-5969 // TODO(@divma): consider including for multipath?
-5970 self.path_data(PathId::ZERO).in_flight.bytes
-5971 }
-5972
-5973 /// Number of bytes worth of non-ack-only packets that may be sent
-5974 #[cfg(test)]
-5975 pub(crate) fn congestion_window(&self) -> u64 {
-5976 let path = self.path_data(PathId::ZERO);
-5977 path.congestion
-5978 .window()
-5979 .saturating_sub(path.in_flight.bytes)
-5980 }
+5965 /// Decrypts a packet, returning the packet number on success
+5966 fn decrypt_packet(
+5967 &mut self,
+5968 now: Instant,
+5969 path_id: PathId,
+5970 packet: &mut Packet,
+5971 ) -> Result<Option<u64>, Option<TransportError>> {
+5972 let result = packet_crypto::decrypt_packet_body(
+5973 packet,
+5974 path_id,
+5975 &self.spaces,
+5976 self.zero_rtt_crypto.as_ref(),
+5977 self.key_phase,
+5978 self.prev_crypto.as_ref(),
+5979 self.next_crypto.as_ref(),
+5980 )?;
5981
-5982 /// Whether no timers but keepalive, idle, rtt, pushnewcid, and key discard are running
-5983 #[cfg(test)]
-5984 pub(crate) fn is_idle(&self) -> bool {
-5985 let current_timers = self.timers.values();
-5986 current_timers
-5987 .into_iter()
-5988 .filter(|(timer, _)| {
-5989 !matches!(
-5990 timer,
-5991 Timer::Conn(ConnTimer::KeepAlive)
-5992 | Timer::PerPath(_, PathTimer::PathKeepAlive)
-5993 | Timer::Conn(ConnTimer::PushNewCid)
-5994 | Timer::Conn(ConnTimer::KeyDiscard)
-5995 )
-5996 })
-5997 .min_by_key(|(_, time)| *time)
-5998 .is_none_or(|(timer, _)| timer == Timer::Conn(ConnTimer::Idle))
-5999 }
-6000
-6001 /// Whether explicit congestion notification is in use on outgoing packets.
-6002 #[cfg(test)]
-6003 pub(crate) fn using_ecn(&self) -> bool {
-6004 self.path_data(PathId::ZERO).sending_ecn
-6005 }
-6006
-6007 /// The number of received bytes in the current path
-6008 #[cfg(test)]
-6009 pub(crate) fn total_recvd(&self) -> u64 {
-6010 self.path_data(PathId::ZERO).total_recvd
-6011 }
-6012
-6013 #[cfg(test)]
-6014 pub(crate) fn active_local_cid_seq(&self) -> (u64, u64) {
-6015 self.local_cid_state
-6016 .get(&PathId::ZERO)
-6017 .unwrap()
-6018 .active_seq()
-6019 }
-6020
-6021 #[cfg(test)]
-6022 #[track_caller]
-6023 pub(crate) fn active_local_path_cid_seq(&self, path_id: u32) -> (u64, u64) {
-6024 self.local_cid_state
-6025 .get(&PathId(path_id))
-6026 .unwrap()
-6027 .active_seq()
-6028 }
-6029
-6030 /// Instruct the peer to replace previously issued CIDs by sending a NEW_CONNECTION_ID frame
-6031 /// with updated `retire_prior_to` field set to `v`
-6032 #[cfg(test)]
-6033 pub(crate) fn rotate_local_cid(&mut self, v: u64, now: Instant) {
-6034 let n = self
-6035 .local_cid_state
-6036 .get_mut(&PathId::ZERO)
-6037 .unwrap()
-6038 .assign_retire_seq(v);
-6039 self.endpoint_events
-6040 .push_back(EndpointEventInner::NeedIdentifiers(PathId::ZERO, now, n));
-6041 }
-6042
-6043 /// Check the current active remote CID sequence for `PathId::ZERO`
-6044 #[cfg(test)]
-6045 pub(crate) fn active_rem_cid_seq(&self) -> u64 {
-6046 self.rem_cids.get(&PathId::ZERO).unwrap().active_seq()
-6047 }
-6048
-6049 /// Returns the detected maximum udp payload size for the current path
-6050 #[cfg(test)]
-6051 pub(crate) fn path_mtu(&self, path_id: PathId) -> u16 {
-6052 self.path_data(path_id).current_mtu()
-6053 }
-6054
-6055 /// Triggers path validation on all paths
-6056 #[cfg(test)]
-6057 pub(crate) fn trigger_path_validation(&mut self) {
-6058 for path in self.paths.values_mut() {
-6059 path.data.send_new_challenge = true;
-6060 }
-6061 }
-6062
-6063 /// Whether we have 1-RTT data to send
-6064 ///
-6065 /// This checks for frames that can only be sent in the data space (1-RTT):
-6066 /// - Pending PATH_CHALLENGE frames on the active and previous path if just migrated.
-6067 /// - Pending PATH_RESPONSE frames.
-6068 /// - Pending data to send in STREAM frames.
-6069 /// - Pending DATAGRAM frames to send.
-6070 ///
-6071 /// See also [`PacketSpace::can_send`] which keeps track of all other frame types that
-6072 /// may need to be sent.
-6073 fn can_send_1rtt(&self, path_id: PathId, max_size: usize) -> SendableFrames {
-6074 let path_exclusive = self.paths.get(&path_id).is_some_and(|path| {
-6075 path.data.send_new_challenge
-6076 || path
-6077 .prev
-6078 .as_ref()
-6079 .is_some_and(|(_, path)| path.send_new_challenge)
-6080 || !path.data.path_responses.is_empty()
-6081 });
-6082 let other = self.streams.can_send_stream_data()
-6083 || self
-6084 .datagrams
-6085 .outgoing
-6086 .front()
-6087 .is_some_and(|x| x.size(true) <= max_size);
-6088 SendableFrames {
-6089 acks: false,
-6090 other,
-6091 close: false,
-6092 path_exclusive,
-6093 }
-6094 }
-6095
-6096 /// Terminate the connection instantly, without sending a close packet
-6097 fn kill(&mut self, reason: ConnectionError) {
-6098 self.close_common();
-6099 self.state.move_to_drained(Some(reason));
-6100 self.endpoint_events.push_back(EndpointEventInner::Drained);
-6101 }
-6102
-6103 /// Storage size required for the largest packet that can be transmitted on all currently
-6104 /// available paths
-6105 ///
-6106 /// Buffers passed to [`Connection::poll_transmit`] should be at least this large.
-6107 ///
-6108 /// When multipath is enabled, this value is the minimum MTU across all available paths.
-6109 pub fn current_mtu(&self) -> u16 {
-6110 self.paths
-6111 .iter()
-6112 .filter(|&(path_id, _path_state)| !self.abandoned_paths.contains(path_id))
-6113 .map(|(_path_id, path_state)| path_state.data.current_mtu())
-6114 .min()
-6115 .expect("There is always at least one available path")
-6116 }
-6117
-6118 /// Size of non-frame data for a 1-RTT packet
-6119 ///
-6120 /// Quantifies space consumed by the QUIC header and AEAD tag. All other bytes in a packet are
-6121 /// frames. Changes if the length of the remote connection ID changes, which is expected to be
-6122 /// rare. If `pn` is specified, may additionally change unpredictably due to variations in
-6123 /// latency and packet loss.
-6124 fn predict_1rtt_overhead(&mut self, pn: u64, path: PathId) -> usize {
-6125 let pn_len = PacketNumber::new(
-6126 pn,
-6127 self.spaces[SpaceId::Data]
-6128 .for_path(path)
-6129 .largest_acked_packet
-6130 .unwrap_or(0),
-6131 )
-6132 .len();
-6133
-6134 // 1 byte for flags
-6135 1 + self
-6136 .rem_cids
-6137 .get(&path)
-6138 .map(|cids| cids.active().len())
-6139 .unwrap_or(20) // Max CID len in QUIC v1
-6140 + pn_len
-6141 + self.tag_len_1rtt()
-6142 }
-6143
-6144 fn predict_1rtt_overhead_no_pn(&self) -> usize {
-6145 let pn_len = 4;
-6146
-6147 let cid_len = self
-6148 .rem_cids
-6149 .values()
-6150 .map(|cids| cids.active().len())
-6151 .max()
-6152 .unwrap_or(20); // Max CID len in QUIC v1
-6153
-6154 // 1 byte for flags
-6155 1 + cid_len + pn_len + self.tag_len_1rtt()
+5982 let result = match result {
+5983 Some(r) => r,
+5984 None => return Ok(None),
+5985 };
+5986
+5987 if result.outgoing_key_update_acked {
+5988 if let Some(prev) = self.prev_crypto.as_mut() {
+5989 prev.end_packet = Some((result.number, now));
+5990 self.set_key_discard_timer(now, packet.header.space());
+5991 }
+5992 }
+5993
+5994 if result.incoming_key_update {
+5995 trace!("key update authenticated");
+5996 self.update_keys(Some((result.number, now)), true);
+5997 self.set_key_discard_timer(now, packet.header.space());
+5998 }
+5999
+6000 Ok(Some(result.number))
+6001 }
+6002
+6003 fn update_keys(&mut self, end_packet: Option<(u64, Instant)>, remote: bool) {
+6004 trace!("executing key update");
+6005 // Generate keys for the key phase after the one we're switching to, store them in
+6006 // `next_crypto`, make the contents of `next_crypto` current, and move the current keys into
+6007 // `prev_crypto`.
+6008 let new = self
+6009 .crypto
+6010 .next_1rtt_keys()
+6011 .expect("only called for `Data` packets");
+6012 self.key_phase_size = new
+6013 .local
+6014 .confidentiality_limit()
+6015 .saturating_sub(KEY_UPDATE_MARGIN);
+6016 let old = mem::replace(
+6017 &mut self.spaces[SpaceId::Data]
+6018 .crypto
+6019 .as_mut()
+6020 .unwrap() // safe because update_keys() can only be triggered by short packets
+6021 .packet,
+6022 mem::replace(self.next_crypto.as_mut().unwrap(), new),
+6023 );
+6024 self.spaces[SpaceId::Data]
+6025 .iter_paths_mut()
+6026 .for_each(|s| s.sent_with_keys = 0);
+6027 self.prev_crypto = Some(PrevCrypto {
+6028 crypto: old,
+6029 end_packet,
+6030 update_unacked: remote,
+6031 });
+6032 self.key_phase = !self.key_phase;
+6033 }
+6034
+6035 fn peer_supports_ack_frequency(&self) -> bool {
+6036 self.peer_params.min_ack_delay.is_some()
+6037 }
+6038
+6039 /// Send an IMMEDIATE_ACK frame to the remote endpoint
+6040 ///
+6041 /// According to the spec, this will result in an error if the remote endpoint does not support
+6042 /// the Acknowledgement Frequency extension
+6043 pub(crate) fn immediate_ack(&mut self, path_id: PathId) {
+6044 debug_assert_eq!(
+6045 self.highest_space,
+6046 SpaceId::Data,
+6047 "immediate ack must be written in the data space"
+6048 );
+6049 self.spaces[self.highest_space]
+6050 .for_path(path_id)
+6051 .immediate_ack_pending = true;
+6052 }
+6053
+6054 /// Decodes a packet, returning its decrypted payload, so it can be inspected in tests
+6055 #[cfg(test)]
+6056 pub(crate) fn decode_packet(&self, event: &ConnectionEvent) -> Option<Vec<u8>> {
+6057 let (path_id, first_decode, remaining) = match &event.0 {
+6058 ConnectionEventInner::Datagram(DatagramConnectionEvent {
+6059 path_id,
+6060 first_decode,
+6061 remaining,
+6062 ..
+6063 }) => (path_id, first_decode, remaining),
+6064 _ => return None,
+6065 };
+6066
+6067 if remaining.is_some() {
+6068 panic!("Packets should never be coalesced in tests");
+6069 }
+6070
+6071 let decrypted_header = packet_crypto::unprotect_header(
+6072 first_decode.clone(),
+6073 &self.spaces,
+6074 self.zero_rtt_crypto.as_ref(),
+6075 self.peer_params.stateless_reset_token,
+6076 )?;
+6077
+6078 let mut packet = decrypted_header.packet?;
+6079 packet_crypto::decrypt_packet_body(
+6080 &mut packet,
+6081 *path_id,
+6082 &self.spaces,
+6083 self.zero_rtt_crypto.as_ref(),
+6084 self.key_phase,
+6085 self.prev_crypto.as_ref(),
+6086 self.next_crypto.as_ref(),
+6087 )
+6088 .ok()?;
+6089
+6090 Some(packet.payload.to_vec())
+6091 }
+6092
+6093 /// The number of bytes of packets containing retransmittable frames that have not been
+6094 /// acknowledged or declared lost.
+6095 #[cfg(test)]
+6096 pub(crate) fn bytes_in_flight(&self) -> u64 {
+6097 // TODO(@divma): consider including for multipath?
+6098 self.path_data(PathId::ZERO).in_flight.bytes
+6099 }
+6100
+6101 /// Number of bytes worth of non-ack-only packets that may be sent
+6102 #[cfg(test)]
+6103 pub(crate) fn congestion_window(&self) -> u64 {
+6104 let path = self.path_data(PathId::ZERO);
+6105 path.congestion
+6106 .window()
+6107 .saturating_sub(path.in_flight.bytes)
+6108 }
+6109
+6110 /// Whether no timers but keepalive, idle, rtt, pushnewcid, and key discard are running
+6111 #[cfg(test)]
+6112 pub(crate) fn is_idle(&self) -> bool {
+6113 let current_timers = self.timers.values();
+6114 current_timers
+6115 .into_iter()
+6116 .filter(|(timer, _)| {
+6117 !matches!(
+6118 timer,
+6119 Timer::Conn(ConnTimer::KeepAlive)
+6120 | Timer::PerPath(_, PathTimer::PathKeepAlive)
+6121 | Timer::Conn(ConnTimer::PushNewCid)
+6122 | Timer::Conn(ConnTimer::KeyDiscard)
+6123 )
+6124 })
+6125 .min_by_key(|(_, time)| *time)
+6126 .is_none_or(|(timer, _)| timer == Timer::Conn(ConnTimer::Idle))
+6127 }
+6128
+6129 /// Whether explicit congestion notification is in use on outgoing packets.
+6130 #[cfg(test)]
+6131 pub(crate) fn using_ecn(&self) -> bool {
+6132 self.path_data(PathId::ZERO).sending_ecn
+6133 }
+6134
+6135 /// The number of received bytes in the current path
+6136 #[cfg(test)]
+6137 pub(crate) fn total_recvd(&self) -> u64 {
+6138 self.path_data(PathId::ZERO).total_recvd
+6139 }
+6140
+6141 #[cfg(test)]
+6142 pub(crate) fn active_local_cid_seq(&self) -> (u64, u64) {
+6143 self.local_cid_state
+6144 .get(&PathId::ZERO)
+6145 .unwrap()
+6146 .active_seq()
+6147 }
+6148
+6149 #[cfg(test)]
+6150 #[track_caller]
+6151 pub(crate) fn active_local_path_cid_seq(&self, path_id: u32) -> (u64, u64) {
+6152 self.local_cid_state
+6153 .get(&PathId(path_id))
+6154 .unwrap()
+6155 .active_seq()
6156 }
6157
-6158 fn tag_len_1rtt(&self) -> usize {
-6159 let key = match self.spaces[SpaceId::Data].crypto.as_ref() {
-6160 Some(crypto) => Some(&*crypto.packet.local),
-6161 None => self.zero_rtt_crypto.as_ref().map(|x| &*x.packet),
-6162 };
-6163 // If neither Data nor 0-RTT keys are available, make a reasonable tag length guess. As of
-6164 // this writing, all QUIC cipher suites use 16-byte tags. We could return `None` instead,
-6165 // but that would needlessly prevent sending datagrams during 0-RTT.
-6166 key.map_or(16, |x| x.tag_len())
-6167 }
-6168
-6169 /// Mark the path as validated, and enqueue NEW_TOKEN frames to be sent as appropriate
-6170 fn on_path_validated(&mut self, path_id: PathId) {
-6171 self.path_data_mut(path_id).validated = true;
-6172 let ConnectionSide::Server { server_config } = &self.side else {
-6173 return;
-6174 };
-6175 let network_path = self.path_data(path_id).network_path;
-6176 let new_tokens = &mut self.spaces[SpaceId::Data as usize].pending.new_tokens;
-6177 new_tokens.clear();
-6178 for _ in 0..server_config.validation_token.sent {
-6179 new_tokens.push(network_path);
-6180 }
+6158 /// Instruct the peer to replace previously issued CIDs by sending a NEW_CONNECTION_ID frame
+6159 /// with updated `retire_prior_to` field set to `v`
+6160 #[cfg(test)]
+6161 pub(crate) fn rotate_local_cid(&mut self, v: u64, now: Instant) {
+6162 let n = self
+6163 .local_cid_state
+6164 .get_mut(&PathId::ZERO)
+6165 .unwrap()
+6166 .assign_retire_seq(v);
+6167 self.endpoint_events
+6168 .push_back(EndpointEventInner::NeedIdentifiers(PathId::ZERO, now, n));
+6169 }
+6170
+6171 /// Check the current active remote CID sequence for `PathId::ZERO`
+6172 #[cfg(test)]
+6173 pub(crate) fn active_rem_cid_seq(&self) -> u64 {
+6174 self.rem_cids.get(&PathId::ZERO).unwrap().active_seq()
+6175 }
+6176
+6177 /// Returns the detected maximum udp payload size for the current path
+6178 #[cfg(test)]
+6179 pub(crate) fn path_mtu(&self, path_id: PathId) -> u16 {
+6180 self.path_data(path_id).current_mtu()
6181 }
6182
-6183 /// Handle new path status information: PATH_STATUS_AVAILABLE, PATH_STATUS_BACKUP
-6184 fn on_path_status(&mut self, path_id: PathId, status: PathStatus, status_seq_no: VarInt) {
-6185 if let Some(path) = self.paths.get_mut(&path_id) {
-6186 path.data.status.remote_update(status, status_seq_no);
-6187 } else {
-6188 debug!("PATH_STATUS_AVAILABLE received unknown path {:?}", path_id);
-6189 }
-6190 self.events.push_back(
-6191 PathEvent::RemoteStatus {
-6192 id: path_id,
-6193 status,
-6194 }
-6195 .into(),
-6196 );
-6197 }
-6198
-6199 /// Returns the maximum [`PathId`] to be used for sending in this connection.
-6200 ///
-6201 /// This is calculated as minimum between the local and remote's maximums when multipath is
-6202 /// enabled, or `None` when disabled.
-6203 ///
-6204 /// For data that's received, we should use [`Self::local_max_path_id`] instead.
-6205 /// The reasoning is that the remote might already have updated to its own newer
-6206 /// [`Self::max_path_id`] after sending out a `MAX_PATH_ID` frame, but it got re-ordered.
-6207 fn max_path_id(&self) -> Option<PathId> {
-6208 if self.is_multipath_negotiated() {
-6209 Some(self.remote_max_path_id.min(self.local_max_path_id))
-6210 } else {
-6211 None
-6212 }
-6213 }
-6214
-6215 /// Add addresses the local endpoint considers are reachable for nat traversal
-6216 pub fn add_nat_traversal_address(&mut self, address: SocketAddr) -> Result<(), iroh_hp::Error> {
-6217 if let Some(added) = self.iroh_hp.add_local_address(address)? {
-6218 self.spaces[SpaceId::Data].pending.add_address.insert(added);
-6219 };
-6220 Ok(())
-6221 }
-6222
-6223 /// Removes an address the endpoing no longer considers reachable for nat traversal
-6224 ///
-6225 /// Addresses not present in the set will be silently ignored.
-6226 pub fn remove_nat_traversal_address(
-6227 &mut self,
-6228 address: SocketAddr,
-6229 ) -> Result<(), iroh_hp::Error> {
-6230 if let Some(removed) = self.iroh_hp.remove_local_address(address)? {
-6231 self.spaces[SpaceId::Data]
-6232 .pending
-6233 .remove_address
-6234 .insert(removed);
-6235 }
-6236 Ok(())
-6237 }
-6238
-6239 /// Get the current local nat traversal addresses
-6240 pub fn get_local_nat_traversal_addresses(&self) -> Result<Vec<SocketAddr>, iroh_hp::Error> {
-6241 self.iroh_hp.get_local_nat_traversal_addresses()
-6242 }
-6243
-6244 /// Get the currently advertised nat traversal addresses by the server
-6245 pub fn get_remote_nat_traversal_addresses(&self) -> Result<Vec<SocketAddr>, iroh_hp::Error> {
-6246 Ok(self
-6247 .iroh_hp
-6248 .client_side()?
-6249 .get_remote_nat_traversal_addresses())
-6250 }
-6251
-6252 /// Attempts to open a path for nat traversal.
-6253 ///
-6254 /// `ipv6` indicates if the path should be opened using an IPV6 remote. If the address is
-6255 /// ignored, it will return `None`.
-6256 ///
-6257 /// On success returns the [`PathId`] and remote address of the path, as well as whether the path
-6258 /// existed for the adjusted remote.
-6259 fn open_nat_traversal_path(
-6260 &mut self,
-6261 now: Instant,
-6262 (ip, port): (IpAddr, u16),
-6263 ipv6: bool,
-6264 ) -> Result<Option<(PathId, SocketAddr, bool)>, PathError> {
-6265 // If this endpoint is an IPv6 endpoint we use IPv6 addresses for all remotes.
-6266 let remote = match ip {
-6267 IpAddr::V4(addr) if ipv6 => SocketAddr::new(addr.to_ipv6_mapped().into(), port),
-6268 IpAddr::V4(addr) => SocketAddr::new(addr.into(), port),
-6269 IpAddr::V6(_) if ipv6 => SocketAddr::new(ip, port),
-6270 IpAddr::V6(_) => {
-6271 trace!("not using IPv6 nat candidate for IPv4 socket");
-6272 return Ok(None);
-6273 }
-6274 };
-6275 // TODO(matheus23): Probe the correct 4-tuple, instead of only a remote address?
-6276 // By specifying None, we do two things: 1. open_path_ensure won't generate two
-6277 // paths to the same remote and 2. we let the OS choose which interface to use for
-6278 // sending on that path.
-6279 let network_path = FourTuple {
-6280 remote,
-6281 local_ip: None,
-6282 };
-6283 match self.open_path_ensure(network_path, PathStatus::Backup, now) {
-6284 Ok((path_id, path_was_known)) => {
-6285 if path_was_known {
-6286 trace!(%path_id, %remote, "nat traversal: path existed for remote");
-6287 }
-6288 Ok(Some((path_id, remote, path_was_known)))
-6289 }
-6290 Err(e) => {
-6291 debug!(%remote, %e, "nat traversal: failed to probe remote");
-6292 Err(e)
-6293 }
-6294 }
+6183 /// Triggers path validation on all paths
+6184 #[cfg(test)]
+6185 pub(crate) fn trigger_path_validation(&mut self) {
+6186 for path in self.paths.values_mut() {
+6187 path.data.send_new_challenge = true;
+6188 }
+6189 }
+6190
+6191 /// Whether we have 1-RTT data to send
+6192 ///
+6193 /// This checks for frames that can only be sent in the data space (1-RTT):
+6194 /// - Pending PATH_CHALLENGE frames on the active and previous path if just migrated.
+6195 /// - Pending PATH_RESPONSE frames.
+6196 /// - Pending data to send in STREAM frames.
+6197 /// - Pending DATAGRAM frames to send.
+6198 ///
+6199 /// See also [`PacketSpace::can_send`] which keeps track of all other frame types that
+6200 /// may need to be sent.
+6201 fn can_send_1rtt(&self, path_id: PathId, max_size: usize) -> SendableFrames {
+6202 let path_exclusive = self.paths.get(&path_id).is_some_and(|path| {
+6203 path.data.send_new_challenge
+6204 || path
+6205 .prev
+6206 .as_ref()
+6207 .is_some_and(|(_, path)| path.send_new_challenge)
+6208 || !path.data.path_responses.is_empty()
+6209 });
+6210 let other = self.streams.can_send_stream_data()
+6211 || self
+6212 .datagrams
+6213 .outgoing
+6214 .front()
+6215 .is_some_and(|x| x.size(true) <= max_size);
+6216 SendableFrames {
+6217 acks: false,
+6218 other,
+6219 close: false,
+6220 path_exclusive,
+6221 }
+6222 }
+6223
+6224 /// Terminate the connection instantly, without sending a close packet
+6225 fn kill(&mut self, reason: ConnectionError) {
+6226 self.close_common();
+6227 self.state.move_to_drained(Some(reason));
+6228 self.endpoint_events.push_back(EndpointEventInner::Drained);
+6229 }
+6230
+6231 /// Storage size required for the largest packet that can be transmitted on all currently
+6232 /// available paths
+6233 ///
+6234 /// Buffers passed to [`Connection::poll_transmit`] should be at least this large.
+6235 ///
+6236 /// When multipath is enabled, this value is the minimum MTU across all available paths.
+6237 pub fn current_mtu(&self) -> u16 {
+6238 self.paths
+6239 .iter()
+6240 .filter(|&(path_id, _path_state)| !self.abandoned_paths.contains(path_id))
+6241 .map(|(_path_id, path_state)| path_state.data.current_mtu())
+6242 .min()
+6243 .expect("There is always at least one available path")
+6244 }
+6245
+6246 /// Size of non-frame data for a 1-RTT packet
+6247 ///
+6248 /// Quantifies space consumed by the QUIC header and AEAD tag. All other bytes in a packet are
+6249 /// frames. Changes if the length of the remote connection ID changes, which is expected to be
+6250 /// rare. If `pn` is specified, may additionally change unpredictably due to variations in
+6251 /// latency and packet loss.
+6252 fn predict_1rtt_overhead(&mut self, pn: u64, path: PathId) -> usize {
+6253 let pn_len = PacketNumber::new(
+6254 pn,
+6255 self.spaces[SpaceId::Data]
+6256 .for_path(path)
+6257 .largest_acked_packet
+6258 .unwrap_or(0),
+6259 )
+6260 .len();
+6261
+6262 // 1 byte for flags
+6263 1 + self
+6264 .rem_cids
+6265 .get(&path)
+6266 .map(|cids| cids.active().len())
+6267 .unwrap_or(20) // Max CID len in QUIC v1
+6268 + pn_len
+6269 + self.tag_len_1rtt()
+6270 }
+6271
+6272 fn predict_1rtt_overhead_no_pn(&self) -> usize {
+6273 let pn_len = 4;
+6274
+6275 let cid_len = self
+6276 .rem_cids
+6277 .values()
+6278 .map(|cids| cids.active().len())
+6279 .max()
+6280 .unwrap_or(20); // Max CID len in QUIC v1
+6281
+6282 // 1 byte for flags
+6283 1 + cid_len + pn_len + self.tag_len_1rtt()
+6284 }
+6285
+6286 fn tag_len_1rtt(&self) -> usize {
+6287 let key = match self.spaces[SpaceId::Data].crypto.as_ref() {
+6288 Some(crypto) => Some(&*crypto.packet.local),
+6289 None => self.zero_rtt_crypto.as_ref().map(|x| &*x.packet),
+6290 };
+6291 // If neither Data nor 0-RTT keys are available, make a reasonable tag length guess. As of
+6292 // this writing, all QUIC cipher suites use 16-byte tags. We could return `None` instead,
+6293 // but that would needlessly prevent sending datagrams during 0-RTT.
+6294 key.map_or(16, |x| x.tag_len())
6295 }
6296
-6297 /// Initiates a new nat traversal round
-6298 ///
-6299 /// A nat traversal round involves advertising the client's local addresses in `REACH_OUT`
-6300 /// frames, and initiating probing of the known remote addresses. When a new round is
-6301 /// initiated, the previous one is cancelled, and paths that have not been opened are closed.
-6302 ///
-6303 /// Returns the server addresses that are now being probed.
-6304 /// If addresses fail due to spurious errors, these might succeed later and not be returned in
-6305 /// this set.
-6306 pub fn initiate_nat_traversal_round(
-6307 &mut self,
-6308 now: Instant,
-6309 ) -> Result<Vec<SocketAddr>, iroh_hp::Error> {
-6310 if self.state.is_closed() {
-6311 return Err(iroh_hp::Error::Closed);
-6312 }
-6313
-6314 let client_state = self.iroh_hp.client_side_mut()?;
-6315 let iroh_hp::NatTraversalRound {
-6316 new_round,
-6317 reach_out_at,
-6318 addresses_to_probe,
-6319 prev_round_path_ids,
-6320 } = client_state.initiate_nat_traversal_round()?;
-6321
-6322 self.spaces[SpaceId::Data].pending.reach_out = Some((new_round, reach_out_at));
-6323
-6324 for path_id in prev_round_path_ids {
-6325 // TODO(@divma): this sounds reasonable but we need if this actually works for the
-6326 // purposes of the protocol
-6327 let validated = self
-6328 .path(path_id)
-6329 .map(|path| path.validated)
-6330 .unwrap_or(false);
-6331
-6332 if !validated {
-6333 let _ = self.close_path(
-6334 now,
-6335 path_id,
-6336 TransportErrorCode::APPLICATION_ABANDON_PATH.into(),
-6337 );
-6338 }
-6339 }
-6340
-6341 let mut err = None;
+6297 /// Mark the path as validated, and enqueue NEW_TOKEN frames to be sent as appropriate
+6298 fn on_path_validated(&mut self, path_id: PathId) {
+6299 self.path_data_mut(path_id).validated = true;
+6300 let ConnectionSide::Server { server_config } = &self.side else {
+6301 return;
+6302 };
+6303 let network_path = self.path_data(path_id).network_path;
+6304 let new_tokens = &mut self.spaces[SpaceId::Data as usize].pending.new_tokens;
+6305 new_tokens.clear();
+6306 for _ in 0..server_config.validation_token.sent {
+6307 new_tokens.push(network_path);
+6308 }
+6309 }
+6310
+6311 /// Handle new path status information: PATH_STATUS_AVAILABLE, PATH_STATUS_BACKUP
+6312 fn on_path_status(&mut self, path_id: PathId, status: PathStatus, status_seq_no: VarInt) {
+6313 if let Some(path) = self.paths.get_mut(&path_id) {
+6314 path.data.status.remote_update(status, status_seq_no);
+6315 } else {
+6316 debug!("PATH_STATUS_AVAILABLE received unknown path {:?}", path_id);
+6317 }
+6318 self.events.push_back(
+6319 PathEvent::RemoteStatus {
+6320 id: path_id,
+6321 status,
+6322 }
+6323 .into(),
+6324 );
+6325 }
+6326
+6327 /// Returns the maximum [`PathId`] to be used for sending in this connection.
+6328 ///
+6329 /// This is calculated as minimum between the local and remote's maximums when multipath is
+6330 /// enabled, or `None` when disabled.
+6331 ///
+6332 /// For data that's received, we should use [`Self::local_max_path_id`] instead.
+6333 /// The reasoning is that the remote might already have updated to its own newer
+6334 /// [`Self::max_path_id`] after sending out a `MAX_PATH_ID` frame, but it got re-ordered.
+6335 fn max_path_id(&self) -> Option<PathId> {
+6336 if self.is_multipath_negotiated() {
+6337 Some(self.remote_max_path_id.min(self.local_max_path_id))
+6338 } else {
+6339 None
+6340 }
+6341 }
6342
-6343 let mut path_ids = Vec::with_capacity(addresses_to_probe.len());
-6344 let mut probed_addresses = Vec::with_capacity(addresses_to_probe.len());
-6345 let ipv6 = self
-6346 .paths
-6347 .values()
-6348 .any(|p| p.data.network_path.remote.is_ipv6());
-6349
-6350 for (id, address) in addresses_to_probe {
-6351 match self.open_nat_traversal_path(now, address, ipv6) {
-6352 Ok(None) => {}
-6353 Ok(Some((path_id, remote, path_was_known))) => {
-6354 if !path_was_known {
-6355 path_ids.push(path_id);
-6356 probed_addresses.push(remote);
-6357 }
-6358 }
-6359 Err(e) => {
-6360 self.iroh_hp
-6361 .client_side_mut()
-6362 .expect("validated")
-6363 .report_in_continuation(id, e);
-6364 err.get_or_insert(e);
-6365 }
-6366 }
-6367 }
-6368
-6369 if let Some(err) = err {
-6370 // We failed to probe any addresses, bail out
-6371 if probed_addresses.is_empty() {
-6372 return Err(iroh_hp::Error::Multipath(err));
-6373 }
-6374 }
-6375
-6376 self.iroh_hp
-6377 .client_side_mut()
-6378 .expect("connection side validated")
-6379 .set_round_path_ids(path_ids);
-6380
-6381 Ok(probed_addresses)
-6382 }
-6383
-6384 /// Attempts to continue a nat traversal round by trying to open paths for pending client probes.
-6385 ///
-6386 /// If there was nothing to do, it returns `None`. Otherwise it returns whether the path was
-6387 /// successfully open.
-6388 fn continue_nat_traversal_round(&mut self, now: Instant) -> Option<bool> {
-6389 let client_state = self.iroh_hp.client_side_mut().ok()?;
-6390 let (id, address) = client_state.continue_nat_traversal_round()?;
-6391 let ipv6 = self
-6392 .paths
-6393 .values()
-6394 .any(|p| p.data.network_path.remote.is_ipv6());
-6395 let open_result = self.open_nat_traversal_path(now, address, ipv6);
-6396 let client_state = self.iroh_hp.client_side_mut().expect("validated");
-6397 match open_result {
-6398 Ok(None) => Some(true),
-6399 Ok(Some((path_id, _remote, path_was_known))) => {
-6400 if !path_was_known {
-6401 client_state.add_round_path_id(path_id);
-6402 }
-6403 Some(true)
-6404 }
-6405 Err(e) => {
-6406 client_state.report_in_continuation(id, e);
-6407 Some(false)
-6408 }
-6409 }
-6410 }
-6411}
-6412
-6413impl fmt::Debug for Connection {
-6414 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
-6415 f.debug_struct("Connection")
-6416 .field("handshake_cid", &self.handshake_cid)
-6417 .finish()
-6418 }
-6419}
-6420
-6421#[derive(Debug, Copy, Clone, PartialEq, Eq)]
-6422enum PathBlocked {
-6423 No,
-6424 AntiAmplification,
-6425 Congestion,
-6426 Pacing,
-6427}
-6428
-6429/// Fields of `Connection` specific to it being client-side or server-side
-6430enum ConnectionSide {
-6431 Client {
-6432 /// Sent in every outgoing Initial packet. Always empty after Initial keys are discarded
-6433 token: Bytes,
-6434 token_store: Arc<dyn TokenStore>,
-6435 server_name: String,
-6436 },
-6437 Server {
-6438 server_config: Arc<ServerConfig>,
-6439 },
-6440}
+6343 /// Add addresses the local endpoint considers are reachable for nat traversal
+6344 pub fn add_nat_traversal_address(&mut self, address: SocketAddr) -> Result<(), iroh_hp::Error> {
+6345 if let Some(added) = self.iroh_hp.add_local_address(address)? {
+6346 self.spaces[SpaceId::Data].pending.add_address.insert(added);
+6347 };
+6348 Ok(())
+6349 }
+6350
+6351 /// Removes an address the endpoing no longer considers reachable for nat traversal
+6352 ///
+6353 /// Addresses not present in the set will be silently ignored.
+6354 pub fn remove_nat_traversal_address(
+6355 &mut self,
+6356 address: SocketAddr,
+6357 ) -> Result<(), iroh_hp::Error> {
+6358 if let Some(removed) = self.iroh_hp.remove_local_address(address)? {
+6359 self.spaces[SpaceId::Data]
+6360 .pending
+6361 .remove_address
+6362 .insert(removed);
+6363 }
+6364 Ok(())
+6365 }
+6366
+6367 /// Get the current local nat traversal addresses
+6368 pub fn get_local_nat_traversal_addresses(&self) -> Result<Vec<SocketAddr>, iroh_hp::Error> {
+6369 self.iroh_hp.get_local_nat_traversal_addresses()
+6370 }
+6371
+6372 /// Get the currently advertised nat traversal addresses by the server
+6373 pub fn get_remote_nat_traversal_addresses(&self) -> Result<Vec<SocketAddr>, iroh_hp::Error> {
+6374 Ok(self
+6375 .iroh_hp
+6376 .client_side()?
+6377 .get_remote_nat_traversal_addresses())
+6378 }
+6379
+6380 /// Attempts to open a path for nat traversal.
+6381 ///
+6382 /// `ipv6` indicates if the path should be opened using an IPV6 remote. If the address is
+6383 /// ignored, it will return `None`.
+6384 ///
+6385 /// On success returns the [`PathId`] and remote address of the path, as well as whether the path
+6386 /// existed for the adjusted remote.
+6387 fn open_nat_traversal_path(
+6388 &mut self,
+6389 now: Instant,
+6390 (ip, port): (IpAddr, u16),
+6391 ipv6: bool,
+6392 ) -> Result<Option<(PathId, SocketAddr, bool)>, PathError> {
+6393 // If this endpoint is an IPv6 endpoint we use IPv6 addresses for all remotes.
+6394 let remote = match ip {
+6395 IpAddr::V4(addr) if ipv6 => SocketAddr::new(addr.to_ipv6_mapped().into(), port),
+6396 IpAddr::V4(addr) => SocketAddr::new(addr.into(), port),
+6397 IpAddr::V6(_) if ipv6 => SocketAddr::new(ip, port),
+6398 IpAddr::V6(_) => {
+6399 trace!("not using IPv6 nat candidate for IPv4 socket");
+6400 return Ok(None);
+6401 }
+6402 };
+6403 // TODO(matheus23): Probe the correct 4-tuple, instead of only a remote address?
+6404 // By specifying None, we do two things: 1. open_path_ensure won't generate two
+6405 // paths to the same remote and 2. we let the OS choose which interface to use for
+6406 // sending on that path.
+6407 let network_path = FourTuple {
+6408 remote,
+6409 local_ip: None,
+6410 };
+6411 match self.open_path_ensure(network_path, PathStatus::Backup, now) {
+6412 Ok((path_id, path_was_known)) => {
+6413 if path_was_known {
+6414 trace!(%path_id, %remote, "nat traversal: path existed for remote");
+6415 }
+6416 Ok(Some((path_id, remote, path_was_known)))
+6417 }
+6418 Err(e) => {
+6419 debug!(%remote, %e, "nat traversal: failed to probe remote");
+6420 Err(e)
+6421 }
+6422 }
+6423 }
+6424
+6425 /// Initiates a new nat traversal round
+6426 ///
+6427 /// A nat traversal round involves advertising the client's local addresses in `REACH_OUT`
+6428 /// frames, and initiating probing of the known remote addresses. When a new round is
+6429 /// initiated, the previous one is cancelled, and paths that have not been opened are closed.
+6430 ///
+6431 /// Returns the server addresses that are now being probed.
+6432 /// If addresses fail due to spurious errors, these might succeed later and not be returned in
+6433 /// this set.
+6434 pub fn initiate_nat_traversal_round(
+6435 &mut self,
+6436 now: Instant,
+6437 ) -> Result<Vec<SocketAddr>, iroh_hp::Error> {
+6438 if self.state.is_closed() {
+6439 return Err(iroh_hp::Error::Closed);
+6440 }
6441
-6442impl ConnectionSide {
-6443 fn remote_may_migrate(&self, state: &State) -> bool {
-6444 match self {
-6445 Self::Server { server_config } => server_config.migration,
-6446 Self::Client { .. } => {
-6447 if let Some(hs) = state.as_handshake() {
-6448 hs.allow_server_migration
-6449 } else {
-6450 false
-6451 }
-6452 }
-6453 }
-6454 }
-6455
-6456 fn is_client(&self) -> bool {
-6457 self.side().is_client()
-6458 }
+6442 let client_state = self.iroh_hp.client_side_mut()?;
+6443 let iroh_hp::NatTraversalRound {
+6444 new_round,
+6445 reach_out_at,
+6446 addresses_to_probe,
+6447 prev_round_path_ids,
+6448 } = client_state.initiate_nat_traversal_round()?;
+6449
+6450 self.spaces[SpaceId::Data].pending.reach_out = Some((new_round, reach_out_at));
+6451
+6452 for path_id in prev_round_path_ids {
+6453 // TODO(@divma): this sounds reasonable but we need if this actually works for the
+6454 // purposes of the protocol
+6455 let validated = self
+6456 .path(path_id)
+6457 .map(|path| path.validated)
+6458 .unwrap_or(false);
6459
-6460 fn is_server(&self) -> bool {
-6461 self.side().is_server()
-6462 }
-6463
-6464 fn side(&self) -> Side {
-6465 match *self {
-6466 Self::Client { .. } => Side::Client,
-6467 Self::Server { .. } => Side::Server,
-6468 }
-6469 }
-6470}
-6471
-6472impl From<SideArgs> for ConnectionSide {
-6473 fn from(side: SideArgs) -> Self {
-6474 match side {
-6475 SideArgs::Client {
-6476 token_store,
-6477 server_name,
-6478 } => Self::Client {
-6479 token: token_store.take(&server_name).unwrap_or_default(),
-6480 token_store,
-6481 server_name,
-6482 },
-6483 SideArgs::Server {
-6484 server_config,
-6485 pref_addr_cid: _,
-6486 path_validated: _,
-6487 } => Self::Server { server_config },
-6488 }
-6489 }
-6490}
-6491
-6492/// Parameters to `Connection::new` specific to it being client-side or server-side
-6493pub(crate) enum SideArgs {
-6494 Client {
-6495 token_store: Arc<dyn TokenStore>,
-6496 server_name: String,
-6497 },
-6498 Server {
-6499 server_config: Arc<ServerConfig>,
-6500 pref_addr_cid: Option<ConnectionId>,
-6501 path_validated: bool,
-6502 },
-6503}
-6504
-6505impl SideArgs {
-6506 pub(crate) fn pref_addr_cid(&self) -> Option<ConnectionId> {
-6507 match *self {
-6508 Self::Client { .. } => None,
-6509 Self::Server { pref_addr_cid, .. } => pref_addr_cid,
-6510 }
-6511 }
-6512
-6513 pub(crate) fn path_validated(&self) -> bool {
-6514 match *self {
-6515 Self::Client { .. } => true,
-6516 Self::Server { path_validated, .. } => path_validated,
-6517 }
-6518 }
-6519
-6520 pub(crate) fn side(&self) -> Side {
-6521 match *self {
-6522 Self::Client { .. } => Side::Client,
-6523 Self::Server { .. } => Side::Server,
-6524 }
-6525 }
-6526}
-6527
-6528/// Reasons why a connection might be lost
-6529#[derive(Debug, Error, Clone, PartialEq, Eq)]
-6530pub enum ConnectionError {
-6531 /// The peer doesn't implement any supported version
-6532 #[error("peer doesn't implement any supported version")]
-6533 VersionMismatch,
-6534 /// The peer violated the QUIC specification as understood by this implementation
-6535 #[error(transparent)]
-6536 TransportError(#[from] TransportError),
-6537 /// The peer's QUIC stack aborted the connection automatically
-6538 #[error("aborted by peer: {0}")]
-6539 ConnectionClosed(frame::ConnectionClose),
-6540 /// The peer closed the connection
-6541 #[error("closed by peer: {0}")]
-6542 ApplicationClosed(frame::ApplicationClose),
-6543 /// The peer is unable to continue processing this connection, usually due to having restarted
-6544 #[error("reset by peer")]
-6545 Reset,
-6546 /// Communication with the peer has lapsed for longer than the negotiated idle timeout
-6547 ///
-6548 /// If neither side is sending keep-alives, a connection will time out after a long enough idle
-6549 /// period even if the peer is still reachable. See also [`TransportConfig::max_idle_timeout()`]
-6550 /// and [`TransportConfig::keep_alive_interval()`].
-6551 #[error("timed out")]
-6552 TimedOut,
-6553 /// The local application closed the connection
-6554 #[error("closed")]
-6555 LocallyClosed,
-6556 /// The connection could not be created because not enough of the CID space is available
-6557 ///
-6558 /// Try using longer connection IDs.
-6559 #[error("CIDs exhausted")]
-6560 CidsExhausted,
-6561}
-6562
-6563impl From<Close> for ConnectionError {
-6564 fn from(x: Close) -> Self {
-6565 match x {
-6566 Close::Connection(reason) => Self::ConnectionClosed(reason),
-6567 Close::Application(reason) => Self::ApplicationClosed(reason),
-6568 }
-6569 }
-6570}
-6571
-6572// For compatibility with API consumers
-6573impl From<ConnectionError> for io::Error {
-6574 fn from(x: ConnectionError) -> Self {
-6575 use ConnectionError::*;
-6576 let kind = match x {
-6577 TimedOut => io::ErrorKind::TimedOut,
-6578 Reset => io::ErrorKind::ConnectionReset,
-6579 ApplicationClosed(_) | ConnectionClosed(_) => io::ErrorKind::ConnectionAborted,
-6580 TransportError(_) | VersionMismatch | LocallyClosed | CidsExhausted => {
-6581 io::ErrorKind::Other
-6582 }
-6583 };
-6584 Self::new(kind, x)
-6585 }
-6586}
+6460 if !validated {
+6461 let _ = self.close_path(
+6462 now,
+6463 path_id,
+6464 TransportErrorCode::APPLICATION_ABANDON_PATH.into(),
+6465 );
+6466 }
+6467 }
+6468
+6469 let mut err = None;
+6470
+6471 let mut path_ids = Vec::with_capacity(addresses_to_probe.len());
+6472 let mut probed_addresses = Vec::with_capacity(addresses_to_probe.len());
+6473 let ipv6 = self
+6474 .paths
+6475 .values()
+6476 .any(|p| p.data.network_path.remote.is_ipv6());
+6477
+6478 for (id, address) in addresses_to_probe {
+6479 match self.open_nat_traversal_path(now, address, ipv6) {
+6480 Ok(None) => {}
+6481 Ok(Some((path_id, remote, path_was_known))) => {
+6482 if !path_was_known {
+6483 path_ids.push(path_id);
+6484 probed_addresses.push(remote);
+6485 }
+6486 }
+6487 Err(e) => {
+6488 self.iroh_hp
+6489 .client_side_mut()
+6490 .expect("validated")
+6491 .report_in_continuation(id, e);
+6492 err.get_or_insert(e);
+6493 }
+6494 }
+6495 }
+6496
+6497 if let Some(err) = err {
+6498 // We failed to probe any addresses, bail out
+6499 if probed_addresses.is_empty() {
+6500 return Err(iroh_hp::Error::Multipath(err));
+6501 }
+6502 }
+6503
+6504 self.iroh_hp
+6505 .client_side_mut()
+6506 .expect("connection side validated")
+6507 .set_round_path_ids(path_ids);
+6508
+6509 Ok(probed_addresses)
+6510 }
+6511
+6512 /// Attempts to continue a nat traversal round by trying to open paths for pending client probes.
+6513 ///
+6514 /// If there was nothing to do, it returns `None`. Otherwise it returns whether the path was
+6515 /// successfully open.
+6516 fn continue_nat_traversal_round(&mut self, now: Instant) -> Option<bool> {
+6517 let client_state = self.iroh_hp.client_side_mut().ok()?;
+6518 let (id, address) = client_state.continue_nat_traversal_round()?;
+6519 let ipv6 = self
+6520 .paths
+6521 .values()
+6522 .any(|p| p.data.network_path.remote.is_ipv6());
+6523 let open_result = self.open_nat_traversal_path(now, address, ipv6);
+6524 let client_state = self.iroh_hp.client_side_mut().expect("validated");
+6525 match open_result {
+6526 Ok(None) => Some(true),
+6527 Ok(Some((path_id, _remote, path_was_known))) => {
+6528 if !path_was_known {
+6529 client_state.add_round_path_id(path_id);
+6530 }
+6531 Some(true)
+6532 }
+6533 Err(e) => {
+6534 client_state.report_in_continuation(id, e);
+6535 Some(false)
+6536 }
+6537 }
+6538 }
+6539}
+6540
+6541impl fmt::Debug for Connection {
+6542 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+6543 f.debug_struct("Connection")
+6544 .field("handshake_cid", &self.handshake_cid)
+6545 .finish()
+6546 }
+6547}
+6548
+6549#[derive(Debug, Copy, Clone, PartialEq, Eq)]
+6550enum PathBlocked {
+6551 No,
+6552 AntiAmplification,
+6553 Congestion,
+6554 Pacing,
+6555}
+6556
+6557/// Fields of `Connection` specific to it being client-side or server-side
+6558enum ConnectionSide {
+6559 Client {
+6560 /// Sent in every outgoing Initial packet. Always empty after Initial keys are discarded
+6561 token: Bytes,
+6562 token_store: Arc<dyn TokenStore>,
+6563 server_name: String,
+6564 },
+6565 Server {
+6566 server_config: Arc<ServerConfig>,
+6567 },
+6568}
+6569
+6570impl ConnectionSide {
+6571 fn remote_may_migrate(&self, state: &State) -> bool {
+6572 match self {
+6573 Self::Server { server_config } => server_config.migration,
+6574 Self::Client { .. } => {
+6575 if let Some(hs) = state.as_handshake() {
+6576 hs.allow_server_migration
+6577 } else {
+6578 false
+6579 }
+6580 }
+6581 }
+6582 }
+6583
+6584 fn is_client(&self) -> bool {
+6585 self.side().is_client()
+6586 }
6587
-6588/// Errors that might trigger a path being closed
-6589// TODO(@divma): maybe needs to be reworked based on what we want to do with the public API
-6590#[derive(Debug, Error, PartialEq, Eq, Clone, Copy)]
-6591pub enum PathError {
-6592 /// The extension was not negotiated with the peer
-6593 #[error("multipath extension not negotiated")]
-6594 MultipathNotNegotiated,
-6595 /// Paths can only be opened client-side
-6596 #[error("the server side may not open a path")]
-6597 ServerSideNotAllowed,
-6598 /// Current limits do not allow us to open more paths
-6599 #[error("maximum number of concurrent paths reached")]
-6600 MaxPathIdReached,
-6601 /// No remote CIDs available to open a new path
-6602 #[error("remoted CIDs exhausted")]
-6603 RemoteCidsExhausted,
-6604 /// Path could not be validated and will be abandoned
-6605 #[error("path validation failed")]
-6606 ValidationFailed,
-6607 /// The remote address for the path is not supported by the endpoint
-6608 #[error("invalid remote address")]
-6609 InvalidRemoteAddress(SocketAddr),
-6610}
-6611
-6612/// Errors triggered when abandoning a path
-6613#[derive(Debug, Error, Clone, Eq, PartialEq)]
-6614pub enum ClosePathError {
-6615 /// The path is already closed or was never opened
-6616 #[error("closed path")]
-6617 ClosedPath,
-6618 /// This is the last path, which can not be abandoned
-6619 #[error("last open path")]
-6620 LastOpenPath,
-6621}
-6622
-6623/// Error when the multipath extension was not negotiated, but attempted to be used.
-6624#[derive(Debug, Error, Clone, Copy)]
-6625#[error("Multipath extension not negotiated")]
-6626pub struct MultipathNotNegotiated {
-6627 _private: (),
-6628}
-6629
-6630/// Events of interest to the application
-6631#[derive(Debug)]
-6632pub enum Event {
-6633 /// The connection's handshake data is ready
-6634 HandshakeDataReady,
-6635 /// The connection was successfully established
-6636 Connected,
-6637 /// The TLS handshake was confirmed
-6638 HandshakeConfirmed,
-6639 /// The connection was lost
-6640 ///
-6641 /// Emitted if the peer closes the connection or an error is encountered.
-6642 ConnectionLost {
-6643 /// Reason that the connection was closed
-6644 reason: ConnectionError,
-6645 },
-6646 /// Stream events
-6647 Stream(StreamEvent),
-6648 /// One or more application datagrams have been received
-6649 DatagramReceived,
-6650 /// One or more application datagrams have been sent after blocking
-6651 DatagramsUnblocked,
-6652 /// (Multi)Path events
-6653 Path(PathEvent),
-6654 /// Iroh's nat traversal events
-6655 NatTraversal(iroh_hp::Event),
-6656}
-6657
-6658impl From<PathEvent> for Event {
-6659 fn from(source: PathEvent) -> Self {
-6660 Self::Path(source)
-6661 }
-6662}
-6663
-6664fn get_max_ack_delay(params: &TransportParameters) -> Duration {
-6665 Duration::from_micros(params.max_ack_delay.0 * 1000)
-6666}
-6667
-6668// Prevents overflow and improves behavior in extreme circumstances
-6669const MAX_BACKOFF_EXPONENT: u32 = 16;
-6670
-6671/// Minimal remaining size to allow packet coalescing, excluding cryptographic tag
-6672///
-6673/// This must be at least as large as the header for a well-formed empty packet to be coalesced,
-6674/// plus some space for frames. We only care about handshake headers because short header packets
-6675/// necessarily have smaller headers, and initial packets are only ever the first packet in a
-6676/// datagram (because we coalesce in ascending packet space order and the only reason to split a
-6677/// packet is when packet space changes).
-6678const MIN_PACKET_SPACE: usize = MAX_HANDSHAKE_OR_0RTT_HEADER_SIZE + 32;
-6679
-6680/// Largest amount of space that could be occupied by a Handshake or 0-RTT packet's header
-6681///
-6682/// Excludes packet-type-specific fields such as packet number or Initial token
-6683// https://www.rfc-editor.org/rfc/rfc9000.html#name-0-rtt: flags + version + dcid len + dcid +
-6684// scid len + scid + length + pn
-6685const MAX_HANDSHAKE_OR_0RTT_HEADER_SIZE: usize =
-6686 1 + 4 + 1 + MAX_CID_SIZE + 1 + MAX_CID_SIZE + VarInt::from_u32(u16::MAX as u32).size() + 4;
-6687
-6688/// Perform key updates this many packets before the AEAD confidentiality limit.
-6689///
-6690/// Chosen arbitrarily, intended to be large enough to prevent spurious connection loss.
-6691const KEY_UPDATE_MARGIN: u64 = 10_000;
-6692
-6693#[derive(Default)]
-6694struct SentFrames {
-6695 retransmits: ThinRetransmits,
-6696 /// The packet number of the largest acknowledged packet for each path
-6697 largest_acked: FxHashMap<PathId, u64>,
-6698 stream_frames: StreamMetaVec,
-6699 /// Whether the packet contains non-retransmittable frames (like datagrams)
-6700 non_retransmits: bool,
-6701 /// If the datagram containing these frames should be padded to the min MTU
-6702 requires_padding: bool,
-6703}
-6704
-6705impl SentFrames {
-6706 /// Returns whether the packet contains only ACKs
-6707 fn is_ack_only(&self, streams: &StreamsState) -> bool {
-6708 !self.largest_acked.is_empty()
-6709 && !self.non_retransmits
-6710 && self.stream_frames.is_empty()
-6711 && self.retransmits.is_empty(streams)
-6712 }
-6713
-6714 fn retransmits_mut(&mut self) -> &mut Retransmits {
-6715 self.retransmits.get_or_create()
-6716 }
-6717
-6718 fn record_sent_frame(&mut self, frame: frame::EncodableFrame<'_>) {
-6719 use frame::EncodableFrame::*;
-6720 match frame {
-6721 PathAck(path_ack_encoder) => {
-6722 if let Some(max) = path_ack_encoder.ranges.max() {
-6723 self.largest_acked.insert(path_ack_encoder.path_id, max);
-6724 }
-6725 }
-6726 Ack(ack_encoder) => {
-6727 if let Some(max) = ack_encoder.ranges.max() {
-6728 self.largest_acked.insert(PathId::ZERO, max);
-6729 }
-6730 }
-6731 Close(_) => { /* non retransmittable, but after this we don't really care */ }
-6732 PathResponse(_) => self.non_retransmits = true,
-6733 HandshakeDone(_) => self.retransmits_mut().handshake_done = true,
-6734 ReachOut(frame::ReachOut { round, ip, port }) => self
-6735 .retransmits_mut()
-6736 .reach_out
-6737 .get_or_insert_with(|| (round, Vec::new()))
-6738 .1
-6739 .push((ip, port)),
-6740 ObservedAddr(_) => self.retransmits_mut().observed_addr = true,
-6741 Ping(_) => self.non_retransmits = true,
-6742 ImmediateAck(_) => self.non_retransmits = true,
-6743 AckFrequency(_) => self.retransmits_mut().ack_frequency = true,
-6744 PathChallenge(_) => self.non_retransmits = true,
-6745 Crypto(crypto) => self.retransmits_mut().crypto.push_back(crypto),
-6746 PathAbandon(path_abandon) => {
-6747 self.retransmits_mut()
-6748 .path_abandon
-6749 .entry(path_abandon.path_id)
-6750 .or_insert(path_abandon.error_code);
-6751 }
-6752 PathStatusAvailable(frame::PathStatusAvailable { path_id, .. })
-6753 | PathStatusBackup(frame::PathStatusBackup { path_id, .. }) => {
-6754 self.retransmits_mut().path_status.insert(path_id);
-6755 }
-6756 MaxPathId(_) => self.retransmits_mut().max_path_id = true,
-6757 PathsBlocked(_) => self.retransmits_mut().paths_blocked = true,
-6758 PathCidsBlocked(path_cids_blocked) => {
-6759 self.retransmits_mut()
-6760 .path_cids_blocked
-6761 .insert(path_cids_blocked.path_id);
-6762 }
-6763 ResetStream(reset) => self
-6764 .retransmits_mut()
-6765 .reset_stream
-6766 .push((reset.id, reset.error_code)),
-6767 StopSending(stop_sending) => self.retransmits_mut().stop_sending.push(stop_sending),
-6768 NewConnectionId(new_cid) => self.retransmits_mut().new_cids.push(new_cid.issued()),
-6769 RetireConnectionId(retire_cid) => self
-6770 .retransmits_mut()
-6771 .retire_cids
-6772 .push((retire_cid.path_id.unwrap_or_default(), retire_cid.sequence)),
-6773 Datagram(_) => self.non_retransmits = true,
-6774 NewToken(_) => {}
-6775 AddAddress(add_address) => {
-6776 self.retransmits_mut().add_address.insert(add_address);
-6777 }
-6778 RemoveAddress(remove_address) => {
-6779 self.retransmits_mut().remove_address.insert(remove_address);
-6780 }
-6781 StreamMeta(stream_meta_encoder) => self.stream_frames.push(stream_meta_encoder.meta),
-6782 MaxData(_) => self.retransmits_mut().max_data = true,
-6783 MaxStreamData(max) => {
-6784 self.retransmits_mut().max_stream_data.insert(max.id);
-6785 }
-6786 MaxStreams(max_streams) => {
-6787 self.retransmits_mut().max_stream_id[max_streams.dir as usize] = true
-6788 }
-6789 }
-6790 }
-6791}
-6792
-6793/// Compute the negotiated idle timeout based on local and remote max_idle_timeout transport parameters.
-6794///
-6795/// According to the definition of max_idle_timeout, a value of `0` means the timeout is disabled; see <https://www.rfc-editor.org/rfc/rfc9000#section-18.2-4.4.1.>
-6796///
-6797/// According to the negotiation procedure, either the minimum of the timeouts or one specified is used as the negotiated value; see <https://www.rfc-editor.org/rfc/rfc9000#section-10.1-2.>
-6798///
-6799/// Returns the negotiated idle timeout as a `Duration`, or `None` when both endpoints have opted out of idle timeout.
-6800fn negotiate_max_idle_timeout(x: Option<VarInt>, y: Option<VarInt>) -> Option<Duration> {
-6801 match (x, y) {
-6802 (Some(VarInt(0)) | None, Some(VarInt(0)) | None) => None,
-6803 (Some(VarInt(0)) | None, Some(y)) => Some(Duration::from_millis(y.0)),
-6804 (Some(x), Some(VarInt(0)) | None) => Some(Duration::from_millis(x.0)),
-6805 (Some(x), Some(y)) => Some(Duration::from_millis(cmp::min(x, y).0)),
-6806 }
-6807}
-6808
-6809#[cfg(test)]
-6810mod tests {
-6811 use super::*;
-6812
-6813 #[test]
-6814 fn negotiate_max_idle_timeout_commutative() {
-6815 let test_params = [
-6816 (None, None, None),
-6817 (None, Some(VarInt(0)), None),
-6818 (None, Some(VarInt(2)), Some(Duration::from_millis(2))),
-6819 (Some(VarInt(0)), Some(VarInt(0)), None),
-6820 (
-6821 Some(VarInt(2)),
-6822 Some(VarInt(0)),
-6823 Some(Duration::from_millis(2)),
-6824 ),
-6825 (
-6826 Some(VarInt(1)),
-6827 Some(VarInt(4)),
-6828 Some(Duration::from_millis(1)),
-6829 ),
-6830 ];
-6831
-6832 for (left, right, result) in test_params {
-6833 assert_eq!(negotiate_max_idle_timeout(left, right), result);
-6834 assert_eq!(negotiate_max_idle_timeout(right, left), result);
-6835 }
-6836 }
-6837}
\ No newline at end of file
+6588 fn is_server(&self) -> bool {
+6589 self.side().is_server()
+6590 }
+6591
+6592 fn side(&self) -> Side {
+6593 match *self {
+6594 Self::Client { .. } => Side::Client,
+6595 Self::Server { .. } => Side::Server,
+6596 }
+6597 }
+6598}
+6599
+6600impl From<SideArgs> for ConnectionSide {
+6601 fn from(side: SideArgs) -> Self {
+6602 match side {
+6603 SideArgs::Client {
+6604 token_store,
+6605 server_name,
+6606 } => Self::Client {
+6607 token: token_store.take(&server_name).unwrap_or_default(),
+6608 token_store,
+6609 server_name,
+6610 },
+6611 SideArgs::Server {
+6612 server_config,
+6613 pref_addr_cid: _,
+6614 path_validated: _,
+6615 } => Self::Server { server_config },
+6616 }
+6617 }
+6618}
+6619
+6620/// Parameters to `Connection::new` specific to it being client-side or server-side
+6621pub(crate) enum SideArgs {
+6622 Client {
+6623 token_store: Arc<dyn TokenStore>,
+6624 server_name: String,
+6625 },
+6626 Server {
+6627 server_config: Arc<ServerConfig>,
+6628 pref_addr_cid: Option<ConnectionId>,
+6629 path_validated: bool,
+6630 },
+6631}
+6632
+6633impl SideArgs {
+6634 pub(crate) fn pref_addr_cid(&self) -> Option<ConnectionId> {
+6635 match *self {
+6636 Self::Client { .. } => None,
+6637 Self::Server { pref_addr_cid, .. } => pref_addr_cid,
+6638 }
+6639 }
+6640
+6641 pub(crate) fn path_validated(&self) -> bool {
+6642 match *self {
+6643 Self::Client { .. } => true,
+6644 Self::Server { path_validated, .. } => path_validated,
+6645 }
+6646 }
+6647
+6648 pub(crate) fn side(&self) -> Side {
+6649 match *self {
+6650 Self::Client { .. } => Side::Client,
+6651 Self::Server { .. } => Side::Server,
+6652 }
+6653 }
+6654}
+6655
+6656/// Reasons why a connection might be lost
+6657#[derive(Debug, Error, Clone, PartialEq, Eq)]
+6658pub enum ConnectionError {
+6659 /// The peer doesn't implement any supported version
+6660 #[error("peer doesn't implement any supported version")]
+6661 VersionMismatch,
+6662 /// The peer violated the QUIC specification as understood by this implementation
+6663 #[error(transparent)]
+6664 TransportError(#[from] TransportError),
+6665 /// The peer's QUIC stack aborted the connection automatically
+6666 #[error("aborted by peer: {0}")]
+6667 ConnectionClosed(frame::ConnectionClose),
+6668 /// The peer closed the connection
+6669 #[error("closed by peer: {0}")]
+6670 ApplicationClosed(frame::ApplicationClose),
+6671 /// The peer is unable to continue processing this connection, usually due to having restarted
+6672 #[error("reset by peer")]
+6673 Reset,
+6674 /// Communication with the peer has lapsed for longer than the negotiated idle timeout
+6675 ///
+6676 /// If neither side is sending keep-alives, a connection will time out after a long enough idle
+6677 /// period even if the peer is still reachable. See also [`TransportConfig::max_idle_timeout()`]
+6678 /// and [`TransportConfig::keep_alive_interval()`].
+6679 #[error("timed out")]
+6680 TimedOut,
+6681 /// The local application closed the connection
+6682 #[error("closed")]
+6683 LocallyClosed,
+6684 /// The connection could not be created because not enough of the CID space is available
+6685 ///
+6686 /// Try using longer connection IDs.
+6687 #[error("CIDs exhausted")]
+6688 CidsExhausted,
+6689}
+6690
+6691impl From<Close> for ConnectionError {
+6692 fn from(x: Close) -> Self {
+6693 match x {
+6694 Close::Connection(reason) => Self::ConnectionClosed(reason),
+6695 Close::Application(reason) => Self::ApplicationClosed(reason),
+6696 }
+6697 }
+6698}
+6699
+6700// For compatibility with API consumers
+6701impl From<ConnectionError> for io::Error {
+6702 fn from(x: ConnectionError) -> Self {
+6703 use ConnectionError::*;
+6704 let kind = match x {
+6705 TimedOut => io::ErrorKind::TimedOut,
+6706 Reset => io::ErrorKind::ConnectionReset,
+6707 ApplicationClosed(_) | ConnectionClosed(_) => io::ErrorKind::ConnectionAborted,
+6708 TransportError(_) | VersionMismatch | LocallyClosed | CidsExhausted => {
+6709 io::ErrorKind::Other
+6710 }
+6711 };
+6712 Self::new(kind, x)
+6713 }
+6714}
+6715
+6716/// Errors that might trigger a path being closed
+6717// TODO(@divma): maybe needs to be reworked based on what we want to do with the public API
+6718#[derive(Debug, Error, PartialEq, Eq, Clone, Copy)]
+6719pub enum PathError {
+6720 /// The extension was not negotiated with the peer
+6721 #[error("multipath extension not negotiated")]
+6722 MultipathNotNegotiated,
+6723 /// Paths can only be opened client-side
+6724 #[error("the server side may not open a path")]
+6725 ServerSideNotAllowed,
+6726 /// Current limits do not allow us to open more paths
+6727 #[error("maximum number of concurrent paths reached")]
+6728 MaxPathIdReached,
+6729 /// No remote CIDs available to open a new path
+6730 #[error("remoted CIDs exhausted")]
+6731 RemoteCidsExhausted,
+6732 /// Path could not be validated and will be abandoned
+6733 #[error("path validation failed")]
+6734 ValidationFailed,
+6735 /// The remote address for the path is not supported by the endpoint
+6736 #[error("invalid remote address")]
+6737 InvalidRemoteAddress(SocketAddr),
+6738}
+6739
+6740/// Errors triggered when abandoning a path
+6741#[derive(Debug, Error, Clone, Eq, PartialEq)]
+6742pub enum ClosePathError {
+6743 /// The path is already closed or was never opened
+6744 #[error("closed path")]
+6745 ClosedPath,
+6746 /// This is the last path, which can not be abandoned
+6747 #[error("last open path")]
+6748 LastOpenPath,
+6749}
+6750
+6751/// Error when the multipath extension was not negotiated, but attempted to be used.
+6752#[derive(Debug, Error, Clone, Copy)]
+6753#[error("Multipath extension not negotiated")]
+6754pub struct MultipathNotNegotiated {
+6755 _private: (),
+6756}
+6757
+6758/// Events of interest to the application
+6759#[derive(Debug)]
+6760pub enum Event {
+6761 /// The connection's handshake data is ready
+6762 HandshakeDataReady,
+6763 /// The connection was successfully established
+6764 Connected,
+6765 /// The TLS handshake was confirmed
+6766 HandshakeConfirmed,
+6767 /// The connection was lost
+6768 ///
+6769 /// Emitted if the peer closes the connection or an error is encountered.
+6770 ConnectionLost {
+6771 /// Reason that the connection was closed
+6772 reason: ConnectionError,
+6773 },
+6774 /// Stream events
+6775 Stream(StreamEvent),
+6776 /// One or more application datagrams have been received
+6777 DatagramReceived,
+6778 /// One or more application datagrams have been sent after blocking
+6779 DatagramsUnblocked,
+6780 /// (Multi)Path events
+6781 Path(PathEvent),
+6782 /// Iroh's nat traversal events
+6783 NatTraversal(iroh_hp::Event),
+6784}
+6785
+6786impl From<PathEvent> for Event {
+6787 fn from(source: PathEvent) -> Self {
+6788 Self::Path(source)
+6789 }
+6790}
+6791
+6792fn get_max_ack_delay(params: &TransportParameters) -> Duration {
+6793 Duration::from_micros(params.max_ack_delay.0 * 1000)
+6794}
+6795
+6796// Prevents overflow and improves behavior in extreme circumstances
+6797const MAX_BACKOFF_EXPONENT: u32 = 16;
+6798
+6799/// Minimal remaining size to allow packet coalescing, excluding cryptographic tag
+6800///
+6801/// This must be at least as large as the header for a well-formed empty packet to be coalesced,
+6802/// plus some space for frames. We only care about handshake headers because short header packets
+6803/// necessarily have smaller headers, and initial packets are only ever the first packet in a
+6804/// datagram (because we coalesce in ascending packet space order and the only reason to split a
+6805/// packet is when packet space changes).
+6806const MIN_PACKET_SPACE: usize = MAX_HANDSHAKE_OR_0RTT_HEADER_SIZE + 32;
+6807
+6808/// Largest amount of space that could be occupied by a Handshake or 0-RTT packet's header
+6809///
+6810/// Excludes packet-type-specific fields such as packet number or Initial token
+6811// https://www.rfc-editor.org/rfc/rfc9000.html#name-0-rtt: flags + version + dcid len + dcid +
+6812// scid len + scid + length + pn
+6813const MAX_HANDSHAKE_OR_0RTT_HEADER_SIZE: usize =
+6814 1 + 4 + 1 + MAX_CID_SIZE + 1 + MAX_CID_SIZE + VarInt::from_u32(u16::MAX as u32).size() + 4;
+6815
+6816/// Perform key updates this many packets before the AEAD confidentiality limit.
+6817///
+6818/// Chosen arbitrarily, intended to be large enough to prevent spurious connection loss.
+6819const KEY_UPDATE_MARGIN: u64 = 10_000;
+6820
+6821#[derive(Default)]
+6822struct SentFrames {
+6823 retransmits: ThinRetransmits,
+6824 /// The packet number of the largest acknowledged packet for each path
+6825 largest_acked: FxHashMap<PathId, u64>,
+6826 stream_frames: StreamMetaVec,
+6827 /// Whether the packet contains non-retransmittable frames (like datagrams)
+6828 non_retransmits: bool,
+6829 /// If the datagram containing these frames should be padded to the min MTU
+6830 requires_padding: bool,
+6831}
+6832
+6833impl SentFrames {
+6834 /// Returns whether the packet contains only ACKs
+6835 fn is_ack_only(&self, streams: &StreamsState) -> bool {
+6836 !self.largest_acked.is_empty()
+6837 && !self.non_retransmits
+6838 && self.stream_frames.is_empty()
+6839 && self.retransmits.is_empty(streams)
+6840 }
+6841
+6842 fn retransmits_mut(&mut self) -> &mut Retransmits {
+6843 self.retransmits.get_or_create()
+6844 }
+6845
+6846 fn record_sent_frame(&mut self, frame: frame::EncodableFrame<'_>) {
+6847 use frame::EncodableFrame::*;
+6848 match frame {
+6849 PathAck(path_ack_encoder) => {
+6850 if let Some(max) = path_ack_encoder.ranges.max() {
+6851 self.largest_acked.insert(path_ack_encoder.path_id, max);
+6852 }
+6853 }
+6854 Ack(ack_encoder) => {
+6855 if let Some(max) = ack_encoder.ranges.max() {
+6856 self.largest_acked.insert(PathId::ZERO, max);
+6857 }
+6858 }
+6859 Close(_) => { /* non retransmittable, but after this we don't really care */ }
+6860 PathResponse(_) => self.non_retransmits = true,
+6861 HandshakeDone(_) => self.retransmits_mut().handshake_done = true,
+6862 ReachOut(frame::ReachOut { round, ip, port }) => self
+6863 .retransmits_mut()
+6864 .reach_out
+6865 .get_or_insert_with(|| (round, Vec::new()))
+6866 .1
+6867 .push((ip, port)),
+6868 ObservedAddr(_) => self.retransmits_mut().observed_addr = true,
+6869 Ping(_) => self.non_retransmits = true,
+6870 ImmediateAck(_) => self.non_retransmits = true,
+6871 AckFrequency(_) => self.retransmits_mut().ack_frequency = true,
+6872 PathChallenge(_) => self.non_retransmits = true,
+6873 Crypto(crypto) => self.retransmits_mut().crypto.push_back(crypto),
+6874 PathAbandon(path_abandon) => {
+6875 self.retransmits_mut()
+6876 .path_abandon
+6877 .entry(path_abandon.path_id)
+6878 .or_insert(path_abandon.error_code);
+6879 }
+6880 PathStatusAvailable(frame::PathStatusAvailable { path_id, .. })
+6881 | PathStatusBackup(frame::PathStatusBackup { path_id, .. }) => {
+6882 self.retransmits_mut().path_status.insert(path_id);
+6883 }
+6884 MaxPathId(_) => self.retransmits_mut().max_path_id = true,
+6885 PathsBlocked(_) => self.retransmits_mut().paths_blocked = true,
+6886 PathCidsBlocked(path_cids_blocked) => {
+6887 self.retransmits_mut()
+6888 .path_cids_blocked
+6889 .insert(path_cids_blocked.path_id);
+6890 }
+6891 ResetStream(reset) => self
+6892 .retransmits_mut()
+6893 .reset_stream
+6894 .push((reset.id, reset.error_code)),
+6895 StopSending(stop_sending) => self.retransmits_mut().stop_sending.push(stop_sending),
+6896 NewConnectionId(new_cid) => self.retransmits_mut().new_cids.push(new_cid.issued()),
+6897 RetireConnectionId(retire_cid) => self
+6898 .retransmits_mut()
+6899 .retire_cids
+6900 .push((retire_cid.path_id.unwrap_or_default(), retire_cid.sequence)),
+6901 Datagram(_) => self.non_retransmits = true,
+6902 NewToken(_) => {}
+6903 AddAddress(add_address) => {
+6904 self.retransmits_mut().add_address.insert(add_address);
+6905 }
+6906 RemoveAddress(remove_address) => {
+6907 self.retransmits_mut().remove_address.insert(remove_address);
+6908 }
+6909 StreamMeta(stream_meta_encoder) => self.stream_frames.push(stream_meta_encoder.meta),
+6910 MaxData(_) => self.retransmits_mut().max_data = true,
+6911 MaxStreamData(max) => {
+6912 self.retransmits_mut().max_stream_data.insert(max.id);
+6913 }
+6914 MaxStreams(max_streams) => {
+6915 self.retransmits_mut().max_stream_id[max_streams.dir as usize] = true
+6916 }
+6917 }
+6918 }
+6919}
+6920
+6921/// Compute the negotiated idle timeout based on local and remote max_idle_timeout transport parameters.
+6922///
+6923/// According to the definition of max_idle_timeout, a value of `0` means the timeout is disabled; see <https://www.rfc-editor.org/rfc/rfc9000#section-18.2-4.4.1.>
+6924///
+6925/// According to the negotiation procedure, either the minimum of the timeouts or one specified is used as the negotiated value; see <https://www.rfc-editor.org/rfc/rfc9000#section-10.1-2.>
+6926///
+6927/// Returns the negotiated idle timeout as a `Duration`, or `None` when both endpoints have opted out of idle timeout.
+6928fn negotiate_max_idle_timeout(x: Option<VarInt>, y: Option<VarInt>) -> Option<Duration> {
+6929 match (x, y) {
+6930 (Some(VarInt(0)) | None, Some(VarInt(0)) | None) => None,
+6931 (Some(VarInt(0)) | None, Some(y)) => Some(Duration::from_millis(y.0)),
+6932 (Some(x), Some(VarInt(0)) | None) => Some(Duration::from_millis(x.0)),
+6933 (Some(x), Some(y)) => Some(Duration::from_millis(cmp::min(x, y).0)),
+6934 }
+6935}
+6936
+6937#[cfg(test)]
+6938mod tests {
+6939 use super::*;
+6940
+6941 #[test]
+6942 fn negotiate_max_idle_timeout_commutative() {
+6943 let test_params = [
+6944 (None, None, None),
+6945 (None, Some(VarInt(0)), None),
+6946 (None, Some(VarInt(2)), Some(Duration::from_millis(2))),
+6947 (Some(VarInt(0)), Some(VarInt(0)), None),
+6948 (
+6949 Some(VarInt(2)),
+6950 Some(VarInt(0)),
+6951 Some(Duration::from_millis(2)),
+6952 ),
+6953 (
+6954 Some(VarInt(1)),
+6955 Some(VarInt(4)),
+6956 Some(Duration::from_millis(1)),
+6957 ),
+6958 ];
+6959
+6960 for (left, right, result) in test_params {
+6961 assert_eq!(negotiate_max_idle_timeout(left, right), result);
+6962 assert_eq!(negotiate_max_idle_timeout(right, left), result);
+6963 }
+6964 }
+6965}
\ No newline at end of file
diff --git a/pr/255/docs/src/iroh_quinn_proto/connection/packet_builder.rs.html b/pr/255/docs/src/iroh_quinn_proto/connection/packet_builder.rs.html
index cb07743e9..923cf2568 100644
--- a/pr/255/docs/src/iroh_quinn_proto/connection/packet_builder.rs.html
+++ b/pr/255/docs/src/iroh_quinn_proto/connection/packet_builder.rs.html
@@ -282,7 +282,7 @@
282 conn: &mut Connection,
283 path_id: PathId,
284 pad_datagram: PadDatagram,
-285 ) {
+285 ) -> u64 {
286 match pad_datagram {
287 PadDatagram::No => (),
288 PadDatagram::ToSize(size) => self.pad_to(size),
@@ -329,121 +329,123 @@
329 conn.set_loss_detection_timer(now, path_id);
330 conn.path_data_mut(path_id).pacing.on_transmit(size);
331 }
-332 }
-333
-334 /// Encrypt packet, returning the length of the packet and whether padding was added
-335 pub(super) fn finish(
-336 mut self,
-337 conn: &mut Connection,
-338 now: Instant,
-339 ) -> (usize, bool, SentFrames) {
-340 debug_assert!(
-341 self.buf.len() <= self.buf.datagram_max_offset() - self.tag_len,
-342 "packet exceeds maximum size"
-343 );
-344 let pad = self.buf.len() < self.min_size;
-345 if pad {
-346 let padding = self.min_size - self.buf.len();
-347 trace!("PADDING * {}", padding);
-348 self.buf.put_bytes(0, padding);
-349 self.qlog.frame_padding(padding);
-350 }
-351
-352 let space = &conn.spaces[self.space];
-353 let (header_crypto, packet_crypto) = if let Some(ref crypto) = space.crypto {
-354 (&*crypto.header.local, &*crypto.packet.local)
-355 } else if self.space == SpaceId::Data {
-356 let zero_rtt = conn.zero_rtt_crypto.as_ref().unwrap();
-357 (&*zero_rtt.header, &*zero_rtt.packet)
-358 } else {
-359 unreachable!("tried to send {:?} packet without keys", self.space);
-360 };
-361
-362 debug_assert_eq!(
-363 packet_crypto.tag_len(),
-364 self.tag_len,
-365 "Mismatching crypto tag len"
-366 );
-367
-368 self.buf.put_bytes(0, packet_crypto.tag_len());
-369 let encode_start = self.partial_encode.start;
-370 let packet_buf = &mut self.buf.as_mut_slice()[encode_start..];
-371 // for packet protection, PathId::ZERO and no path are equivalent.
-372 self.partial_encode.finish(
-373 packet_buf,
-374 header_crypto,
-375 Some((self.exact_number, self.path, packet_crypto)),
-376 );
-377
-378 let packet_len = self.buf.len() - encode_start;
-379 trace!(size = %packet_len, short_header = %self.short_header, "wrote packet");
-380 self.qlog.finalize(packet_len);
-381 conn.qlog.emit_packet_sent(self.qlog, now);
-382 (packet_len, pad, self.sent_frames)
-383 }
-384
-385 /// The number of additional bytes the current packet would take up if it was finished now
-386 ///
-387 /// This will include any padding which is required to make the size large enough to be
-388 /// encrypted correctly.
-389 pub(super) fn predict_packet_end(&self) -> usize {
-390 self.buf.len().max(self.min_size) + self.tag_len - self.buf.len()
-391 }
-392
-393 /// Returns the remaining space in the packet that can be taken up by QUIC frames
-394 ///
-395 /// This leaves space in the datagram for the cryptographic tag that needs to be written
-396 /// when the packet is finished.
-397 pub(super) fn frame_space_remaining(&self) -> usize {
-398 let max_offset = self.buf.datagram_max_offset() - self.tag_len;
-399 max_offset.saturating_sub(self.buf.len())
-400 }
-401
-402 pub(crate) fn require_padding(&mut self) {
-403 self.sent_frames.requires_padding = true;
-404 }
-405
-406 pub(crate) fn retransmits_mut(&mut self) -> &mut Retransmits {
-407 self.sent_frames.retransmits_mut()
-408 }
-409}
-410
-411#[derive(Debug, Copy, Clone)]
-412pub(super) enum PadDatagram {
-413 /// Do not pad the datagram
-414 No,
-415 /// To a specific size
-416 ToSize(u16),
-417 /// Pad to the current MTU/segment size
-418 ///
-419 /// For the first datagram in a transmit the MTU is the same as the
-420 /// [`TransmitBuf::segment_size`].
-421 ToSegmentSize,
-422 /// Pad to [`MIN_INITIAL_SIZE`], the minimal QUIC MTU of 1200 bytes
-423 ToMinMtu,
-424}
-425
-426impl std::ops::BitOrAssign for PadDatagram {
-427 fn bitor_assign(&mut self, rhs: Self) {
-428 *self = *self | rhs;
-429 }
-430}
-431
-432impl std::ops::BitOr for PadDatagram {
-433 type Output = Self;
-434
-435 fn bitor(self, rhs: Self) -> Self::Output {
-436 match (self, rhs) {
-437 (Self::No, rhs) => rhs,
-438 (Self::ToSize(size), Self::No) => Self::ToSize(size),
-439 (Self::ToSize(a), Self::ToSize(b)) => Self::ToSize(a.max(b)),
-440 (Self::ToSize(_), Self::ToSegmentSize) => Self::ToSegmentSize,
-441 (Self::ToSize(_), Self::ToMinMtu) => Self::ToMinMtu,
-442 (Self::ToSegmentSize, Self::No) => Self::ToSegmentSize,
-443 (Self::ToSegmentSize, Self::ToSize(_)) => Self::ToSegmentSize,
-444 (Self::ToSegmentSize, Self::ToSegmentSize) => Self::ToSegmentSize,
-445 (Self::ToSegmentSize, Self::ToMinMtu) => Self::ToMinMtu,
-446 (Self::ToMinMtu, _) => Self::ToMinMtu,
-447 }
-448 }
-449}
MultipathNotNegotiated
pub struct MultipathNotNegotiated { /* private fields */ }Expand description
Error when the multipath extension was not negotiated, but attempted to be used.
+Trait Implementations§
Source§impl Clone for MultipathNotNegotiated
impl Clone for MultipathNotNegotiated
Source§fn clone(&self) -> MultipathNotNegotiated
fn clone(&self) -> MultipathNotNegotiated
Returns a duplicate of the value. Read more
1.0.0 · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source. Read moreSource§impl Debug for MultipathNotNegotiated
impl Debug for MultipathNotNegotiated
Source§impl Display for MultipathNotNegotiated
impl Display for MultipathNotNegotiated
Source§impl Error for MultipathNotNegotiated
impl Error for MultipathNotNegotiated
1.30.0 · Source§fn source(&self) -> Option<&(dyn Error + 'static)>
fn source(&self) -> Option<&(dyn Error + 'static)>
Returns the lower-level source of this error, if any. Read more
1.0.0 · Source§fn description(&self) -> &str
fn description(&self) -> &str
👎Deprecated since 1.42.0: use the Display impl or to_string()
impl Copy for MultipathNotNegotiated
Auto Trait Implementations§
impl Freeze for MultipathNotNegotiated
impl RefUnwindSafe for MultipathNotNegotiated
impl Send for MultipathNotNegotiated
impl Sync for MultipathNotNegotiated
impl Unpin for MultipathNotNegotiated
impl UnwindSafe for MultipathNotNegotiated
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> CloneToUninit for Twhere
diff --git a/pr/255/docs/iroh_quinn_proto/struct.TransportError.html b/pr/255/docs/iroh_quinn_proto/struct.TransportError.html
index c4be43306..b6dc15d42 100644
--- a/pr/255/docs/iroh_quinn_proto/struct.TransportError.html
+++ b/pr/255/docs/iroh_quinn_proto/struct.TransportError.html
@@ -11,7 +11,7 @@
§reason: StringHuman-readable explanation of the reason
§crypto: Option<Arc<dyn Error + Send + Sync>>An underlying crypto (e.g. TLS) layer error
Implementations§
Trait Implementations§
Source§impl Error for Error
1.30.0 · Source§fn source(&self) -> Option<&(dyn Error + 'static)>
Returns the lower-level source of this error, if any. Read more1.0.0 · Source§fn description(&self) -> &str
👎Deprecated since 1.42.0: use the Display impl or to_string()Source§impl From<Error> for ConnectionClose
Source§fn from(x: TransportError) -> Self
Converts to this type from the input type.Source§impl From<Error> for ConnectionError
Source§fn from(source: TransportError) -> Self
Converts to this type from the input type.Source§impl From<Error> for TransportError
Trait Implementations§
Source§impl Error for Error
1.30.0 · Source§fn source(&self) -> Option<&(dyn Error + 'static)>
Returns the lower-level source of this error, if any. Read more1.0.0 · Source§fn description(&self) -> &str
👎Deprecated since 1.42.0: use the Display impl or to_string()Source§impl From<Error> for ConnectionClose
Source§fn from(x: TransportError) -> Self
Converts to this type from the input type.Source§impl From<Error> for ConnectionError
Source§fn from(source: TransportError) -> Self
Converts to this type from the input type.Source§impl From<Error> for TransportError
Source§impl Eq for Error
Auto Trait Implementations§
§impl Freeze for Error
§impl !RefUnwindSafe for Error
§impl Send for Error
§impl Sync for Error
§impl Unpin for Error
§impl !UnwindSafe for Error
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
diff --git a/pr/255/docs/src/iroh_quinn_proto/connection/mod.rs.html b/pr/255/docs/src/iroh_quinn_proto/connection/mod.rs.html
index 81ec7afd0..e1b5a58ea 100644
--- a/pr/255/docs/src/iroh_quinn_proto/connection/mod.rs.html
+++ b/pr/255/docs/src/iroh_quinn_proto/connection/mod.rs.html
@@ -343,826 +343,826 @@
343/// Return value for `poll_transmit_path_space`.
344#[derive(Debug)]
345enum PollPathSpaceStatus {
-346 /// Nothing to send on this space, continue to the next one.
-347 NextSpace,
-348 /// We have something to send, and data has been accumulated
-349 /// on the passed in `transmit`.
-350 Send { last_packet_number: Option<u64> },
-351 /// A transmit is ready to be sent out.
-352 SendTransmit {
-353 /// The transmit to send.
-354 transmit: Transmit,
-355 },
-356 /// Nothing to send currently.
-357 NothingToSend {
-358 /// Set to `true` if we consider the current inability to send something because of congestion control
-359 congestion_blocked: bool,
-360 },
-361}
-362
-363impl Connection {
-364 pub(crate) fn new(
-365 endpoint_config: Arc<EndpointConfig>,
-366 config: Arc<TransportConfig>,
-367 init_cid: ConnectionId,
-368 loc_cid: ConnectionId,
-369 rem_cid: ConnectionId,
-370 network_path: FourTuple,
-371 crypto: Box<dyn crypto::Session>,
-372 cid_gen: &dyn ConnectionIdGenerator,
-373 now: Instant,
-374 version: u32,
-375 allow_mtud: bool,
-376 rng_seed: [u8; 32],
-377 side_args: SideArgs,
-378 qlog: QlogSink,
-379 ) -> Self {
-380 let pref_addr_cid = side_args.pref_addr_cid();
-381 let path_validated = side_args.path_validated();
-382 let connection_side = ConnectionSide::from(side_args);
-383 let side = connection_side.side();
-384 let mut rng = StdRng::from_seed(rng_seed);
-385 let initial_space = {
-386 let mut space = PacketSpace::new(now, SpaceId::Initial, &mut rng);
-387 space.crypto = Some(crypto.initial_keys(init_cid, side));
-388 space
-389 };
-390 let handshake_space = PacketSpace::new(now, SpaceId::Handshake, &mut rng);
-391 #[cfg(test)]
-392 let data_space = match config.deterministic_packet_numbers {
-393 true => PacketSpace::new_deterministic(now, SpaceId::Data),
-394 false => PacketSpace::new(now, SpaceId::Data, &mut rng),
-395 };
-396 #[cfg(not(test))]
-397 let data_space = PacketSpace::new(now, SpaceId::Data, &mut rng);
-398 let state = State::handshake(state::Handshake {
-399 rem_cid_set: side.is_server(),
-400 expected_token: Bytes::new(),
-401 client_hello: None,
-402 allow_server_migration: side.is_client(),
-403 });
-404 let local_cid_state = FxHashMap::from_iter([(
-405 PathId::ZERO,
-406 CidState::new(
-407 cid_gen.cid_len(),
-408 cid_gen.cid_lifetime(),
-409 now,
-410 if pref_addr_cid.is_some() { 2 } else { 1 },
-411 ),
-412 )]);
-413
-414 let mut path = PathData::new(network_path, allow_mtud, None, 0, now, &config);
-415 // TODO(@divma): consider if we want to delay this until the path is validated
-416 path.open = true;
-417 let mut this = Self {
-418 endpoint_config,
-419 crypto,
-420 handshake_cid: loc_cid,
-421 rem_handshake_cid: rem_cid,
-422 local_cid_state,
-423 paths: BTreeMap::from_iter([(
-424 PathId::ZERO,
-425 PathState {
-426 data: path,
-427 prev: None,
-428 },
-429 )]),
-430 path_generation_counter: 0,
-431 allow_mtud,
-432 state,
-433 side: connection_side,
-434 zero_rtt_enabled: false,
-435 zero_rtt_crypto: None,
-436 key_phase: false,
-437 // A small initial key phase size ensures peers that don't handle key updates correctly
-438 // fail sooner rather than later. It's okay for both peers to do this, as the first one
-439 // to perform an update will reset the other's key phase size in `update_keys`, and a
-440 // simultaneous key update by both is just like a regular key update with a really fast
-441 // response. Inspired by quic-go's similar behavior of performing the first key update
-442 // at the 100th short-header packet.
-443 key_phase_size: rng.random_range(10..1000),
-444 peer_params: TransportParameters::default(),
-445 orig_rem_cid: rem_cid,
-446 initial_dst_cid: init_cid,
-447 retry_src_cid: None,
-448 events: VecDeque::new(),
-449 endpoint_events: VecDeque::new(),
-450 spin_enabled: config.allow_spin && rng.random_ratio(7, 8),
-451 spin: false,
-452 spaces: [initial_space, handshake_space, data_space],
-453 highest_space: SpaceId::Initial,
-454 prev_crypto: None,
-455 next_crypto: None,
-456 accepted_0rtt: false,
-457 permit_idle_reset: true,
-458 idle_timeout: match config.max_idle_timeout {
-459 None | Some(VarInt(0)) => None,
-460 Some(dur) => Some(Duration::from_millis(dur.0)),
-461 },
-462 timers: TimerTable::default(),
-463 authentication_failures: 0,
-464 close: false,
-465
-466 ack_frequency: AckFrequencyState::new(get_max_ack_delay(
-467 &TransportParameters::default(),
-468 )),
-469
-470 app_limited: false,
-471 receiving_ecn: false,
-472 total_authed_packets: 0,
+346 /// We have something to send, and data has been accumulated
+347 /// on the passed in `transmit`.
+348 Send { last_packet_number: Option<u64> },
+349 /// A transmit is ready to be sent out.
+350 SendTransmit {
+351 /// The transmit to send.
+352 transmit: Transmit,
+353 },
+354 /// Nothing to send currently on this space/path.
+355 NothingToSend {
+356 /// Set to `true` if we consider the current inability to send something because of congestion control
+357 congestion_blocked: bool,
+358 },
+359}
+360
+361impl Connection {
+362 pub(crate) fn new(
+363 endpoint_config: Arc<EndpointConfig>,
+364 config: Arc<TransportConfig>,
+365 init_cid: ConnectionId,
+366 loc_cid: ConnectionId,
+367 rem_cid: ConnectionId,
+368 network_path: FourTuple,
+369 crypto: Box<dyn crypto::Session>,
+370 cid_gen: &dyn ConnectionIdGenerator,
+371 now: Instant,
+372 version: u32,
+373 allow_mtud: bool,
+374 rng_seed: [u8; 32],
+375 side_args: SideArgs,
+376 qlog: QlogSink,
+377 ) -> Self {
+378 let pref_addr_cid = side_args.pref_addr_cid();
+379 let path_validated = side_args.path_validated();
+380 let connection_side = ConnectionSide::from(side_args);
+381 let side = connection_side.side();
+382 let mut rng = StdRng::from_seed(rng_seed);
+383 let initial_space = {
+384 let mut space = PacketSpace::new(now, SpaceId::Initial, &mut rng);
+385 space.crypto = Some(crypto.initial_keys(init_cid, side));
+386 space
+387 };
+388 let handshake_space = PacketSpace::new(now, SpaceId::Handshake, &mut rng);
+389 #[cfg(test)]
+390 let data_space = match config.deterministic_packet_numbers {
+391 true => PacketSpace::new_deterministic(now, SpaceId::Data),
+392 false => PacketSpace::new(now, SpaceId::Data, &mut rng),
+393 };
+394 #[cfg(not(test))]
+395 let data_space = PacketSpace::new(now, SpaceId::Data, &mut rng);
+396 let state = State::handshake(state::Handshake {
+397 rem_cid_set: side.is_server(),
+398 expected_token: Bytes::new(),
+399 client_hello: None,
+400 allow_server_migration: side.is_client(),
+401 });
+402 let local_cid_state = FxHashMap::from_iter([(
+403 PathId::ZERO,
+404 CidState::new(
+405 cid_gen.cid_len(),
+406 cid_gen.cid_lifetime(),
+407 now,
+408 if pref_addr_cid.is_some() { 2 } else { 1 },
+409 ),
+410 )]);
+411
+412 let mut path = PathData::new(network_path, allow_mtud, None, 0, now, &config);
+413 // TODO(@divma): consider if we want to delay this until the path is validated
+414 path.open = true;
+415 let mut this = Self {
+416 endpoint_config,
+417 crypto,
+418 handshake_cid: loc_cid,
+419 rem_handshake_cid: rem_cid,
+420 local_cid_state,
+421 paths: BTreeMap::from_iter([(
+422 PathId::ZERO,
+423 PathState {
+424 data: path,
+425 prev: None,
+426 },
+427 )]),
+428 path_generation_counter: 0,
+429 allow_mtud,
+430 state,
+431 side: connection_side,
+432 zero_rtt_enabled: false,
+433 zero_rtt_crypto: None,
+434 key_phase: false,
+435 // A small initial key phase size ensures peers that don't handle key updates correctly
+436 // fail sooner rather than later. It's okay for both peers to do this, as the first one
+437 // to perform an update will reset the other's key phase size in `update_keys`, and a
+438 // simultaneous key update by both is just like a regular key update with a really fast
+439 // response. Inspired by quic-go's similar behavior of performing the first key update
+440 // at the 100th short-header packet.
+441 key_phase_size: rng.random_range(10..1000),
+442 peer_params: TransportParameters::default(),
+443 orig_rem_cid: rem_cid,
+444 initial_dst_cid: init_cid,
+445 retry_src_cid: None,
+446 events: VecDeque::new(),
+447 endpoint_events: VecDeque::new(),
+448 spin_enabled: config.allow_spin && rng.random_ratio(7, 8),
+449 spin: false,
+450 spaces: [initial_space, handshake_space, data_space],
+451 highest_space: SpaceId::Initial,
+452 prev_crypto: None,
+453 next_crypto: None,
+454 accepted_0rtt: false,
+455 permit_idle_reset: true,
+456 idle_timeout: match config.max_idle_timeout {
+457 None | Some(VarInt(0)) => None,
+458 Some(dur) => Some(Duration::from_millis(dur.0)),
+459 },
+460 timers: TimerTable::default(),
+461 authentication_failures: 0,
+462 close: false,
+463
+464 ack_frequency: AckFrequencyState::new(get_max_ack_delay(
+465 &TransportParameters::default(),
+466 )),
+467
+468 app_limited: false,
+469 receiving_ecn: false,
+470 total_authed_packets: 0,
+471
+472 next_observed_addr_seq_no: 0u32.into(),
473
-474 next_observed_addr_seq_no: 0u32.into(),
-475
-476 streams: StreamsState::new(
-477 side,
-478 config.max_concurrent_uni_streams,
-479 config.max_concurrent_bidi_streams,
-480 config.send_window,
-481 config.receive_window,
-482 config.stream_receive_window,
-483 ),
-484 datagrams: DatagramState::default(),
-485 config,
-486 rem_cids: FxHashMap::from_iter([(PathId::ZERO, CidQueue::new(rem_cid))]),
-487 rng,
-488 stats: ConnectionStats::default(),
-489 path_stats: Default::default(),
-490 version,
-491
-492 // peer params are not yet known, so multipath is not enabled
-493 max_concurrent_paths: NonZeroU32::MIN,
-494 local_max_path_id: PathId::ZERO,
-495 remote_max_path_id: PathId::ZERO,
-496 max_path_id_with_cids: PathId::ZERO,
-497 abandoned_paths: Default::default(),
-498
-499 // iroh's nat traversal
-500 iroh_hp: Default::default(),
-501 qlog,
-502 };
-503 if path_validated {
-504 this.on_path_validated(PathId::ZERO);
-505 }
-506 if side.is_client() {
-507 // Kick off the connection
-508 this.write_crypto();
-509 this.init_0rtt(now);
-510 }
-511 this.qlog
-512 .emit_tuple_assigned(PathId::ZERO, network_path, now);
-513 this
-514 }
-515
-516 /// Returns the next time at which `handle_timeout` should be called
-517 ///
-518 /// The value returned may change after:
-519 /// - the application performed some I/O on the connection
-520 /// - a call was made to `handle_event`
-521 /// - a call to `poll_transmit` returned `Some`
-522 /// - a call was made to `handle_timeout`
-523 #[must_use]
-524 pub fn poll_timeout(&mut self) -> Option<Instant> {
-525 self.timers.peek()
-526 }
-527
-528 /// Returns application-facing events
-529 ///
-530 /// Connections should be polled for events after:
-531 /// - a call was made to `handle_event`
-532 /// - a call was made to `handle_timeout`
-533 #[must_use]
-534 pub fn poll(&mut self) -> Option<Event> {
-535 if let Some(x) = self.events.pop_front() {
-536 return Some(x);
-537 }
-538
-539 if let Some(event) = self.streams.poll() {
-540 return Some(Event::Stream(event));
-541 }
-542
-543 if let Some(reason) = self.state.take_error() {
-544 return Some(Event::ConnectionLost { reason });
-545 }
-546
-547 None
-548 }
-549
-550 /// Return endpoint-facing events
-551 #[must_use]
-552 pub fn poll_endpoint_events(&mut self) -> Option<EndpointEvent> {
-553 self.endpoint_events.pop_front().map(EndpointEvent)
-554 }
-555
-556 /// Provide control over streams
-557 #[must_use]
-558 pub fn streams(&mut self) -> Streams<'_> {
-559 Streams {
-560 state: &mut self.streams,
-561 conn_state: &self.state,
-562 }
-563 }
-564
-565 /// Provide control over streams
-566 #[must_use]
-567 pub fn recv_stream(&mut self, id: StreamId) -> RecvStream<'_> {
-568 assert!(id.dir() == Dir::Bi || id.initiator() != self.side.side());
-569 RecvStream {
-570 id,
-571 state: &mut self.streams,
-572 pending: &mut self.spaces[SpaceId::Data].pending,
-573 }
-574 }
-575
-576 /// Provide control over streams
-577 #[must_use]
-578 pub fn send_stream(&mut self, id: StreamId) -> SendStream<'_> {
-579 assert!(id.dir() == Dir::Bi || id.initiator() == self.side.side());
-580 SendStream {
-581 id,
-582 state: &mut self.streams,
-583 pending: &mut self.spaces[SpaceId::Data].pending,
-584 conn_state: &self.state,
-585 }
-586 }
-587
-588 /// Opens a new path only if no path on the same network path currently exists.
-589 ///
-590 /// This comparison will use [`FourTuple::is_probably_same_path`] on the given `network_path`
-591 /// and pass it existing path's network paths.
-592 ///
-593 /// This means that you can pass `local_ip: None` to make the comparison only compare
-594 /// remote addresses.
-595 ///
-596 /// This avoids having to guess which local interface will be used to communicate with the
-597 /// remote, should it not be known yet. We assume that if we already have a path to the remote,
-598 /// the OS is likely to use the same interface to talk to said remote.
-599 ///
-600 /// See also [`open_path`]. Returns `(path_id, true)` if the path already existed. `(path_id,
-601 /// false)` if was opened.
-602 ///
-603 /// [`open_path`]: Connection::open_path
-604 pub fn open_path_ensure(
-605 &mut self,
-606 network_path: FourTuple,
-607 initial_status: PathStatus,
-608 now: Instant,
-609 ) -> Result<(PathId, bool), PathError> {
-610 let existing_open_path = self.paths.iter().find(|(id, path)| {
-611 network_path.is_probably_same_path(&path.data.network_path)
-612 && !self.abandoned_paths.contains(*id)
-613 });
-614 match existing_open_path {
-615 Some((path_id, _state)) => Ok((*path_id, true)),
-616 None => Ok((self.open_path(network_path, initial_status, now)?, false)),
-617 }
-618 }
-619
-620 /// Opens a new path
-621 ///
-622 /// Further errors might occur and they will be emitted in [`PathEvent::LocallyClosed`] events.
-623 /// When the path is opened it will be reported as an [`PathEvent::Opened`].
-624 pub fn open_path(
-625 &mut self,
-626 network_path: FourTuple,
-627 initial_status: PathStatus,
-628 now: Instant,
-629 ) -> Result<PathId, PathError> {
-630 if !self.is_multipath_negotiated() {
-631 return Err(PathError::MultipathNotNegotiated);
-632 }
-633 if self.side().is_server() {
-634 return Err(PathError::ServerSideNotAllowed);
-635 }
-636
-637 let max_abandoned = self.abandoned_paths.iter().max().copied();
-638 let max_used = self.paths.keys().last().copied();
-639 let path_id = max_abandoned
-640 .max(max_used)
-641 .unwrap_or(PathId::ZERO)
-642 .saturating_add(1u8);
-643
-644 if Some(path_id) > self.max_path_id() {
-645 return Err(PathError::MaxPathIdReached);
-646 }
-647 if path_id > self.remote_max_path_id {
-648 self.spaces[SpaceId::Data].pending.paths_blocked = true;
-649 return Err(PathError::MaxPathIdReached);
-650 }
-651 if self.rem_cids.get(&path_id).map(CidQueue::active).is_none() {
-652 self.spaces[SpaceId::Data]
-653 .pending
-654 .path_cids_blocked
-655 .insert(path_id);
-656 return Err(PathError::RemoteCidsExhausted);
-657 }
-658
-659 let path = self.ensure_path(path_id, network_path, now, None);
-660 path.status.local_update(initial_status);
-661
-662 Ok(path_id)
-663 }
-664
-665 /// Closes a path by sending a PATH_ABANDON frame
-666 ///
-667 /// This will not allow closing the last path. It does allow closing paths which have
-668 /// not yet been opened, as e.g. is the case when receiving a PATH_ABANDON from the peer
-669 /// for a path that was never opened locally.
-670 pub fn close_path(
-671 &mut self,
-672 now: Instant,
-673 path_id: PathId,
-674 error_code: VarInt,
-675 ) -> Result<(), ClosePathError> {
-676 if self.abandoned_paths.contains(&path_id)
-677 || Some(path_id) > self.max_path_id()
-678 || !self.paths.contains_key(&path_id)
-679 {
-680 return Err(ClosePathError::ClosedPath);
-681 }
-682 if self
-683 .paths
-684 .iter()
-685 // Would there be any remaining, non-abandoned, validated paths
-686 .any(|(id, path)| {
-687 *id != path_id && !self.abandoned_paths.contains(id) && path.data.validated
-688 })
-689 .not()
-690 {
-691 return Err(ClosePathError::LastOpenPath);
-692 }
-693
-694 // Send PATH_ABANDON
-695 self.spaces[SpaceId::Data]
-696 .pending
-697 .path_abandon
-698 .insert(path_id, error_code.into());
-699
-700 // Remove pending NEW CIDs for this path
-701 let pending_space = &mut self.spaces[SpaceId::Data].pending;
-702 pending_space.new_cids.retain(|cid| cid.path_id != path_id);
-703 pending_space.path_cids_blocked.retain(|&id| id != path_id);
-704 pending_space.path_status.retain(|&id| id != path_id);
-705
-706 // Cleanup retransmits across ALL paths (CIDs for path_id may have been transmitted on other paths)
-707 for space in self.spaces[SpaceId::Data].iter_paths_mut() {
-708 for sent_packet in space.sent_packets.values_mut() {
-709 if let Some(retransmits) = sent_packet.retransmits.get_mut() {
-710 retransmits.new_cids.retain(|cid| cid.path_id != path_id);
-711 retransmits.path_cids_blocked.retain(|&id| id != path_id);
-712 retransmits.path_status.retain(|&id| id != path_id);
-713 }
-714 }
-715 }
-716
-717 // Consider remotely issued CIDs as retired.
-718 // Technically we don't have to do this just yet. We only need to do this *after*
-719 // the ABANDON_PATH frame is sent, allowing us to still send it on the
-720 // to-be-abandoned path. However it is recommended to send it on another path, and
-721 // we do not allow abandoning the last path anyway.
-722 self.rem_cids.remove(&path_id);
-723 self.endpoint_events
-724 .push_back(EndpointEventInner::RetireResetToken(path_id));
+474 streams: StreamsState::new(
+475 side,
+476 config.max_concurrent_uni_streams,
+477 config.max_concurrent_bidi_streams,
+478 config.send_window,
+479 config.receive_window,
+480 config.stream_receive_window,
+481 ),
+482 datagrams: DatagramState::default(),
+483 config,
+484 rem_cids: FxHashMap::from_iter([(PathId::ZERO, CidQueue::new(rem_cid))]),
+485 rng,
+486 stats: ConnectionStats::default(),
+487 path_stats: Default::default(),
+488 version,
+489
+490 // peer params are not yet known, so multipath is not enabled
+491 max_concurrent_paths: NonZeroU32::MIN,
+492 local_max_path_id: PathId::ZERO,
+493 remote_max_path_id: PathId::ZERO,
+494 max_path_id_with_cids: PathId::ZERO,
+495 abandoned_paths: Default::default(),
+496
+497 // iroh's nat traversal
+498 iroh_hp: Default::default(),
+499 qlog,
+500 };
+501 if path_validated {
+502 this.on_path_validated(PathId::ZERO);
+503 }
+504 if side.is_client() {
+505 // Kick off the connection
+506 this.write_crypto();
+507 this.init_0rtt(now);
+508 }
+509 this.qlog
+510 .emit_tuple_assigned(PathId::ZERO, network_path, now);
+511 this
+512 }
+513
+514 /// Returns the next time at which `handle_timeout` should be called
+515 ///
+516 /// The value returned may change after:
+517 /// - the application performed some I/O on the connection
+518 /// - a call was made to `handle_event`
+519 /// - a call to `poll_transmit` returned `Some`
+520 /// - a call was made to `handle_timeout`
+521 #[must_use]
+522 pub fn poll_timeout(&mut self) -> Option<Instant> {
+523 self.timers.peek()
+524 }
+525
+526 /// Returns application-facing events
+527 ///
+528 /// Connections should be polled for events after:
+529 /// - a call was made to `handle_event`
+530 /// - a call was made to `handle_timeout`
+531 #[must_use]
+532 pub fn poll(&mut self) -> Option<Event> {
+533 if let Some(x) = self.events.pop_front() {
+534 return Some(x);
+535 }
+536
+537 if let Some(event) = self.streams.poll() {
+538 return Some(Event::Stream(event));
+539 }
+540
+541 if let Some(reason) = self.state.take_error() {
+542 return Some(Event::ConnectionLost { reason });
+543 }
+544
+545 None
+546 }
+547
+548 /// Return endpoint-facing events
+549 #[must_use]
+550 pub fn poll_endpoint_events(&mut self) -> Option<EndpointEvent> {
+551 self.endpoint_events.pop_front().map(EndpointEvent)
+552 }
+553
+554 /// Provide control over streams
+555 #[must_use]
+556 pub fn streams(&mut self) -> Streams<'_> {
+557 Streams {
+558 state: &mut self.streams,
+559 conn_state: &self.state,
+560 }
+561 }
+562
+563 /// Provide control over streams
+564 #[must_use]
+565 pub fn recv_stream(&mut self, id: StreamId) -> RecvStream<'_> {
+566 assert!(id.dir() == Dir::Bi || id.initiator() != self.side.side());
+567 RecvStream {
+568 id,
+569 state: &mut self.streams,
+570 pending: &mut self.spaces[SpaceId::Data].pending,
+571 }
+572 }
+573
+574 /// Provide control over streams
+575 #[must_use]
+576 pub fn send_stream(&mut self, id: StreamId) -> SendStream<'_> {
+577 assert!(id.dir() == Dir::Bi || id.initiator() == self.side.side());
+578 SendStream {
+579 id,
+580 state: &mut self.streams,
+581 pending: &mut self.spaces[SpaceId::Data].pending,
+582 conn_state: &self.state,
+583 }
+584 }
+585
+586 /// Opens a new path only if no path on the same network path currently exists.
+587 ///
+588 /// This comparison will use [`FourTuple::is_probably_same_path`] on the given `network_path`
+589 /// and pass it existing path's network paths.
+590 ///
+591 /// This means that you can pass `local_ip: None` to make the comparison only compare
+592 /// remote addresses.
+593 ///
+594 /// This avoids having to guess which local interface will be used to communicate with the
+595 /// remote, should it not be known yet. We assume that if we already have a path to the remote,
+596 /// the OS is likely to use the same interface to talk to said remote.
+597 ///
+598 /// See also [`open_path`]. Returns `(path_id, true)` if the path already existed. `(path_id,
+599 /// false)` if was opened.
+600 ///
+601 /// [`open_path`]: Connection::open_path
+602 pub fn open_path_ensure(
+603 &mut self,
+604 network_path: FourTuple,
+605 initial_status: PathStatus,
+606 now: Instant,
+607 ) -> Result<(PathId, bool), PathError> {
+608 let existing_open_path = self.paths.iter().find(|(id, path)| {
+609 network_path.is_probably_same_path(&path.data.network_path)
+610 && !self.abandoned_paths.contains(*id)
+611 });
+612 match existing_open_path {
+613 Some((path_id, _state)) => Ok((*path_id, true)),
+614 None => Ok((self.open_path(network_path, initial_status, now)?, false)),
+615 }
+616 }
+617
+618 /// Opens a new path
+619 ///
+620 /// Further errors might occur and they will be emitted in [`PathEvent::LocallyClosed`] events.
+621 /// When the path is opened it will be reported as an [`PathEvent::Opened`].
+622 pub fn open_path(
+623 &mut self,
+624 network_path: FourTuple,
+625 initial_status: PathStatus,
+626 now: Instant,
+627 ) -> Result<PathId, PathError> {
+628 if !self.is_multipath_negotiated() {
+629 return Err(PathError::MultipathNotNegotiated);
+630 }
+631 if self.side().is_server() {
+632 return Err(PathError::ServerSideNotAllowed);
+633 }
+634
+635 let max_abandoned = self.abandoned_paths.iter().max().copied();
+636 let max_used = self.paths.keys().last().copied();
+637 let path_id = max_abandoned
+638 .max(max_used)
+639 .unwrap_or(PathId::ZERO)
+640 .saturating_add(1u8);
+641
+642 if Some(path_id) > self.max_path_id() {
+643 return Err(PathError::MaxPathIdReached);
+644 }
+645 if path_id > self.remote_max_path_id {
+646 self.spaces[SpaceId::Data].pending.paths_blocked = true;
+647 return Err(PathError::MaxPathIdReached);
+648 }
+649 if self.rem_cids.get(&path_id).map(CidQueue::active).is_none() {
+650 self.spaces[SpaceId::Data]
+651 .pending
+652 .path_cids_blocked
+653 .insert(path_id);
+654 return Err(PathError::RemoteCidsExhausted);
+655 }
+656
+657 let path = self.ensure_path(path_id, network_path, now, None);
+658 path.status.local_update(initial_status);
+659
+660 Ok(path_id)
+661 }
+662
+663 /// Closes a path by sending a PATH_ABANDON frame
+664 ///
+665 /// This will not allow closing the last path. It does allow closing paths which have
+666 /// not yet been opened, as e.g. is the case when receiving a PATH_ABANDON from the peer
+667 /// for a path that was never opened locally.
+668 pub fn close_path(
+669 &mut self,
+670 now: Instant,
+671 path_id: PathId,
+672 error_code: VarInt,
+673 ) -> Result<(), ClosePathError> {
+674 if self.abandoned_paths.contains(&path_id)
+675 || Some(path_id) > self.max_path_id()
+676 || !self.paths.contains_key(&path_id)
+677 {
+678 return Err(ClosePathError::ClosedPath);
+679 }
+680 if self
+681 .paths
+682 .iter()
+683 // Would there be any remaining, non-abandoned, validated paths
+684 .any(|(id, path)| {
+685 *id != path_id && !self.abandoned_paths.contains(id) && path.data.validated
+686 })
+687 .not()
+688 {
+689 return Err(ClosePathError::LastOpenPath);
+690 }
+691
+692 // Send PATH_ABANDON
+693 self.spaces[SpaceId::Data]
+694 .pending
+695 .path_abandon
+696 .insert(path_id, error_code.into());
+697
+698 // Remove pending NEW CIDs for this path
+699 let pending_space = &mut self.spaces[SpaceId::Data].pending;
+700 pending_space.new_cids.retain(|cid| cid.path_id != path_id);
+701 pending_space.path_cids_blocked.retain(|&id| id != path_id);
+702 pending_space.path_status.retain(|&id| id != path_id);
+703
+704 // Cleanup retransmits across ALL paths (CIDs for path_id may have been transmitted on other paths)
+705 for space in self.spaces[SpaceId::Data].iter_paths_mut() {
+706 for sent_packet in space.sent_packets.values_mut() {
+707 if let Some(retransmits) = sent_packet.retransmits.get_mut() {
+708 retransmits.new_cids.retain(|cid| cid.path_id != path_id);
+709 retransmits.path_cids_blocked.retain(|&id| id != path_id);
+710 retransmits.path_status.retain(|&id| id != path_id);
+711 }
+712 }
+713 }
+714
+715 // Consider remotely issued CIDs as retired.
+716 // Technically we don't have to do this just yet. We only need to do this *after*
+717 // the ABANDON_PATH frame is sent, allowing us to still send it on the
+718 // to-be-abandoned path. However it is recommended to send it on another path, and
+719 // we do not allow abandoning the last path anyway.
+720 self.rem_cids.remove(&path_id);
+721 self.endpoint_events
+722 .push_back(EndpointEventInner::RetireResetToken(path_id));
+723
+724 self.abandoned_paths.insert(path_id);
725
-726 self.abandoned_paths.insert(path_id);
+726 self.set_max_path_id(now, self.local_max_path_id.saturating_add(1u8));
727
-728 self.set_max_path_id(now, self.local_max_path_id.saturating_add(1u8));
-729
-730 Ok(())
-731 }
-732
-733 /// Gets the [`PathData`] for a known [`PathId`].
-734 ///
-735 /// Will panic if the path_id does not reference any known path.
-736 #[track_caller]
-737 fn path_data(&self, path_id: PathId) -> &PathData {
-738 if let Some(data) = self.paths.get(&path_id) {
-739 &data.data
-740 } else {
-741 panic!(
-742 "unknown path: {path_id}, currently known paths: {:?}",
-743 self.paths.keys().collect::<Vec<_>>()
-744 );
-745 }
-746 }
-747
-748 /// Gets a reference to the [`PathData`] for a [`PathId`]
-749 fn path(&self, path_id: PathId) -> Option<&PathData> {
-750 self.paths.get(&path_id).map(|path_state| &path_state.data)
-751 }
-752
-753 /// Gets a mutable reference to the [`PathData`] for a [`PathId`]
-754 fn path_mut(&mut self, path_id: PathId) -> Option<&mut PathData> {
-755 self.paths
-756 .get_mut(&path_id)
-757 .map(|path_state| &mut path_state.data)
-758 }
-759
-760 /// Returns all known paths.
-761 ///
-762 /// There is no guarantee any of these paths are open or usable.
-763 pub fn paths(&self) -> Vec<PathId> {
-764 self.paths.keys().copied().collect()
-765 }
-766
-767 /// Gets the local [`PathStatus`] for a known [`PathId`]
-768 pub fn path_status(&self, path_id: PathId) -> Result<PathStatus, ClosedPath> {
-769 self.path(path_id)
-770 .map(PathData::local_status)
-771 .ok_or(ClosedPath { _private: () })
-772 }
-773
-774 /// Returns the path's network path represented as a 4-tuple.
-775 pub fn network_path(&self, path_id: PathId) -> Result<FourTuple, ClosedPath> {
-776 self.path(path_id)
-777 .map(|path| path.network_path)
-778 .ok_or(ClosedPath { _private: () })
-779 }
-780
-781 /// Sets the [`PathStatus`] for a known [`PathId`]
-782 ///
-783 /// Returns the previous path status on success.
-784 pub fn set_path_status(
-785 &mut self,
-786 path_id: PathId,
-787 status: PathStatus,
-788 ) -> Result<PathStatus, SetPathStatusError> {
-789 if !self.is_multipath_negotiated() {
-790 return Err(SetPathStatusError::MultipathNotNegotiated);
-791 }
-792 let path = self
-793 .path_mut(path_id)
-794 .ok_or(SetPathStatusError::ClosedPath)?;
-795 let prev = match path.status.local_update(status) {
-796 Some(prev) => {
-797 self.spaces[SpaceId::Data]
-798 .pending
-799 .path_status
-800 .insert(path_id);
-801 prev
-802 }
-803 None => path.local_status(),
-804 };
-805 Ok(prev)
-806 }
-807
-808 /// Returns the remote path status
-809 // TODO(flub): Probably should also be some kind of path event? Not even sure if I like
-810 // this as an API, but for now it allows me to write a test easily.
-811 // TODO(flub): Technically this should be a Result<Option<PathSTatus>>?
-812 pub fn remote_path_status(&self, path_id: PathId) -> Option<PathStatus> {
-813 self.path(path_id).and_then(|path| path.remote_status())
-814 }
-815
-816 /// Sets the max_idle_timeout for a specific path
+728 Ok(())
+729 }
+730
+731 /// Gets the [`PathData`] for a known [`PathId`].
+732 ///
+733 /// Will panic if the path_id does not reference any known path.
+734 #[track_caller]
+735 fn path_data(&self, path_id: PathId) -> &PathData {
+736 if let Some(data) = self.paths.get(&path_id) {
+737 &data.data
+738 } else {
+739 panic!(
+740 "unknown path: {path_id}, currently known paths: {:?}",
+741 self.paths.keys().collect::<Vec<_>>()
+742 );
+743 }
+744 }
+745
+746 /// Gets a reference to the [`PathData`] for a [`PathId`]
+747 fn path(&self, path_id: PathId) -> Option<&PathData> {
+748 self.paths.get(&path_id).map(|path_state| &path_state.data)
+749 }
+750
+751 /// Gets a mutable reference to the [`PathData`] for a [`PathId`]
+752 fn path_mut(&mut self, path_id: PathId) -> Option<&mut PathData> {
+753 self.paths
+754 .get_mut(&path_id)
+755 .map(|path_state| &mut path_state.data)
+756 }
+757
+758 /// Returns all known paths.
+759 ///
+760 /// There is no guarantee any of these paths are open or usable.
+761 pub fn paths(&self) -> Vec<PathId> {
+762 self.paths.keys().copied().collect()
+763 }
+764
+765 /// Gets the local [`PathStatus`] for a known [`PathId`]
+766 pub fn path_status(&self, path_id: PathId) -> Result<PathStatus, ClosedPath> {
+767 self.path(path_id)
+768 .map(PathData::local_status)
+769 .ok_or(ClosedPath { _private: () })
+770 }
+771
+772 /// Returns the path's network path represented as a 4-tuple.
+773 pub fn network_path(&self, path_id: PathId) -> Result<FourTuple, ClosedPath> {
+774 self.path(path_id)
+775 .map(|path| path.network_path)
+776 .ok_or(ClosedPath { _private: () })
+777 }
+778
+779 /// Sets the [`PathStatus`] for a known [`PathId`]
+780 ///
+781 /// Returns the previous path status on success.
+782 pub fn set_path_status(
+783 &mut self,
+784 path_id: PathId,
+785 status: PathStatus,
+786 ) -> Result<PathStatus, SetPathStatusError> {
+787 if !self.is_multipath_negotiated() {
+788 return Err(SetPathStatusError::MultipathNotNegotiated);
+789 }
+790 let path = self
+791 .path_mut(path_id)
+792 .ok_or(SetPathStatusError::ClosedPath)?;
+793 let prev = match path.status.local_update(status) {
+794 Some(prev) => {
+795 self.spaces[SpaceId::Data]
+796 .pending
+797 .path_status
+798 .insert(path_id);
+799 prev
+800 }
+801 None => path.local_status(),
+802 };
+803 Ok(prev)
+804 }
+805
+806 /// Returns the remote path status
+807 // TODO(flub): Probably should also be some kind of path event? Not even sure if I like
+808 // this as an API, but for now it allows me to write a test easily.
+809 // TODO(flub): Technically this should be a Result<Option<PathSTatus>>?
+810 pub fn remote_path_status(&self, path_id: PathId) -> Option<PathStatus> {
+811 self.path(path_id).and_then(|path| path.remote_status())
+812 }
+813
+814 /// Sets the max_idle_timeout for a specific path
+815 ///
+816 /// See [`TransportConfig::default_path_max_idle_timeout`] for details.
817 ///
-818 /// See [`TransportConfig::default_path_max_idle_timeout`] for details.
-819 ///
-820 /// Returns the previous value of the setting.
-821 pub fn set_path_max_idle_timeout(
-822 &mut self,
-823 path_id: PathId,
-824 timeout: Option<Duration>,
-825 ) -> Result<Option<Duration>, ClosedPath> {
-826 let path = self
-827 .paths
-828 .get_mut(&path_id)
-829 .ok_or(ClosedPath { _private: () })?;
-830 Ok(std::mem::replace(&mut path.data.idle_timeout, timeout))
-831 }
-832
-833 /// Sets the keep_alive_interval for a specific path
+818 /// Returns the previous value of the setting.
+819 pub fn set_path_max_idle_timeout(
+820 &mut self,
+821 path_id: PathId,
+822 timeout: Option<Duration>,
+823 ) -> Result<Option<Duration>, ClosedPath> {
+824 let path = self
+825 .paths
+826 .get_mut(&path_id)
+827 .ok_or(ClosedPath { _private: () })?;
+828 Ok(std::mem::replace(&mut path.data.idle_timeout, timeout))
+829 }
+830
+831 /// Sets the keep_alive_interval for a specific path
+832 ///
+833 /// See [`TransportConfig::default_path_keep_alive_interval`] for details.
834 ///
-835 /// See [`TransportConfig::default_path_keep_alive_interval`] for details.
-836 ///
-837 /// Returns the previous value of the setting.
-838 pub fn set_path_keep_alive_interval(
-839 &mut self,
-840 path_id: PathId,
-841 interval: Option<Duration>,
-842 ) -> Result<Option<Duration>, ClosedPath> {
-843 let path = self
-844 .paths
-845 .get_mut(&path_id)
-846 .ok_or(ClosedPath { _private: () })?;
-847 Ok(std::mem::replace(&mut path.data.keep_alive, interval))
-848 }
-849
-850 /// Gets the [`PathData`] for a known [`PathId`].
-851 ///
-852 /// Will panic if the path_id does not reference any known path.
-853 #[track_caller]
-854 fn path_data_mut(&mut self, path_id: PathId) -> &mut PathData {
-855 &mut self.paths.get_mut(&path_id).expect("known path").data
-856 }
-857
-858 /// Find an open, validated path that's on the same network path as the given network path.
-859 ///
-860 /// Returns the first path matching, even if there's multiple.
-861 fn find_validated_path_on_network_path(
-862 &self,
-863 network_path: FourTuple,
-864 ) -> Option<(&PathId, &PathState)> {
-865 self.paths.iter().find(|(path_id, path_state)| {
-866 path_state.data.validated
-867 // Would this use the same network path, if network_path were used to send right now?
-868 && network_path.is_probably_same_path(&path_state.data.network_path)
-869 && !self.abandoned_paths.contains(path_id)
-870 })
-871 // TODO(@divma): we might want to ensure the path has been recently active to consider the
-872 // address validated
-873 // matheus23: Perhaps looking at !self.abandoned_paths.contains(path_id) is enough, given keep-alives?
-874 }
-875
-876 fn ensure_path(
-877 &mut self,
-878 path_id: PathId,
-879 network_path: FourTuple,
-880 now: Instant,
-881 pn: Option<u64>,
-882 ) -> &mut PathData {
-883 let valid_path = self.find_validated_path_on_network_path(network_path);
-884 let validated = valid_path.is_some();
-885 let initial_rtt = valid_path.map(|(_, path)| path.data.rtt.conservative());
-886 let vacant_entry = match self.paths.entry(path_id) {
-887 btree_map::Entry::Vacant(vacant_entry) => vacant_entry,
-888 btree_map::Entry::Occupied(occupied_entry) => {
-889 return &mut occupied_entry.into_mut().data;
-890 }
-891 };
-892
-893 debug!(%validated, %path_id, %network_path, "path added");
-894 let peer_max_udp_payload_size =
-895 u16::try_from(self.peer_params.max_udp_payload_size.into_inner()).unwrap_or(u16::MAX);
-896 self.path_generation_counter = self.path_generation_counter.wrapping_add(1);
-897 let mut data = PathData::new(
-898 network_path,
-899 self.allow_mtud,
-900 Some(peer_max_udp_payload_size),
-901 self.path_generation_counter,
-902 now,
-903 &self.config,
-904 );
-905
-906 data.validated = validated;
-907 if let Some(initial_rtt) = initial_rtt {
-908 data.rtt.reset_initial_rtt(initial_rtt);
-909 }
-910
-911 let pto = self.ack_frequency.max_ack_delay_for_pto() + data.rtt.pto_base();
-912 self.timers.set(
-913 Timer::PerPath(path_id, PathTimer::PathOpen),
-914 now + 3 * pto,
-915 self.qlog.with_time(now),
-916 );
-917
-918 // for the path to be opened we need to send a packet on the path. Sending a challenge
-919 // guarantees this
-920 data.send_new_challenge = true;
+835 /// Returns the previous value of the setting.
+836 pub fn set_path_keep_alive_interval(
+837 &mut self,
+838 path_id: PathId,
+839 interval: Option<Duration>,
+840 ) -> Result<Option<Duration>, ClosedPath> {
+841 let path = self
+842 .paths
+843 .get_mut(&path_id)
+844 .ok_or(ClosedPath { _private: () })?;
+845 Ok(std::mem::replace(&mut path.data.keep_alive, interval))
+846 }
+847
+848 /// Gets the [`PathData`] for a known [`PathId`].
+849 ///
+850 /// Will panic if the path_id does not reference any known path.
+851 #[track_caller]
+852 fn path_data_mut(&mut self, path_id: PathId) -> &mut PathData {
+853 &mut self.paths.get_mut(&path_id).expect("known path").data
+854 }
+855
+856 /// Find an open, validated path that's on the same network path as the given network path.
+857 ///
+858 /// Returns the first path matching, even if there's multiple.
+859 fn find_validated_path_on_network_path(
+860 &self,
+861 network_path: FourTuple,
+862 ) -> Option<(&PathId, &PathState)> {
+863 self.paths.iter().find(|(path_id, path_state)| {
+864 path_state.data.validated
+865 // Would this use the same network path, if network_path were used to send right now?
+866 && network_path.is_probably_same_path(&path_state.data.network_path)
+867 && !self.abandoned_paths.contains(path_id)
+868 })
+869 // TODO(@divma): we might want to ensure the path has been recently active to consider the
+870 // address validated
+871 // matheus23: Perhaps looking at !self.abandoned_paths.contains(path_id) is enough, given keep-alives?
+872 }
+873
+874 fn ensure_path(
+875 &mut self,
+876 path_id: PathId,
+877 network_path: FourTuple,
+878 now: Instant,
+879 pn: Option<u64>,
+880 ) -> &mut PathData {
+881 let valid_path = self.find_validated_path_on_network_path(network_path);
+882 let validated = valid_path.is_some();
+883 let initial_rtt = valid_path.map(|(_, path)| path.data.rtt.conservative());
+884 let vacant_entry = match self.paths.entry(path_id) {
+885 btree_map::Entry::Vacant(vacant_entry) => vacant_entry,
+886 btree_map::Entry::Occupied(occupied_entry) => {
+887 return &mut occupied_entry.into_mut().data;
+888 }
+889 };
+890
+891 debug!(%validated, %path_id, %network_path, "path added");
+892 let peer_max_udp_payload_size =
+893 u16::try_from(self.peer_params.max_udp_payload_size.into_inner()).unwrap_or(u16::MAX);
+894 self.path_generation_counter = self.path_generation_counter.wrapping_add(1);
+895 let mut data = PathData::new(
+896 network_path,
+897 self.allow_mtud,
+898 Some(peer_max_udp_payload_size),
+899 self.path_generation_counter,
+900 now,
+901 &self.config,
+902 );
+903
+904 data.validated = validated;
+905 if let Some(initial_rtt) = initial_rtt {
+906 data.rtt.reset_initial_rtt(initial_rtt);
+907 }
+908
+909 let pto = self.ack_frequency.max_ack_delay_for_pto() + data.rtt.pto_base();
+910 self.timers.set(
+911 Timer::PerPath(path_id, PathTimer::PathOpen),
+912 now + 3 * pto,
+913 self.qlog.with_time(now),
+914 );
+915
+916 // for the path to be opened we need to send a packet on the path. Sending a challenge
+917 // guarantees this
+918 data.send_new_challenge = true;
+919
+920 let path = vacant_entry.insert(PathState { data, prev: None });
921
-922 let path = vacant_entry.insert(PathState { data, prev: None });
-923
-924 let mut pn_space = spaces::PacketNumberSpace::new(now, SpaceId::Data, &mut self.rng);
-925 if let Some(pn) = pn {
-926 pn_space.dedup.insert(pn);
-927 }
-928 self.spaces[SpaceId::Data]
-929 .number_spaces
-930 .insert(path_id, pn_space);
-931 self.qlog.emit_tuple_assigned(path_id, network_path, now);
-932 &mut path.data
-933 }
-934
-935 /// Returns packets to transmit
-936 ///
-937 /// Connections should be polled for transmit after:
-938 /// - the application performed some I/O on the connection
-939 /// - a call was made to `handle_event`
-940 /// - a call was made to `handle_timeout`
-941 ///
-942 /// `max_datagrams` specifies how many datagrams can be returned inside a
-943 /// single Transmit using GSO. This must be at least 1.
-944 #[must_use]
-945 pub fn poll_transmit(
-946 &mut self,
-947 now: Instant,
-948 max_datagrams: NonZeroUsize,
-949 buf: &mut Vec<u8>,
-950 ) -> Option<Transmit> {
-951 if let Some(probing) = self
-952 .iroh_hp
-953 .server_side_mut()
-954 .ok()
-955 .and_then(iroh_hp::ServerState::next_probe)
-956 {
-957 let destination = probing.remote();
-958 trace!(%destination, "RAND_DATA packet");
-959 let token: u64 = self.rng.random();
-960 buf.put_u64(token);
-961 probing.finish(token);
-962 return Some(Transmit {
-963 destination,
-964 ecn: None,
-965 size: 8,
-966 segment_size: None,
-967 src_ip: None,
-968 });
-969 }
-970
-971 let max_datagrams = match self.config.enable_segmentation_offload {
-972 false => NonZeroUsize::MIN,
-973 true => max_datagrams,
-974 };
-975
-976 // Each call to poll_transmit can only send datagrams to one destination, because
-977 // all datagrams in a GSO batch are for the same destination. Therefore only
-978 // datagrams for one Path ID are produced for each poll_transmit call.
-979
-980 // TODO(flub): this is wishful thinking and not actually implemented, but perhaps it
-981 // should be:
-982
-983 // First, if we have to send a close, select a path for that.
-984 // Next, all paths that have a PATH_CHALLENGE or PATH_RESPONSE pending.
-985
-986 // For all, open, validated and AVAILABLE paths:
-987 // - Is the path congestion blocked or pacing blocked?
-988 // - call maybe_queue_ to ensure a tail-loss probe would be sent?
-989 // - do we need to send a close message?
-990 // - call can_send
-991 // Once there's nothing more to send on the AVAILABLE paths, do the same for BACKUP paths
-992
-993 // Check whether we need to send a close message
-994 let close = match self.state.as_type() {
-995 StateType::Drained => {
-996 self.app_limited = true;
-997 return None;
-998 }
-999 StateType::Draining | StateType::Closed => {
-1000 // self.close is only reset once the associated packet had been
-1001 // encoded successfully
-1002 if !self.close {
-1003 self.app_limited = true;
-1004 return None;
-1005 }
-1006 true
-1007 }
-1008 _ => false,
-1009 };
-1010
-1011 // Check whether we need to send an ACK_FREQUENCY frame
-1012 if let Some(config) = &self.config.ack_frequency_config {
-1013 let rtt = self
-1014 .paths
-1015 .values()
-1016 .map(|p| p.data.rtt.get())
-1017 .min()
-1018 .expect("one path exists");
-1019 self.spaces[SpaceId::Data].pending.ack_frequency = self
-1020 .ack_frequency
-1021 .should_send_ack_frequency(rtt, config, &self.peer_params)
-1022 && self.highest_space == SpaceId::Data
-1023 && self.peer_supports_ack_frequency();
-1024 }
-1025
-1026 // If there is any open, validated and available path we only want to send frames to
-1027 // any backup path that must be sent on that backup path exclusively.
-1028 let have_available_path = self.paths.iter().any(|(id, path)| {
-1029 path.data.validated
-1030 && path.data.local_status() == PathStatus::Available
-1031 && self.rem_cids.contains_key(id)
-1032 });
-1033
-1034 // TODO: how to avoid the allocation? Cannot use a for loop because of borrowing
-1035 let path_ids: Vec<_> = self.paths.keys().copied().collect();
-1036
-1037 let mut transmit = TransmitBuf::new(
-1038 buf,
-1039 max_datagrams,
-1040 self.path_data(path_ids[0]).current_mtu().into(),
-1041 );
+922 let mut pn_space = spaces::PacketNumberSpace::new(now, SpaceId::Data, &mut self.rng);
+923 if let Some(pn) = pn {
+924 pn_space.dedup.insert(pn);
+925 }
+926 self.spaces[SpaceId::Data]
+927 .number_spaces
+928 .insert(path_id, pn_space);
+929 self.qlog.emit_tuple_assigned(path_id, network_path, now);
+930 &mut path.data
+931 }
+932
+933 /// Returns packets to transmit
+934 ///
+935 /// Connections should be polled for transmit after:
+936 /// - the application performed some I/O on the connection
+937 /// - a call was made to `handle_event`
+938 /// - a call was made to `handle_timeout`
+939 ///
+940 /// `max_datagrams` specifies how many datagrams can be returned inside a
+941 /// single Transmit using GSO. This must be at least 1.
+942 #[must_use]
+943 pub fn poll_transmit(
+944 &mut self,
+945 now: Instant,
+946 max_datagrams: NonZeroUsize,
+947 buf: &mut Vec<u8>,
+948 ) -> Option<Transmit> {
+949 if let Some(probing) = self
+950 .iroh_hp
+951 .server_side_mut()
+952 .ok()
+953 .and_then(iroh_hp::ServerState::next_probe)
+954 {
+955 let destination = probing.remote();
+956 trace!(%destination, "RAND_DATA packet");
+957 let token: u64 = self.rng.random();
+958 buf.put_u64(token);
+959 probing.finish(token);
+960 return Some(Transmit {
+961 destination,
+962 ecn: None,
+963 size: 8,
+964 segment_size: None,
+965 src_ip: None,
+966 });
+967 }
+968
+969 let max_datagrams = match self.config.enable_segmentation_offload {
+970 false => NonZeroUsize::MIN,
+971 true => max_datagrams,
+972 };
+973
+974 // Each call to poll_transmit can only send datagrams to one destination, because
+975 // all datagrams in a GSO batch are for the same destination. Therefore only
+976 // datagrams for one Path ID are produced for each poll_transmit call.
+977
+978 // TODO(flub): this is wishful thinking and not actually implemented, but perhaps it
+979 // should be:
+980
+981 // First, if we have to send a close, select a path for that.
+982 // Next, all paths that have a PATH_CHALLENGE or PATH_RESPONSE pending.
+983
+984 // For all, open, validated and AVAILABLE paths:
+985 // - Is the path congestion blocked or pacing blocked?
+986 // - call maybe_queue_ to ensure a tail-loss probe would be sent?
+987 // - do we need to send a close message?
+988 // - call can_send
+989 // Once there's nothing more to send on the AVAILABLE paths, do the same for BACKUP paths
+990
+991 // Check whether we need to send a close message
+992 let close = match self.state.as_type() {
+993 StateType::Drained => {
+994 self.app_limited = true;
+995 return None;
+996 }
+997 StateType::Draining | StateType::Closed => {
+998 // self.close is only reset once the associated packet had been
+999 // encoded successfully
+1000 if !self.close {
+1001 self.app_limited = true;
+1002 return None;
+1003 }
+1004 true
+1005 }
+1006 _ => false,
+1007 };
+1008
+1009 // Check whether we need to send an ACK_FREQUENCY frame
+1010 if let Some(config) = &self.config.ack_frequency_config {
+1011 let rtt = self
+1012 .paths
+1013 .values()
+1014 .map(|p| p.data.rtt.get())
+1015 .min()
+1016 .expect("one path exists");
+1017 self.spaces[SpaceId::Data].pending.ack_frequency = self
+1018 .ack_frequency
+1019 .should_send_ack_frequency(rtt, config, &self.peer_params)
+1020 && self.highest_space == SpaceId::Data
+1021 && self.peer_supports_ack_frequency();
+1022 }
+1023
+1024 // If there is any open, validated and available path we only want to send frames to
+1025 // any backup path that must be sent on that backup path exclusively.
+1026 let have_available_path = self.paths.iter().any(|(id, path)| {
+1027 path.data.validated
+1028 && path.data.local_status() == PathStatus::Available
+1029 && self.rem_cids.contains_key(id)
+1030 });
+1031
+1032 // TODO: how to avoid the allocation? Cannot use a for loop because of borrowing
+1033 let path_ids: Vec<_> = self.paths.keys().copied().collect();
+1034
+1035 let mut transmit = TransmitBuf::new(
+1036 buf,
+1037 max_datagrams,
+1038 self.path_data(path_ids[0]).current_mtu().into(),
+1039 );
+1040
+1041 let mut congestion_blocked = false;
1042
-1043 let mut congestion_blocked = false;
-1044
-1045 for &path_id in &path_ids {
-1046 // Update per path state
-1047 transmit.set_segment_size(self.path_data(path_id).current_mtu().into());
-1048
-1049 if let Some(challenge) = self.send_prev_path_challenge(now, &mut transmit, path_id) {
-1050 return Some(challenge);
-1051 }
-1052
-1053 match self.poll_transmit_path(now, &mut transmit, path_id, have_available_path, close) {
-1054 PollPathStatus::SendTransmit { transmit } => {
-1055 return Some(transmit);
-1056 }
-1057 PollPathStatus::Send => {
-1058 let transmit = self.build_transmit(path_id, transmit);
-1059 return Some(transmit);
-1060 }
-1061 PollPathStatus::NothingToSend {
-1062 congestion_blocked: cb,
-1063 } => {
-1064 // TODO: congestion blocked should be per path? how to deal with this better
-1065
-1066 // move to next path
-1067 if cb {
-1068 // note congestion block
-1069 congestion_blocked = true;
-1070 }
-1071 }
-1072 }
-1073 // Nothing more to send.
-1074 trace!(%path_id, "nothing to send on path");
-1075 }
-1076
-1077 // We didn't produce any application data packet
-1078 debug_assert!(
-1079 transmit.is_empty(),
-1080 "there was data in the transmit, but it was not sent"
-1081 );
-1082 self.app_limited = !congestion_blocked;
-1083
-1084 if self.state.is_established() {
-1085 // Try MTU probing now
-1086 for path_id in path_ids {
-1087 // Update per path state
-1088 transmit.set_segment_size(self.path_data(path_id).current_mtu().into());
-1089 self.poll_transmit_mtu_probe(now, &mut transmit, path_id);
-1090 if !transmit.is_empty() {
-1091 let transmit = self.build_transmit(path_id, transmit);
-1092 return Some(transmit);
-1093 }
-1094 }
-1095 }
-1096
-1097 None
-1098 }
-1099
-1100 fn build_transmit(&mut self, path_id: PathId, transmit: TransmitBuf<'_>) -> Transmit {
-1101 debug_assert!(
-1102 !transmit.is_empty(),
-1103 "must not be called with an empty transmit buffer"
-1104 );
-1105
-1106 let network_path = self.path_data(path_id).network_path;
-1107 trace!(
-1108 segment_size = transmit.segment_size(),
-1109 last_datagram_len = transmit.len() % transmit.segment_size(),
-1110 %network_path,
-1111 "sending {} bytes in {} datagrams",
-1112 transmit.len(),
-1113 transmit.num_datagrams()
-1114 );
-1115 self.path_data_mut(path_id)
-1116 .inc_total_sent(transmit.len() as u64);
-1117
-1118 self.stats
-1119 .udp_tx
-1120 .on_sent(transmit.num_datagrams() as u64, transmit.len());
-1121
-1122 Transmit {
-1123 destination: network_path.remote,
-1124 size: transmit.len(),
-1125 ecn: if self.path_data(path_id).sending_ecn {
-1126 Some(EcnCodepoint::Ect0)
-1127 } else {
-1128 None
-1129 },
-1130 segment_size: match transmit.num_datagrams() {
-1131 1 => None,
-1132 _ => Some(transmit.segment_size()),
-1133 },
-1134 src_ip: network_path.local_ip,
-1135 }
-1136 }
-1137
-1138 /// poll_transmit logic for a specific path
-1139 #[must_use]
-1140 fn poll_transmit_path(
-1141 &mut self,
-1142 now: Instant,
-1143 transmit: &mut TransmitBuf<'_>,
-1144 path_id: PathId,
-1145 have_available_path: bool,
-1146 close: bool,
-1147 ) -> PollPathStatus {
-1148 // Check if there is at least one active CID to use for sending
-1149 let Some(remote_cid) = self.rem_cids.get(&path_id).map(CidQueue::active) else {
-1150 self.on_remote_cids_exhausted(now, path_id);
-1151
-1152 return PollPathStatus::NothingToSend {
-1153 congestion_blocked: false,
-1154 };
-1155 };
-1156
-1157 // Whether this packet can be coalesced with another one in the same datagram.
-1158 let mut coalesce = true;
-1159
-1160 // Whether the last packet in the datagram must be padded so the datagram takes up
-1161 // to at least MIN_INITIAL_SIZE, or to the maximum segment size if this is smaller.
-1162 let mut pad_datagram = PadDatagram::No;
-1163
-1164 // The packet number of the last built packet.
-1165 let mut last_packet_number = None;
+1043 for &path_id in &path_ids {
+1044 // Update per path state
+1045 transmit.set_segment_size(self.path_data(path_id).current_mtu().into());
+1046
+1047 if let Some(challenge) = self.send_prev_path_challenge(now, &mut transmit, path_id) {
+1048 return Some(challenge);
+1049 }
+1050
+1051 match self.poll_transmit_path(now, &mut transmit, path_id, have_available_path, close) {
+1052 PollPathStatus::SendTransmit { transmit } => {
+1053 return Some(transmit);
+1054 }
+1055 PollPathStatus::Send => {
+1056 let transmit = self.build_transmit(path_id, transmit);
+1057 return Some(transmit);
+1058 }
+1059 PollPathStatus::NothingToSend {
+1060 congestion_blocked: cb,
+1061 } => {
+1062 // TODO: congestion blocked should be per path? how to deal with this better
+1063
+1064 // move to next path
+1065 if cb {
+1066 // note congestion block
+1067 congestion_blocked = true;
+1068 }
+1069 }
+1070 }
+1071 // Nothing more to send.
+1072 trace!(%path_id, "nothing to send on path");
+1073 }
+1074
+1075 // We didn't produce any application data packet
+1076 debug_assert!(
+1077 transmit.is_empty(),
+1078 "there was data in the transmit, but it was not sent"
+1079 );
+1080 self.app_limited = !congestion_blocked;
+1081
+1082 if self.state.is_established() {
+1083 // Try MTU probing now
+1084 for path_id in path_ids {
+1085 // Update per path state
+1086 transmit.set_segment_size(self.path_data(path_id).current_mtu().into());
+1087 self.poll_transmit_mtu_probe(now, &mut transmit, path_id);
+1088 if !transmit.is_empty() {
+1089 let transmit = self.build_transmit(path_id, transmit);
+1090 return Some(transmit);
+1091 }
+1092 }
+1093 }
+1094
+1095 None
+1096 }
+1097
+1098 fn build_transmit(&mut self, path_id: PathId, transmit: TransmitBuf<'_>) -> Transmit {
+1099 debug_assert!(
+1100 !transmit.is_empty(),
+1101 "must not be called with an empty transmit buffer"
+1102 );
+1103
+1104 let network_path = self.path_data(path_id).network_path;
+1105 trace!(
+1106 segment_size = transmit.segment_size(),
+1107 last_datagram_len = transmit.len() % transmit.segment_size(),
+1108 %network_path,
+1109 "sending {} bytes in {} datagrams",
+1110 transmit.len(),
+1111 transmit.num_datagrams()
+1112 );
+1113 self.path_data_mut(path_id)
+1114 .inc_total_sent(transmit.len() as u64);
+1115
+1116 self.stats
+1117 .udp_tx
+1118 .on_sent(transmit.num_datagrams() as u64, transmit.len());
+1119
+1120 Transmit {
+1121 destination: network_path.remote,
+1122 size: transmit.len(),
+1123 ecn: if self.path_data(path_id).sending_ecn {
+1124 Some(EcnCodepoint::Ect0)
+1125 } else {
+1126 None
+1127 },
+1128 segment_size: match transmit.num_datagrams() {
+1129 1 => None,
+1130 _ => Some(transmit.segment_size()),
+1131 },
+1132 src_ip: network_path.local_ip,
+1133 }
+1134 }
+1135
+1136 /// poll_transmit logic for a specific path
+1137 #[must_use]
+1138 fn poll_transmit_path(
+1139 &mut self,
+1140 now: Instant,
+1141 transmit: &mut TransmitBuf<'_>,
+1142 path_id: PathId,
+1143 have_available_path: bool,
+1144 close: bool,
+1145 ) -> PollPathStatus {
+1146 // Check if there is at least one active CID to use for sending
+1147 let Some(remote_cid) = self.rem_cids.get(&path_id).map(CidQueue::active) else {
+1148 self.on_remote_cids_exhausted(now, path_id);
+1149
+1150 return PollPathStatus::NothingToSend {
+1151 congestion_blocked: false,
+1152 };
+1153 };
+1154
+1155 // Whether the last packet in the datagram must be padded so the datagram takes up
+1156 // to at least MIN_INITIAL_SIZE, or to the maximum segment size if this is smaller.
+1157 let mut pad_datagram = PadDatagram::No;
+1158
+1159 // The packet number of the last built packet.
+1160 let mut last_packet_number = None;
+1161
+1162 // Whether this packet can be coalesced with another one in the same datagram.
+1163 let mut coalesce = true;
+1164
+1165 let mut congestion_blocked = false;
1166
1167 // Iterate over the available spaces
1168 for space_id in SpaceId::iter() {
@@ -1170,5668 +1170,5796 @@
1170 if path_id != PathId::ZERO && space_id != SpaceId::Data {
1171 continue;
1172 }
-1173
-1174 let res = self.poll_transmit_path_space(
-1175 now,
-1176 transmit,
-1177 path_id,
-1178 space_id,
-1179 remote_cid,
-1180 have_available_path,
-1181 close,
-1182 &mut coalesce,
-1183 &mut pad_datagram,
-1184 );
-1185 match res {
-1186 PollPathSpaceStatus::Send {
-1187 last_packet_number: lp,
-1188 } => {
-1189 last_packet_number = lp;
-1190 break;
-1191 }
-1192 PollPathSpaceStatus::SendTransmit { transmit } => {
-1193 return PollPathStatus::SendTransmit { transmit };
-1194 }
-1195 PollPathSpaceStatus::NothingToSend { congestion_blocked } => {
-1196 return PollPathStatus::NothingToSend { congestion_blocked };
-1197 }
-1198 PollPathSpaceStatus::NextSpace => {
-1199 // moving onto the next space
-1200 }
-1201 }
-1202 }
-1203
-1204 if let Some(last_packet_number) = last_packet_number {
-1205 // Note that when sending in multiple packet spaces the last packet number will
-1206 // be the one from the highest packet space.
-1207 self.path_data_mut(path_id).congestion.on_sent(
-1208 now,
-1209 transmit.len() as u64,
-1210 last_packet_number,
-1211 );
-1212 }
-1213
-1214 self.qlog.emit_recovery_metrics(
-1215 path_id,
-1216 &mut self.paths.get_mut(&path_id).unwrap().data,
-1217 now,
-1218 );
-1219
-1220 if transmit.is_empty() {
-1221 PollPathStatus::NothingToSend {
-1222 congestion_blocked: false,
+1173 // We only send up to the highest space
+1174 if space_id >= self.highest_space {
+1175 break;
+1176 }
+1177
+1178 // Check if we can send in this space for lower spaces.
+1179 let sendable_frames = if space_id < SpaceId::Data {
+1180 let sendable_frames = self.space_can_send_non_data(space_id, path_id, close);
+1181 let path_should_send =
+1182 self.path_should_send(path_id, space_id, have_available_path, &sendable_frames);
+1183
+1184 if !path_should_send {
+1185 if self.spaces[space_id].crypto.is_some() {
+1186 trace!(?space_id, %path_id, "nothing to send in space");
+1187 }
+1188 continue;
+1189 }
+1190 Some(sendable_frames)
+1191 } else {
+1192 None
+1193 };
+1194
+1195 let res = self.poll_transmit_path_space(
+1196 now,
+1197 transmit,
+1198 path_id,
+1199 space_id,
+1200 remote_cid,
+1201 have_available_path,
+1202 close,
+1203 sendable_frames,
+1204 &mut coalesce,
+1205 &mut pad_datagram,
+1206 );
+1207 match res {
+1208 PollPathSpaceStatus::Send {
+1209 last_packet_number: lp,
+1210 } => {
+1211 last_packet_number = lp;
+1212 break;
+1213 }
+1214 PollPathSpaceStatus::SendTransmit { transmit } => {
+1215 return PollPathStatus::SendTransmit { transmit };
+1216 }
+1217 PollPathSpaceStatus::NothingToSend {
+1218 congestion_blocked: cb,
+1219 } => {
+1220 congestion_blocked |= cb;
+1221 continue;
+1222 }
1223 }
-1224 } else {
-1225 PollPathStatus::Send
-1226 }
-1227 }
-1228
-1229 /// poll_transmit logic for a path_id - space_id combination
-1230 #[must_use]
-1231 fn poll_transmit_path_space(
-1232 &mut self,
-1233 now: Instant,
-1234 transmit: &mut TransmitBuf<'_>,
-1235 path_id: PathId,
-1236 space_id: SpaceId,
-1237 remote_cid: ConnectionId,
-1238 have_available_path: bool,
-1239 close: bool,
-1240 coalesce: &mut bool,
-1241 pad_datagram: &mut PadDatagram,
-1242 ) -> PollPathSpaceStatus {
-1243 let mut last_packet_number = None;
-1244
-1245 loop {
-1246 // Determine if anything can be sent in this packet number space (SpaceId + PathId).
-1247 let max_packet_size = if transmit.datagram_remaining_mut() > 0 {
-1248 // We are trying to coalesce another packet into this datagram.
-1249 transmit.datagram_remaining_mut()
-1250 } else {
-1251 // A new datagram needs to be started.
-1252 transmit.segment_size()
-1253 };
-1254
-1255 let can_send = self.space_can_send(space_id, path_id, max_packet_size, close);
-1256
-1257 let path_should_send = {
-1258 let path_exclusive_only = space_id == SpaceId::Data
-1259 && have_available_path
-1260 && self.path_data(path_id).local_status() == PathStatus::Backup;
-1261 let path_should_send = if path_exclusive_only {
-1262 can_send.path_exclusive
-1263 } else {
-1264 !can_send.is_empty()
-1265 };
-1266 let needs_loss_probe = self.spaces[space_id].for_path(path_id).loss_probes > 0;
-1267 path_should_send || needs_loss_probe
-1268 };
-1269
-1270 if !path_should_send && space_id < SpaceId::Data {
-1271 if self.spaces[space_id].crypto.is_some() {
-1272 trace!(?space_id, %path_id, "nothing to send in space");
-1273 }
-1274 return PollPathSpaceStatus::NextSpace;
-1275 }
-1276
-1277 let send_blocked = if path_should_send && transmit.datagram_remaining_mut() == 0 {
-1278 // Only check congestion control if a new datagram is needed.
-1279 self.path_congestion_check(space_id, path_id, transmit, &can_send, now)
-1280 } else {
-1281 PathBlocked::No
-1282 };
-1283
-1284 // Whether congestion control stopped the next packet from being sent. Further
-1285 // packets could still be built, as e.g. tail-loss probes are not congestion
-1286 // limited.
-1287 let congestion_blocked = if send_blocked != PathBlocked::No {
-1288 trace!(?space_id, %path_id, ?send_blocked, "congestion blocked");
-1289 true
-1290 } else {
-1291 false
-1292 };
-1293 if send_blocked != PathBlocked::No && space_id < SpaceId::Data {
-1294 // Higher spaces might still have tail-loss probes to send, which are not
-1295 // congestion blocked.
-1296 return PollPathSpaceStatus::NextSpace;
-1297 }
-1298 if !path_should_send || send_blocked != PathBlocked::No {
-1299 // Nothing more to send on this path, check the next path if possible.
-1300
-1301 // If there are any datagrams in the transmit, packets for another path can
-1302 // not be built.
-1303 if transmit.num_datagrams() > 0 {
-1304 return PollPathSpaceStatus::Send { last_packet_number };
-1305 }
-1306
-1307 return PollPathSpaceStatus::NothingToSend { congestion_blocked };
-1308 }
-1309
-1310 if transmit.datagram_remaining_mut() == 0 {
-1311 if transmit.num_datagrams() >= transmit.max_datagrams().get() {
-1312 // No more datagrams allowed
-1313 return PollPathSpaceStatus::Send { last_packet_number };
-1314 }
-1315
-1316 match self.spaces[space_id].for_path(path_id).loss_probes {
-1317 0 => transmit.start_new_datagram(),
-1318 _ => {
-1319 // We need something to send for a tail-loss probe.
-1320 let request_immediate_ack =
-1321 space_id == SpaceId::Data && self.peer_supports_ack_frequency();
-1322 self.spaces[space_id].maybe_queue_probe(
-1323 path_id,
-1324 request_immediate_ack,
-1325 &self.streams,
-1326 );
-1327
-1328 self.spaces[space_id].for_path(path_id).loss_probes -= 1;
-1329
-1330 // Clamp the datagram to at most the minimum MTU to ensure that loss
-1331 // probes can get through and enable recovery even if the path MTU
-1332 // has shrank unexpectedly.
-1333 transmit.start_new_datagram_with_size(std::cmp::min(
-1334 usize::from(INITIAL_MTU),
-1335 transmit.segment_size(),
-1336 ));
-1337 }
-1338 }
-1339 trace!(count = transmit.num_datagrams(), "new datagram started");
-1340 *coalesce = true;
-1341 *pad_datagram = PadDatagram::No;
-1342 }
-1343
-1344 // If coalescing another packet into the existing datagram, there should
-1345 // still be enough space for a whole packet.
-1346 if transmit.datagram_start_offset() < transmit.len() {
-1347 debug_assert!(transmit.datagram_remaining_mut() >= MIN_PACKET_SPACE);
-1348 }
-1349
-1350 //
-1351 // From here on, we've determined that a packet will definitely be sent.
-1352 //
+1224 }
+1225
+1226 if let Some(last_packet_number) = last_packet_number {
+1227 // Note that when sending in multiple packet spaces the last packet number will
+1228 // be the one from the highest packet space.
+1229 self.path_data_mut(path_id).congestion.on_sent(
+1230 now,
+1231 transmit.len() as u64,
+1232 last_packet_number,
+1233 );
+1234 }
+1235
+1236 self.qlog.emit_recovery_metrics(
+1237 path_id,
+1238 &mut self.paths.get_mut(&path_id).unwrap().data,
+1239 now,
+1240 );
+1241
+1242 if transmit.is_empty() {
+1243 PollPathStatus::NothingToSend { congestion_blocked }
+1244 } else {
+1245 PollPathStatus::Send
+1246 }
+1247 }
+1248
+1249 /// Determine if anything can be sent in this packet number space (SpaceId + PathId).
+1250 /// poll_transmit logic for a path_id - space_id combination
+1251 #[must_use]
+1252 fn poll_transmit_path_space(
+1253 &mut self,
+1254 now: Instant,
+1255 transmit: &mut TransmitBuf<'_>,
+1256 path_id: PathId,
+1257 space_id: SpaceId,
+1258 remote_cid: ConnectionId,
+1259 have_available_path: bool,
+1260 close: bool,
+1261 sendable_frames: Option<SendableFrames>,
+1262 coalesce: &mut bool,
+1263 pad_datagram: &mut PadDatagram,
+1264 ) -> PollPathSpaceStatus {
+1265 let mut last_packet_number = None;
+1266
+1267 // Build datagrams until the transmit is full.
+1268 while transmit.num_datagrams() < transmit.max_datagrams().get() {
+1269 let max_packet_size = if transmit.datagram_remaining_mut() > 0 {
+1270 // We are trying to coalesce another packet into this datagram.
+1271 debug_assert!(*coalesce, "coalescing must be set");
+1272 transmit.datagram_remaining_mut()
+1273 } else {
+1274 // A new datagram needs to be started.
+1275 transmit.segment_size()
+1276 };
+1277
+1278 let sendable_frames = sendable_frames
+1279 .unwrap_or_else(|| self.space_can_send(space_id, path_id, max_packet_size, close));
+1280 let path_should_send =
+1281 self.path_should_send(path_id, space_id, have_available_path, &sendable_frames);
+1282
+1283 let send_blocked = if path_should_send && transmit.datagram_remaining_mut() == 0 {
+1284 // Only check congestion control if a new datagram is needed.
+1285 self.path_congestion_check(space_id, path_id, transmit, &sendable_frames, now)
+1286 } else {
+1287 PathBlocked::No
+1288 };
+1289
+1290 // Whether congestion control stopped the next packet from being sent. Further
+1291 // packets could still be built, as e.g. tail-loss probes are not congestion limited.
+1292 let congestion_blocked = if send_blocked != PathBlocked::No {
+1293 trace!(?space_id, %path_id, ?send_blocked, "congestion blocked");
+1294 true
+1295 } else {
+1296 false
+1297 };
+1298 if send_blocked != PathBlocked::No && space_id < SpaceId::Data {
+1299 // Higher spaces might still have tail-loss probes to send, which are not
+1300 // congestion blocked.
+1301 return PollPathSpaceStatus::NothingToSend { congestion_blocked };
+1302 }
+1303 if !path_should_send || send_blocked != PathBlocked::No {
+1304 // Nothing more to send on this path, check the next path if possible.
+1305
+1306 // If there are any datagrams in the transmit, packets for another path can
+1307 // not be built.
+1308 if transmit.num_datagrams() > 0 {
+1309 return PollPathSpaceStatus::Send { last_packet_number };
+1310 }
+1311
+1312 return PollPathSpaceStatus::NothingToSend { congestion_blocked };
+1313 }
+1314
+1315 // If coalescing another packet into the existing datagram, there should
+1316 // still be enough space for a whole packet.
+1317 if *coalesce && transmit.datagram_start_offset() < transmit.len() {
+1318 debug_assert!(transmit.datagram_remaining_mut() >= MIN_PACKET_SPACE);
+1319 }
+1320
+1321 if transmit.datagram_remaining_mut() == 0 {
+1322 // If the datagram is full, start a new datagram.
+1323 // Only needed if we are building a new datagram
+1324 match self.spaces[space_id].for_path(path_id).loss_probes {
+1325 0 => transmit.start_new_datagram(),
+1326 _ => {
+1327 // We need something to send for a tail-loss probe.
+1328 let request_immediate_ack =
+1329 space_id == SpaceId::Data && self.peer_supports_ack_frequency();
+1330 self.spaces[space_id].maybe_queue_probe(
+1331 path_id,
+1332 request_immediate_ack,
+1333 &self.streams,
+1334 );
+1335
+1336 self.spaces[space_id].for_path(path_id).loss_probes -= 1;
+1337
+1338 // Clamp the datagram to at most the minimum MTU to ensure that loss
+1339 // probes can get through and enable recovery even if the path MTU
+1340 // has shrank unexpectedly.
+1341 transmit.start_new_datagram_with_size(std::cmp::min(
+1342 usize::from(INITIAL_MTU),
+1343 transmit.segment_size(),
+1344 ));
+1345 }
+1346 }
+1347 trace!(count = transmit.num_datagrams(), "new datagram started");
+1348 *coalesce = true;
+1349 *pad_datagram = PadDatagram::No;
+1350 }
+1351
+1352 let is_close = sendable_frames.close;
1353
-1354 if self.spaces[SpaceId::Initial].crypto.is_some()
-1355 && space_id == SpaceId::Handshake
-1356 && self.side.is_client()
-1357 {
-1358 // A client stops both sending and processing Initial packets when it
-1359 // sends its first Handshake packet.
-1360 self.discard_space(now, SpaceId::Initial);
-1361 }
-1362 if let Some(ref mut prev) = self.prev_crypto {
-1363 prev.update_unacked = false;
-1364 }
-1365
-1366 let Some(mut builder) = PacketBuilder::new(
-1367 now,
-1368 space_id,
-1369 path_id,
-1370 remote_cid,
+1354 // Send an off-path PATH_RESPONSE. Prioritized over on-path data to ensure that
+1355 // path validation can occur while the link is saturated.
+1356 if !is_close && space_id == SpaceId::Data {
+1357 if let Some(transmit) = self.build_off_path_challenge(
+1358 now,
+1359 path_id,
+1360 space_id,
+1361 remote_cid,
+1362 sendable_frames.other,
+1363 transmit,
+1364 ) {
+1365 return PollPathSpaceStatus::SendTransmit { transmit };
+1366 }
+1367 }
+1368
+1369 let n = self.poll_transmit_path_space_datagram(
+1370 now,
1371 transmit,
-1372 can_send.other,
-1373 self,
-1374 ) else {
-1375 return PollPathSpaceStatus::NothingToSend { congestion_blocked };
-1376 };
-1377 last_packet_number = Some(builder.exact_number);
-1378 *coalesce = *coalesce && !builder.short_header;
-1379
-1380 if space_id == SpaceId::Initial && (self.side.is_client() || can_send.other) {
-1381 // https://www.rfc-editor.org/rfc/rfc9000.html#section-14.1
-1382 *pad_datagram |= PadDatagram::ToMinMtu;
-1383 }
-1384 if space_id == SpaceId::Data && self.config.pad_to_mtu {
-1385 *pad_datagram |= PadDatagram::ToSegmentSize;
+1372 path_id,
+1373 space_id,
+1374 remote_cid,
+1375 have_available_path,
+1376 close,
+1377 sendable_frames,
+1378 coalesce,
+1379 pad_datagram,
+1380 );
+1381 if let Some(n) = n {
+1382 last_packet_number.replace(n);
+1383 } else {
+1384 // error, unable to send
+1385 return PollPathSpaceStatus::NothingToSend { congestion_blocked };
1386 }
1387
-1388 if can_send.close {
-1389 trace!("sending CONNECTION_CLOSE");
-1390 // Encode ACKs before the ConnectionClose message, to give the receiver
-1391 // a better approximate on what data has been processed. This is
-1392 // especially important with ack delay, since the peer might not
-1393 // have gotten any other ACK for the data earlier on.
-1394 let is_multipath_negotiated = self.is_multipath_negotiated();
-1395 for path_id in self.spaces[space_id]
-1396 .number_spaces
-1397 .iter()
-1398 .filter(|(_, pns)| !pns.pending_acks.ranges().is_empty())
-1399 .map(|(&path_id, _)| path_id)
-1400 .collect::<Vec<_>>()
-1401 {
-1402 Self::populate_acks(
-1403 now,
-1404 self.receiving_ecn,
-1405 path_id,
-1406 space_id,
-1407 &mut self.spaces[space_id],
-1408 is_multipath_negotiated,
-1409 &mut builder,
-1410 &mut self.stats.frame_tx,
-1411 );
-1412 }
-1413
-1414 // Since there only 64 ACK frames there will always be enough space
-1415 // to encode the ConnectionClose frame too. However we still have the
-1416 // check here to prevent crashes if something changes.
-1417 debug_assert!(
-1418 builder.frame_space_remaining() > frame::ConnectionClose::SIZE_BOUND,
-1419 "ACKs should leave space for ConnectionClose"
-1420 );
-1421 let stats = &mut self.stats.frame_tx;
-1422 if frame::ConnectionClose::SIZE_BOUND < builder.frame_space_remaining() {
-1423 let max_frame_size = builder.frame_space_remaining();
-1424 let close: Close = match self.state.as_type() {
-1425 StateType::Closed => {
-1426 let reason: Close =
-1427 self.state.as_closed().expect("checked").clone().into();
-1428 if space_id == SpaceId::Data || reason.is_transport_layer() {
-1429 reason
-1430 } else {
-1431 TransportError::APPLICATION_ERROR("").into()
-1432 }
-1433 }
-1434 StateType::Draining => TransportError::NO_ERROR("").into(),
-1435 _ => unreachable!(
-1436 "tried to make a close packet when the connection wasn't closed"
-1437 ),
-1438 };
-1439 builder.write_frame(close.encoder(max_frame_size), stats);
-1440 }
-1441 builder.finish_and_track(now, self, path_id, *pad_datagram);
-1442 if space_id == self.highest_space {
-1443 // Don't send another close packet. Even with multipath we only send
-1444 // CONNECTION_CLOSE on a single path since we expect our paths to work.
-1445 self.close = false;
-1446 // `CONNECTION_CLOSE` is the final packet
-1447 return PollPathSpaceStatus::Send { last_packet_number };
-1448 } else {
-1449 // Send a close frame in every possible space for robustness, per
-1450 // RFC9000 "Immediate Close during the Handshake". Don't bother trying
-1451 // to send anything else.
-1452 return PollPathSpaceStatus::NextSpace;
-1453 }
-1454 }
-1455
-1456 // Send an off-path PATH_RESPONSE. Prioritized over on-path data to ensure that
-1457 // path validation can occur while the link is saturated.
-1458 if space_id == SpaceId::Data && builder.buf.num_datagrams() == 1 {
-1459 let path = self.path_data_mut(path_id);
-1460 if let Some((token, network_path)) =
-1461 path.path_responses.pop_off_path(path.network_path)
-1462 {
-1463 // TODO(flub): We need to use the right CID! We shouldn't use the same
-1464 // CID as the current active one for the path. Though see also
-1465 // https://github.com/quinn-rs/quinn/issues/2184
-1466 //
-1467 let stats = &mut self.stats.frame_tx;
-1468 let frame = frame::PathResponse(token);
-1469 builder.write_frame_with_log_msg(frame, stats, Some("(off-path)"));
-1470 builder.finish_and_track(now, self, path_id, PadDatagram::ToMinMtu);
-1471 self.stats.udp_tx.on_sent(1, transmit.len());
-1472 return PollPathSpaceStatus::SendTransmit {
-1473 transmit: Transmit {
-1474 destination: network_path.remote,
-1475 size: transmit.len(),
-1476 ecn: None,
-1477 segment_size: None,
-1478 src_ip: network_path.local_ip,
-1479 },
-1480 };
-1481 }
-1482 }
-1483
-1484 let path_exclusive_only =
-1485 have_available_path && self.path_data(path_id).local_status() == PathStatus::Backup;
-1486 self.populate_packet(now, space_id, path_id, path_exclusive_only, &mut builder);
-1487
-1488 // ACK-only packets should only be sent when explicitly allowed. If we write them due to
-1489 // any other reason, there is a bug which leads to one component announcing write
-1490 // readiness while not writing any data. This degrades performance. The condition is
-1491 // only checked if the full MTU is available and when potentially large fixed-size
-1492 // frames aren't queued, so that lack of space in the datagram isn't the reason for just
-1493 // writing ACKs.
-1494 debug_assert!(
-1495 !(builder.sent_frames().is_ack_only(&self.streams)
-1496 && !can_send.acks
-1497 && can_send.other
-1498 && builder.buf.segment_size()
-1499 == self.path_data(path_id).current_mtu() as usize
-1500 && self.datagrams.outgoing.is_empty()),
-1501 "SendableFrames was {can_send:?}, but only ACKs have been written"
-1502 );
-1503 if builder.sent_frames().requires_padding {
-1504 *pad_datagram |= PadDatagram::ToMinMtu;
-1505 }
-1506
-1507 for (path_id, _pn) in builder.sent_frames().largest_acked.iter() {
-1508 self.spaces[space_id]
-1509 .for_path(*path_id)
-1510 .pending_acks
-1511 .acks_sent();
-1512 self.timers.stop(
-1513 Timer::PerPath(*path_id, PathTimer::MaxAckDelay),
-1514 self.qlog.with_time(now),
-1515 );
-1516 }
-1517
-1518 // Now we need to finish the packet. Before we do so we need to know if we will
-1519 // be coalescing the next packet into this one, or will be ending the datagram
-1520 // as well. Because if this is the last packet in the datagram more padding
-1521 // might be needed because of the packet type, or to fill the GSO segment size.
-1522
-1523 // Are we allowed to coalesce AND is there enough space for another *packet* in
-1524 // this datagram AND is there another packet to send in this or the next space?
-1525 if *coalesce
-1526 && builder
-1527 .buf
-1528 .datagram_remaining_mut()
-1529 .saturating_sub(builder.predict_packet_end())
-1530 > MIN_PACKET_SPACE
-1531 && self.has_next_send_space(space_id, path_id, builder.buf, close)
-1532 {
-1533 // We can append/coalesce the next packet into the current
-1534 // datagram. Finish the current packet without adding extra padding.
-1535 builder.finish_and_track(now, self, path_id, PadDatagram::No);
-1536 } else {
-1537 // We need a new datagram for the next packet. Finish the current
-1538 // packet with padding.
-1539 if builder.buf.num_datagrams() > 1 && matches!(pad_datagram, PadDatagram::No) {
-1540 // If too many padding bytes would be required to continue the
-1541 // GSO batch after this packet, end the GSO batch here. Ensures
-1542 // that fixed-size frames with heterogeneous sizes
-1543 // (e.g. application datagrams) won't inadvertently waste large
-1544 // amounts of bandwidth. The exact threshold is a bit arbitrary
-1545 // and might benefit from further tuning, though there's no
-1546 // universally optimal value.
-1547 const MAX_PADDING: usize = 32;
-1548 if builder.buf.datagram_remaining_mut()
-1549 > builder.predict_packet_end() + MAX_PADDING
-1550 {
-1551 trace!(
-1552 "GSO truncated by demand for {} padding bytes",
-1553 builder.buf.datagram_remaining_mut() - builder.predict_packet_end()
-1554 );
-1555 builder.finish_and_track(now, self, path_id, PadDatagram::No);
-1556 return PollPathSpaceStatus::Send { last_packet_number };
-1557 }
-1558
-1559 // Pad the current datagram to GSO segment size so it can be
-1560 // included in the GSO batch.
-1561 builder.finish_and_track(now, self, path_id, PadDatagram::ToSegmentSize);
-1562 } else {
-1563 builder.finish_and_track(now, self, path_id, *pad_datagram);
-1564 }
-1565 if transmit.num_datagrams() == 1 {
-1566 transmit.clip_datagram_size();
-1567 }
-1568 }
-1569 }
-1570 }
-1571
-1572 fn on_remote_cids_exhausted(&mut self, now: Instant, path_id: PathId) {
-1573 if self.abandoned_paths.contains(&path_id) {
-1574 trace!(%path_id, "remote CIDs retired for abandoned path");
-1575 return;
-1576 }
-1577
-1578 let error = PathError::RemoteCidsExhausted;
-1579 debug!(?error, %path_id, "no active CID for path");
-1580 self.events
-1581 .push_back(Event::Path(PathEvent::LocallyClosed { id: path_id, error }));
-1582 // Locally we should have refused to open this path, the remote should
-1583 // have given us CIDs for this path before opening it. So we can always
-1584 // abandon this here.
-1585 self.close_path(
-1586 now,
-1587 path_id,
-1588 TransportErrorCode::NO_CID_AVAILABLE_FOR_PATH.into(),
-1589 )
-1590 .ok();
-1591 self.spaces[SpaceId::Data]
-1592 .pending
-1593 .path_cids_blocked
-1594 .insert(path_id);
-1595 }
-1596
-1597 fn poll_transmit_mtu_probe(
-1598 &mut self,
-1599 now: Instant,
-1600 transmit: &mut TransmitBuf<'_>,
-1601 path_id: PathId,
-1602 ) {
-1603 let Some((active_cid, probe_size)) = self.get_mtu_probe_data(now, path_id) else {
-1604 return;
-1605 };
-1606
-1607 // We are definitely sending a DPLPMTUD probe.
-1608 debug_assert_eq!(transmit.num_datagrams(), 0);
-1609 transmit.start_new_datagram_with_size(probe_size as usize);
-1610
-1611 let Some(mut builder) = PacketBuilder::new(
-1612 now,
-1613 SpaceId::Data,
-1614 path_id,
-1615 active_cid,
-1616 transmit,
-1617 true,
-1618 self,
-1619 ) else {
-1620 return;
-1621 };
-1622
-1623 // We implement MTU probes as ping packets padded up to the probe size
-1624 trace!(?probe_size, "writing MTUD probe");
-1625 builder.write_frame(frame::Ping, &mut self.stats.frame_tx);
-1626
-1627 // If supported by the peer, we want no delays to the probe's ACK
-1628 if self.peer_supports_ack_frequency() {
-1629 builder.write_frame(frame::ImmediateAck, &mut self.stats.frame_tx);
-1630 }
-1631
-1632 builder.finish_and_track(now, self, path_id, PadDatagram::ToSize(probe_size));
-1633
-1634 self.path_stats
-1635 .entry(path_id)
-1636 .or_default()
-1637 .sent_plpmtud_probes += 1;
-1638 }
-1639
-1640 fn get_mtu_probe_data(&mut self, now: Instant, path_id: PathId) -> Option<(ConnectionId, u16)> {
-1641 // We MTU probe all paths for which all of the following is true:
-1642 // - We have an active destination CID for the path.
-1643 // - The remote address *and* path are validated.
-1644 // - The path is not abandoned.
-1645 // - The MTU Discovery subsystem wants to probe the path.
-1646 let active_cid = self.rem_cids.get(&path_id).map(CidQueue::active)?;
-1647 let is_eligible = self.path_data(path_id).validated
-1648 && !self.path_data(path_id).is_validating_path()
-1649 && !self.abandoned_paths.contains(&path_id);
-1650
-1651 if !is_eligible {
-1652 return None;
+1388 if is_close {
+1389 if space_id < self.highest_space {
+1390 // Send a close frame in every possible space for robustness, per
+1391 // RFC9000 "Immediate Close during the Handshake". Don't bother trying
+1392 // to send anything else.
+1393 continue;
+1394 } else {
+1395 // Don't send another close packet, we are in the highest space
+1396
+1397 // Even with multipath we only send
+1398 // CONNECTION_CLOSE on a single path since we expect our paths to work.
+1399 self.close = false;
+1400 // `CONNECTION_CLOSE` is the final packet
+1401 break;
+1402 }
+1403 }
+1404
+1405 // Clip the first datagram
+1406 if transmit.num_datagrams() == 1 {
+1407 transmit.clip_datagram_size();
+1408 }
+1409 }
+1410
+1411 if last_packet_number.is_some() {
+1412 // No more datagrams allowed
+1413 PollPathSpaceStatus::Send { last_packet_number }
+1414 } else {
+1415 PollPathSpaceStatus::NothingToSend {
+1416 congestion_blocked: false,
+1417 }
+1418 }
+1419 }
+1420
+1421 /// Tries to build a single datagram
+1422 fn poll_transmit_path_space_datagram(
+1423 &mut self,
+1424 now: Instant,
+1425 transmit: &mut TransmitBuf<'_>,
+1426 path_id: PathId,
+1427 space_id: SpaceId,
+1428 remote_cid: ConnectionId,
+1429 have_available_path: bool,
+1430 close: bool,
+1431 sendable_frames: SendableFrames,
+1432 coalesce: &mut bool,
+1433 pad_datagram: &mut PadDatagram,
+1434 ) -> Option<u64> {
+1435 if self.spaces[SpaceId::Initial].crypto.is_some()
+1436 && space_id == SpaceId::Handshake
+1437 && self.side.is_client()
+1438 {
+1439 // A client stops both sending and processing Initial packets when it
+1440 // sends its first Handshake packet.
+1441 self.discard_space(now, SpaceId::Initial);
+1442 }
+1443 if let Some(ref mut prev) = self.prev_crypto {
+1444 prev.update_unacked = false;
+1445 }
+1446
+1447 let mut builder = PacketBuilder::new(
+1448 now,
+1449 space_id,
+1450 path_id,
+1451 remote_cid,
+1452 transmit,
+1453 sendable_frames.other,
+1454 self,
+1455 )?;
+1456 *coalesce = *coalesce && !builder.short_header;
+1457
+1458 if space_id == SpaceId::Initial && (self.side.is_client() || sendable_frames.other) {
+1459 // https://www.rfc-editor.org/rfc/rfc9000.html#section-14.1
+1460 *pad_datagram |= PadDatagram::ToMinMtu;
+1461 }
+1462 if space_id == SpaceId::Data && self.config.pad_to_mtu {
+1463 *pad_datagram |= PadDatagram::ToSegmentSize;
+1464 }
+1465
+1466 if sendable_frames.close {
+1467 trace!("sending CONNECTION_CLOSE");
+1468 // Encode ACKs before the ConnectionClose message, to give the receiver
+1469 // a better approximate on what data has been processed. This is
+1470 // especially important with ack delay, since the peer might not
+1471 // have gotten any other ACK for the data earlier on.
+1472 let is_multipath_negotiated = self.is_multipath_negotiated();
+1473 for path_id in self.spaces[space_id]
+1474 .number_spaces
+1475 .iter()
+1476 .filter(|(_, pns)| !pns.pending_acks.ranges().is_empty())
+1477 .map(|(&path_id, _)| path_id)
+1478 .collect::<Vec<_>>()
+1479 {
+1480 Self::populate_acks(
+1481 now,
+1482 self.receiving_ecn,
+1483 path_id,
+1484 space_id,
+1485 &mut self.spaces[space_id],
+1486 is_multipath_negotiated,
+1487 &mut builder,
+1488 &mut self.stats.frame_tx,
+1489 );
+1490 }
+1491
+1492 // Since there only 64 ACK frames there will always be enough space
+1493 // to encode the ConnectionClose frame too. However we still have the
+1494 // check here to prevent crashes if something changes.
+1495 debug_assert!(
+1496 builder.frame_space_remaining() > frame::ConnectionClose::SIZE_BOUND,
+1497 "ACKs should leave space for ConnectionClose"
+1498 );
+1499 let stats = &mut self.stats.frame_tx;
+1500 if frame::ConnectionClose::SIZE_BOUND < builder.frame_space_remaining() {
+1501 let max_frame_size = builder.frame_space_remaining();
+1502 let close: Close = match self.state.as_type() {
+1503 StateType::Closed => {
+1504 let reason: Close = self.state.as_closed().expect("checked").clone().into();
+1505 if space_id == SpaceId::Data || reason.is_transport_layer() {
+1506 reason
+1507 } else {
+1508 TransportError::APPLICATION_ERROR("").into()
+1509 }
+1510 }
+1511 StateType::Draining => TransportError::NO_ERROR("").into(),
+1512 _ => unreachable!(
+1513 "tried to make a close packet when the connection wasn't closed"
+1514 ),
+1515 };
+1516 builder.write_frame(close.encoder(max_frame_size), stats);
+1517 }
+1518 return Some(builder.finish_and_track(now, self, path_id, *pad_datagram));
+1519 }
+1520
+1521 let path_exclusive_only =
+1522 have_available_path && self.path_data(path_id).local_status() == PathStatus::Backup;
+1523 self.populate_packet(now, space_id, path_id, path_exclusive_only, &mut builder);
+1524
+1525 // ACK-only packets should only be sent when explicitly allowed. If we write them due to
+1526 // any other reason, there is a bug which leads to one component announcing write
+1527 // readiness while not writing any data. This degrades performance. The condition is
+1528 // only checked if the full MTU is available and when potentially large fixed-size
+1529 // frames aren't queued, so that lack of space in the datagram isn't the reason for just
+1530 // writing ACKs.
+1531 debug_assert!(
+1532 !(builder.sent_frames().is_ack_only(&self.streams)
+1533 && !sendable_frames.acks
+1534 && sendable_frames.other
+1535 && builder.buf.segment_size() == self.path_data(path_id).current_mtu() as usize
+1536 && self.datagrams.outgoing.is_empty()),
+1537 "SendableFrames was {sendable_frames:?}, but only ACKs have been written"
+1538 );
+1539 if builder.sent_frames().requires_padding {
+1540 *pad_datagram |= PadDatagram::ToMinMtu;
+1541 }
+1542
+1543 for (path_id, _pn) in builder.sent_frames().largest_acked.iter() {
+1544 self.spaces[space_id]
+1545 .for_path(*path_id)
+1546 .pending_acks
+1547 .acks_sent();
+1548 self.timers.stop(
+1549 Timer::PerPath(*path_id, PathTimer::MaxAckDelay),
+1550 self.qlog.with_time(now),
+1551 );
+1552 }
+1553
+1554 // Now we need to finish the packet. Before we do so we need to know if we will
+1555 // be coalescing the next packet into this one, or will be ending the datagram
+1556 // as well. Because if this is the last packet in the datagram more padding
+1557 // might be needed because of the packet type, or to fill the GSO segment size.
+1558 let padding =
+1559 self.calculate_padding(path_id, space_id, *coalesce, close, *pad_datagram, &builder);
+1560
+1561 Some(builder.finish_and_track(now, self, path_id, padding))
+1562 }
+1563
+1564 fn build_off_path_challenge(
+1565 &mut self,
+1566 now: Instant,
+1567 path_id: PathId,
+1568 space_id: SpaceId,
+1569 remote_cid: ConnectionId,
+1570 ack_eliciting: bool,
+1571 transmit: &mut TransmitBuf<'_>,
+1572 ) -> Option<Transmit> {
+1573 let mut builder = PacketBuilder::new(
+1574 now,
+1575 space_id,
+1576 path_id,
+1577 remote_cid,
+1578 transmit,
+1579 ack_eliciting,
+1580 self,
+1581 )?;
+1582 let path = self.path_data_mut(path_id);
+1583 if let Some((token, network_path)) = path.path_responses.pop_off_path(path.network_path) {
+1584 // TODO(flub): We need to use the right CID! We shouldn't use the same
+1585 // CID as the current active one for the path. Though see also
+1586 // https://github.com/quinn-rs/quinn/issues/2184
+1587 //
+1588 let stats = &mut self.stats.frame_tx;
+1589 let frame = frame::PathResponse(token);
+1590 builder.write_frame_with_log_msg(frame, stats, Some("(off-path)"));
+1591 builder.finish_and_track(now, self, path_id, PadDatagram::ToMinMtu);
+1592 self.stats.udp_tx.on_sent(1, transmit.len());
+1593 return Some(Transmit {
+1594 destination: network_path.remote,
+1595 size: transmit.len(),
+1596 ecn: None,
+1597 segment_size: None,
+1598 src_ip: network_path.local_ip,
+1599 });
+1600 }
+1601 None
+1602 }
+1603
+1604 fn calculate_padding(
+1605 &mut self,
+1606 path_id: PathId,
+1607 space_id: SpaceId,
+1608 coalesce: bool,
+1609 close: bool,
+1610 pad_datagram: PadDatagram,
+1611 builder: &PacketBuilder<'_, '_>,
+1612 ) -> PadDatagram {
+1613 // Are we allowed to coalesce AND is there enough space for another *packet* in
+1614 // this datagram AND is there another packet to send in this or the next space?
+1615 if coalesce
+1616 && builder
+1617 .buf
+1618 .datagram_remaining_mut()
+1619 .saturating_sub(builder.predict_packet_end())
+1620 > MIN_PACKET_SPACE
+1621 && self.has_next_send_space(space_id, path_id, builder.buf, close)
+1622 {
+1623 // We can append/coalesce the next packet into the current
+1624 // datagram. Finish the current packet without adding extra padding.
+1625 PadDatagram::No
+1626 } else {
+1627 // We need a new datagram for the next packet. Finish the current
+1628 // packet with padding.
+1629 if builder.buf.num_datagrams() > 1 && matches!(pad_datagram, PadDatagram::No) {
+1630 // If too many padding bytes would be required to continue the
+1631 // GSO batch after this packet, end the GSO batch here. Ensures
+1632 // that fixed-size frames with heterogeneous sizes
+1633 // (e.g. application datagrams) won't inadvertently waste large
+1634 // amounts of bandwidth. The exact threshold is a bit arbitrary
+1635 // and might benefit from further tuning, though there's no
+1636 // universally optimal value.
+1637 const MAX_PADDING: usize = 32;
+1638 if builder.buf.datagram_remaining_mut() > builder.predict_packet_end() + MAX_PADDING
+1639 {
+1640 trace!(
+1641 "GSO truncated by demand for {} padding bytes",
+1642 builder.buf.datagram_remaining_mut() - builder.predict_packet_end()
+1643 );
+1644 PadDatagram::No
+1645 } else {
+1646 // Pad the current datagram to GSO segment size so it can be
+1647 // included in the GSO batch.
+1648 PadDatagram::ToSegmentSize
+1649 }
+1650 } else {
+1651 pad_datagram
+1652 }
1653 }
-1654 let next_pn = self.spaces[SpaceId::Data]
-1655 .for_path(path_id)
-1656 .peek_tx_number();
-1657 let probe_size = self
-1658 .path_data_mut(path_id)
-1659 .mtud
-1660 .poll_transmit(now, next_pn)?;
-1661
-1662 Some((active_cid, probe_size))
-1663 }
-1664
-1665 /// Returns if there is anext packet space which has data to send
-1666 ///
-1667 /// This takes into account the space available to frames in the next datagram.
-1668 // TODO(flub): This duplication is not nice.
-1669 fn has_next_send_space(
-1670 &mut self,
-1671 current_space_id: SpaceId,
-1672 path_id: PathId,
-1673 buf: &TransmitBuf<'_>,
-1674 close: bool,
-1675 ) -> bool {
-1676 // Number of bytes available for frames if this is a 1-RTT packet. We're guaranteed
-1677 // to be able to send an individual frame at least this large in the next 1-RTT
-1678 // packet. This could be generalized to support every space, but it's only needed to
-1679 // handle large fixed-size frames, which only exist in 1-RTT (application
-1680 // datagrams). We don't account for coalesced packets potentially occupying space
-1681 // because frames can always spill into the next datagram.
-1682 let mut space_id = current_space_id;
-1683 loop {
-1684 let can_send = self.space_can_send(space_id, path_id, buf.segment_size(), close);
-1685 if !can_send.is_empty() || (close && self.spaces[space_id].crypto.is_some()) {
-1686 return true;
-1687 }
-1688 let Some(next_space) = space_id.next() else {
-1689 break;
-1690 };
-1691 space_id = next_space;
-1692 }
-1693 false
-1694 }
-1695
-1696 /// Checks if creating a new datagram would be blocked by congestion control
-1697 fn path_congestion_check(
-1698 &mut self,
-1699 space_id: SpaceId,
-1700 path_id: PathId,
-1701 transmit: &TransmitBuf<'_>,
-1702 can_send: &SendableFrames,
-1703 now: Instant,
-1704 ) -> PathBlocked {
-1705 // Anti-amplification is only based on `total_sent`, which gets updated after
-1706 // the transmit is sent. Therefore we pass the amount of bytes for datagrams
-1707 // that are already created, as well as 1 byte for starting another datagram. If
-1708 // there is any anti-amplification budget left, we always allow a full MTU to be
-1709 // sent (see https://github.com/quinn-rs/quinn/issues/1082).
-1710 if self.side().is_server()
-1711 && self
-1712 .path_data(path_id)
-1713 .anti_amplification_blocked(transmit.len() as u64 + 1)
-1714 {
-1715 trace!(?space_id, %path_id, "blocked by anti-amplification");
-1716 return PathBlocked::AntiAmplification;
-1717 }
-1718
-1719 // Congestion control check.
-1720 // Tail loss probes must not be blocked by congestion, or a deadlock could arise.
-1721 let bytes_to_send = transmit.segment_size() as u64;
-1722 let need_loss_probe = self.spaces[space_id].for_path(path_id).loss_probes > 0;
-1723
-1724 if can_send.other && !need_loss_probe && !can_send.close {
-1725 let path = self.path_data(path_id);
-1726 if path.in_flight.bytes + bytes_to_send >= path.congestion.window() {
-1727 trace!(?space_id, %path_id, "blocked by congestion control");
-1728 return PathBlocked::Congestion;
-1729 }
-1730 }
-1731
-1732 // Pacing check.
-1733 if let Some(delay) = self.path_data_mut(path_id).pacing_delay(bytes_to_send, now) {
-1734 self.timers.set(
-1735 Timer::PerPath(path_id, PathTimer::Pacing),
-1736 delay,
-1737 self.qlog.with_time(now),
-1738 );
-1739 // Loss probes and CONNECTION_CLOSE should be subject to pacing, even though
-1740 // they are not congestion controlled.
-1741 trace!(?space_id, %path_id, "blocked by pacing");
-1742 return PathBlocked::Pacing;
-1743 }
-1744
-1745 PathBlocked::No
-1746 }
-1747
-1748 /// Send PATH_CHALLENGE for a previous path if necessary
-1749 ///
-1750 /// QUIC-TRANSPORT section 9.3.3
-1751 /// <https://www.rfc-editor.org/rfc/rfc9000.html#name-off-path-packet-forwarding>
-1752 fn send_prev_path_challenge(
-1753 &mut self,
-1754 now: Instant,
-1755 buf: &mut TransmitBuf<'_>,
-1756 path_id: PathId,
-1757 ) -> Option<Transmit> {
-1758 let (prev_cid, prev_path) = self.paths.get_mut(&path_id)?.prev.as_mut()?;
-1759 // TODO (matheus23): We could use !prev_path.is_validating() here instead to
-1760 // (possibly) also re-send challenges when they get lost.
-1761 if !prev_path.send_new_challenge {
-1762 return None;
-1763 };
-1764 prev_path.send_new_challenge = false;
-1765 let network_path = prev_path.network_path;
-1766 let token = self.rng.random();
-1767 let info = paths::SentChallengeInfo {
-1768 sent_instant: now,
-1769 network_path,
-1770 };
-1771 prev_path.challenges_sent.insert(token, info);
-1772 debug_assert_eq!(
-1773 self.highest_space,
-1774 SpaceId::Data,
-1775 "PATH_CHALLENGE queued without 1-RTT keys"
-1776 );
-1777 buf.start_new_datagram_with_size(MIN_INITIAL_SIZE as usize);
-1778
-1779 // Use the previous CID to avoid linking the new path with the previous path. We
-1780 // don't bother accounting for possible retirement of that prev_cid because this is
-1781 // sent once, immediately after migration, when the CID is known to be valid. Even
-1782 // if a post-migration packet caused the CID to be retired, it's fair to pretend
-1783 // this is sent first.
-1784 debug_assert_eq!(buf.datagram_start_offset(), 0);
-1785 let mut builder =
-1786 PacketBuilder::new(now, SpaceId::Data, path_id, *prev_cid, buf, false, self)?;
-1787 let challenge = frame::PathChallenge(token);
-1788 let stats = &mut self.stats.frame_tx;
-1789 builder.write_frame_with_log_msg(challenge, stats, Some("validating previous path"));
-1790
-1791 // An endpoint MUST expand datagrams that contain a PATH_CHALLENGE frame
-1792 // to at least the smallest allowed maximum datagram size of 1200 bytes,
-1793 // unless the anti-amplification limit for the path does not permit
-1794 // sending a datagram of this size
-1795 builder.pad_to(MIN_INITIAL_SIZE);
-1796
-1797 builder.finish(self, now);
-1798 self.stats.udp_tx.on_sent(1, buf.len());
-1799
-1800 Some(Transmit {
-1801 destination: network_path.remote,
-1802 size: buf.len(),
-1803 ecn: None,
-1804 segment_size: None,
-1805 src_ip: network_path.local_ip,
-1806 })
-1807 }
-1808
-1809 /// Indicate what types of frames are ready to send for the given space
-1810 ///
-1811 /// *packet_size* is the number of bytes available to build the next packet. *close*
-1812 /// *indicates whether a CONNECTION_CLOSE frame needs to be sent.
-1813 fn space_can_send(
-1814 &mut self,
-1815 space_id: SpaceId,
-1816 path_id: PathId,
-1817 packet_size: usize,
-1818 close: bool,
-1819 ) -> SendableFrames {
-1820 let pn = self.spaces[SpaceId::Data]
-1821 .for_path(path_id)
-1822 .peek_tx_number();
-1823 let frame_space_1rtt = packet_size.saturating_sub(self.predict_1rtt_overhead(pn, path_id));
-1824 if self.spaces[space_id].crypto.is_none()
-1825 && (space_id != SpaceId::Data
-1826 || self.zero_rtt_crypto.is_none()
-1827 || self.side.is_server())
-1828 {
-1829 // No keys available for this space
-1830 return SendableFrames::empty();
-1831 }
-1832 let mut can_send = self.spaces[space_id].can_send(path_id, &self.streams);
-1833 if space_id == SpaceId::Data {
-1834 can_send |= self.can_send_1rtt(path_id, frame_space_1rtt);
-1835 }
-1836
-1837 can_send.close = close && self.spaces[space_id].crypto.is_some();
-1838
-1839 can_send
-1840 }
-1841
-1842 /// Process `ConnectionEvent`s generated by the associated `Endpoint`
-1843 ///
-1844 /// Will execute protocol logic upon receipt of a connection event, in turn preparing signals
-1845 /// (including application `Event`s, `EndpointEvent`s and outgoing datagrams) that should be
-1846 /// extracted through the relevant methods.
-1847 pub fn handle_event(&mut self, event: ConnectionEvent) {
-1848 use ConnectionEventInner::*;
-1849 match event.0 {
-1850 Datagram(DatagramConnectionEvent {
-1851 now,
-1852 network_path,
-1853 path_id,
-1854 ecn,
-1855 first_decode,
-1856 remaining,
-1857 }) => {
-1858 let span = trace_span!("pkt", %path_id);
-1859 let _guard = span.enter();
-1860
-1861 if self.update_network_path_or_discard(network_path, path_id) {
-1862 // A return value of true indicates we should discard this packet.
-1863 return;
-1864 }
-1865
-1866 let was_anti_amplification_blocked = self
-1867 .path(path_id)
-1868 .map(|path| path.anti_amplification_blocked(1))
-1869 .unwrap_or(true); // if we don't know about this path it's eagerly considered as unvalidated
-1870 // TODO(@divma): revisit this
-1871
-1872 self.stats.udp_rx.datagrams += 1;
-1873 self.stats.udp_rx.bytes += first_decode.len() as u64;
-1874 let data_len = first_decode.len();
-1875
-1876 self.handle_decode(now, network_path, path_id, ecn, first_decode);
-1877 // The current `path` might have changed inside `handle_decode` since the packet
-1878 // could have triggered a migration. The packet might also belong to an unknown
-1879 // path and have been rejected. Make sure the data received is accounted for the
-1880 // most recent path by accessing `path` after `handle_decode`.
-1881 if let Some(path) = self.path_mut(path_id) {
-1882 path.inc_total_recvd(data_len as u64);
-1883 }
-1884
-1885 if let Some(data) = remaining {
-1886 self.stats.udp_rx.bytes += data.len() as u64;
-1887 self.handle_coalesced(now, network_path, path_id, ecn, data);
-1888 }
-1889
-1890 if let Some(path) = self.paths.get_mut(&path_id) {
-1891 self.qlog
-1892 .emit_recovery_metrics(path_id, &mut path.data, now);
-1893 }
-1894
-1895 if was_anti_amplification_blocked {
-1896 // A prior attempt to set the loss detection timer may have failed due to
-1897 // anti-amplification, so ensure it's set now. Prevents a handshake deadlock if
-1898 // the server's first flight is lost.
-1899 self.set_loss_detection_timer(now, path_id);
-1900 }
-1901 }
-1902 NewIdentifiers(ids, now, cid_len, cid_lifetime) => {
-1903 let path_id = ids.first().map(|issued| issued.path_id).unwrap_or_default();
-1904 debug_assert!(ids.iter().all(|issued| issued.path_id == path_id));
-1905 let cid_state = self
-1906 .local_cid_state
-1907 .entry(path_id)
-1908 .or_insert_with(|| CidState::new(cid_len, cid_lifetime, now, 0));
-1909 cid_state.new_cids(&ids, now);
-1910
-1911 ids.into_iter().rev().for_each(|frame| {
-1912 self.spaces[SpaceId::Data].pending.new_cids.push(frame);
-1913 });
-1914 // Always update Timer::PushNewCid
-1915 self.reset_cid_retirement(now);
-1916 }
-1917 }
-1918 }
-1919
-1920 /// Updates the network path for `path_id`.
-1921 ///
-1922 /// Returns true if a packet coming in for this `path_id` over given `network_path` should be discarded.
-1923 /// Returns false if the path was updated and the packet doesn't need to be discarded.
-1924 fn update_network_path_or_discard(&mut self, network_path: FourTuple, path_id: PathId) -> bool {
-1925 let remote_may_migrate = self.side.remote_may_migrate(&self.state);
-1926 let local_ip_may_migrate = self.side.is_client();
-1927 // If this packet could initiate a migration and we're a client or a server that
-1928 // forbids migration, drop the datagram. This could be relaxed to heuristically
-1929 // permit NAT-rebinding-like migration.
-1930 if let Some(known_path) = self.path_mut(path_id) {
-1931 if network_path.remote != known_path.network_path.remote && !remote_may_migrate {
-1932 trace!(
-1933 %path_id,
-1934 %network_path,
-1935 %known_path.network_path,
-1936 "discarding packet from unrecognized peer"
-1937 );
-1938 return true;
-1939 }
-1940
-1941 if known_path.network_path.local_ip.is_some()
-1942 && network_path.local_ip.is_some()
-1943 && known_path.network_path.local_ip != network_path.local_ip
-1944 && !local_ip_may_migrate
-1945 {
-1946 trace!(
-1947 %path_id,
-1948 %network_path,
-1949 %known_path.network_path,
-1950 "discarding packet sent to incorrect interface"
-1951 );
-1952 return true;
-1953 }
-1954 // If the datagram indicates that we've changed our local IP, we update it.
-1955 // This is alluded to in Section 5.2 of the Multipath RFC draft 18:
-1956 // https://www.ietf.org/archive/id/draft-ietf-quic-multipath-18.html#name-using-multiple-paths-on-the
-1957 // > Client receives the packet, recognizes a path migration, updates the source address of path 2 to 192.0.2.1.
-1958 if let Some(local_ip) = network_path.local_ip {
-1959 if known_path
-1960 .network_path
-1961 .local_ip
-1962 .is_some_and(|ip| ip != local_ip)
-1963 {
-1964 debug!(
-1965 %path_id,
-1966 %network_path,
-1967 %known_path.network_path,
-1968 "path's local address seemingly migrated"
-1969 );
-1970 }
-1971 // We update the address without path validation on the client side.
-1972 // https://www.ietf.org/archive/id/draft-ietf-quic-multipath-18.html#section-5.1
-1973 // > Servers observing a 4-tuple change will perform path validation (see Section 9 of [QUIC-TRANSPORT]).
-1974 // This sounds like it's *only* the server endpoints that do this.
-1975 // TODO(matheus23): We should still consider doing a proper migration on the client side in the future.
-1976 // For now, this preserves the behavior of this code pre 4-tuple tracking.
-1977 known_path.network_path.local_ip = Some(local_ip);
-1978 }
-1979 }
-1980 false
-1981 }
-1982
-1983 /// Process timer expirations
-1984 ///
-1985 /// Executes protocol logic, potentially preparing signals (including application `Event`s,
-1986 /// `EndpointEvent`s and outgoing datagrams) that should be extracted through the relevant
-1987 /// methods.
-1988 ///
-1989 /// It is most efficient to call this immediately after the system clock reaches the latest
-1990 /// `Instant` that was output by `poll_timeout`; however spurious extra calls will simply
-1991 /// no-op and therefore are safe.
-1992 pub fn handle_timeout(&mut self, now: Instant) {
-1993 while let Some((timer, _time)) = self.timers.expire_before(now, &self.qlog) {
-1994 // TODO(@divma): remove `at` when the unicorn is born
-1995 trace!(?timer, at=?now, "timeout");
-1996 match timer {
-1997 Timer::Conn(timer) => match timer {
-1998 ConnTimer::Close => {
-1999 self.state.move_to_drained(None);
-2000 self.endpoint_events.push_back(EndpointEventInner::Drained);
-2001 }
-2002 ConnTimer::Idle => {
-2003 self.kill(ConnectionError::TimedOut);
-2004 }
-2005 ConnTimer::KeepAlive => {
-2006 trace!("sending keep-alive");
-2007 self.ping();
-2008 }
-2009 ConnTimer::KeyDiscard => {
-2010 self.zero_rtt_crypto = None;
-2011 self.prev_crypto = None;
-2012 }
-2013 ConnTimer::PushNewCid => {
-2014 while let Some((path_id, when)) = self.next_cid_retirement() {
-2015 if when > now {
-2016 break;
-2017 }
-2018 match self.local_cid_state.get_mut(&path_id) {
-2019 None => error!(%path_id, "No local CID state for path"),
-2020 Some(cid_state) => {
-2021 // Update `retire_prior_to` field in NEW_CONNECTION_ID frame
-2022 let num_new_cid = cid_state.on_cid_timeout().into();
-2023 if !self.state.is_closed() {
-2024 trace!(
-2025 "push a new CID to peer RETIRE_PRIOR_TO field {}",
-2026 cid_state.retire_prior_to()
-2027 );
-2028 self.endpoint_events.push_back(
-2029 EndpointEventInner::NeedIdentifiers(
-2030 path_id,
-2031 now,
-2032 num_new_cid,
-2033 ),
-2034 );
-2035 }
-2036 }
-2037 }
-2038 }
-2039 }
-2040 },
-2041 // TODO: add path_id as span somehow
-2042 Timer::PerPath(path_id, timer) => {
-2043 let span = trace_span!("per-path timer fired", %path_id, ?timer);
-2044 let _guard = span.enter();
-2045 match timer {
-2046 PathTimer::PathIdle => {
-2047 self.close_path(now, path_id, TransportErrorCode::NO_ERROR.into())
-2048 .ok();
-2049 }
-2050
-2051 PathTimer::PathKeepAlive => {
-2052 trace!("sending keep-alive on path");
-2053 self.ping_path(path_id).ok();
-2054 }
-2055 PathTimer::LossDetection => {
-2056 self.on_loss_detection_timeout(now, path_id);
-2057 self.qlog.emit_recovery_metrics(
-2058 path_id,
-2059 &mut self.paths.get_mut(&path_id).unwrap().data,
-2060 now,
-2061 );
-2062 }
-2063 PathTimer::PathValidation => {
-2064 let Some(path) = self.paths.get_mut(&path_id) else {
-2065 continue;
-2066 };
-2067 self.timers.stop(
-2068 Timer::PerPath(path_id, PathTimer::PathChallengeLost),
-2069 self.qlog.with_time(now),
-2070 );
-2071 debug!("path validation failed");
-2072 if let Some((_, prev)) = path.prev.take() {
-2073 path.data = prev;
-2074 }
-2075 path.data.challenges_sent.clear();
-2076 path.data.send_new_challenge = false;
-2077 }
-2078 PathTimer::PathChallengeLost => {
-2079 let Some(path) = self.paths.get_mut(&path_id) else {
-2080 continue;
-2081 };
-2082 trace!("path challenge deemed lost");
-2083 path.data.send_new_challenge = true;
-2084 }
-2085 PathTimer::PathOpen => {
-2086 let Some(path) = self.paths.get_mut(&path_id) else {
-2087 continue;
-2088 };
-2089 path.data.challenges_sent.clear();
-2090 path.data.send_new_challenge = false;
-2091 self.timers.stop(
-2092 Timer::PerPath(path_id, PathTimer::PathChallengeLost),
-2093 self.qlog.with_time(now),
-2094 );
-2095 debug!("new path validation failed");
-2096 if let Err(err) = self.close_path(
-2097 now,
-2098 path_id,
-2099 TransportErrorCode::PATH_UNSTABLE_OR_POOR.into(),
-2100 ) {
-2101 warn!(?err, "failed closing path");
-2102 }
-2103
-2104 self.events.push_back(Event::Path(PathEvent::LocallyClosed {
-2105 id: path_id,
-2106 error: PathError::ValidationFailed,
-2107 }));
-2108 }
-2109 PathTimer::Pacing => trace!("pacing timer expired"),
-2110 PathTimer::MaxAckDelay => {
-2111 trace!("max ack delay reached");
-2112 // This timer is only armed in the Data space
-2113 self.spaces[SpaceId::Data]
-2114 .for_path(path_id)
-2115 .pending_acks
-2116 .on_max_ack_delay_timeout()
-2117 }
-2118 PathTimer::DiscardPath => {
-2119 // The path was abandoned and 3*PTO has expired since. Clean up all
-2120 // remaining state and install stateless reset token.
-2121 self.timers.stop_per_path(path_id, self.qlog.with_time(now));
-2122 if let Some(loc_cid_state) = self.local_cid_state.remove(&path_id) {
-2123 let (min_seq, max_seq) = loc_cid_state.active_seq();
-2124 for seq in min_seq..=max_seq {
-2125 self.endpoint_events.push_back(
-2126 EndpointEventInner::RetireConnectionId(
-2127 now, path_id, seq, false,
-2128 ),
-2129 );
-2130 }
-2131 }
-2132 self.discard_path(path_id, now);
-2133 }
-2134 }
-2135 }
-2136 }
-2137 }
-2138 }
-2139
-2140 /// Close a connection immediately
-2141 ///
-2142 /// This does not ensure delivery of outstanding data. It is the application's responsibility to
-2143 /// call this only when all important communications have been completed, e.g. by calling
-2144 /// [`SendStream::finish`] on outstanding streams and waiting for the corresponding
-2145 /// [`StreamEvent::Finished`] event.
-2146 ///
-2147 /// If [`Streams::send_streams`] returns 0, all outstanding stream data has been
-2148 /// delivered. There may still be data from the peer that has not been received.
-2149 ///
-2150 /// [`StreamEvent::Finished`]: crate::StreamEvent::Finished
-2151 pub fn close(&mut self, now: Instant, error_code: VarInt, reason: Bytes) {
-2152 self.close_inner(
-2153 now,
-2154 Close::Application(frame::ApplicationClose { error_code, reason }),
-2155 )
-2156 }
-2157
-2158 fn close_inner(&mut self, now: Instant, reason: Close) {
-2159 let was_closed = self.state.is_closed();
-2160 if !was_closed {
-2161 self.close_common();
-2162 self.set_close_timer(now);
-2163 self.close = true;
-2164 self.state.move_to_closed_local(reason);
-2165 }
-2166 }
-2167
-2168 /// Control datagrams
-2169 pub fn datagrams(&mut self) -> Datagrams<'_> {
-2170 Datagrams { conn: self }
-2171 }
-2172
-2173 /// Returns connection statistics
-2174 pub fn stats(&mut self) -> ConnectionStats {
-2175 self.stats.clone()
-2176 }
-2177
-2178 /// Returns path statistics
-2179 pub fn path_stats(&mut self, path_id: PathId) -> Option<PathStats> {
-2180 let path = self.paths.get(&path_id)?;
-2181 let stats = self.path_stats.entry(path_id).or_default();
-2182 stats.rtt = path.data.rtt.get();
-2183 stats.cwnd = path.data.congestion.window();
-2184 stats.current_mtu = path.data.mtud.current_mtu();
-2185 Some(*stats)
-2186 }
-2187
-2188 /// Ping the remote endpoint
-2189 ///
-2190 /// Causes an ACK-eliciting packet to be transmitted on the connection.
-2191 pub fn ping(&mut self) {
-2192 // TODO(flub): This is very brute-force: it pings *all* the paths. Instead it would
-2193 // be nice if we could only send a single packet for this.
-2194 for path_data in self.spaces[self.highest_space].number_spaces.values_mut() {
-2195 path_data.ping_pending = true;
-2196 }
-2197 }
-2198
-2199 /// Ping the remote endpoint over a specific path
-2200 ///
-2201 /// Causes an ACK-eliciting packet to be transmitted on the path.
-2202 pub fn ping_path(&mut self, path: PathId) -> Result<(), ClosedPath> {
-2203 let path_data = self.spaces[self.highest_space]
-2204 .number_spaces
-2205 .get_mut(&path)
-2206 .ok_or(ClosedPath { _private: () })?;
-2207 path_data.ping_pending = true;
-2208 Ok(())
-2209 }
-2210
-2211 /// Update traffic keys spontaneously
-2212 ///
-2213 /// This can be useful for testing key updates, as they otherwise only happen infrequently.
-2214 pub fn force_key_update(&mut self) {
-2215 if !self.state.is_established() {
-2216 debug!("ignoring forced key update in illegal state");
-2217 return;
-2218 }
-2219 if self.prev_crypto.is_some() {
-2220 // We already just updated, or are currently updating, the keys. Concurrent key updates
-2221 // are illegal.
-2222 debug!("ignoring redundant forced key update");
-2223 return;
-2224 }
-2225 self.update_keys(None, false);
-2226 }
-2227
-2228 /// Get a session reference
-2229 pub fn crypto_session(&self) -> &dyn crypto::Session {
-2230 &*self.crypto
-2231 }
-2232
-2233 /// Whether the connection is in the process of being established
-2234 ///
-2235 /// If this returns `false`, the connection may be either established or closed, signaled by the
-2236 /// emission of a `Connected` or `ConnectionLost` message respectively.
-2237 pub fn is_handshaking(&self) -> bool {
-2238 self.state.is_handshake()
-2239 }
-2240
-2241 /// Whether the connection is closed
-2242 ///
-2243 /// Closed connections cannot transport any further data. A connection becomes closed when
-2244 /// either peer application intentionally closes it, or when either transport layer detects an
-2245 /// error such as a time-out or certificate validation failure.
-2246 ///
-2247 /// A `ConnectionLost` event is emitted with details when the connection becomes closed.
-2248 pub fn is_closed(&self) -> bool {
-2249 self.state.is_closed()
-2250 }
-2251
-2252 /// Whether there is no longer any need to keep the connection around
-2253 ///
-2254 /// Closed connections become drained after a brief timeout to absorb any remaining in-flight
-2255 /// packets from the peer. All drained connections have been closed.
-2256 pub fn is_drained(&self) -> bool {
-2257 self.state.is_drained()
-2258 }
-2259
-2260 /// For clients, if the peer accepted the 0-RTT data packets
-2261 ///
-2262 /// The value is meaningless until after the handshake completes.
-2263 pub fn accepted_0rtt(&self) -> bool {
-2264 self.accepted_0rtt
-2265 }
-2266
-2267 /// Whether 0-RTT is/was possible during the handshake
-2268 pub fn has_0rtt(&self) -> bool {
-2269 self.zero_rtt_enabled
-2270 }
-2271
-2272 /// Whether there are any pending retransmits
-2273 pub fn has_pending_retransmits(&self) -> bool {
-2274 !self.spaces[SpaceId::Data].pending.is_empty(&self.streams)
-2275 }
-2276
-2277 /// Look up whether we're the client or server of this Connection
-2278 pub fn side(&self) -> Side {
-2279 self.side.side()
-2280 }
-2281
-2282 /// Get the address observed by the remote over the given path
-2283 pub fn path_observed_address(&self, path_id: PathId) -> Result<Option<SocketAddr>, ClosedPath> {
-2284 self.path(path_id)
-2285 .map(|path_data| {
-2286 path_data
-2287 .last_observed_addr_report
-2288 .as_ref()
-2289 .map(|observed| observed.socket_addr())
-2290 })
-2291 .ok_or(ClosedPath { _private: () })
-2292 }
-2293
-2294 /// Current best estimate of this connection's latency (round-trip-time)
-2295 pub fn rtt(&self, path_id: PathId) -> Option<Duration> {
-2296 self.path(path_id).map(|d| d.rtt.get())
-2297 }
-2298
-2299 /// Current state of this connection's congestion controller, for debugging purposes
-2300 pub fn congestion_state(&self, path_id: PathId) -> Option<&dyn Controller> {
-2301 self.path(path_id).map(|d| d.congestion.as_ref())
-2302 }
-2303
-2304 /// Modify the number of remotely initiated streams that may be concurrently open
-2305 ///
-2306 /// No streams may be opened by the peer unless fewer than `count` are already open. Large
-2307 /// `count`s increase both minimum and worst-case memory consumption.
-2308 pub fn set_max_concurrent_streams(&mut self, dir: Dir, count: VarInt) {
-2309 self.streams.set_max_concurrent(dir, count);
-2310 // If the limit was reduced, then a flow control update previously deemed insignificant may
-2311 // now be significant.
-2312 let pending = &mut self.spaces[SpaceId::Data].pending;
-2313 self.streams.queue_max_stream_id(pending);
+1654 }
+1655
+1656 fn path_should_send(
+1657 &mut self,
+1658 path_id: PathId,
+1659 space_id: SpaceId,
+1660 have_available_path: bool,
+1661 sendable_frames: &SendableFrames,
+1662 ) -> bool {
+1663 let path_exclusive_only = space_id == SpaceId::Data
+1664 && have_available_path
+1665 && self.path_data(path_id).local_status() == PathStatus::Backup;
+1666 let path_should_send = if path_exclusive_only {
+1667 sendable_frames.path_exclusive
+1668 } else {
+1669 !sendable_frames.is_empty()
+1670 };
+1671 let needs_loss_probe = self.spaces[space_id].for_path(path_id).loss_probes > 0;
+1672 path_should_send || needs_loss_probe
+1673 }
+1674
+1675 fn on_remote_cids_exhausted(&mut self, now: Instant, path_id: PathId) {
+1676 if self.abandoned_paths.contains(&path_id) {
+1677 trace!(%path_id, "remote CIDs retired for abandoned path");
+1678 return;
+1679 }
+1680
+1681 let error = PathError::RemoteCidsExhausted;
+1682 debug!(?error, %path_id, "no active CID for path");
+1683 self.events
+1684 .push_back(Event::Path(PathEvent::LocallyClosed { id: path_id, error }));
+1685 // Locally we should have refused to open this path, the remote should
+1686 // have given us CIDs for this path before opening it. So we can always
+1687 // abandon this here.
+1688 self.close_path(
+1689 now,
+1690 path_id,
+1691 TransportErrorCode::NO_CID_AVAILABLE_FOR_PATH.into(),
+1692 )
+1693 .ok();
+1694 self.spaces[SpaceId::Data]
+1695 .pending
+1696 .path_cids_blocked
+1697 .insert(path_id);
+1698 }
+1699
+1700 fn poll_transmit_mtu_probe(
+1701 &mut self,
+1702 now: Instant,
+1703 transmit: &mut TransmitBuf<'_>,
+1704 path_id: PathId,
+1705 ) {
+1706 let Some((active_cid, probe_size)) = self.get_mtu_probe_data(now, path_id) else {
+1707 return;
+1708 };
+1709
+1710 // We are definitely sending a DPLPMTUD probe.
+1711 debug_assert_eq!(transmit.num_datagrams(), 0);
+1712 transmit.start_new_datagram_with_size(probe_size as usize);
+1713
+1714 let Some(mut builder) = PacketBuilder::new(
+1715 now,
+1716 SpaceId::Data,
+1717 path_id,
+1718 active_cid,
+1719 transmit,
+1720 true,
+1721 self,
+1722 ) else {
+1723 return;
+1724 };
+1725
+1726 // We implement MTU probes as ping packets padded up to the probe size
+1727 trace!(?probe_size, "writing MTUD probe");
+1728 builder.write_frame(frame::Ping, &mut self.stats.frame_tx);
+1729
+1730 // If supported by the peer, we want no delays to the probe's ACK
+1731 if self.peer_supports_ack_frequency() {
+1732 builder.write_frame(frame::ImmediateAck, &mut self.stats.frame_tx);
+1733 }
+1734
+1735 builder.finish_and_track(now, self, path_id, PadDatagram::ToSize(probe_size));
+1736
+1737 self.path_stats
+1738 .entry(path_id)
+1739 .or_default()
+1740 .sent_plpmtud_probes += 1;
+1741 }
+1742
+1743 fn get_mtu_probe_data(&mut self, now: Instant, path_id: PathId) -> Option<(ConnectionId, u16)> {
+1744 // We MTU probe all paths for which all of the following is true:
+1745 // - We have an active destination CID for the path.
+1746 // - The remote address *and* path are validated.
+1747 // - The path is not abandoned.
+1748 // - The MTU Discovery subsystem wants to probe the path.
+1749 let active_cid = self.rem_cids.get(&path_id).map(CidQueue::active)?;
+1750 let is_eligible = self.path_data(path_id).validated
+1751 && !self.path_data(path_id).is_validating_path()
+1752 && !self.abandoned_paths.contains(&path_id);
+1753
+1754 if !is_eligible {
+1755 return None;
+1756 }
+1757 let next_pn = self.spaces[SpaceId::Data]
+1758 .for_path(path_id)
+1759 .peek_tx_number();
+1760 let probe_size = self
+1761 .path_data_mut(path_id)
+1762 .mtud
+1763 .poll_transmit(now, next_pn)?;
+1764
+1765 Some((active_cid, probe_size))
+1766 }
+1767
+1768 /// Returns if there is anext packet space which has data to send
+1769 ///
+1770 /// This takes into account the space available to frames in the next datagram.
+1771 // TODO(flub): This duplication is not nice.
+1772 fn has_next_send_space(
+1773 &mut self,
+1774 current_space_id: SpaceId,
+1775 path_id: PathId,
+1776 buf: &TransmitBuf<'_>,
+1777 close: bool,
+1778 ) -> bool {
+1779 // Number of bytes available for frames if this is a 1-RTT packet. We're guaranteed
+1780 // to be able to send an individual frame at least this large in the next 1-RTT
+1781 // packet. This could be generalized to support every space, but it's only needed to
+1782 // handle large fixed-size frames, which only exist in 1-RTT (application
+1783 // datagrams). We don't account for coalesced packets potentially occupying space
+1784 // because frames can always spill into the next datagram.
+1785 let mut space_id = current_space_id;
+1786 loop {
+1787 let can_send = self.space_can_send(space_id, path_id, buf.segment_size(), close);
+1788 if !can_send.is_empty() || (close && self.spaces[space_id].crypto.is_some()) {
+1789 return true;
+1790 }
+1791 let Some(next_space) = space_id.next() else {
+1792 break;
+1793 };
+1794 space_id = next_space;
+1795 }
+1796 false
+1797 }
+1798
+1799 /// Checks if creating a new datagram would be blocked by congestion control
+1800 fn path_congestion_check(
+1801 &mut self,
+1802 space_id: SpaceId,
+1803 path_id: PathId,
+1804 transmit: &TransmitBuf<'_>,
+1805 can_send: &SendableFrames,
+1806 now: Instant,
+1807 ) -> PathBlocked {
+1808 // Anti-amplification is only based on `total_sent`, which gets updated after
+1809 // the transmit is sent. Therefore we pass the amount of bytes for datagrams
+1810 // that are already created, as well as 1 byte for starting another datagram. If
+1811 // there is any anti-amplification budget left, we always allow a full MTU to be
+1812 // sent (see https://github.com/quinn-rs/quinn/issues/1082).
+1813 if self.side().is_server()
+1814 && self
+1815 .path_data(path_id)
+1816 .anti_amplification_blocked(transmit.len() as u64 + 1)
+1817 {
+1818 trace!(?space_id, %path_id, "blocked by anti-amplification");
+1819 return PathBlocked::AntiAmplification;
+1820 }
+1821
+1822 // Congestion control check.
+1823 // Tail loss probes must not be blocked by congestion, or a deadlock could arise.
+1824 let bytes_to_send = transmit.segment_size() as u64;
+1825 let need_loss_probe = self.spaces[space_id].for_path(path_id).loss_probes > 0;
+1826
+1827 if can_send.other && !need_loss_probe && !can_send.close {
+1828 let path = self.path_data(path_id);
+1829 if path.in_flight.bytes + bytes_to_send >= path.congestion.window() {
+1830 trace!(?space_id, %path_id, "blocked by congestion control");
+1831 return PathBlocked::Congestion;
+1832 }
+1833 }
+1834
+1835 // Pacing check.
+1836 if let Some(delay) = self.path_data_mut(path_id).pacing_delay(bytes_to_send, now) {
+1837 self.timers.set(
+1838 Timer::PerPath(path_id, PathTimer::Pacing),
+1839 delay,
+1840 self.qlog.with_time(now),
+1841 );
+1842 // Loss probes and CONNECTION_CLOSE should be subject to pacing, even though
+1843 // they are not congestion controlled.
+1844 trace!(?space_id, %path_id, "blocked by pacing");
+1845 return PathBlocked::Pacing;
+1846 }
+1847
+1848 PathBlocked::No
+1849 }
+1850
+1851 /// Send PATH_CHALLENGE for a previous path if necessary
+1852 ///
+1853 /// QUIC-TRANSPORT section 9.3.3
+1854 /// <https://www.rfc-editor.org/rfc/rfc9000.html#name-off-path-packet-forwarding>
+1855 fn send_prev_path_challenge(
+1856 &mut self,
+1857 now: Instant,
+1858 buf: &mut TransmitBuf<'_>,
+1859 path_id: PathId,
+1860 ) -> Option<Transmit> {
+1861 let (prev_cid, prev_path) = self.paths.get_mut(&path_id)?.prev.as_mut()?;
+1862 // TODO (matheus23): We could use !prev_path.is_validating() here instead to
+1863 // (possibly) also re-send challenges when they get lost.
+1864 if !prev_path.send_new_challenge {
+1865 return None;
+1866 };
+1867 prev_path.send_new_challenge = false;
+1868 let network_path = prev_path.network_path;
+1869 let token = self.rng.random();
+1870 let info = paths::SentChallengeInfo {
+1871 sent_instant: now,
+1872 network_path,
+1873 };
+1874 prev_path.challenges_sent.insert(token, info);
+1875 debug_assert_eq!(
+1876 self.highest_space,
+1877 SpaceId::Data,
+1878 "PATH_CHALLENGE queued without 1-RTT keys"
+1879 );
+1880 buf.start_new_datagram_with_size(MIN_INITIAL_SIZE as usize);
+1881
+1882 // Use the previous CID to avoid linking the new path with the previous path. We
+1883 // don't bother accounting for possible retirement of that prev_cid because this is
+1884 // sent once, immediately after migration, when the CID is known to be valid. Even
+1885 // if a post-migration packet caused the CID to be retired, it's fair to pretend
+1886 // this is sent first.
+1887 debug_assert_eq!(buf.datagram_start_offset(), 0);
+1888 let mut builder =
+1889 PacketBuilder::new(now, SpaceId::Data, path_id, *prev_cid, buf, false, self)?;
+1890 let challenge = frame::PathChallenge(token);
+1891 let stats = &mut self.stats.frame_tx;
+1892 builder.write_frame_with_log_msg(challenge, stats, Some("validating previous path"));
+1893
+1894 // An endpoint MUST expand datagrams that contain a PATH_CHALLENGE frame
+1895 // to at least the smallest allowed maximum datagram size of 1200 bytes,
+1896 // unless the anti-amplification limit for the path does not permit
+1897 // sending a datagram of this size
+1898 builder.pad_to(MIN_INITIAL_SIZE);
+1899
+1900 builder.finish(self, now);
+1901 self.stats.udp_tx.on_sent(1, buf.len());
+1902
+1903 Some(Transmit {
+1904 destination: network_path.remote,
+1905 size: buf.len(),
+1906 ecn: None,
+1907 segment_size: None,
+1908 src_ip: network_path.local_ip,
+1909 })
+1910 }
+1911
+1912 /// Indicate what types of frames are ready to send for the given space
+1913 ///
+1914 /// *packet_size* is the number of bytes available to build the next packet.
+1915 /// *close* indicates whether a CONNECTION_CLOSE frame needs to be sent.
+1916 fn space_can_send(
+1917 &mut self,
+1918 space_id: SpaceId,
+1919 path_id: PathId,
+1920 packet_size: usize,
+1921 close: bool,
+1922 ) -> SendableFrames {
+1923 let space = &mut self.spaces[space_id];
+1924 let space_has_crypto = space.crypto.is_none();
+1925
+1926 if !space_has_crypto
+1927 && (space_id != SpaceId::Data
+1928 || self.zero_rtt_crypto.is_none()
+1929 || self.side.is_server())
+1930 {
+1931 // Nothing send in this space
+1932 return SendableFrames::empty();
+1933 }
+1934
+1935 let mut can_send = space.can_send(path_id, &self.streams);
+1936
+1937 if space_id == SpaceId::Data {
+1938 // Check for 1RTT space.
+1939 let pn = space.for_path(path_id).peek_tx_number();
+1940 let frame_space_1rtt =
+1941 packet_size.saturating_sub(self.predict_1rtt_overhead(pn, path_id));
+1942 can_send |= self.can_send_1rtt(path_id, frame_space_1rtt);
+1943 }
+1944
+1945 can_send.close = close && space_has_crypto;
+1946
+1947 can_send
+1948 }
+1949
+1950 fn space_can_send_non_data(
+1951 &mut self,
+1952 space_id: SpaceId,
+1953 path_id: PathId,
+1954 close: bool,
+1955 ) -> SendableFrames {
+1956 debug_assert!(space_id != SpaceId::Data);
+1957 let space = &mut self.spaces[space_id];
+1958 let space_has_crypto = space.crypto.is_none();
+1959
+1960 if !space_has_crypto && (self.zero_rtt_crypto.is_none() || self.side.is_server()) {
+1961 // Nothing send in this space
+1962 return SendableFrames::empty();
+1963 }
+1964
+1965 let mut can_send = space.can_send(path_id, &self.streams);
+1966 can_send.close = close && space_has_crypto;
+1967 can_send
+1968 }
+1969
+1970 /// Process `ConnectionEvent`s generated by the associated `Endpoint`
+1971 ///
+1972 /// Will execute protocol logic upon receipt of a connection event, in turn preparing signals
+1973 /// (including application `Event`s, `EndpointEvent`s and outgoing datagrams) that should be
+1974 /// extracted through the relevant methods.
+1975 pub fn handle_event(&mut self, event: ConnectionEvent) {
+1976 use ConnectionEventInner::*;
+1977 match event.0 {
+1978 Datagram(DatagramConnectionEvent {
+1979 now,
+1980 network_path,
+1981 path_id,
+1982 ecn,
+1983 first_decode,
+1984 remaining,
+1985 }) => {
+1986 let span = trace_span!("pkt", %path_id);
+1987 let _guard = span.enter();
+1988
+1989 if self.update_network_path_or_discard(network_path, path_id) {
+1990 // A return value of true indicates we should discard this packet.
+1991 return;
+1992 }
+1993
+1994 let was_anti_amplification_blocked = self
+1995 .path(path_id)
+1996 .map(|path| path.anti_amplification_blocked(1))
+1997 .unwrap_or(true); // if we don't know about this path it's eagerly considered as unvalidated
+1998 // TODO(@divma): revisit this
+1999
+2000 self.stats.udp_rx.datagrams += 1;
+2001 self.stats.udp_rx.bytes += first_decode.len() as u64;
+2002 let data_len = first_decode.len();
+2003
+2004 self.handle_decode(now, network_path, path_id, ecn, first_decode);
+2005 // The current `path` might have changed inside `handle_decode` since the packet
+2006 // could have triggered a migration. The packet might also belong to an unknown
+2007 // path and have been rejected. Make sure the data received is accounted for the
+2008 // most recent path by accessing `path` after `handle_decode`.
+2009 if let Some(path) = self.path_mut(path_id) {
+2010 path.inc_total_recvd(data_len as u64);
+2011 }
+2012
+2013 if let Some(data) = remaining {
+2014 self.stats.udp_rx.bytes += data.len() as u64;
+2015 self.handle_coalesced(now, network_path, path_id, ecn, data);
+2016 }
+2017
+2018 if let Some(path) = self.paths.get_mut(&path_id) {
+2019 self.qlog
+2020 .emit_recovery_metrics(path_id, &mut path.data, now);
+2021 }
+2022
+2023 if was_anti_amplification_blocked {
+2024 // A prior attempt to set the loss detection timer may have failed due to
+2025 // anti-amplification, so ensure it's set now. Prevents a handshake deadlock if
+2026 // the server's first flight is lost.
+2027 self.set_loss_detection_timer(now, path_id);
+2028 }
+2029 }
+2030 NewIdentifiers(ids, now, cid_len, cid_lifetime) => {
+2031 let path_id = ids.first().map(|issued| issued.path_id).unwrap_or_default();
+2032 debug_assert!(ids.iter().all(|issued| issued.path_id == path_id));
+2033 let cid_state = self
+2034 .local_cid_state
+2035 .entry(path_id)
+2036 .or_insert_with(|| CidState::new(cid_len, cid_lifetime, now, 0));
+2037 cid_state.new_cids(&ids, now);
+2038
+2039 ids.into_iter().rev().for_each(|frame| {
+2040 self.spaces[SpaceId::Data].pending.new_cids.push(frame);
+2041 });
+2042 // Always update Timer::PushNewCid
+2043 self.reset_cid_retirement(now);
+2044 }
+2045 }
+2046 }
+2047
+2048 /// Updates the network path for `path_id`.
+2049 ///
+2050 /// Returns true if a packet coming in for this `path_id` over given `network_path` should be discarded.
+2051 /// Returns false if the path was updated and the packet doesn't need to be discarded.
+2052 fn update_network_path_or_discard(&mut self, network_path: FourTuple, path_id: PathId) -> bool {
+2053 let remote_may_migrate = self.side.remote_may_migrate(&self.state);
+2054 let local_ip_may_migrate = self.side.is_client();
+2055 // If this packet could initiate a migration and we're a client or a server that
+2056 // forbids migration, drop the datagram. This could be relaxed to heuristically
+2057 // permit NAT-rebinding-like migration.
+2058 if let Some(known_path) = self.path_mut(path_id) {
+2059 if network_path.remote != known_path.network_path.remote && !remote_may_migrate {
+2060 trace!(
+2061 %path_id,
+2062 %network_path,
+2063 %known_path.network_path,
+2064 "discarding packet from unrecognized peer"
+2065 );
+2066 return true;
+2067 }
+2068
+2069 if known_path.network_path.local_ip.is_some()
+2070 && network_path.local_ip.is_some()
+2071 && known_path.network_path.local_ip != network_path.local_ip
+2072 && !local_ip_may_migrate
+2073 {
+2074 trace!(
+2075 %path_id,
+2076 %network_path,
+2077 %known_path.network_path,
+2078 "discarding packet sent to incorrect interface"
+2079 );
+2080 return true;
+2081 }
+2082 // If the datagram indicates that we've changed our local IP, we update it.
+2083 // This is alluded to in Section 5.2 of the Multipath RFC draft 18:
+2084 // https://www.ietf.org/archive/id/draft-ietf-quic-multipath-18.html#name-using-multiple-paths-on-the
+2085 // > Client receives the packet, recognizes a path migration, updates the source address of path 2 to 192.0.2.1.
+2086 if let Some(local_ip) = network_path.local_ip {
+2087 if known_path
+2088 .network_path
+2089 .local_ip
+2090 .is_some_and(|ip| ip != local_ip)
+2091 {
+2092 debug!(
+2093 %path_id,
+2094 %network_path,
+2095 %known_path.network_path,
+2096 "path's local address seemingly migrated"
+2097 );
+2098 }
+2099 // We update the address without path validation on the client side.
+2100 // https://www.ietf.org/archive/id/draft-ietf-quic-multipath-18.html#section-5.1
+2101 // > Servers observing a 4-tuple change will perform path validation (see Section 9 of [QUIC-TRANSPORT]).
+2102 // This sounds like it's *only* the server endpoints that do this.
+2103 // TODO(matheus23): We should still consider doing a proper migration on the client side in the future.
+2104 // For now, this preserves the behavior of this code pre 4-tuple tracking.
+2105 known_path.network_path.local_ip = Some(local_ip);
+2106 }
+2107 }
+2108 false
+2109 }
+2110
+2111 /// Process timer expirations
+2112 ///
+2113 /// Executes protocol logic, potentially preparing signals (including application `Event`s,
+2114 /// `EndpointEvent`s and outgoing datagrams) that should be extracted through the relevant
+2115 /// methods.
+2116 ///
+2117 /// It is most efficient to call this immediately after the system clock reaches the latest
+2118 /// `Instant` that was output by `poll_timeout`; however spurious extra calls will simply
+2119 /// no-op and therefore are safe.
+2120 pub fn handle_timeout(&mut self, now: Instant) {
+2121 while let Some((timer, _time)) = self.timers.expire_before(now, &self.qlog) {
+2122 // TODO(@divma): remove `at` when the unicorn is born
+2123 trace!(?timer, at=?now, "timeout");
+2124 match timer {
+2125 Timer::Conn(timer) => match timer {
+2126 ConnTimer::Close => {
+2127 self.state.move_to_drained(None);
+2128 self.endpoint_events.push_back(EndpointEventInner::Drained);
+2129 }
+2130 ConnTimer::Idle => {
+2131 self.kill(ConnectionError::TimedOut);
+2132 }
+2133 ConnTimer::KeepAlive => {
+2134 trace!("sending keep-alive");
+2135 self.ping();
+2136 }
+2137 ConnTimer::KeyDiscard => {
+2138 self.zero_rtt_crypto = None;
+2139 self.prev_crypto = None;
+2140 }
+2141 ConnTimer::PushNewCid => {
+2142 while let Some((path_id, when)) = self.next_cid_retirement() {
+2143 if when > now {
+2144 break;
+2145 }
+2146 match self.local_cid_state.get_mut(&path_id) {
+2147 None => error!(%path_id, "No local CID state for path"),
+2148 Some(cid_state) => {
+2149 // Update `retire_prior_to` field in NEW_CONNECTION_ID frame
+2150 let num_new_cid = cid_state.on_cid_timeout().into();
+2151 if !self.state.is_closed() {
+2152 trace!(
+2153 "push a new CID to peer RETIRE_PRIOR_TO field {}",
+2154 cid_state.retire_prior_to()
+2155 );
+2156 self.endpoint_events.push_back(
+2157 EndpointEventInner::NeedIdentifiers(
+2158 path_id,
+2159 now,
+2160 num_new_cid,
+2161 ),
+2162 );
+2163 }
+2164 }
+2165 }
+2166 }
+2167 }
+2168 },
+2169 // TODO: add path_id as span somehow
+2170 Timer::PerPath(path_id, timer) => {
+2171 let span = trace_span!("per-path timer fired", %path_id, ?timer);
+2172 let _guard = span.enter();
+2173 match timer {
+2174 PathTimer::PathIdle => {
+2175 self.close_path(now, path_id, TransportErrorCode::NO_ERROR.into())
+2176 .ok();
+2177 }
+2178
+2179 PathTimer::PathKeepAlive => {
+2180 trace!("sending keep-alive on path");
+2181 self.ping_path(path_id).ok();
+2182 }
+2183 PathTimer::LossDetection => {
+2184 self.on_loss_detection_timeout(now, path_id);
+2185 self.qlog.emit_recovery_metrics(
+2186 path_id,
+2187 &mut self.paths.get_mut(&path_id).unwrap().data,
+2188 now,
+2189 );
+2190 }
+2191 PathTimer::PathValidation => {
+2192 let Some(path) = self.paths.get_mut(&path_id) else {
+2193 continue;
+2194 };
+2195 self.timers.stop(
+2196 Timer::PerPath(path_id, PathTimer::PathChallengeLost),
+2197 self.qlog.with_time(now),
+2198 );
+2199 debug!("path validation failed");
+2200 if let Some((_, prev)) = path.prev.take() {
+2201 path.data = prev;
+2202 }
+2203 path.data.challenges_sent.clear();
+2204 path.data.send_new_challenge = false;
+2205 }
+2206 PathTimer::PathChallengeLost => {
+2207 let Some(path) = self.paths.get_mut(&path_id) else {
+2208 continue;
+2209 };
+2210 trace!("path challenge deemed lost");
+2211 path.data.send_new_challenge = true;
+2212 }
+2213 PathTimer::PathOpen => {
+2214 let Some(path) = self.paths.get_mut(&path_id) else {
+2215 continue;
+2216 };
+2217 path.data.challenges_sent.clear();
+2218 path.data.send_new_challenge = false;
+2219 self.timers.stop(
+2220 Timer::PerPath(path_id, PathTimer::PathChallengeLost),
+2221 self.qlog.with_time(now),
+2222 );
+2223 debug!("new path validation failed");
+2224 if let Err(err) = self.close_path(
+2225 now,
+2226 path_id,
+2227 TransportErrorCode::PATH_UNSTABLE_OR_POOR.into(),
+2228 ) {
+2229 warn!(?err, "failed closing path");
+2230 }
+2231
+2232 self.events.push_back(Event::Path(PathEvent::LocallyClosed {
+2233 id: path_id,
+2234 error: PathError::ValidationFailed,
+2235 }));
+2236 }
+2237 PathTimer::Pacing => trace!("pacing timer expired"),
+2238 PathTimer::MaxAckDelay => {
+2239 trace!("max ack delay reached");
+2240 // This timer is only armed in the Data space
+2241 self.spaces[SpaceId::Data]
+2242 .for_path(path_id)
+2243 .pending_acks
+2244 .on_max_ack_delay_timeout()
+2245 }
+2246 PathTimer::DiscardPath => {
+2247 // The path was abandoned and 3*PTO has expired since. Clean up all
+2248 // remaining state and install stateless reset token.
+2249 self.timers.stop_per_path(path_id, self.qlog.with_time(now));
+2250 if let Some(loc_cid_state) = self.local_cid_state.remove(&path_id) {
+2251 let (min_seq, max_seq) = loc_cid_state.active_seq();
+2252 for seq in min_seq..=max_seq {
+2253 self.endpoint_events.push_back(
+2254 EndpointEventInner::RetireConnectionId(
+2255 now, path_id, seq, false,
+2256 ),
+2257 );
+2258 }
+2259 }
+2260 self.discard_path(path_id, now);
+2261 }
+2262 }
+2263 }
+2264 }
+2265 }
+2266 }
+2267
+2268 /// Close a connection immediately
+2269 ///
+2270 /// This does not ensure delivery of outstanding data. It is the application's responsibility to
+2271 /// call this only when all important communications have been completed, e.g. by calling
+2272 /// [`SendStream::finish`] on outstanding streams and waiting for the corresponding
+2273 /// [`StreamEvent::Finished`] event.
+2274 ///
+2275 /// If [`Streams::send_streams`] returns 0, all outstanding stream data has been
+2276 /// delivered. There may still be data from the peer that has not been received.
+2277 ///
+2278 /// [`StreamEvent::Finished`]: crate::StreamEvent::Finished
+2279 pub fn close(&mut self, now: Instant, error_code: VarInt, reason: Bytes) {
+2280 self.close_inner(
+2281 now,
+2282 Close::Application(frame::ApplicationClose { error_code, reason }),
+2283 )
+2284 }
+2285
+2286 fn close_inner(&mut self, now: Instant, reason: Close) {
+2287 let was_closed = self.state.is_closed();
+2288 if !was_closed {
+2289 self.close_common();
+2290 self.set_close_timer(now);
+2291 self.close = true;
+2292 self.state.move_to_closed_local(reason);
+2293 }
+2294 }
+2295
+2296 /// Control datagrams
+2297 pub fn datagrams(&mut self) -> Datagrams<'_> {
+2298 Datagrams { conn: self }
+2299 }
+2300
+2301 /// Returns connection statistics
+2302 pub fn stats(&mut self) -> ConnectionStats {
+2303 self.stats.clone()
+2304 }
+2305
+2306 /// Returns path statistics
+2307 pub fn path_stats(&mut self, path_id: PathId) -> Option<PathStats> {
+2308 let path = self.paths.get(&path_id)?;
+2309 let stats = self.path_stats.entry(path_id).or_default();
+2310 stats.rtt = path.data.rtt.get();
+2311 stats.cwnd = path.data.congestion.window();
+2312 stats.current_mtu = path.data.mtud.current_mtu();
+2313 Some(*stats)
2314 }
2315
-2316 /// Modify the number of open paths allowed when multipath is enabled
+2316 /// Ping the remote endpoint
2317 ///
-2318 /// When reducing the number of concurrent paths this will only affect delaying sending
-2319 /// new MAX_PATH_ID frames until fewer than this number of paths are possible. To
-2320 /// actively reduce paths they must be closed using [`Connection::close_path`], which
-2321 /// can also be used to close not-yet-opened paths.
-2322 ///
-2323 /// If multipath is not negotiated (see the [`TransportConfig`]) this can not enable
-2324 /// multipath and will fail.
-2325 pub fn set_max_concurrent_paths(
-2326 &mut self,
-2327 now: Instant,
-2328 count: NonZeroU32,
-2329 ) -> Result<(), MultipathNotNegotiated> {
-2330 if !self.is_multipath_negotiated() {
-2331 return Err(MultipathNotNegotiated { _private: () });
-2332 }
-2333 self.max_concurrent_paths = count;
-2334
-2335 let in_use_count = self
-2336 .local_max_path_id
-2337 .next()
-2338 .saturating_sub(self.abandoned_paths.len() as u32)
-2339 .as_u32();
-2340 let extra_needed = count.get().saturating_sub(in_use_count);
-2341 let new_max_path_id = self.local_max_path_id.saturating_add(extra_needed);
-2342
-2343 self.set_max_path_id(now, new_max_path_id);
-2344
-2345 Ok(())
-2346 }
-2347
-2348 /// If needed, issues a new MAX_PATH_ID frame and new CIDs for any newly allowed paths
-2349 fn set_max_path_id(&mut self, now: Instant, max_path_id: PathId) {
-2350 if max_path_id <= self.local_max_path_id {
+2318 /// Causes an ACK-eliciting packet to be transmitted on the connection.
+2319 pub fn ping(&mut self) {
+2320 // TODO(flub): This is very brute-force: it pings *all* the paths. Instead it would
+2321 // be nice if we could only send a single packet for this.
+2322 for path_data in self.spaces[self.highest_space].number_spaces.values_mut() {
+2323 path_data.ping_pending = true;
+2324 }
+2325 }
+2326
+2327 /// Ping the remote endpoint over a specific path
+2328 ///
+2329 /// Causes an ACK-eliciting packet to be transmitted on the path.
+2330 pub fn ping_path(&mut self, path: PathId) -> Result<(), ClosedPath> {
+2331 let path_data = self.spaces[self.highest_space]
+2332 .number_spaces
+2333 .get_mut(&path)
+2334 .ok_or(ClosedPath { _private: () })?;
+2335 path_data.ping_pending = true;
+2336 Ok(())
+2337 }
+2338
+2339 /// Update traffic keys spontaneously
+2340 ///
+2341 /// This can be useful for testing key updates, as they otherwise only happen infrequently.
+2342 pub fn force_key_update(&mut self) {
+2343 if !self.state.is_established() {
+2344 debug!("ignoring forced key update in illegal state");
+2345 return;
+2346 }
+2347 if self.prev_crypto.is_some() {
+2348 // We already just updated, or are currently updating, the keys. Concurrent key updates
+2349 // are illegal.
+2350 debug!("ignoring redundant forced key update");
2351 return;
2352 }
-2353
-2354 self.local_max_path_id = max_path_id;
-2355 self.spaces[SpaceId::Data].pending.max_path_id = true;
-2356
-2357 self.issue_first_path_cids(now);
-2358 }
-2359
-2360 /// Current number of remotely initiated streams that may be concurrently open
-2361 ///
-2362 /// If the target for this limit is reduced using [`set_max_concurrent_streams`](Self::set_max_concurrent_streams),
-2363 /// it will not change immediately, even if fewer streams are open. Instead, it will
-2364 /// decrement by one for each time a remotely initiated stream of matching directionality is closed.
-2365 pub fn max_concurrent_streams(&self, dir: Dir) -> u64 {
-2366 self.streams.max_concurrent(dir)
+2353 self.update_keys(None, false);
+2354 }
+2355
+2356 /// Get a session reference
+2357 pub fn crypto_session(&self) -> &dyn crypto::Session {
+2358 &*self.crypto
+2359 }
+2360
+2361 /// Whether the connection is in the process of being established
+2362 ///
+2363 /// If this returns `false`, the connection may be either established or closed, signaled by the
+2364 /// emission of a `Connected` or `ConnectionLost` message respectively.
+2365 pub fn is_handshaking(&self) -> bool {
+2366 self.state.is_handshake()
2367 }
2368
-2369 /// See [`TransportConfig::send_window()`]
-2370 pub fn set_send_window(&mut self, send_window: u64) {
-2371 self.streams.set_send_window(send_window);
-2372 }
-2373
-2374 /// See [`TransportConfig::receive_window()`]
-2375 pub fn set_receive_window(&mut self, receive_window: VarInt) {
-2376 if self.streams.set_receive_window(receive_window) {
-2377 self.spaces[SpaceId::Data].pending.max_data = true;
-2378 }
-2379 }
-2380
-2381 /// Whether the Multipath for QUIC extension is enabled.
-2382 ///
-2383 /// Multipath is only enabled after the handshake is completed and if it was enabled by both
-2384 /// peers.
-2385 pub fn is_multipath_negotiated(&self) -> bool {
-2386 !self.is_handshaking()
-2387 && self.config.max_concurrent_multipath_paths.is_some()
-2388 && self.peer_params.initial_max_path_id.is_some()
-2389 }
-2390
-2391 fn on_ack_received(
-2392 &mut self,
-2393 now: Instant,
-2394 space: SpaceId,
-2395 ack: frame::Ack,
-2396 ) -> Result<(), TransportError> {
-2397 // All ACKs are referencing path 0
-2398 let path = PathId::ZERO;
-2399 self.inner_on_ack_received(now, space, path, ack)
-2400 }
-2401
-2402 fn on_path_ack_received(
-2403 &mut self,
-2404 now: Instant,
-2405 space: SpaceId,
-2406 path_ack: frame::PathAck,
-2407 ) -> Result<(), TransportError> {
-2408 let (ack, path) = path_ack.into_ack();
-2409 self.inner_on_ack_received(now, space, path, ack)
-2410 }
-2411
-2412 /// Handles an ACK frame acknowledging packets sent on *path*.
-2413 fn inner_on_ack_received(
-2414 &mut self,
-2415 now: Instant,
-2416 space: SpaceId,
-2417 path: PathId,
-2418 ack: frame::Ack,
-2419 ) -> Result<(), TransportError> {
-2420 if self.abandoned_paths.contains(&path) {
-2421 // See also https://www.ietf.org/archive/id/draft-ietf-quic-multipath-17.html#section-3.4.3-3
-2422 // > PATH_ACK frames received with an abandoned path ID are silently ignored, as specified in Section 4.
-2423 trace!("silently ignoring PATH_ACK on abandoned path");
-2424 return Ok(());
-2425 }
-2426 if ack.largest >= self.spaces[space].for_path(path).next_packet_number {
-2427 return Err(TransportError::PROTOCOL_VIOLATION("unsent packet acked"));
-2428 }
-2429 let new_largest = {
-2430 let space = &mut self.spaces[space].for_path(path);
-2431 if space.largest_acked_packet.is_none_or(|pn| ack.largest > pn) {
-2432 space.largest_acked_packet = Some(ack.largest);
-2433 if let Some(info) = space.sent_packets.get(ack.largest) {
-2434 // This should always succeed, but a misbehaving peer might ACK a packet we
-2435 // haven't sent. At worst, that will result in us spuriously reducing the
-2436 // congestion window.
-2437 space.largest_acked_packet_sent = info.time_sent;
-2438 }
-2439 true
-2440 } else {
-2441 false
-2442 }
-2443 };
-2444
-2445 if self.detect_spurious_loss(&ack, space, path) {
-2446 self.path_data_mut(path)
-2447 .congestion
-2448 .on_spurious_congestion_event();
-2449 }
-2450
-2451 // Avoid DoS from unreasonably huge ack ranges by filtering out just the new acks.
-2452 let mut newly_acked = ArrayRangeSet::new();
-2453 for range in ack.iter() {
-2454 self.spaces[space].for_path(path).check_ack(range.clone())?;
-2455 for (pn, _) in self.spaces[space]
-2456 .for_path(path)
-2457 .sent_packets
-2458 .iter_range(range)
-2459 {
-2460 newly_acked.insert_one(pn);
-2461 }
-2462 }
-2463
-2464 if newly_acked.is_empty() {
-2465 return Ok(());
-2466 }
-2467
-2468 let mut ack_eliciting_acked = false;
-2469 for packet in newly_acked.elts() {
-2470 if let Some(info) = self.spaces[space].for_path(path).take(packet) {
-2471 for (acked_path_id, acked_pn) in info.largest_acked.iter() {
-2472 // Assume ACKs for all packets below the largest acknowledged in
-2473 // `packet` have been received. This can cause the peer to spuriously
-2474 // retransmit if some of our earlier ACKs were lost, but allows for
-2475 // simpler state tracking. See discussion at
-2476 // https://www.rfc-editor.org/rfc/rfc9000.html#name-limiting-ranges-by-tracking
-2477 if let Some(pns) = self.spaces[space].path_space_mut(*acked_path_id) {
-2478 pns.pending_acks.subtract_below(*acked_pn);
-2479 }
-2480 }
-2481 ack_eliciting_acked |= info.ack_eliciting;
-2482
-2483 // Notify MTU discovery that a packet was acked, because it might be an MTU probe
-2484 let path_data = self.path_data_mut(path);
-2485 let mtu_updated = path_data.mtud.on_acked(space, packet, info.size);
-2486 if mtu_updated {
-2487 path_data
-2488 .congestion
-2489 .on_mtu_update(path_data.mtud.current_mtu());
-2490 }
-2491
-2492 // Notify ack frequency that a packet was acked, because it might contain an ACK_FREQUENCY frame
-2493 self.ack_frequency.on_acked(path, packet);
-2494
-2495 self.on_packet_acked(now, path, info);
-2496 }
-2497 }
-2498
-2499 let largest_ackd = self.spaces[space].for_path(path).largest_acked_packet;
-2500 let app_limited = self.app_limited;
-2501 let path_data = self.path_data_mut(path);
-2502 let in_flight = path_data.in_flight.bytes;
-2503
-2504 path_data
-2505 .congestion
-2506 .on_end_acks(now, in_flight, app_limited, largest_ackd);
-2507
-2508 if new_largest && ack_eliciting_acked {
-2509 let ack_delay = if space != SpaceId::Data {
-2510 Duration::from_micros(0)
-2511 } else {
-2512 cmp::min(
-2513 self.ack_frequency.peer_max_ack_delay,
-2514 Duration::from_micros(ack.delay << self.peer_params.ack_delay_exponent.0),
-2515 )
-2516 };
-2517 let rtt = now.saturating_duration_since(
-2518 self.spaces[space].for_path(path).largest_acked_packet_sent,
-2519 );
-2520
-2521 let next_pn = self.spaces[space].for_path(path).next_packet_number;
-2522 let path_data = self.path_data_mut(path);
-2523 // TODO(@divma): should be a method of path, should be contained in a single place
-2524 path_data.rtt.update(ack_delay, rtt);
-2525 if path_data.first_packet_after_rtt_sample.is_none() {
-2526 path_data.first_packet_after_rtt_sample = Some((space, next_pn));
-2527 }
-2528 }
+2369 /// Whether the connection is closed
+2370 ///
+2371 /// Closed connections cannot transport any further data. A connection becomes closed when
+2372 /// either peer application intentionally closes it, or when either transport layer detects an
+2373 /// error such as a time-out or certificate validation failure.
+2374 ///
+2375 /// A `ConnectionLost` event is emitted with details when the connection becomes closed.
+2376 pub fn is_closed(&self) -> bool {
+2377 self.state.is_closed()
+2378 }
+2379
+2380 /// Whether there is no longer any need to keep the connection around
+2381 ///
+2382 /// Closed connections become drained after a brief timeout to absorb any remaining in-flight
+2383 /// packets from the peer. All drained connections have been closed.
+2384 pub fn is_drained(&self) -> bool {
+2385 self.state.is_drained()
+2386 }
+2387
+2388 /// For clients, if the peer accepted the 0-RTT data packets
+2389 ///
+2390 /// The value is meaningless until after the handshake completes.
+2391 pub fn accepted_0rtt(&self) -> bool {
+2392 self.accepted_0rtt
+2393 }
+2394
+2395 /// Whether 0-RTT is/was possible during the handshake
+2396 pub fn has_0rtt(&self) -> bool {
+2397 self.zero_rtt_enabled
+2398 }
+2399
+2400 /// Whether there are any pending retransmits
+2401 pub fn has_pending_retransmits(&self) -> bool {
+2402 !self.spaces[SpaceId::Data].pending.is_empty(&self.streams)
+2403 }
+2404
+2405 /// Look up whether we're the client or server of this Connection
+2406 pub fn side(&self) -> Side {
+2407 self.side.side()
+2408 }
+2409
+2410 /// Get the address observed by the remote over the given path
+2411 pub fn path_observed_address(&self, path_id: PathId) -> Result<Option<SocketAddr>, ClosedPath> {
+2412 self.path(path_id)
+2413 .map(|path_data| {
+2414 path_data
+2415 .last_observed_addr_report
+2416 .as_ref()
+2417 .map(|observed| observed.socket_addr())
+2418 })
+2419 .ok_or(ClosedPath { _private: () })
+2420 }
+2421
+2422 /// Current best estimate of this connection's latency (round-trip-time)
+2423 pub fn rtt(&self, path_id: PathId) -> Option<Duration> {
+2424 self.path(path_id).map(|d| d.rtt.get())
+2425 }
+2426
+2427 /// Current state of this connection's congestion controller, for debugging purposes
+2428 pub fn congestion_state(&self, path_id: PathId) -> Option<&dyn Controller> {
+2429 self.path(path_id).map(|d| d.congestion.as_ref())
+2430 }
+2431
+2432 /// Modify the number of remotely initiated streams that may be concurrently open
+2433 ///
+2434 /// No streams may be opened by the peer unless fewer than `count` are already open. Large
+2435 /// `count`s increase both minimum and worst-case memory consumption.
+2436 pub fn set_max_concurrent_streams(&mut self, dir: Dir, count: VarInt) {
+2437 self.streams.set_max_concurrent(dir, count);
+2438 // If the limit was reduced, then a flow control update previously deemed insignificant may
+2439 // now be significant.
+2440 let pending = &mut self.spaces[SpaceId::Data].pending;
+2441 self.streams.queue_max_stream_id(pending);
+2442 }
+2443
+2444 /// Modify the number of open paths allowed when multipath is enabled
+2445 ///
+2446 /// When reducing the number of concurrent paths this will only affect delaying sending
+2447 /// new MAX_PATH_ID frames until fewer than this number of paths are possible. To
+2448 /// actively reduce paths they must be closed using [`Connection::close_path`], which
+2449 /// can also be used to close not-yet-opened paths.
+2450 ///
+2451 /// If multipath is not negotiated (see the [`TransportConfig`]) this can not enable
+2452 /// multipath and will fail.
+2453 pub fn set_max_concurrent_paths(
+2454 &mut self,
+2455 now: Instant,
+2456 count: NonZeroU32,
+2457 ) -> Result<(), MultipathNotNegotiated> {
+2458 if !self.is_multipath_negotiated() {
+2459 return Err(MultipathNotNegotiated { _private: () });
+2460 }
+2461 self.max_concurrent_paths = count;
+2462
+2463 let in_use_count = self
+2464 .local_max_path_id
+2465 .next()
+2466 .saturating_sub(self.abandoned_paths.len() as u32)
+2467 .as_u32();
+2468 let extra_needed = count.get().saturating_sub(in_use_count);
+2469 let new_max_path_id = self.local_max_path_id.saturating_add(extra_needed);
+2470
+2471 self.set_max_path_id(now, new_max_path_id);
+2472
+2473 Ok(())
+2474 }
+2475
+2476 /// If needed, issues a new MAX_PATH_ID frame and new CIDs for any newly allowed paths
+2477 fn set_max_path_id(&mut self, now: Instant, max_path_id: PathId) {
+2478 if max_path_id <= self.local_max_path_id {
+2479 return;
+2480 }
+2481
+2482 self.local_max_path_id = max_path_id;
+2483 self.spaces[SpaceId::Data].pending.max_path_id = true;
+2484
+2485 self.issue_first_path_cids(now);
+2486 }
+2487
+2488 /// Current number of remotely initiated streams that may be concurrently open
+2489 ///
+2490 /// If the target for this limit is reduced using [`set_max_concurrent_streams`](Self::set_max_concurrent_streams),
+2491 /// it will not change immediately, even if fewer streams are open. Instead, it will
+2492 /// decrement by one for each time a remotely initiated stream of matching directionality is closed.
+2493 pub fn max_concurrent_streams(&self, dir: Dir) -> u64 {
+2494 self.streams.max_concurrent(dir)
+2495 }
+2496
+2497 /// See [`TransportConfig::send_window()`]
+2498 pub fn set_send_window(&mut self, send_window: u64) {
+2499 self.streams.set_send_window(send_window);
+2500 }
+2501
+2502 /// See [`TransportConfig::receive_window()`]
+2503 pub fn set_receive_window(&mut self, receive_window: VarInt) {
+2504 if self.streams.set_receive_window(receive_window) {
+2505 self.spaces[SpaceId::Data].pending.max_data = true;
+2506 }
+2507 }
+2508
+2509 /// Whether the Multipath for QUIC extension is enabled.
+2510 ///
+2511 /// Multipath is only enabled after the handshake is completed and if it was enabled by both
+2512 /// peers.
+2513 pub fn is_multipath_negotiated(&self) -> bool {
+2514 !self.is_handshaking()
+2515 && self.config.max_concurrent_multipath_paths.is_some()
+2516 && self.peer_params.initial_max_path_id.is_some()
+2517 }
+2518
+2519 fn on_ack_received(
+2520 &mut self,
+2521 now: Instant,
+2522 space: SpaceId,
+2523 ack: frame::Ack,
+2524 ) -> Result<(), TransportError> {
+2525 // All ACKs are referencing path 0
+2526 let path = PathId::ZERO;
+2527 self.inner_on_ack_received(now, space, path, ack)
+2528 }
2529
-2530 // Must be called before crypto/pto_count are clobbered
-2531 self.detect_lost_packets(now, space, path, true);
-2532
-2533 if self.peer_completed_address_validation(path) {
-2534 self.path_data_mut(path).pto_count = 0;
-2535 }
-2536
-2537 // Explicit congestion notification
-2538 // TODO(@divma): this code is a good example of logic that should be contained in a single
-2539 // place but it's split between the path data and the packet number space data, we should
-2540 // find a way to make this work without two lookups
-2541 if self.path_data(path).sending_ecn {
-2542 if let Some(ecn) = ack.ecn {
-2543 // We only examine ECN counters from ACKs that we are certain we received in transmit
-2544 // order, allowing us to compute an increase in ECN counts to compare against the number
-2545 // of newly acked packets that remains well-defined in the presence of arbitrary packet
-2546 // reordering.
-2547 if new_largest {
-2548 let sent = self.spaces[space].for_path(path).largest_acked_packet_sent;
-2549 self.process_ecn(now, space, path, newly_acked.len() as u64, ecn, sent);
-2550 }
-2551 } else {
-2552 // We always start out sending ECN, so any ack that doesn't acknowledge it disables it.
-2553 debug!("ECN not acknowledged by peer");
-2554 self.path_data_mut(path).sending_ecn = false;
-2555 }
+2530 fn on_path_ack_received(
+2531 &mut self,
+2532 now: Instant,
+2533 space: SpaceId,
+2534 path_ack: frame::PathAck,
+2535 ) -> Result<(), TransportError> {
+2536 let (ack, path) = path_ack.into_ack();
+2537 self.inner_on_ack_received(now, space, path, ack)
+2538 }
+2539
+2540 /// Handles an ACK frame acknowledging packets sent on *path*.
+2541 fn inner_on_ack_received(
+2542 &mut self,
+2543 now: Instant,
+2544 space: SpaceId,
+2545 path: PathId,
+2546 ack: frame::Ack,
+2547 ) -> Result<(), TransportError> {
+2548 if self.abandoned_paths.contains(&path) {
+2549 // See also https://www.ietf.org/archive/id/draft-ietf-quic-multipath-17.html#section-3.4.3-3
+2550 // > PATH_ACK frames received with an abandoned path ID are silently ignored, as specified in Section 4.
+2551 trace!("silently ignoring PATH_ACK on abandoned path");
+2552 return Ok(());
+2553 }
+2554 if ack.largest >= self.spaces[space].for_path(path).next_packet_number {
+2555 return Err(TransportError::PROTOCOL_VIOLATION("unsent packet acked"));
2556 }
-2557
-2558 self.set_loss_detection_timer(now, path);
-2559 Ok(())
-2560 }
-2561
-2562 fn detect_spurious_loss(&mut self, ack: &frame::Ack, space: SpaceId, path: PathId) -> bool {
-2563 let lost_packets = &mut self.spaces[space].for_path(path).lost_packets;
-2564
-2565 if lost_packets.is_empty() {
-2566 return false;
-2567 }
-2568
-2569 for range in ack.iter() {
-2570 let spurious_losses: Vec<u64> = lost_packets
-2571 .iter_range(range.clone())
-2572 .map(|(pn, _info)| pn)
-2573 .collect();
-2574
-2575 for pn in spurious_losses {
-2576 lost_packets.remove(pn);
-2577 }
-2578 }
-2579
-2580 // If this ACK frame acknowledged all deemed lost packets,
-2581 // then we have raised a spurious congestion event in the past.
-2582 // We cannot conclude when there are remaining packets,
-2583 // but future ACK frames might indicate a spurious loss detection.
-2584 lost_packets.is_empty()
-2585 }
-2586
-2587 /// Drain lost packets that we reasonably think will never arrive
-2588 ///
-2589 /// The current criterion is copied from `msquic`:
-2590 /// discard packets that were sent earlier than 2 probe timeouts ago.
-2591 fn drain_lost_packets(&mut self, now: Instant, space: SpaceId, path: PathId) {
-2592 let two_pto = 2 * self.path_data(path).rtt.pto_base();
-2593
-2594 let lost_packets = &mut self.spaces[space].for_path(path).lost_packets;
-2595 lost_packets.retain(|_pn, info| now.saturating_duration_since(info.time_sent) <= two_pto);
-2596 }
-2597
-2598 /// Process a new ECN block from an in-order ACK
-2599 fn process_ecn(
-2600 &mut self,
-2601 now: Instant,
-2602 space: SpaceId,
-2603 path: PathId,
-2604 newly_acked: u64,
-2605 ecn: frame::EcnCounts,
-2606 largest_sent_time: Instant,
-2607 ) {
-2608 match self.spaces[space]
-2609 .for_path(path)
-2610 .detect_ecn(newly_acked, ecn)
-2611 {
-2612 Err(e) => {
-2613 debug!("halting ECN due to verification failure: {}", e);
-2614
-2615 self.path_data_mut(path).sending_ecn = false;
-2616 // Wipe out the existing value because it might be garbage and could interfere with
-2617 // future attempts to use ECN on new paths.
-2618 self.spaces[space].for_path(path).ecn_feedback = frame::EcnCounts::ZERO;
-2619 }
-2620 Ok(false) => {}
-2621 Ok(true) => {
-2622 self.path_stats.entry(path).or_default().congestion_events += 1;
-2623 self.path_data_mut(path).congestion.on_congestion_event(
-2624 now,
-2625 largest_sent_time,
-2626 false,
-2627 true,
-2628 0,
-2629 );
-2630 }
-2631 }
-2632 }
-2633
-2634 // Not timing-aware, so it's safe to call this for inferred acks, such as arise from
-2635 // high-latency handshakes
-2636 fn on_packet_acked(&mut self, now: Instant, path_id: PathId, info: SentPacket) {
-2637 self.paths
-2638 .get_mut(&path_id)
-2639 .expect("known path")
-2640 .remove_in_flight(&info);
-2641 let app_limited = self.app_limited;
-2642 let path = self.path_data_mut(path_id);
-2643 if info.ack_eliciting && !path.is_validating_path() {
-2644 // Only pass ACKs to the congestion controller if we are not validating the current
-2645 // path, so as to ignore any ACKs from older paths still coming in.
-2646 let rtt = path.rtt;
-2647 path.congestion
-2648 .on_ack(now, info.time_sent, info.size.into(), app_limited, &rtt);
-2649 }
-2650
-2651 // Update state for confirmed delivery of frames
-2652 if let Some(retransmits) = info.retransmits.get() {
-2653 for (id, _) in retransmits.reset_stream.iter() {
-2654 self.streams.reset_acked(*id);
+2557 let new_largest = {
+2558 let space = &mut self.spaces[space].for_path(path);
+2559 if space.largest_acked_packet.is_none_or(|pn| ack.largest > pn) {
+2560 space.largest_acked_packet = Some(ack.largest);
+2561 if let Some(info) = space.sent_packets.get(ack.largest) {
+2562 // This should always succeed, but a misbehaving peer might ACK a packet we
+2563 // haven't sent. At worst, that will result in us spuriously reducing the
+2564 // congestion window.
+2565 space.largest_acked_packet_sent = info.time_sent;
+2566 }
+2567 true
+2568 } else {
+2569 false
+2570 }
+2571 };
+2572
+2573 if self.detect_spurious_loss(&ack, space, path) {
+2574 self.path_data_mut(path)
+2575 .congestion
+2576 .on_spurious_congestion_event();
+2577 }
+2578
+2579 // Avoid DoS from unreasonably huge ack ranges by filtering out just the new acks.
+2580 let mut newly_acked = ArrayRangeSet::new();
+2581 for range in ack.iter() {
+2582 self.spaces[space].for_path(path).check_ack(range.clone())?;
+2583 for (pn, _) in self.spaces[space]
+2584 .for_path(path)
+2585 .sent_packets
+2586 .iter_range(range)
+2587 {
+2588 newly_acked.insert_one(pn);
+2589 }
+2590 }
+2591
+2592 if newly_acked.is_empty() {
+2593 return Ok(());
+2594 }
+2595
+2596 let mut ack_eliciting_acked = false;
+2597 for packet in newly_acked.elts() {
+2598 if let Some(info) = self.spaces[space].for_path(path).take(packet) {
+2599 for (acked_path_id, acked_pn) in info.largest_acked.iter() {
+2600 // Assume ACKs for all packets below the largest acknowledged in
+2601 // `packet` have been received. This can cause the peer to spuriously
+2602 // retransmit if some of our earlier ACKs were lost, but allows for
+2603 // simpler state tracking. See discussion at
+2604 // https://www.rfc-editor.org/rfc/rfc9000.html#name-limiting-ranges-by-tracking
+2605 if let Some(pns) = self.spaces[space].path_space_mut(*acked_path_id) {
+2606 pns.pending_acks.subtract_below(*acked_pn);
+2607 }
+2608 }
+2609 ack_eliciting_acked |= info.ack_eliciting;
+2610
+2611 // Notify MTU discovery that a packet was acked, because it might be an MTU probe
+2612 let path_data = self.path_data_mut(path);
+2613 let mtu_updated = path_data.mtud.on_acked(space, packet, info.size);
+2614 if mtu_updated {
+2615 path_data
+2616 .congestion
+2617 .on_mtu_update(path_data.mtud.current_mtu());
+2618 }
+2619
+2620 // Notify ack frequency that a packet was acked, because it might contain an ACK_FREQUENCY frame
+2621 self.ack_frequency.on_acked(path, packet);
+2622
+2623 self.on_packet_acked(now, path, info);
+2624 }
+2625 }
+2626
+2627 let largest_ackd = self.spaces[space].for_path(path).largest_acked_packet;
+2628 let app_limited = self.app_limited;
+2629 let path_data = self.path_data_mut(path);
+2630 let in_flight = path_data.in_flight.bytes;
+2631
+2632 path_data
+2633 .congestion
+2634 .on_end_acks(now, in_flight, app_limited, largest_ackd);
+2635
+2636 if new_largest && ack_eliciting_acked {
+2637 let ack_delay = if space != SpaceId::Data {
+2638 Duration::from_micros(0)
+2639 } else {
+2640 cmp::min(
+2641 self.ack_frequency.peer_max_ack_delay,
+2642 Duration::from_micros(ack.delay << self.peer_params.ack_delay_exponent.0),
+2643 )
+2644 };
+2645 let rtt = now.saturating_duration_since(
+2646 self.spaces[space].for_path(path).largest_acked_packet_sent,
+2647 );
+2648
+2649 let next_pn = self.spaces[space].for_path(path).next_packet_number;
+2650 let path_data = self.path_data_mut(path);
+2651 // TODO(@divma): should be a method of path, should be contained in a single place
+2652 path_data.rtt.update(ack_delay, rtt);
+2653 if path_data.first_packet_after_rtt_sample.is_none() {
+2654 path_data.first_packet_after_rtt_sample = Some((space, next_pn));
2655 }
2656 }
2657
-2658 for frame in info.stream_frames {
-2659 self.streams.received_ack_of(frame);
-2660 }
-2661 }
-2662
-2663 fn set_key_discard_timer(&mut self, now: Instant, space: SpaceId) {
-2664 let start = if self.zero_rtt_crypto.is_some() {
-2665 now
-2666 } else {
-2667 self.prev_crypto
-2668 .as_ref()
-2669 .expect("no previous keys")
-2670 .end_packet
-2671 .as_ref()
-2672 .expect("update not acknowledged yet")
-2673 .1
-2674 };
-2675
-2676 // QUIC-MULTIPATH § 2.5 Key Phase Update Process: use largest PTO off all paths.
-2677 self.timers.set(
-2678 Timer::Conn(ConnTimer::KeyDiscard),
-2679 start + self.pto_max_path(space, false) * 3,
-2680 self.qlog.with_time(now),
-2681 );
-2682 }
-2683
-2684 /// Handle a [`PathTimer::LossDetection`] timeout.
-2685 ///
-2686 /// This timer expires for two reasons:
-2687 /// - An ACK-eliciting packet we sent should be considered lost.
-2688 /// - The PTO may have expired and a tail-loss probe needs to be scheduled.
-2689 ///
-2690 /// The former needs us to schedule re-transmission of the lost data.
-2691 ///
-2692 /// The latter means we have not received an ACK for an ack-eliciting packet we sent
-2693 /// within the PTO time-window. We need to schedule a tail-loss probe, an ack-eliciting
-2694 /// packet, to try and elicit new acknowledgements. These new acknowledgements will
-2695 /// indicate whether the previously sent packets were lost or not.
-2696 fn on_loss_detection_timeout(&mut self, now: Instant, path_id: PathId) {
-2697 if let Some((_, pn_space)) = self.loss_time_and_space(path_id) {
-2698 // Time threshold loss Detection
-2699 self.detect_lost_packets(now, pn_space, path_id, false);
-2700 self.set_loss_detection_timer(now, path_id);
-2701 return;
-2702 }
-2703
-2704 let (_, space) = match self.pto_time_and_space(now, path_id) {
-2705 Some(x) => x,
-2706 None => {
-2707 error!(%path_id, "PTO expired while unset");
-2708 return;
-2709 }
-2710 };
-2711 trace!(
-2712 in_flight = self.path_data(path_id).in_flight.bytes,
-2713 count = self.path_data(path_id).pto_count,
-2714 ?space,
-2715 %path_id,
-2716 "PTO fired"
-2717 );
-2718
-2719 let count = match self.path_data(path_id).in_flight.ack_eliciting {
-2720 // A PTO when we're not expecting any ACKs must be due to handshake anti-amplification
-2721 // deadlock preventions
-2722 0 => {
-2723 debug_assert!(!self.peer_completed_address_validation(path_id));
-2724 1
-2725 }
-2726 // Conventional loss probe
-2727 _ => 2,
-2728 };
-2729 let pns = self.spaces[space].for_path(path_id);
-2730 pns.loss_probes = pns.loss_probes.saturating_add(count);
-2731 let path_data = self.path_data_mut(path_id);
-2732 path_data.pto_count = path_data.pto_count.saturating_add(1);
-2733 self.set_loss_detection_timer(now, path_id);
-2734 }
-2735
-2736 /// Detect any lost packets
-2737 ///
-2738 /// There are two cases in which we detects lost packets:
-2739 ///
-2740 /// - We received an ACK packet.
-2741 /// - The [`PathTimer::LossDetection`] timer expired. So there is an un-acknowledged packet
-2742 /// that was followed by an acknowledged packet. The loss timer for this
-2743 /// un-acknowledged packet expired and we need to detect that packet as lost.
-2744 ///
-2745 /// Packets are lost if they are both (See RFC9002 §6.1):
-2746 ///
-2747 /// - Unacknowledged, in flight and sent prior to an acknowledged packet.
-2748 /// - Old enough by either:
-2749 /// - Having a packet number [`TransportConfig::packet_threshold`] lower then the last
-2750 /// acknowledged packet.
-2751 /// - Being sent [`TransportConfig::time_threshold`] * RTT in the past.
-2752 fn detect_lost_packets(
-2753 &mut self,
-2754 now: Instant,
-2755 pn_space: SpaceId,
-2756 path_id: PathId,
-2757 due_to_ack: bool,
-2758 ) {
-2759 let mut lost_packets = Vec::<u64>::new();
-2760 let mut lost_mtu_probe = None;
-2761 let mut in_persistent_congestion = false;
-2762 let mut size_of_lost_packets = 0u64;
-2763 self.spaces[pn_space].for_path(path_id).loss_time = None;
-2764
-2765 // Find all the lost packets, populating all variables initialised above.
-2766
-2767 let path = self.path_data(path_id);
-2768 let in_flight_mtu_probe = path.mtud.in_flight_mtu_probe();
-2769 let loss_delay = path
-2770 .rtt
-2771 .conservative()
-2772 .mul_f32(self.config.time_threshold)
-2773 .max(TIMER_GRANULARITY);
-2774 let first_packet_after_rtt_sample = path.first_packet_after_rtt_sample;
-2775
-2776 let largest_acked_packet = self.spaces[pn_space]
-2777 .for_path(path_id)
-2778 .largest_acked_packet
-2779 .expect("detect_lost_packets only to be called if path received at least one ACK");
-2780 let packet_threshold = self.config.packet_threshold as u64;
-2781
-2782 // InPersistentCongestion: Determine if all packets in the time period before the newest
-2783 // lost packet, including the edges, are marked lost. PTO computation must always
-2784 // include max ACK delay, i.e. operate as if in Data space (see RFC9001 §7.6.1).
-2785 let congestion_period = self
-2786 .pto(SpaceId::Data, path_id)
-2787 .saturating_mul(self.config.persistent_congestion_threshold);
-2788 let mut persistent_congestion_start: Option<Instant> = None;
-2789 let mut prev_packet = None;
-2790 let space = self.spaces[pn_space].for_path(path_id);
-2791
-2792 for (packet, info) in space.sent_packets.iter_range(0..largest_acked_packet) {
-2793 if prev_packet != Some(packet.wrapping_sub(1)) {
-2794 // An intervening packet was acknowledged
-2795 persistent_congestion_start = None;
-2796 }
-2797
-2798 // Packets sent before now - loss_delay are deemed lost.
-2799 // However, we avoid subtraction as it can panic and there's no
-2800 // saturating equivalent of this subtraction operation with a Duration.
-2801 let packet_too_old = now.saturating_duration_since(info.time_sent) >= loss_delay;
-2802 if packet_too_old || largest_acked_packet >= packet + packet_threshold {
-2803 // The packet should be declared lost.
-2804 if Some(packet) == in_flight_mtu_probe {
-2805 // Lost MTU probes are not included in `lost_packets`, because they
-2806 // should not trigger a congestion control response
-2807 lost_mtu_probe = in_flight_mtu_probe;
-2808 } else {
-2809 lost_packets.push(packet);
-2810 size_of_lost_packets += info.size as u64;
-2811 if info.ack_eliciting && due_to_ack {
-2812 match persistent_congestion_start {
-2813 // Two ACK-eliciting packets lost more than
-2814 // congestion_period apart, with no ACKed packets in between
-2815 Some(start) if info.time_sent - start > congestion_period => {
-2816 in_persistent_congestion = true;
-2817 }
-2818 // Persistent congestion must start after the first RTT sample
-2819 None if first_packet_after_rtt_sample
-2820 .is_some_and(|x| x < (pn_space, packet)) =>
-2821 {
-2822 persistent_congestion_start = Some(info.time_sent);
-2823 }
-2824 _ => {}
-2825 }
-2826 }
-2827 }
-2828 } else {
-2829 // The packet should not yet be declared lost.
-2830 if space.loss_time.is_none() {
-2831 // Since we iterate in order the lowest packet number's loss time will
-2832 // always be the earliest.
-2833 space.loss_time = Some(info.time_sent + loss_delay);
-2834 }
-2835 persistent_congestion_start = None;
-2836 }
-2837
-2838 prev_packet = Some(packet);
-2839 }
-2840
-2841 self.handle_lost_packets(
-2842 pn_space,
-2843 path_id,
-2844 now,
-2845 lost_packets,
-2846 lost_mtu_probe,
-2847 loss_delay,
-2848 in_persistent_congestion,
-2849 size_of_lost_packets,
-2850 );
-2851 }
-2852
-2853 /// Drops the path state, declaring any remaining in-flight packets as lost
-2854 fn discard_path(&mut self, path_id: PathId, now: Instant) {
-2855 trace!(%path_id, "dropping path state");
-2856 let path = self.path_data(path_id);
-2857 let in_flight_mtu_probe = path.mtud.in_flight_mtu_probe();
-2858
-2859 let mut size_of_lost_packets = 0u64; // add to path_stats.lost_bytes;
-2860 let lost_pns: Vec<_> = self.spaces[SpaceId::Data]
-2861 .for_path(path_id)
-2862 .sent_packets
-2863 .iter()
-2864 .filter(|(pn, _info)| Some(*pn) != in_flight_mtu_probe)
-2865 .map(|(pn, info)| {
-2866 size_of_lost_packets += info.size as u64;
-2867 pn
-2868 })
-2869 .collect();
-2870
-2871 if !lost_pns.is_empty() {
-2872 trace!(
-2873 %path_id,
-2874 count = lost_pns.len(),
-2875 lost_bytes = size_of_lost_packets,
-2876 "packets lost on path abandon"
-2877 );
-2878 self.handle_lost_packets(
-2879 SpaceId::Data,
-2880 path_id,
-2881 now,
-2882 lost_pns,
-2883 in_flight_mtu_probe,
-2884 Duration::ZERO,
-2885 false,
-2886 size_of_lost_packets,
-2887 );
-2888 }
-2889 self.paths.remove(&path_id);
-2890 self.spaces[SpaceId::Data].number_spaces.remove(&path_id);
-2891
-2892 let path_stats = self.path_stats.remove(&path_id).unwrap_or_default();
-2893 self.events.push_back(
-2894 PathEvent::Abandoned {
-2895 id: path_id,
-2896 path_stats,
-2897 }
-2898 .into(),
-2899 );
-2900 }
-2901
-2902 fn handle_lost_packets(
-2903 &mut self,
-2904 pn_space: SpaceId,
-2905 path_id: PathId,
-2906 now: Instant,
-2907 lost_packets: Vec<u64>,
-2908 lost_mtu_probe: Option<u64>,
-2909 loss_delay: Duration,
-2910 in_persistent_congestion: bool,
-2911 size_of_lost_packets: u64,
-2912 ) {
-2913 debug_assert!(
-2914 {
-2915 let mut sorted = lost_packets.clone();
-2916 sorted.sort();
-2917 sorted == lost_packets
-2918 },
-2919 "lost_packets must be sorted"
-2920 );
-2921
-2922 self.drain_lost_packets(now, pn_space, path_id);
-2923
-2924 // OnPacketsLost
-2925 if let Some(largest_lost) = lost_packets.last().cloned() {
-2926 let old_bytes_in_flight = self.path_data_mut(path_id).in_flight.bytes;
-2927 let largest_lost_sent = self.spaces[pn_space]
-2928 .for_path(path_id)
-2929 .sent_packets
-2930 .get(largest_lost)
-2931 .unwrap()
-2932 .time_sent;
-2933 let path_stats = self.path_stats.entry(path_id).or_default();
-2934 path_stats.lost_packets += lost_packets.len() as u64;
-2935 path_stats.lost_bytes += size_of_lost_packets;
-2936 trace!(
-2937 %path_id,
-2938 count = lost_packets.len(),
-2939 lost_bytes = size_of_lost_packets,
-2940 "packets lost",
-2941 );
-2942
-2943 for &packet in &lost_packets {
-2944 let Some(info) = self.spaces[pn_space].for_path(path_id).take(packet) else {
-2945 continue;
-2946 };
-2947 self.qlog
-2948 .emit_packet_lost(packet, &info, loss_delay, pn_space, now);
-2949 self.paths
-2950 .get_mut(&path_id)
-2951 .unwrap()
-2952 .remove_in_flight(&info);
-2953
-2954 for frame in info.stream_frames {
-2955 self.streams.retransmit(frame);
-2956 }
-2957 self.spaces[pn_space].pending |= info.retransmits;
-2958 self.path_data_mut(path_id)
-2959 .mtud
-2960 .on_non_probe_lost(packet, info.size);
-2961
-2962 self.spaces[pn_space].for_path(path_id).lost_packets.insert(
-2963 packet,
-2964 LostPacket {
-2965 time_sent: info.time_sent,
-2966 },
-2967 );
-2968 }
-2969
-2970 let path = self.path_data_mut(path_id);
-2971 if path.mtud.black_hole_detected(now) {
-2972 path.congestion.on_mtu_update(path.mtud.current_mtu());
-2973 if let Some(max_datagram_size) = self.datagrams().max_size() {
-2974 if self.datagrams.drop_oversized(max_datagram_size)
-2975 && self.datagrams.send_blocked
-2976 {
-2977 self.datagrams.send_blocked = false;
-2978 self.events.push_back(Event::DatagramsUnblocked);
-2979 }
-2980 }
-2981 self.path_stats
-2982 .entry(path_id)
-2983 .or_default()
-2984 .black_holes_detected += 1;
-2985 }
+2658 // Must be called before crypto/pto_count are clobbered
+2659 self.detect_lost_packets(now, space, path, true);
+2660
+2661 if self.peer_completed_address_validation(path) {
+2662 self.path_data_mut(path).pto_count = 0;
+2663 }
+2664
+2665 // Explicit congestion notification
+2666 // TODO(@divma): this code is a good example of logic that should be contained in a single
+2667 // place but it's split between the path data and the packet number space data, we should
+2668 // find a way to make this work without two lookups
+2669 if self.path_data(path).sending_ecn {
+2670 if let Some(ecn) = ack.ecn {
+2671 // We only examine ECN counters from ACKs that we are certain we received in transmit
+2672 // order, allowing us to compute an increase in ECN counts to compare against the number
+2673 // of newly acked packets that remains well-defined in the presence of arbitrary packet
+2674 // reordering.
+2675 if new_largest {
+2676 let sent = self.spaces[space].for_path(path).largest_acked_packet_sent;
+2677 self.process_ecn(now, space, path, newly_acked.len() as u64, ecn, sent);
+2678 }
+2679 } else {
+2680 // We always start out sending ECN, so any ack that doesn't acknowledge it disables it.
+2681 debug!("ECN not acknowledged by peer");
+2682 self.path_data_mut(path).sending_ecn = false;
+2683 }
+2684 }
+2685
+2686 self.set_loss_detection_timer(now, path);
+2687 Ok(())
+2688 }
+2689
+2690 fn detect_spurious_loss(&mut self, ack: &frame::Ack, space: SpaceId, path: PathId) -> bool {
+2691 let lost_packets = &mut self.spaces[space].for_path(path).lost_packets;
+2692
+2693 if lost_packets.is_empty() {
+2694 return false;
+2695 }
+2696
+2697 for range in ack.iter() {
+2698 let spurious_losses: Vec<u64> = lost_packets
+2699 .iter_range(range.clone())
+2700 .map(|(pn, _info)| pn)
+2701 .collect();
+2702
+2703 for pn in spurious_losses {
+2704 lost_packets.remove(pn);
+2705 }
+2706 }
+2707
+2708 // If this ACK frame acknowledged all deemed lost packets,
+2709 // then we have raised a spurious congestion event in the past.
+2710 // We cannot conclude when there are remaining packets,
+2711 // but future ACK frames might indicate a spurious loss detection.
+2712 lost_packets.is_empty()
+2713 }
+2714
+2715 /// Drain lost packets that we reasonably think will never arrive
+2716 ///
+2717 /// The current criterion is copied from `msquic`:
+2718 /// discard packets that were sent earlier than 2 probe timeouts ago.
+2719 fn drain_lost_packets(&mut self, now: Instant, space: SpaceId, path: PathId) {
+2720 let two_pto = 2 * self.path_data(path).rtt.pto_base();
+2721
+2722 let lost_packets = &mut self.spaces[space].for_path(path).lost_packets;
+2723 lost_packets.retain(|_pn, info| now.saturating_duration_since(info.time_sent) <= two_pto);
+2724 }
+2725
+2726 /// Process a new ECN block from an in-order ACK
+2727 fn process_ecn(
+2728 &mut self,
+2729 now: Instant,
+2730 space: SpaceId,
+2731 path: PathId,
+2732 newly_acked: u64,
+2733 ecn: frame::EcnCounts,
+2734 largest_sent_time: Instant,
+2735 ) {
+2736 match self.spaces[space]
+2737 .for_path(path)
+2738 .detect_ecn(newly_acked, ecn)
+2739 {
+2740 Err(e) => {
+2741 debug!("halting ECN due to verification failure: {}", e);
+2742
+2743 self.path_data_mut(path).sending_ecn = false;
+2744 // Wipe out the existing value because it might be garbage and could interfere with
+2745 // future attempts to use ECN on new paths.
+2746 self.spaces[space].for_path(path).ecn_feedback = frame::EcnCounts::ZERO;
+2747 }
+2748 Ok(false) => {}
+2749 Ok(true) => {
+2750 self.path_stats.entry(path).or_default().congestion_events += 1;
+2751 self.path_data_mut(path).congestion.on_congestion_event(
+2752 now,
+2753 largest_sent_time,
+2754 false,
+2755 true,
+2756 0,
+2757 );
+2758 }
+2759 }
+2760 }
+2761
+2762 // Not timing-aware, so it's safe to call this for inferred acks, such as arise from
+2763 // high-latency handshakes
+2764 fn on_packet_acked(&mut self, now: Instant, path_id: PathId, info: SentPacket) {
+2765 self.paths
+2766 .get_mut(&path_id)
+2767 .expect("known path")
+2768 .remove_in_flight(&info);
+2769 let app_limited = self.app_limited;
+2770 let path = self.path_data_mut(path_id);
+2771 if info.ack_eliciting && !path.is_validating_path() {
+2772 // Only pass ACKs to the congestion controller if we are not validating the current
+2773 // path, so as to ignore any ACKs from older paths still coming in.
+2774 let rtt = path.rtt;
+2775 path.congestion
+2776 .on_ack(now, info.time_sent, info.size.into(), app_limited, &rtt);
+2777 }
+2778
+2779 // Update state for confirmed delivery of frames
+2780 if let Some(retransmits) = info.retransmits.get() {
+2781 for (id, _) in retransmits.reset_stream.iter() {
+2782 self.streams.reset_acked(*id);
+2783 }
+2784 }
+2785
+2786 for frame in info.stream_frames {
+2787 self.streams.received_ack_of(frame);
+2788 }
+2789 }
+2790
+2791 fn set_key_discard_timer(&mut self, now: Instant, space: SpaceId) {
+2792 let start = if self.zero_rtt_crypto.is_some() {
+2793 now
+2794 } else {
+2795 self.prev_crypto
+2796 .as_ref()
+2797 .expect("no previous keys")
+2798 .end_packet
+2799 .as_ref()
+2800 .expect("update not acknowledged yet")
+2801 .1
+2802 };
+2803
+2804 // QUIC-MULTIPATH § 2.5 Key Phase Update Process: use largest PTO off all paths.
+2805 self.timers.set(
+2806 Timer::Conn(ConnTimer::KeyDiscard),
+2807 start + self.pto_max_path(space, false) * 3,
+2808 self.qlog.with_time(now),
+2809 );
+2810 }
+2811
+2812 /// Handle a [`PathTimer::LossDetection`] timeout.
+2813 ///
+2814 /// This timer expires for two reasons:
+2815 /// - An ACK-eliciting packet we sent should be considered lost.
+2816 /// - The PTO may have expired and a tail-loss probe needs to be scheduled.
+2817 ///
+2818 /// The former needs us to schedule re-transmission of the lost data.
+2819 ///
+2820 /// The latter means we have not received an ACK for an ack-eliciting packet we sent
+2821 /// within the PTO time-window. We need to schedule a tail-loss probe, an ack-eliciting
+2822 /// packet, to try and elicit new acknowledgements. These new acknowledgements will
+2823 /// indicate whether the previously sent packets were lost or not.
+2824 fn on_loss_detection_timeout(&mut self, now: Instant, path_id: PathId) {
+2825 if let Some((_, pn_space)) = self.loss_time_and_space(path_id) {
+2826 // Time threshold loss Detection
+2827 self.detect_lost_packets(now, pn_space, path_id, false);
+2828 self.set_loss_detection_timer(now, path_id);
+2829 return;
+2830 }
+2831
+2832 let (_, space) = match self.pto_time_and_space(now, path_id) {
+2833 Some(x) => x,
+2834 None => {
+2835 error!(%path_id, "PTO expired while unset");
+2836 return;
+2837 }
+2838 };
+2839 trace!(
+2840 in_flight = self.path_data(path_id).in_flight.bytes,
+2841 count = self.path_data(path_id).pto_count,
+2842 ?space,
+2843 %path_id,
+2844 "PTO fired"
+2845 );
+2846
+2847 let count = match self.path_data(path_id).in_flight.ack_eliciting {
+2848 // A PTO when we're not expecting any ACKs must be due to handshake anti-amplification
+2849 // deadlock preventions
+2850 0 => {
+2851 debug_assert!(!self.peer_completed_address_validation(path_id));
+2852 1
+2853 }
+2854 // Conventional loss probe
+2855 _ => 2,
+2856 };
+2857 let pns = self.spaces[space].for_path(path_id);
+2858 pns.loss_probes = pns.loss_probes.saturating_add(count);
+2859 let path_data = self.path_data_mut(path_id);
+2860 path_data.pto_count = path_data.pto_count.saturating_add(1);
+2861 self.set_loss_detection_timer(now, path_id);
+2862 }
+2863
+2864 /// Detect any lost packets
+2865 ///
+2866 /// There are two cases in which we detects lost packets:
+2867 ///
+2868 /// - We received an ACK packet.
+2869 /// - The [`PathTimer::LossDetection`] timer expired. So there is an un-acknowledged packet
+2870 /// that was followed by an acknowledged packet. The loss timer for this
+2871 /// un-acknowledged packet expired and we need to detect that packet as lost.
+2872 ///
+2873 /// Packets are lost if they are both (See RFC9002 §6.1):
+2874 ///
+2875 /// - Unacknowledged, in flight and sent prior to an acknowledged packet.
+2876 /// - Old enough by either:
+2877 /// - Having a packet number [`TransportConfig::packet_threshold`] lower then the last
+2878 /// acknowledged packet.
+2879 /// - Being sent [`TransportConfig::time_threshold`] * RTT in the past.
+2880 fn detect_lost_packets(
+2881 &mut self,
+2882 now: Instant,
+2883 pn_space: SpaceId,
+2884 path_id: PathId,
+2885 due_to_ack: bool,
+2886 ) {
+2887 let mut lost_packets = Vec::<u64>::new();
+2888 let mut lost_mtu_probe = None;
+2889 let mut in_persistent_congestion = false;
+2890 let mut size_of_lost_packets = 0u64;
+2891 self.spaces[pn_space].for_path(path_id).loss_time = None;
+2892
+2893 // Find all the lost packets, populating all variables initialised above.
+2894
+2895 let path = self.path_data(path_id);
+2896 let in_flight_mtu_probe = path.mtud.in_flight_mtu_probe();
+2897 let loss_delay = path
+2898 .rtt
+2899 .conservative()
+2900 .mul_f32(self.config.time_threshold)
+2901 .max(TIMER_GRANULARITY);
+2902 let first_packet_after_rtt_sample = path.first_packet_after_rtt_sample;
+2903
+2904 let largest_acked_packet = self.spaces[pn_space]
+2905 .for_path(path_id)
+2906 .largest_acked_packet
+2907 .expect("detect_lost_packets only to be called if path received at least one ACK");
+2908 let packet_threshold = self.config.packet_threshold as u64;
+2909
+2910 // InPersistentCongestion: Determine if all packets in the time period before the newest
+2911 // lost packet, including the edges, are marked lost. PTO computation must always
+2912 // include max ACK delay, i.e. operate as if in Data space (see RFC9001 §7.6.1).
+2913 let congestion_period = self
+2914 .pto(SpaceId::Data, path_id)
+2915 .saturating_mul(self.config.persistent_congestion_threshold);
+2916 let mut persistent_congestion_start: Option<Instant> = None;
+2917 let mut prev_packet = None;
+2918 let space = self.spaces[pn_space].for_path(path_id);
+2919
+2920 for (packet, info) in space.sent_packets.iter_range(0..largest_acked_packet) {
+2921 if prev_packet != Some(packet.wrapping_sub(1)) {
+2922 // An intervening packet was acknowledged
+2923 persistent_congestion_start = None;
+2924 }
+2925
+2926 // Packets sent before now - loss_delay are deemed lost.
+2927 // However, we avoid subtraction as it can panic and there's no
+2928 // saturating equivalent of this subtraction operation with a Duration.
+2929 let packet_too_old = now.saturating_duration_since(info.time_sent) >= loss_delay;
+2930 if packet_too_old || largest_acked_packet >= packet + packet_threshold {
+2931 // The packet should be declared lost.
+2932 if Some(packet) == in_flight_mtu_probe {
+2933 // Lost MTU probes are not included in `lost_packets`, because they
+2934 // should not trigger a congestion control response
+2935 lost_mtu_probe = in_flight_mtu_probe;
+2936 } else {
+2937 lost_packets.push(packet);
+2938 size_of_lost_packets += info.size as u64;
+2939 if info.ack_eliciting && due_to_ack {
+2940 match persistent_congestion_start {
+2941 // Two ACK-eliciting packets lost more than
+2942 // congestion_period apart, with no ACKed packets in between
+2943 Some(start) if info.time_sent - start > congestion_period => {
+2944 in_persistent_congestion = true;
+2945 }
+2946 // Persistent congestion must start after the first RTT sample
+2947 None if first_packet_after_rtt_sample
+2948 .is_some_and(|x| x < (pn_space, packet)) =>
+2949 {
+2950 persistent_congestion_start = Some(info.time_sent);
+2951 }
+2952 _ => {}
+2953 }
+2954 }
+2955 }
+2956 } else {
+2957 // The packet should not yet be declared lost.
+2958 if space.loss_time.is_none() {
+2959 // Since we iterate in order the lowest packet number's loss time will
+2960 // always be the earliest.
+2961 space.loss_time = Some(info.time_sent + loss_delay);
+2962 }
+2963 persistent_congestion_start = None;
+2964 }
+2965
+2966 prev_packet = Some(packet);
+2967 }
+2968
+2969 self.handle_lost_packets(
+2970 pn_space,
+2971 path_id,
+2972 now,
+2973 lost_packets,
+2974 lost_mtu_probe,
+2975 loss_delay,
+2976 in_persistent_congestion,
+2977 size_of_lost_packets,
+2978 );
+2979 }
+2980
+2981 /// Drops the path state, declaring any remaining in-flight packets as lost
+2982 fn discard_path(&mut self, path_id: PathId, now: Instant) {
+2983 trace!(%path_id, "dropping path state");
+2984 let path = self.path_data(path_id);
+2985 let in_flight_mtu_probe = path.mtud.in_flight_mtu_probe();
2986
-2987 // Don't apply congestion penalty for lost ack-only packets
-2988 let lost_ack_eliciting =
-2989 old_bytes_in_flight != self.path_data_mut(path_id).in_flight.bytes;
-2990
-2991 if lost_ack_eliciting {
-2992 self.path_stats
-2993 .entry(path_id)
-2994 .or_default()
-2995 .congestion_events += 1;
-2996 self.path_data_mut(path_id).congestion.on_congestion_event(
-2997 now,
-2998 largest_lost_sent,
-2999 in_persistent_congestion,
-3000 false,
-3001 size_of_lost_packets,
-3002 );
-3003 }
-3004 }
-3005
-3006 // Handle a lost MTU probe
-3007 if let Some(packet) = lost_mtu_probe {
-3008 let info = self.spaces[SpaceId::Data]
-3009 .for_path(path_id)
-3010 .take(packet)
-3011 .unwrap(); // safe: lost_mtu_probe is omitted from lost_packets, and
-3012 // therefore must not have been removed yet
-3013 self.paths
-3014 .get_mut(&path_id)
-3015 .unwrap()
-3016 .remove_in_flight(&info);
-3017 self.path_data_mut(path_id).mtud.on_probe_lost();
-3018 self.path_stats
-3019 .entry(path_id)
-3020 .or_default()
-3021 .lost_plpmtud_probes += 1;
-3022 }
-3023 }
-3024
-3025 /// Returns the earliest time packets should be declared lost for all spaces on a path.
-3026 ///
-3027 /// If a path has an acknowledged packet with any prior un-acknowledged packets, the
-3028 /// earliest un-acknowledged packet can be declared lost after a timeout has elapsed.
-3029 /// The time returned is when this packet should be declared lost.
-3030 fn loss_time_and_space(&self, path_id: PathId) -> Option<(Instant, SpaceId)> {
-3031 SpaceId::iter()
-3032 .filter_map(|id| {
-3033 self.spaces[id]
-3034 .number_spaces
-3035 .get(&path_id)
-3036 .and_then(|pns| pns.loss_time)
-3037 .map(|time| (time, id))
-3038 })
-3039 .min_by_key(|&(time, _)| time)
-3040 }
-3041
-3042 /// Returns the earliest next PTO should fire for all spaces on a path.
-3043 fn pto_time_and_space(&mut self, now: Instant, path_id: PathId) -> Option<(Instant, SpaceId)> {
-3044 let path = self.path(path_id)?;
-3045 let pto_count = path.pto_count;
-3046 let backoff = 2u32.pow(pto_count.min(MAX_BACKOFF_EXPONENT));
-3047 let mut duration = path.rtt.pto_base() * backoff;
-3048
-3049 if path_id == PathId::ZERO
-3050 && path.in_flight.ack_eliciting == 0
-3051 && !self.peer_completed_address_validation(PathId::ZERO)
-3052 {
-3053 // Address Validation during Connection Establishment:
-3054 // https://www.rfc-editor.org/rfc/rfc9000.html#section-8.1. To prevent a
-3055 // deadlock if an Initial or Handshake packet from the server is lost and the
-3056 // server can not send more due to its anti-amplification limit the client must
-3057 // send another packet on PTO.
-3058 let space = match self.highest_space {
-3059 SpaceId::Handshake => SpaceId::Handshake,
-3060 _ => SpaceId::Initial,
-3061 };
-3062
-3063 return Some((now + duration, space));
-3064 }
-3065
-3066 let mut result = None;
-3067 for space in SpaceId::iter() {
-3068 let Some(pns) = self.spaces[space].number_spaces.get(&path_id) else {
-3069 continue;
-3070 };
-3071
-3072 if !pns.has_in_flight() {
-3073 continue;
-3074 }
-3075 if space == SpaceId::Data {
-3076 // Skip ApplicationData until handshake completes.
-3077 if self.is_handshaking() {
-3078 return result;
-3079 }
-3080 // Include max_ack_delay and backoff for ApplicationData.
-3081 duration += self.ack_frequency.max_ack_delay_for_pto() * backoff;
-3082 }
-3083 let Some(last_ack_eliciting) = pns.time_of_last_ack_eliciting_packet else {
-3084 continue;
-3085 };
-3086 let pto = last_ack_eliciting + duration;
-3087 if result.is_none_or(|(earliest_pto, _)| pto < earliest_pto) {
-3088 if path.anti_amplification_blocked(1) {
-3089 // Nothing would be able to be sent.
-3090 continue;
-3091 }
-3092 if path.in_flight.ack_eliciting == 0 {
-3093 // Nothing ack-eliciting, no PTO to arm/fire.
-3094 continue;
-3095 }
-3096 result = Some((pto, space));
-3097 }
-3098 }
-3099 result
-3100 }
-3101
-3102 fn peer_completed_address_validation(&self, path: PathId) -> bool {
-3103 // TODO(flub): This logic needs updating for multipath
-3104 if self.side.is_server() || self.state.is_closed() {
-3105 return true;
-3106 }
-3107 // The server is guaranteed to have validated our address if any of our handshake or 1-RTT
-3108 // packets are acknowledged or we've seen HANDSHAKE_DONE and discarded handshake keys.
-3109 self.spaces[SpaceId::Handshake]
-3110 .path_space(PathId::ZERO)
-3111 .and_then(|pns| pns.largest_acked_packet)
-3112 .is_some()
-3113 || self.spaces[SpaceId::Data]
-3114 .path_space(path)
-3115 .and_then(|pns| pns.largest_acked_packet)
-3116 .is_some()
-3117 || (self.spaces[SpaceId::Data].crypto.is_some()
-3118 && self.spaces[SpaceId::Handshake].crypto.is_none())
-3119 }
-3120
-3121 /// Resets the the [`PathTimer::LossDetection`] timer to the next instant it may be needed
-3122 ///
-3123 /// The timer must fire if either:
-3124 /// - An ack-eliciting packet we sent needs to be declared lost.
-3125 /// - A tail-loss probe needs to be sent.
-3126 ///
-3127 /// See [`Connection::on_loss_detection_timeout`] for details.
-3128 fn set_loss_detection_timer(&mut self, now: Instant, path_id: PathId) {
-3129 if self.state.is_closed() {
-3130 // No loss detection takes place on closed connections, and `close_common` already
-3131 // stopped time timer. Ensure we don't restart it inadvertently, e.g. in response to a
-3132 // reordered packet being handled by state-insensitive code.
-3133 return;
-3134 }
-3135
-3136 if let Some((loss_time, _)) = self.loss_time_and_space(path_id) {
-3137 // Time threshold loss detection.
-3138 self.timers.set(
-3139 Timer::PerPath(path_id, PathTimer::LossDetection),
-3140 loss_time,
-3141 self.qlog.with_time(now),
-3142 );
-3143 return;
-3144 }
-3145
-3146 // Determine which PN space to arm PTO for.
-3147 // Calculate PTO duration
-3148 if let Some((timeout, _)) = self.pto_time_and_space(now, path_id) {
-3149 self.timers.set(
-3150 Timer::PerPath(path_id, PathTimer::LossDetection),
-3151 timeout,
-3152 self.qlog.with_time(now),
-3153 );
-3154 } else {
-3155 self.timers.stop(
-3156 Timer::PerPath(path_id, PathTimer::LossDetection),
-3157 self.qlog.with_time(now),
-3158 );
-3159 }
-3160 }
-3161
-3162 /// The maximum probe timeout across all paths
-3163 ///
-3164 /// If `is_closing` is set to `true` it will filter out paths that have not yet been used.
-3165 ///
-3166 /// See [`Connection::pto`]
-3167 fn pto_max_path(&self, space: SpaceId, is_closing: bool) -> Duration {
-3168 match space {
-3169 SpaceId::Initial | SpaceId::Handshake => self.pto(space, PathId::ZERO),
-3170 SpaceId::Data => self
-3171 .paths
-3172 .iter()
-3173 .filter_map(|(path_id, state)| {
-3174 if is_closing && state.data.total_sent == 0 && state.data.total_recvd == 0 {
-3175 // If we are closing and haven't sent anything yet, do not include
-3176 None
-3177 } else {
-3178 let pto = self.pto(space, *path_id);
-3179 Some(pto)
-3180 }
-3181 })
-3182 .max()
-3183 .expect("there should be at least one path"),
-3184 }
-3185 }
-3186
-3187 /// Probe Timeout
-3188 ///
-3189 /// The PTO is logically the time in which you'd expect to receive an acknowledgement
-3190 /// for a packet. So approximately RTT + max_ack_delay.
-3191 fn pto(&self, space: SpaceId, path_id: PathId) -> Duration {
-3192 let max_ack_delay = match space {
-3193 SpaceId::Initial | SpaceId::Handshake => Duration::ZERO,
-3194 SpaceId::Data => self.ack_frequency.max_ack_delay_for_pto(),
-3195 };
-3196 self.path_data(path_id).rtt.pto_base() + max_ack_delay
-3197 }
-3198
-3199 fn on_packet_authenticated(
-3200 &mut self,
-3201 now: Instant,
-3202 space_id: SpaceId,
-3203 path_id: PathId,
-3204 ecn: Option<EcnCodepoint>,
-3205 packet: Option<u64>,
-3206 spin: bool,
-3207 is_1rtt: bool,
-3208 ) {
-3209 self.total_authed_packets += 1;
-3210 if let Some(AbandonState::ExpectingPathAbandon { deadline }) = self
-3211 .paths
-3212 .get(&path_id)
-3213 .map(|path| &path.data.abandon_state)
-3214 {
-3215 if now > *deadline {
-3216 warn!("received data on path which we abandoned more than 3 * PTO ago");
-3217 // The peer failed to respond with a PATH_ABANDON in time.
-3218 if !self.state.is_closed() {
-3219 self.state
-3220 .move_to_closed(TransportError::PROTOCOL_VIOLATION(
-3221 "peer failed to respond with PATH_ABANDON in time",
-3222 ));
-3223 self.close_common();
-3224 self.set_close_timer(now);
-3225 self.close = true;
-3226 }
-3227 return;
-3228 }
-3229 }
-3230
-3231 self.reset_keep_alive(path_id, now);
-3232 self.reset_idle_timeout(now, space_id, path_id);
-3233 self.permit_idle_reset = true;
-3234 self.receiving_ecn |= ecn.is_some();
-3235 if let Some(x) = ecn {
-3236 let space = &mut self.spaces[space_id];
-3237 space.for_path(path_id).ecn_counters += x;
-3238
-3239 if x.is_ce() {
-3240 space
-3241 .for_path(path_id)
-3242 .pending_acks
-3243 .set_immediate_ack_required();
-3244 }
-3245 }
-3246
-3247 let packet = match packet {
-3248 Some(x) => x,
-3249 None => return,
-3250 };
-3251 match &self.side {
-3252 ConnectionSide::Client { .. } => {
-3253 // If we received a handshake packet that authenticated, then we're talking to
-3254 // the real server. From now on we should no longer allow the server to migrate
-3255 // its address.
-3256 if space_id == SpaceId::Handshake {
-3257 if let Some(hs) = self.state.as_handshake_mut() {
-3258 hs.allow_server_migration = false;
-3259 }
-3260 }
-3261 }
-3262 ConnectionSide::Server { .. } => {
-3263 if self.spaces[SpaceId::Initial].crypto.is_some() && space_id == SpaceId::Handshake
-3264 {
-3265 // A server stops sending and processing Initial packets when it receives its first Handshake packet.
-3266 self.discard_space(now, SpaceId::Initial);
-3267 }
-3268 if self.zero_rtt_crypto.is_some() && is_1rtt {
-3269 // Discard 0-RTT keys soon after receiving a 1-RTT packet
-3270 self.set_key_discard_timer(now, space_id)
-3271 }
-3272 }
-3273 }
-3274 let space = self.spaces[space_id].for_path(path_id);
-3275 space.pending_acks.insert_one(packet, now);
-3276 if packet >= space.rx_packet.unwrap_or_default() {
-3277 space.rx_packet = Some(packet);
-3278 // Update outgoing spin bit, inverting iff we're the client
-3279 self.spin = self.side.is_client() ^ spin;
-3280 }
-3281 }
-3282
-3283 /// Resets the idle timeout timers
-3284 ///
-3285 /// Without multipath there is only the connection-wide idle timeout. When multipath is
-3286 /// enabled there is an additional per-path idle timeout.
-3287 fn reset_idle_timeout(&mut self, now: Instant, space: SpaceId, path_id: PathId) {
-3288 // First reset the global idle timeout.
-3289 if let Some(timeout) = self.idle_timeout {
-3290 if self.state.is_closed() {
-3291 self.timers
-3292 .stop(Timer::Conn(ConnTimer::Idle), self.qlog.with_time(now));
-3293 } else {
-3294 let dt = cmp::max(timeout, 3 * self.pto_max_path(space, false));
-3295 self.timers.set(
-3296 Timer::Conn(ConnTimer::Idle),
-3297 now + dt,
-3298 self.qlog.with_time(now),
-3299 );
-3300 }
-3301 }
-3302
-3303 // Now handle the per-path state
-3304 if let Some(timeout) = self.path_data(path_id).idle_timeout {
-3305 if self.state.is_closed() {
-3306 self.timers.stop(
-3307 Timer::PerPath(path_id, PathTimer::PathIdle),
-3308 self.qlog.with_time(now),
-3309 );
-3310 } else {
-3311 let dt = cmp::max(timeout, 3 * self.pto(space, path_id));
-3312 self.timers.set(
-3313 Timer::PerPath(path_id, PathTimer::PathIdle),
-3314 now + dt,
-3315 self.qlog.with_time(now),
-3316 );
-3317 }
-3318 }
-3319 }
-3320
-3321 /// Resets both the [`ConnTimer::KeepAlive`] and [`PathTimer::PathKeepAlive`] timers
-3322 fn reset_keep_alive(&mut self, path_id: PathId, now: Instant) {
-3323 if !self.state.is_established() {
-3324 return;
-3325 }
+2987 let mut size_of_lost_packets = 0u64; // add to path_stats.lost_bytes;
+2988 let lost_pns: Vec<_> = self.spaces[SpaceId::Data]
+2989 .for_path(path_id)
+2990 .sent_packets
+2991 .iter()
+2992 .filter(|(pn, _info)| Some(*pn) != in_flight_mtu_probe)
+2993 .map(|(pn, info)| {
+2994 size_of_lost_packets += info.size as u64;
+2995 pn
+2996 })
+2997 .collect();
+2998
+2999 if !lost_pns.is_empty() {
+3000 trace!(
+3001 %path_id,
+3002 count = lost_pns.len(),
+3003 lost_bytes = size_of_lost_packets,
+3004 "packets lost on path abandon"
+3005 );
+3006 self.handle_lost_packets(
+3007 SpaceId::Data,
+3008 path_id,
+3009 now,
+3010 lost_pns,
+3011 in_flight_mtu_probe,
+3012 Duration::ZERO,
+3013 false,
+3014 size_of_lost_packets,
+3015 );
+3016 }
+3017 self.paths.remove(&path_id);
+3018 self.spaces[SpaceId::Data].number_spaces.remove(&path_id);
+3019
+3020 let path_stats = self.path_stats.remove(&path_id).unwrap_or_default();
+3021 self.events.push_back(
+3022 PathEvent::Abandoned {
+3023 id: path_id,
+3024 path_stats,
+3025 }
+3026 .into(),
+3027 );
+3028 }
+3029
+3030 fn handle_lost_packets(
+3031 &mut self,
+3032 pn_space: SpaceId,
+3033 path_id: PathId,
+3034 now: Instant,
+3035 lost_packets: Vec<u64>,
+3036 lost_mtu_probe: Option<u64>,
+3037 loss_delay: Duration,
+3038 in_persistent_congestion: bool,
+3039 size_of_lost_packets: u64,
+3040 ) {
+3041 debug_assert!(
+3042 {
+3043 let mut sorted = lost_packets.clone();
+3044 sorted.sort();
+3045 sorted == lost_packets
+3046 },
+3047 "lost_packets must be sorted"
+3048 );
+3049
+3050 self.drain_lost_packets(now, pn_space, path_id);
+3051
+3052 // OnPacketsLost
+3053 if let Some(largest_lost) = lost_packets.last().cloned() {
+3054 let old_bytes_in_flight = self.path_data_mut(path_id).in_flight.bytes;
+3055 let largest_lost_sent = self.spaces[pn_space]
+3056 .for_path(path_id)
+3057 .sent_packets
+3058 .get(largest_lost)
+3059 .unwrap()
+3060 .time_sent;
+3061 let path_stats = self.path_stats.entry(path_id).or_default();
+3062 path_stats.lost_packets += lost_packets.len() as u64;
+3063 path_stats.lost_bytes += size_of_lost_packets;
+3064 trace!(
+3065 %path_id,
+3066 count = lost_packets.len(),
+3067 lost_bytes = size_of_lost_packets,
+3068 "packets lost",
+3069 );
+3070
+3071 for &packet in &lost_packets {
+3072 let Some(info) = self.spaces[pn_space].for_path(path_id).take(packet) else {
+3073 continue;
+3074 };
+3075 self.qlog
+3076 .emit_packet_lost(packet, &info, loss_delay, pn_space, now);
+3077 self.paths
+3078 .get_mut(&path_id)
+3079 .unwrap()
+3080 .remove_in_flight(&info);
+3081
+3082 for frame in info.stream_frames {
+3083 self.streams.retransmit(frame);
+3084 }
+3085 self.spaces[pn_space].pending |= info.retransmits;
+3086 self.path_data_mut(path_id)
+3087 .mtud
+3088 .on_non_probe_lost(packet, info.size);
+3089
+3090 self.spaces[pn_space].for_path(path_id).lost_packets.insert(
+3091 packet,
+3092 LostPacket {
+3093 time_sent: info.time_sent,
+3094 },
+3095 );
+3096 }
+3097
+3098 let path = self.path_data_mut(path_id);
+3099 if path.mtud.black_hole_detected(now) {
+3100 path.congestion.on_mtu_update(path.mtud.current_mtu());
+3101 if let Some(max_datagram_size) = self.datagrams().max_size() {
+3102 if self.datagrams.drop_oversized(max_datagram_size)
+3103 && self.datagrams.send_blocked
+3104 {
+3105 self.datagrams.send_blocked = false;
+3106 self.events.push_back(Event::DatagramsUnblocked);
+3107 }
+3108 }
+3109 self.path_stats
+3110 .entry(path_id)
+3111 .or_default()
+3112 .black_holes_detected += 1;
+3113 }
+3114
+3115 // Don't apply congestion penalty for lost ack-only packets
+3116 let lost_ack_eliciting =
+3117 old_bytes_in_flight != self.path_data_mut(path_id).in_flight.bytes;
+3118
+3119 if lost_ack_eliciting {
+3120 self.path_stats
+3121 .entry(path_id)
+3122 .or_default()
+3123 .congestion_events += 1;
+3124 self.path_data_mut(path_id).congestion.on_congestion_event(
+3125 now,
+3126 largest_lost_sent,
+3127 in_persistent_congestion,
+3128 false,
+3129 size_of_lost_packets,
+3130 );
+3131 }
+3132 }
+3133
+3134 // Handle a lost MTU probe
+3135 if let Some(packet) = lost_mtu_probe {
+3136 let info = self.spaces[SpaceId::Data]
+3137 .for_path(path_id)
+3138 .take(packet)
+3139 .unwrap(); // safe: lost_mtu_probe is omitted from lost_packets, and
+3140 // therefore must not have been removed yet
+3141 self.paths
+3142 .get_mut(&path_id)
+3143 .unwrap()
+3144 .remove_in_flight(&info);
+3145 self.path_data_mut(path_id).mtud.on_probe_lost();
+3146 self.path_stats
+3147 .entry(path_id)
+3148 .or_default()
+3149 .lost_plpmtud_probes += 1;
+3150 }
+3151 }
+3152
+3153 /// Returns the earliest time packets should be declared lost for all spaces on a path.
+3154 ///
+3155 /// If a path has an acknowledged packet with any prior un-acknowledged packets, the
+3156 /// earliest un-acknowledged packet can be declared lost after a timeout has elapsed.
+3157 /// The time returned is when this packet should be declared lost.
+3158 fn loss_time_and_space(&self, path_id: PathId) -> Option<(Instant, SpaceId)> {
+3159 SpaceId::iter()
+3160 .filter_map(|id| {
+3161 self.spaces[id]
+3162 .number_spaces
+3163 .get(&path_id)
+3164 .and_then(|pns| pns.loss_time)
+3165 .map(|time| (time, id))
+3166 })
+3167 .min_by_key(|&(time, _)| time)
+3168 }
+3169
+3170 /// Returns the earliest next PTO should fire for all spaces on a path.
+3171 fn pto_time_and_space(&mut self, now: Instant, path_id: PathId) -> Option<(Instant, SpaceId)> {
+3172 let path = self.path(path_id)?;
+3173 let pto_count = path.pto_count;
+3174 let backoff = 2u32.pow(pto_count.min(MAX_BACKOFF_EXPONENT));
+3175 let mut duration = path.rtt.pto_base() * backoff;
+3176
+3177 if path_id == PathId::ZERO
+3178 && path.in_flight.ack_eliciting == 0
+3179 && !self.peer_completed_address_validation(PathId::ZERO)
+3180 {
+3181 // Address Validation during Connection Establishment:
+3182 // https://www.rfc-editor.org/rfc/rfc9000.html#section-8.1. To prevent a
+3183 // deadlock if an Initial or Handshake packet from the server is lost and the
+3184 // server can not send more due to its anti-amplification limit the client must
+3185 // send another packet on PTO.
+3186 let space = match self.highest_space {
+3187 SpaceId::Handshake => SpaceId::Handshake,
+3188 _ => SpaceId::Initial,
+3189 };
+3190
+3191 return Some((now + duration, space));
+3192 }
+3193
+3194 let mut result = None;
+3195 for space in SpaceId::iter() {
+3196 let Some(pns) = self.spaces[space].number_spaces.get(&path_id) else {
+3197 continue;
+3198 };
+3199
+3200 if !pns.has_in_flight() {
+3201 continue;
+3202 }
+3203 if space == SpaceId::Data {
+3204 // Skip ApplicationData until handshake completes.
+3205 if self.is_handshaking() {
+3206 return result;
+3207 }
+3208 // Include max_ack_delay and backoff for ApplicationData.
+3209 duration += self.ack_frequency.max_ack_delay_for_pto() * backoff;
+3210 }
+3211 let Some(last_ack_eliciting) = pns.time_of_last_ack_eliciting_packet else {
+3212 continue;
+3213 };
+3214 let pto = last_ack_eliciting + duration;
+3215 if result.is_none_or(|(earliest_pto, _)| pto < earliest_pto) {
+3216 if path.anti_amplification_blocked(1) {
+3217 // Nothing would be able to be sent.
+3218 continue;
+3219 }
+3220 if path.in_flight.ack_eliciting == 0 {
+3221 // Nothing ack-eliciting, no PTO to arm/fire.
+3222 continue;
+3223 }
+3224 result = Some((pto, space));
+3225 }
+3226 }
+3227 result
+3228 }
+3229
+3230 fn peer_completed_address_validation(&self, path: PathId) -> bool {
+3231 // TODO(flub): This logic needs updating for multipath
+3232 if self.side.is_server() || self.state.is_closed() {
+3233 return true;
+3234 }
+3235 // The server is guaranteed to have validated our address if any of our handshake or 1-RTT
+3236 // packets are acknowledged or we've seen HANDSHAKE_DONE and discarded handshake keys.
+3237 self.spaces[SpaceId::Handshake]
+3238 .path_space(PathId::ZERO)
+3239 .and_then(|pns| pns.largest_acked_packet)
+3240 .is_some()
+3241 || self.spaces[SpaceId::Data]
+3242 .path_space(path)
+3243 .and_then(|pns| pns.largest_acked_packet)
+3244 .is_some()
+3245 || (self.spaces[SpaceId::Data].crypto.is_some()
+3246 && self.spaces[SpaceId::Handshake].crypto.is_none())
+3247 }
+3248
+3249 /// Resets the the [`PathTimer::LossDetection`] timer to the next instant it may be needed
+3250 ///
+3251 /// The timer must fire if either:
+3252 /// - An ack-eliciting packet we sent needs to be declared lost.
+3253 /// - A tail-loss probe needs to be sent.
+3254 ///
+3255 /// See [`Connection::on_loss_detection_timeout`] for details.
+3256 fn set_loss_detection_timer(&mut self, now: Instant, path_id: PathId) {
+3257 if self.state.is_closed() {
+3258 // No loss detection takes place on closed connections, and `close_common` already
+3259 // stopped time timer. Ensure we don't restart it inadvertently, e.g. in response to a
+3260 // reordered packet being handled by state-insensitive code.
+3261 return;
+3262 }
+3263
+3264 if let Some((loss_time, _)) = self.loss_time_and_space(path_id) {
+3265 // Time threshold loss detection.
+3266 self.timers.set(
+3267 Timer::PerPath(path_id, PathTimer::LossDetection),
+3268 loss_time,
+3269 self.qlog.with_time(now),
+3270 );
+3271 return;
+3272 }
+3273
+3274 // Determine which PN space to arm PTO for.
+3275 // Calculate PTO duration
+3276 if let Some((timeout, _)) = self.pto_time_and_space(now, path_id) {
+3277 self.timers.set(
+3278 Timer::PerPath(path_id, PathTimer::LossDetection),
+3279 timeout,
+3280 self.qlog.with_time(now),
+3281 );
+3282 } else {
+3283 self.timers.stop(
+3284 Timer::PerPath(path_id, PathTimer::LossDetection),
+3285 self.qlog.with_time(now),
+3286 );
+3287 }
+3288 }
+3289
+3290 /// The maximum probe timeout across all paths
+3291 ///
+3292 /// If `is_closing` is set to `true` it will filter out paths that have not yet been used.
+3293 ///
+3294 /// See [`Connection::pto`]
+3295 fn pto_max_path(&self, space: SpaceId, is_closing: bool) -> Duration {
+3296 match space {
+3297 SpaceId::Initial | SpaceId::Handshake => self.pto(space, PathId::ZERO),
+3298 SpaceId::Data => self
+3299 .paths
+3300 .iter()
+3301 .filter_map(|(path_id, state)| {
+3302 if is_closing && state.data.total_sent == 0 && state.data.total_recvd == 0 {
+3303 // If we are closing and haven't sent anything yet, do not include
+3304 None
+3305 } else {
+3306 let pto = self.pto(space, *path_id);
+3307 Some(pto)
+3308 }
+3309 })
+3310 .max()
+3311 .expect("there should be at least one path"),
+3312 }
+3313 }
+3314
+3315 /// Probe Timeout
+3316 ///
+3317 /// The PTO is logically the time in which you'd expect to receive an acknowledgement
+3318 /// for a packet. So approximately RTT + max_ack_delay.
+3319 fn pto(&self, space: SpaceId, path_id: PathId) -> Duration {
+3320 let max_ack_delay = match space {
+3321 SpaceId::Initial | SpaceId::Handshake => Duration::ZERO,
+3322 SpaceId::Data => self.ack_frequency.max_ack_delay_for_pto(),
+3323 };
+3324 self.path_data(path_id).rtt.pto_base() + max_ack_delay
+3325 }
3326
-3327 if let Some(interval) = self.config.keep_alive_interval {
-3328 self.timers.set(
-3329 Timer::Conn(ConnTimer::KeepAlive),
-3330 now + interval,
-3331 self.qlog.with_time(now),
-3332 );
-3333 }
-3334
-3335 if let Some(interval) = self.path_data(path_id).keep_alive {
-3336 self.timers.set(
-3337 Timer::PerPath(path_id, PathTimer::PathKeepAlive),
-3338 now + interval,
-3339 self.qlog.with_time(now),
-3340 );
-3341 }
-3342 }
-3343
-3344 /// Sets the timer for when a previously issued CID should be retired next
-3345 fn reset_cid_retirement(&mut self, now: Instant) {
-3346 if let Some((_path, t)) = self.next_cid_retirement() {
-3347 self.timers.set(
-3348 Timer::Conn(ConnTimer::PushNewCid),
-3349 t,
-3350 self.qlog.with_time(now),
-3351 );
-3352 }
-3353 }
-3354
-3355 /// The next time when a previously issued CID should be retired
-3356 fn next_cid_retirement(&self) -> Option<(PathId, Instant)> {
-3357 self.local_cid_state
-3358 .iter()
-3359 .filter_map(|(path_id, cid_state)| cid_state.next_timeout().map(|t| (*path_id, t)))
-3360 .min_by_key(|(_path_id, timeout)| *timeout)
-3361 }
-3362
-3363 /// Handle the already-decrypted first packet from the client
-3364 ///
-3365 /// Decrypting the first packet in the `Endpoint` allows stateless packet handling to be more
-3366 /// efficient.
-3367 pub(crate) fn handle_first_packet(
-3368 &mut self,
-3369 now: Instant,
-3370 network_path: FourTuple,
-3371 ecn: Option<EcnCodepoint>,
-3372 packet_number: u64,
-3373 packet: InitialPacket,
-3374 remaining: Option<BytesMut>,
-3375 ) -> Result<(), ConnectionError> {
-3376 let span = trace_span!("first recv");
-3377 let _guard = span.enter();
-3378 debug_assert!(self.side.is_server());
-3379 let len = packet.header_data.len() + packet.payload.len();
-3380 let path_id = PathId::ZERO;
-3381 self.path_data_mut(path_id).total_recvd = len as u64;
-3382
-3383 if let Some(hs) = self.state.as_handshake_mut() {
-3384 hs.expected_token = packet.header.token.clone();
-3385 } else {
-3386 unreachable!("first packet must be delivered in Handshake state");
-3387 }
-3388
-3389 // The first packet is always on PathId::ZERO
-3390 self.on_packet_authenticated(
-3391 now,
-3392 SpaceId::Initial,
-3393 path_id,
-3394 ecn,
-3395 Some(packet_number),
-3396 false,
-3397 false,
-3398 );
-3399
-3400 let packet: Packet = packet.into();
-3401
-3402 let mut qlog = QlogRecvPacket::new(len);
-3403 qlog.header(&packet.header, Some(packet_number), path_id);
-3404
-3405 self.process_decrypted_packet(
-3406 now,
-3407 network_path,
-3408 path_id,
-3409 Some(packet_number),
-3410 packet,
-3411 &mut qlog,
-3412 )?;
-3413 self.qlog.emit_packet_received(qlog, now);
-3414 if let Some(data) = remaining {
-3415 self.handle_coalesced(now, network_path, path_id, ecn, data);
-3416 }
-3417
-3418 self.qlog.emit_recovery_metrics(
-3419 path_id,
-3420 &mut self.paths.get_mut(&path_id).unwrap().data,
-3421 now,
-3422 );
-3423
-3424 Ok(())
-3425 }
-3426
-3427 fn init_0rtt(&mut self, now: Instant) {
-3428 let (header, packet) = match self.crypto.early_crypto() {
-3429 Some(x) => x,
-3430 None => return,
-3431 };
-3432 if self.side.is_client() {
-3433 match self.crypto.transport_parameters() {
-3434 Ok(params) => {
-3435 let params = params
-3436 .expect("crypto layer didn't supply transport parameters with ticket");
-3437 // Certain values must not be cached
-3438 let params = TransportParameters {
-3439 initial_src_cid: None,
-3440 original_dst_cid: None,
-3441 preferred_address: None,
-3442 retry_src_cid: None,
-3443 stateless_reset_token: None,
-3444 min_ack_delay: None,
-3445 ack_delay_exponent: TransportParameters::default().ack_delay_exponent,
-3446 max_ack_delay: TransportParameters::default().max_ack_delay,
-3447 initial_max_path_id: None,
-3448 ..params
-3449 };
-3450 self.set_peer_params(params);
-3451 self.qlog.emit_peer_transport_params_restored(self, now);
-3452 }
-3453 Err(e) => {
-3454 error!("session ticket has malformed transport parameters: {}", e);
-3455 return;
-3456 }
-3457 }
-3458 }
-3459 trace!("0-RTT enabled");
-3460 self.zero_rtt_enabled = true;
-3461 self.zero_rtt_crypto = Some(ZeroRttCrypto { header, packet });
-3462 }
-3463
-3464 fn read_crypto(
-3465 &mut self,
-3466 space: SpaceId,
-3467 crypto: &frame::Crypto,
-3468 payload_len: usize,
-3469 ) -> Result<(), TransportError> {
-3470 let expected = if !self.state.is_handshake() {
-3471 SpaceId::Data
-3472 } else if self.highest_space == SpaceId::Initial {
-3473 SpaceId::Initial
-3474 } else {
-3475 // On the server, self.highest_space can be Data after receiving the client's first
-3476 // flight, but we expect Handshake CRYPTO until the handshake is complete.
-3477 SpaceId::Handshake
-3478 };
-3479 // We can't decrypt Handshake packets when highest_space is Initial, CRYPTO frames in 0-RTT
-3480 // packets are illegal, and we don't process 1-RTT packets until the handshake is
-3481 // complete. Therefore, we will never see CRYPTO data from a later-than-expected space.
-3482 debug_assert!(space <= expected, "received out-of-order CRYPTO data");
-3483
-3484 let end = crypto.offset + crypto.data.len() as u64;
-3485 if space < expected && end > self.spaces[space].crypto_stream.bytes_read() {
-3486 warn!(
-3487 "received new {:?} CRYPTO data when expecting {:?}",
-3488 space, expected
-3489 );
-3490 return Err(TransportError::PROTOCOL_VIOLATION(
-3491 "new data at unexpected encryption level",
-3492 ));
-3493 }
-3494
-3495 let space = &mut self.spaces[space];
-3496 let max = end.saturating_sub(space.crypto_stream.bytes_read());
-3497 if max > self.config.crypto_buffer_size as u64 {
-3498 return Err(TransportError::CRYPTO_BUFFER_EXCEEDED(""));
-3499 }
-3500
-3501 space
-3502 .crypto_stream
-3503 .insert(crypto.offset, crypto.data.clone(), payload_len);
-3504 while let Some(chunk) = space.crypto_stream.read(usize::MAX, true) {
-3505 trace!("consumed {} CRYPTO bytes", chunk.bytes.len());
-3506 if self.crypto.read_handshake(&chunk.bytes)? {
-3507 self.events.push_back(Event::HandshakeDataReady);
-3508 }
-3509 }
+3327 fn on_packet_authenticated(
+3328 &mut self,
+3329 now: Instant,
+3330 space_id: SpaceId,
+3331 path_id: PathId,
+3332 ecn: Option<EcnCodepoint>,
+3333 packet: Option<u64>,
+3334 spin: bool,
+3335 is_1rtt: bool,
+3336 ) {
+3337 self.total_authed_packets += 1;
+3338 if let Some(AbandonState::ExpectingPathAbandon { deadline }) = self
+3339 .paths
+3340 .get(&path_id)
+3341 .map(|path| &path.data.abandon_state)
+3342 {
+3343 if now > *deadline {
+3344 warn!("received data on path which we abandoned more than 3 * PTO ago");
+3345 // The peer failed to respond with a PATH_ABANDON in time.
+3346 if !self.state.is_closed() {
+3347 self.state
+3348 .move_to_closed(TransportError::PROTOCOL_VIOLATION(
+3349 "peer failed to respond with PATH_ABANDON in time",
+3350 ));
+3351 self.close_common();
+3352 self.set_close_timer(now);
+3353 self.close = true;
+3354 }
+3355 return;
+3356 }
+3357 }
+3358
+3359 self.reset_keep_alive(path_id, now);
+3360 self.reset_idle_timeout(now, space_id, path_id);
+3361 self.permit_idle_reset = true;
+3362 self.receiving_ecn |= ecn.is_some();
+3363 if let Some(x) = ecn {
+3364 let space = &mut self.spaces[space_id];
+3365 space.for_path(path_id).ecn_counters += x;
+3366
+3367 if x.is_ce() {
+3368 space
+3369 .for_path(path_id)
+3370 .pending_acks
+3371 .set_immediate_ack_required();
+3372 }
+3373 }
+3374
+3375 let packet = match packet {
+3376 Some(x) => x,
+3377 None => return,
+3378 };
+3379 match &self.side {
+3380 ConnectionSide::Client { .. } => {
+3381 // If we received a handshake packet that authenticated, then we're talking to
+3382 // the real server. From now on we should no longer allow the server to migrate
+3383 // its address.
+3384 if space_id == SpaceId::Handshake {
+3385 if let Some(hs) = self.state.as_handshake_mut() {
+3386 hs.allow_server_migration = false;
+3387 }
+3388 }
+3389 }
+3390 ConnectionSide::Server { .. } => {
+3391 if self.spaces[SpaceId::Initial].crypto.is_some() && space_id == SpaceId::Handshake
+3392 {
+3393 // A server stops sending and processing Initial packets when it receives its first Handshake packet.
+3394 self.discard_space(now, SpaceId::Initial);
+3395 }
+3396 if self.zero_rtt_crypto.is_some() && is_1rtt {
+3397 // Discard 0-RTT keys soon after receiving a 1-RTT packet
+3398 self.set_key_discard_timer(now, space_id)
+3399 }
+3400 }
+3401 }
+3402 let space = self.spaces[space_id].for_path(path_id);
+3403 space.pending_acks.insert_one(packet, now);
+3404 if packet >= space.rx_packet.unwrap_or_default() {
+3405 space.rx_packet = Some(packet);
+3406 // Update outgoing spin bit, inverting iff we're the client
+3407 self.spin = self.side.is_client() ^ spin;
+3408 }
+3409 }
+3410
+3411 /// Resets the idle timeout timers
+3412 ///
+3413 /// Without multipath there is only the connection-wide idle timeout. When multipath is
+3414 /// enabled there is an additional per-path idle timeout.
+3415 fn reset_idle_timeout(&mut self, now: Instant, space: SpaceId, path_id: PathId) {
+3416 // First reset the global idle timeout.
+3417 if let Some(timeout) = self.idle_timeout {
+3418 if self.state.is_closed() {
+3419 self.timers
+3420 .stop(Timer::Conn(ConnTimer::Idle), self.qlog.with_time(now));
+3421 } else {
+3422 let dt = cmp::max(timeout, 3 * self.pto_max_path(space, false));
+3423 self.timers.set(
+3424 Timer::Conn(ConnTimer::Idle),
+3425 now + dt,
+3426 self.qlog.with_time(now),
+3427 );
+3428 }
+3429 }
+3430
+3431 // Now handle the per-path state
+3432 if let Some(timeout) = self.path_data(path_id).idle_timeout {
+3433 if self.state.is_closed() {
+3434 self.timers.stop(
+3435 Timer::PerPath(path_id, PathTimer::PathIdle),
+3436 self.qlog.with_time(now),
+3437 );
+3438 } else {
+3439 let dt = cmp::max(timeout, 3 * self.pto(space, path_id));
+3440 self.timers.set(
+3441 Timer::PerPath(path_id, PathTimer::PathIdle),
+3442 now + dt,
+3443 self.qlog.with_time(now),
+3444 );
+3445 }
+3446 }
+3447 }
+3448
+3449 /// Resets both the [`ConnTimer::KeepAlive`] and [`PathTimer::PathKeepAlive`] timers
+3450 fn reset_keep_alive(&mut self, path_id: PathId, now: Instant) {
+3451 if !self.state.is_established() {
+3452 return;
+3453 }
+3454
+3455 if let Some(interval) = self.config.keep_alive_interval {
+3456 self.timers.set(
+3457 Timer::Conn(ConnTimer::KeepAlive),
+3458 now + interval,
+3459 self.qlog.with_time(now),
+3460 );
+3461 }
+3462
+3463 if let Some(interval) = self.path_data(path_id).keep_alive {
+3464 self.timers.set(
+3465 Timer::PerPath(path_id, PathTimer::PathKeepAlive),
+3466 now + interval,
+3467 self.qlog.with_time(now),
+3468 );
+3469 }
+3470 }
+3471
+3472 /// Sets the timer for when a previously issued CID should be retired next
+3473 fn reset_cid_retirement(&mut self, now: Instant) {
+3474 if let Some((_path, t)) = self.next_cid_retirement() {
+3475 self.timers.set(
+3476 Timer::Conn(ConnTimer::PushNewCid),
+3477 t,
+3478 self.qlog.with_time(now),
+3479 );
+3480 }
+3481 }
+3482
+3483 /// The next time when a previously issued CID should be retired
+3484 fn next_cid_retirement(&self) -> Option<(PathId, Instant)> {
+3485 self.local_cid_state
+3486 .iter()
+3487 .filter_map(|(path_id, cid_state)| cid_state.next_timeout().map(|t| (*path_id, t)))
+3488 .min_by_key(|(_path_id, timeout)| *timeout)
+3489 }
+3490
+3491 /// Handle the already-decrypted first packet from the client
+3492 ///
+3493 /// Decrypting the first packet in the `Endpoint` allows stateless packet handling to be more
+3494 /// efficient.
+3495 pub(crate) fn handle_first_packet(
+3496 &mut self,
+3497 now: Instant,
+3498 network_path: FourTuple,
+3499 ecn: Option<EcnCodepoint>,
+3500 packet_number: u64,
+3501 packet: InitialPacket,
+3502 remaining: Option<BytesMut>,
+3503 ) -> Result<(), ConnectionError> {
+3504 let span = trace_span!("first recv");
+3505 let _guard = span.enter();
+3506 debug_assert!(self.side.is_server());
+3507 let len = packet.header_data.len() + packet.payload.len();
+3508 let path_id = PathId::ZERO;
+3509 self.path_data_mut(path_id).total_recvd = len as u64;
3510
-3511 Ok(())
-3512 }
-3513
-3514 fn write_crypto(&mut self) {
-3515 loop {
-3516 let space = self.highest_space;
-3517 let mut outgoing = Vec::new();
-3518 if let Some(crypto) = self.crypto.write_handshake(&mut outgoing) {
-3519 match space {
-3520 SpaceId::Initial => {
-3521 self.upgrade_crypto(SpaceId::Handshake, crypto);
-3522 }
-3523 SpaceId::Handshake => {
-3524 self.upgrade_crypto(SpaceId::Data, crypto);
-3525 }
-3526 _ => unreachable!("got updated secrets during 1-RTT"),
-3527 }
-3528 }
-3529 if outgoing.is_empty() {
-3530 if space == self.highest_space {
-3531 break;
-3532 } else {
-3533 // Keys updated, check for more data to send
-3534 continue;
-3535 }
-3536 }
-3537 let offset = self.spaces[space].crypto_offset;
-3538 let outgoing = Bytes::from(outgoing);
-3539 if let Some(hs) = self.state.as_handshake_mut() {
-3540 if space == SpaceId::Initial && offset == 0 && self.side.is_client() {
-3541 hs.client_hello = Some(outgoing.clone());
-3542 }
-3543 }
-3544 self.spaces[space].crypto_offset += outgoing.len() as u64;
-3545 trace!("wrote {} {:?} CRYPTO bytes", outgoing.len(), space);
-3546 self.spaces[space].pending.crypto.push_back(frame::Crypto {
-3547 offset,
-3548 data: outgoing,
-3549 });
-3550 }
-3551 }
-3552
-3553 /// Switch to stronger cryptography during handshake
-3554 fn upgrade_crypto(&mut self, space: SpaceId, crypto: Keys) {
-3555 debug_assert!(
-3556 self.spaces[space].crypto.is_none(),
-3557 "already reached packet space {space:?}"
-3558 );
-3559 trace!("{:?} keys ready", space);
-3560 if space == SpaceId::Data {
-3561 // Precompute the first key update
-3562 self.next_crypto = Some(
-3563 self.crypto
-3564 .next_1rtt_keys()
-3565 .expect("handshake should be complete"),
-3566 );
-3567 }
-3568
-3569 self.spaces[space].crypto = Some(crypto);
-3570 debug_assert!(space as usize > self.highest_space as usize);
-3571 self.highest_space = space;
-3572 if space == SpaceId::Data && self.side.is_client() {
-3573 // Discard 0-RTT keys because 1-RTT keys are available.
-3574 self.zero_rtt_crypto = None;
-3575 }
-3576 }
-3577
-3578 fn discard_space(&mut self, now: Instant, space_id: SpaceId) {
-3579 debug_assert!(space_id != SpaceId::Data);
-3580 trace!("discarding {:?} keys", space_id);
-3581 if space_id == SpaceId::Initial {
-3582 // No longer needed
-3583 if let ConnectionSide::Client { token, .. } = &mut self.side {
-3584 *token = Bytes::new();
+3511 if let Some(hs) = self.state.as_handshake_mut() {
+3512 hs.expected_token = packet.header.token.clone();
+3513 } else {
+3514 unreachable!("first packet must be delivered in Handshake state");
+3515 }
+3516
+3517 // The first packet is always on PathId::ZERO
+3518 self.on_packet_authenticated(
+3519 now,
+3520 SpaceId::Initial,
+3521 path_id,
+3522 ecn,
+3523 Some(packet_number),
+3524 false,
+3525 false,
+3526 );
+3527
+3528 let packet: Packet = packet.into();
+3529
+3530 let mut qlog = QlogRecvPacket::new(len);
+3531 qlog.header(&packet.header, Some(packet_number), path_id);
+3532
+3533 self.process_decrypted_packet(
+3534 now,
+3535 network_path,
+3536 path_id,
+3537 Some(packet_number),
+3538 packet,
+3539 &mut qlog,
+3540 )?;
+3541 self.qlog.emit_packet_received(qlog, now);
+3542 if let Some(data) = remaining {
+3543 self.handle_coalesced(now, network_path, path_id, ecn, data);
+3544 }
+3545
+3546 self.qlog.emit_recovery_metrics(
+3547 path_id,
+3548 &mut self.paths.get_mut(&path_id).unwrap().data,
+3549 now,
+3550 );
+3551
+3552 Ok(())
+3553 }
+3554
+3555 fn init_0rtt(&mut self, now: Instant) {
+3556 let (header, packet) = match self.crypto.early_crypto() {
+3557 Some(x) => x,
+3558 None => return,
+3559 };
+3560 if self.side.is_client() {
+3561 match self.crypto.transport_parameters() {
+3562 Ok(params) => {
+3563 let params = params
+3564 .expect("crypto layer didn't supply transport parameters with ticket");
+3565 // Certain values must not be cached
+3566 let params = TransportParameters {
+3567 initial_src_cid: None,
+3568 original_dst_cid: None,
+3569 preferred_address: None,
+3570 retry_src_cid: None,
+3571 stateless_reset_token: None,
+3572 min_ack_delay: None,
+3573 ack_delay_exponent: TransportParameters::default().ack_delay_exponent,
+3574 max_ack_delay: TransportParameters::default().max_ack_delay,
+3575 initial_max_path_id: None,
+3576 ..params
+3577 };
+3578 self.set_peer_params(params);
+3579 self.qlog.emit_peer_transport_params_restored(self, now);
+3580 }
+3581 Err(e) => {
+3582 error!("session ticket has malformed transport parameters: {}", e);
+3583 return;
+3584 }
3585 }
3586 }
-3587 let space = &mut self.spaces[space_id];
-3588 space.crypto = None;
-3589 let pns = space.for_path(PathId::ZERO);
-3590 pns.time_of_last_ack_eliciting_packet = None;
-3591 pns.loss_time = None;
-3592 pns.loss_probes = 0;
-3593 let sent_packets = mem::take(&mut pns.sent_packets);
-3594 let path = self.paths.get_mut(&PathId::ZERO).unwrap();
-3595 for (_, packet) in sent_packets.into_iter() {
-3596 path.data.remove_in_flight(&packet);
-3597 }
-3598
-3599 self.set_loss_detection_timer(now, PathId::ZERO)
-3600 }
-3601
-3602 fn handle_coalesced(
-3603 &mut self,
-3604 now: Instant,
-3605 network_path: FourTuple,
-3606 path_id: PathId,
-3607 ecn: Option<EcnCodepoint>,
-3608 data: BytesMut,
-3609 ) {
-3610 self.path_data_mut(path_id)
-3611 .inc_total_recvd(data.len() as u64);
-3612 let mut remaining = Some(data);
-3613 let cid_len = self
-3614 .local_cid_state
-3615 .values()
-3616 .map(|cid_state| cid_state.cid_len())
-3617 .next()
-3618 .expect("one cid_state must exist");
-3619 while let Some(data) = remaining {
-3620 match PartialDecode::new(
-3621 data,
-3622 &FixedLengthConnectionIdParser::new(cid_len),
-3623 &[self.version],
-3624 self.endpoint_config.grease_quic_bit,
-3625 ) {
-3626 Ok((partial_decode, rest)) => {
-3627 remaining = rest;
-3628 self.handle_decode(now, network_path, path_id, ecn, partial_decode);
-3629 }
-3630 Err(e) => {
-3631 trace!("malformed header: {}", e);
-3632 return;
-3633 }
-3634 }
-3635 }
-3636 }
-3637
-3638 fn handle_decode(
-3639 &mut self,
-3640 now: Instant,
-3641 network_path: FourTuple,
-3642 path_id: PathId,
-3643 ecn: Option<EcnCodepoint>,
-3644 partial_decode: PartialDecode,
-3645 ) {
-3646 let qlog = QlogRecvPacket::new(partial_decode.len());
-3647 if let Some(decoded) = packet_crypto::unprotect_header(
-3648 partial_decode,
-3649 &self.spaces,
-3650 self.zero_rtt_crypto.as_ref(),
-3651 self.peer_params.stateless_reset_token,
-3652 ) {
-3653 self.handle_packet(
-3654 now,
-3655 network_path,
-3656 path_id,
-3657 ecn,
-3658 decoded.packet,
-3659 decoded.stateless_reset,
-3660 qlog,
-3661 );
-3662 }
-3663 }
-3664
-3665 fn handle_packet(
-3666 &mut self,
-3667 now: Instant,
-3668 network_path: FourTuple,
-3669 path_id: PathId,
-3670 ecn: Option<EcnCodepoint>,
-3671 packet: Option<Packet>,
-3672 stateless_reset: bool,
-3673 mut qlog: QlogRecvPacket,
-3674 ) {
-3675 self.stats.udp_rx.ios += 1;
-3676 if let Some(ref packet) = packet {
-3677 trace!(
-3678 "got {:?} packet ({} bytes) from {} using id {}",
-3679 packet.header.space(),
-3680 packet.payload.len() + packet.header_data.len(),
-3681 network_path,
-3682 packet.header.dst_cid(),
-3683 );
-3684 }
-3685
-3686 if self.is_handshaking() {
-3687 if path_id != PathId::ZERO {
-3688 debug!(%network_path, %path_id, "discarding multipath packet during handshake");
-3689 return;
-3690 }
-3691 if network_path != self.path_data_mut(path_id).network_path {
-3692 if let Some(hs) = self.state.as_handshake() {
-3693 if hs.allow_server_migration {
-3694 trace!(%network_path, prev = %self.path_data(path_id).network_path, "server migrated to new remote");
-3695 self.path_data_mut(path_id).network_path = network_path;
-3696 self.qlog.emit_tuple_assigned(path_id, network_path, now);
-3697 } else {
-3698 debug!("discarding packet with unexpected remote during handshake");
-3699 return;
-3700 }
-3701 } else {
-3702 debug!("discarding packet with unexpected remote during handshake");
-3703 return;
-3704 }
-3705 }
-3706 }
-3707
-3708 let was_closed = self.state.is_closed();
-3709 let was_drained = self.state.is_drained();
-3710
-3711 let decrypted = match packet {
-3712 None => Err(None),
-3713 Some(mut packet) => self
-3714 .decrypt_packet(now, path_id, &mut packet)
-3715 .map(move |number| (packet, number)),
-3716 };
-3717 let result = match decrypted {
-3718 _ if stateless_reset => {
-3719 debug!("got stateless reset");
-3720 Err(ConnectionError::Reset)
-3721 }
-3722 Err(Some(e)) => {
-3723 warn!("illegal packet: {}", e);
-3724 Err(e.into())
-3725 }
-3726 Err(None) => {
-3727 debug!("failed to authenticate packet");
-3728 self.authentication_failures += 1;
-3729 let integrity_limit = self.spaces[self.highest_space]
-3730 .crypto
-3731 .as_ref()
-3732 .unwrap()
-3733 .packet
-3734 .local
-3735 .integrity_limit();
-3736 if self.authentication_failures > integrity_limit {
-3737 Err(TransportError::AEAD_LIMIT_REACHED("integrity limit violated").into())
-3738 } else {
-3739 return;
-3740 }
-3741 }
-3742 Ok((packet, number)) => {
-3743 qlog.header(&packet.header, number, path_id);
-3744 let span = match number {
-3745 Some(pn) => trace_span!("recv", space = ?packet.header.space(), pn),
-3746 None => trace_span!("recv", space = ?packet.header.space()),
-3747 };
-3748 let _guard = span.enter();
-3749
-3750 let dedup = self.spaces[packet.header.space()]
-3751 .path_space_mut(path_id)
-3752 .map(|pns| &mut pns.dedup);
-3753 if number.zip(dedup).is_some_and(|(n, d)| d.insert(n)) {
-3754 debug!("discarding possible duplicate packet");
-3755 self.qlog.emit_packet_received(qlog, now);
-3756 return;
-3757 } else if self.state.is_handshake() && packet.header.is_short() {
-3758 // TODO: SHOULD buffer these to improve reordering tolerance.
-3759 trace!("dropping short packet during handshake");
-3760 self.qlog.emit_packet_received(qlog, now);
-3761 return;
-3762 } else {
-3763 if let Header::Initial(InitialHeader { ref token, .. }) = packet.header {
-3764 if let Some(hs) = self.state.as_handshake() {
-3765 if self.side.is_server() && token != &hs.expected_token {
-3766 // Clients must send the same retry token in every Initial. Initial
-3767 // packets can be spoofed, so we discard rather than killing the
-3768 // connection.
-3769 warn!("discarding Initial with invalid retry token");
-3770 self.qlog.emit_packet_received(qlog, now);
-3771 return;
-3772 }
-3773 }
-3774 }
-3775
-3776 if !self.state.is_closed() {
-3777 let spin = match packet.header {
-3778 Header::Short { spin, .. } => spin,
-3779 _ => false,
-3780 };
-3781
-3782 if self.side().is_server() && !self.abandoned_paths.contains(&path_id) {
-3783 // Only the client is allowed to open paths
-3784 self.ensure_path(path_id, network_path, now, number);
-3785 }
-3786 if self.paths.contains_key(&path_id) {
-3787 self.on_packet_authenticated(
-3788 now,
-3789 packet.header.space(),
-3790 path_id,
-3791 ecn,
-3792 number,
-3793 spin,
-3794 packet.header.is_1rtt(),
-3795 );
-3796 }
-3797 }
-3798
-3799 let res = self.process_decrypted_packet(
-3800 now,
-3801 network_path,
-3802 path_id,
-3803 number,
-3804 packet,
-3805 &mut qlog,
-3806 );
-3807
-3808 self.qlog.emit_packet_received(qlog, now);
-3809 res
-3810 }
-3811 }
-3812 };
+3587 trace!("0-RTT enabled");
+3588 self.zero_rtt_enabled = true;
+3589 self.zero_rtt_crypto = Some(ZeroRttCrypto { header, packet });
+3590 }
+3591
+3592 fn read_crypto(
+3593 &mut self,
+3594 space: SpaceId,
+3595 crypto: &frame::Crypto,
+3596 payload_len: usize,
+3597 ) -> Result<(), TransportError> {
+3598 let expected = if !self.state.is_handshake() {
+3599 SpaceId::Data
+3600 } else if self.highest_space == SpaceId::Initial {
+3601 SpaceId::Initial
+3602 } else {
+3603 // On the server, self.highest_space can be Data after receiving the client's first
+3604 // flight, but we expect Handshake CRYPTO until the handshake is complete.
+3605 SpaceId::Handshake
+3606 };
+3607 // We can't decrypt Handshake packets when highest_space is Initial, CRYPTO frames in 0-RTT
+3608 // packets are illegal, and we don't process 1-RTT packets until the handshake is
+3609 // complete. Therefore, we will never see CRYPTO data from a later-than-expected space.
+3610 debug_assert!(space <= expected, "received out-of-order CRYPTO data");
+3611
+3612 let end = crypto.offset + crypto.data.len() as u64;
+3613 if space < expected && end > self.spaces[space].crypto_stream.bytes_read() {
+3614 warn!(
+3615 "received new {:?} CRYPTO data when expecting {:?}",
+3616 space, expected
+3617 );
+3618 return Err(TransportError::PROTOCOL_VIOLATION(
+3619 "new data at unexpected encryption level",
+3620 ));
+3621 }
+3622
+3623 let space = &mut self.spaces[space];
+3624 let max = end.saturating_sub(space.crypto_stream.bytes_read());
+3625 if max > self.config.crypto_buffer_size as u64 {
+3626 return Err(TransportError::CRYPTO_BUFFER_EXCEEDED(""));
+3627 }
+3628
+3629 space
+3630 .crypto_stream
+3631 .insert(crypto.offset, crypto.data.clone(), payload_len);
+3632 while let Some(chunk) = space.crypto_stream.read(usize::MAX, true) {
+3633 trace!("consumed {} CRYPTO bytes", chunk.bytes.len());
+3634 if self.crypto.read_handshake(&chunk.bytes)? {
+3635 self.events.push_back(Event::HandshakeDataReady);
+3636 }
+3637 }
+3638
+3639 Ok(())
+3640 }
+3641
+3642 fn write_crypto(&mut self) {
+3643 loop {
+3644 let space = self.highest_space;
+3645 let mut outgoing = Vec::new();
+3646 if let Some(crypto) = self.crypto.write_handshake(&mut outgoing) {
+3647 match space {
+3648 SpaceId::Initial => {
+3649 self.upgrade_crypto(SpaceId::Handshake, crypto);
+3650 }
+3651 SpaceId::Handshake => {
+3652 self.upgrade_crypto(SpaceId::Data, crypto);
+3653 }
+3654 _ => unreachable!("got updated secrets during 1-RTT"),
+3655 }
+3656 }
+3657 if outgoing.is_empty() {
+3658 if space == self.highest_space {
+3659 break;
+3660 } else {
+3661 // Keys updated, check for more data to send
+3662 continue;
+3663 }
+3664 }
+3665 let offset = self.spaces[space].crypto_offset;
+3666 let outgoing = Bytes::from(outgoing);
+3667 if let Some(hs) = self.state.as_handshake_mut() {
+3668 if space == SpaceId::Initial && offset == 0 && self.side.is_client() {
+3669 hs.client_hello = Some(outgoing.clone());
+3670 }
+3671 }
+3672 self.spaces[space].crypto_offset += outgoing.len() as u64;
+3673 trace!("wrote {} {:?} CRYPTO bytes", outgoing.len(), space);
+3674 self.spaces[space].pending.crypto.push_back(frame::Crypto {
+3675 offset,
+3676 data: outgoing,
+3677 });
+3678 }
+3679 }
+3680
+3681 /// Switch to stronger cryptography during handshake
+3682 fn upgrade_crypto(&mut self, space: SpaceId, crypto: Keys) {
+3683 debug_assert!(
+3684 self.spaces[space].crypto.is_none(),
+3685 "already reached packet space {space:?}"
+3686 );
+3687 trace!("{:?} keys ready", space);
+3688 if space == SpaceId::Data {
+3689 // Precompute the first key update
+3690 self.next_crypto = Some(
+3691 self.crypto
+3692 .next_1rtt_keys()
+3693 .expect("handshake should be complete"),
+3694 );
+3695 }
+3696
+3697 self.spaces[space].crypto = Some(crypto);
+3698 debug_assert!(space as usize > self.highest_space as usize);
+3699 self.highest_space = space;
+3700 if space == SpaceId::Data && self.side.is_client() {
+3701 // Discard 0-RTT keys because 1-RTT keys are available.
+3702 self.zero_rtt_crypto = None;
+3703 }
+3704 }
+3705
+3706 fn discard_space(&mut self, now: Instant, space_id: SpaceId) {
+3707 debug_assert!(space_id != SpaceId::Data);
+3708 trace!("discarding {:?} keys", space_id);
+3709 if space_id == SpaceId::Initial {
+3710 // No longer needed
+3711 if let ConnectionSide::Client { token, .. } = &mut self.side {
+3712 *token = Bytes::new();
+3713 }
+3714 }
+3715 let space = &mut self.spaces[space_id];
+3716 space.crypto = None;
+3717 let pns = space.for_path(PathId::ZERO);
+3718 pns.time_of_last_ack_eliciting_packet = None;
+3719 pns.loss_time = None;
+3720 pns.loss_probes = 0;
+3721 let sent_packets = mem::take(&mut pns.sent_packets);
+3722 let path = self.paths.get_mut(&PathId::ZERO).unwrap();
+3723 for (_, packet) in sent_packets.into_iter() {
+3724 path.data.remove_in_flight(&packet);
+3725 }
+3726
+3727 self.set_loss_detection_timer(now, PathId::ZERO)
+3728 }
+3729
+3730 fn handle_coalesced(
+3731 &mut self,
+3732 now: Instant,
+3733 network_path: FourTuple,
+3734 path_id: PathId,
+3735 ecn: Option<EcnCodepoint>,
+3736 data: BytesMut,
+3737 ) {
+3738 self.path_data_mut(path_id)
+3739 .inc_total_recvd(data.len() as u64);
+3740 let mut remaining = Some(data);
+3741 let cid_len = self
+3742 .local_cid_state
+3743 .values()
+3744 .map(|cid_state| cid_state.cid_len())
+3745 .next()
+3746 .expect("one cid_state must exist");
+3747 while let Some(data) = remaining {
+3748 match PartialDecode::new(
+3749 data,
+3750 &FixedLengthConnectionIdParser::new(cid_len),
+3751 &[self.version],
+3752 self.endpoint_config.grease_quic_bit,
+3753 ) {
+3754 Ok((partial_decode, rest)) => {
+3755 remaining = rest;
+3756 self.handle_decode(now, network_path, path_id, ecn, partial_decode);
+3757 }
+3758 Err(e) => {
+3759 trace!("malformed header: {}", e);
+3760 return;
+3761 }
+3762 }
+3763 }
+3764 }
+3765
+3766 fn handle_decode(
+3767 &mut self,
+3768 now: Instant,
+3769 network_path: FourTuple,
+3770 path_id: PathId,
+3771 ecn: Option<EcnCodepoint>,
+3772 partial_decode: PartialDecode,
+3773 ) {
+3774 let qlog = QlogRecvPacket::new(partial_decode.len());
+3775 if let Some(decoded) = packet_crypto::unprotect_header(
+3776 partial_decode,
+3777 &self.spaces,
+3778 self.zero_rtt_crypto.as_ref(),
+3779 self.peer_params.stateless_reset_token,
+3780 ) {
+3781 self.handle_packet(
+3782 now,
+3783 network_path,
+3784 path_id,
+3785 ecn,
+3786 decoded.packet,
+3787 decoded.stateless_reset,
+3788 qlog,
+3789 );
+3790 }
+3791 }
+3792
+3793 fn handle_packet(
+3794 &mut self,
+3795 now: Instant,
+3796 network_path: FourTuple,
+3797 path_id: PathId,
+3798 ecn: Option<EcnCodepoint>,
+3799 packet: Option<Packet>,
+3800 stateless_reset: bool,
+3801 mut qlog: QlogRecvPacket,
+3802 ) {
+3803 self.stats.udp_rx.ios += 1;
+3804 if let Some(ref packet) = packet {
+3805 trace!(
+3806 "got {:?} packet ({} bytes) from {} using id {}",
+3807 packet.header.space(),
+3808 packet.payload.len() + packet.header_data.len(),
+3809 network_path,
+3810 packet.header.dst_cid(),
+3811 );
+3812 }
3813
-3814 // State transitions for error cases
-3815 if let Err(conn_err) = result {
-3816 match conn_err {
-3817 ConnectionError::ApplicationClosed(reason) => self.state.move_to_closed(reason),
-3818 ConnectionError::ConnectionClosed(reason) => self.state.move_to_closed(reason),
-3819 ConnectionError::Reset
-3820 | ConnectionError::TransportError(TransportError {
-3821 code: TransportErrorCode::AEAD_LIMIT_REACHED,
-3822 ..
-3823 }) => {
-3824 self.state.move_to_drained(Some(conn_err));
-3825 }
-3826 ConnectionError::TimedOut => {
-3827 unreachable!("timeouts aren't generated by packet processing");
-3828 }
-3829 ConnectionError::TransportError(err) => {
-3830 debug!("closing connection due to transport error: {}", err);
-3831 self.state.move_to_closed(err);
+3814 if self.is_handshaking() {
+3815 if path_id != PathId::ZERO {
+3816 debug!(%network_path, %path_id, "discarding multipath packet during handshake");
+3817 return;
+3818 }
+3819 if network_path != self.path_data_mut(path_id).network_path {
+3820 if let Some(hs) = self.state.as_handshake() {
+3821 if hs.allow_server_migration {
+3822 trace!(%network_path, prev = %self.path_data(path_id).network_path, "server migrated to new remote");
+3823 self.path_data_mut(path_id).network_path = network_path;
+3824 self.qlog.emit_tuple_assigned(path_id, network_path, now);
+3825 } else {
+3826 debug!("discarding packet with unexpected remote during handshake");
+3827 return;
+3828 }
+3829 } else {
+3830 debug!("discarding packet with unexpected remote during handshake");
+3831 return;
3832 }
-3833 ConnectionError::VersionMismatch => {
-3834 self.state.move_to_draining(Some(conn_err));
-3835 }
-3836 ConnectionError::LocallyClosed => {
-3837 unreachable!("LocallyClosed isn't generated by packet processing");
-3838 }
-3839 ConnectionError::CidsExhausted => {
-3840 unreachable!("CidsExhausted isn't generated by packet processing");
-3841 }
-3842 };
-3843 }
-3844
-3845 if !was_closed && self.state.is_closed() {
-3846 self.close_common();
-3847 if !self.state.is_drained() {
-3848 self.set_close_timer(now);
+3833 }
+3834 }
+3835
+3836 let was_closed = self.state.is_closed();
+3837 let was_drained = self.state.is_drained();
+3838
+3839 let decrypted = match packet {
+3840 None => Err(None),
+3841 Some(mut packet) => self
+3842 .decrypt_packet(now, path_id, &mut packet)
+3843 .map(move |number| (packet, number)),
+3844 };
+3845 let result = match decrypted {
+3846 _ if stateless_reset => {
+3847 debug!("got stateless reset");
+3848 Err(ConnectionError::Reset)
3849 }
-3850 }
-3851 if !was_drained && self.state.is_drained() {
-3852 self.endpoint_events.push_back(EndpointEventInner::Drained);
-3853 // Close timer may have been started previously, e.g. if we sent a close and got a
-3854 // stateless reset in response
-3855 self.timers
-3856 .stop(Timer::Conn(ConnTimer::Close), self.qlog.with_time(now));
-3857 }
-3858
-3859 // Transmit CONNECTION_CLOSE if necessary
-3860 if matches!(self.state.as_type(), StateType::Closed) {
-3861 // If there is no PathData for this PathId the packet was for a brand new
-3862 // path. It was a valid packet however, so the remote is valid and we want to
-3863 // send CONNECTION_CLOSE.
-3864 let path_remote = self
-3865 .paths
-3866 .get(&path_id)
-3867 .map(|p| p.data.network_path)
-3868 .unwrap_or(network_path);
-3869 self.close = network_path == path_remote;
-3870 }
-3871 }
-3872
-3873 fn process_decrypted_packet(
-3874 &mut self,
-3875 now: Instant,
-3876 network_path: FourTuple,
-3877 path_id: PathId,
-3878 number: Option<u64>,
-3879 packet: Packet,
-3880 qlog: &mut QlogRecvPacket,
-3881 ) -> Result<(), ConnectionError> {
-3882 if !self.paths.contains_key(&path_id) {
-3883 // There is a chance this is a server side, first (for this path) packet, which would
-3884 // be a protocol violation. It's more likely, however, that this is a packet of a
-3885 // pruned path
-3886 trace!(%path_id, ?number, "discarding packet for unknown path");
-3887 return Ok(());
-3888 }
-3889 let state = match self.state.as_type() {
-3890 StateType::Established => {
-3891 match packet.header.space() {
-3892 SpaceId::Data => self.process_payload(
-3893 now,
-3894 network_path,
-3895 path_id,
-3896 number.unwrap(),
-3897 packet,
-3898 qlog,
-3899 )?,
-3900 _ if packet.header.has_frames() => {
-3901 self.process_early_payload(now, path_id, packet, qlog)?
-3902 }
-3903 _ => {
-3904 trace!("discarding unexpected pre-handshake packet");
-3905 }
-3906 }
-3907 return Ok(());
-3908 }
-3909 StateType::Closed => {
-3910 for result in frame::Iter::new(packet.payload.freeze())? {
-3911 let frame = match result {
-3912 Ok(frame) => frame,
-3913 Err(err) => {
-3914 debug!("frame decoding error: {err:?}");
-3915 continue;
-3916 }
-3917 };
-3918 qlog.frame(&frame);
-3919
-3920 if let Frame::Padding = frame {
-3921 continue;
-3922 };
-3923
-3924 self.stats.frame_rx.record(frame.ty());
-3925
-3926 if let Frame::Close(_error) = frame {
-3927 self.state.move_to_draining(None);
-3928 break;
-3929 }
-3930 }
-3931 return Ok(());
-3932 }
-3933 StateType::Draining | StateType::Drained => return Ok(()),
-3934 StateType::Handshake => self.state.as_handshake_mut().expect("checked"),
-3935 };
-3936
-3937 match packet.header {
-3938 Header::Retry {
-3939 src_cid: rem_cid, ..
-3940 } => {
-3941 debug_assert_eq!(path_id, PathId::ZERO);
-3942 if self.side.is_server() {
-3943 return Err(TransportError::PROTOCOL_VIOLATION("client sent Retry").into());
-3944 }
-3945
-3946 let is_valid_retry = self
-3947 .rem_cids
-3948 .get(&path_id)
-3949 .map(|cids| cids.active())
-3950 .map(|orig_dst_cid| {
-3951 self.crypto.is_valid_retry(
-3952 orig_dst_cid,
-3953 &packet.header_data,
-3954 &packet.payload,
-3955 )
-3956 })
-3957 .unwrap_or_default();
-3958 if self.total_authed_packets > 1
-3959 || packet.payload.len() <= 16 // token + 16 byte tag
-3960 || !is_valid_retry
-3961 {
-3962 trace!("discarding invalid Retry");
-3963 // - After the client has received and processed an Initial or Retry
-3964 // packet from the server, it MUST discard any subsequent Retry
-3965 // packets that it receives.
-3966 // - A client MUST discard a Retry packet with a zero-length Retry Token
-3967 // field.
-3968 // - Clients MUST discard Retry packets that have a Retry Integrity Tag
-3969 // that cannot be validated
-3970 return Ok(());
-3971 }
+3850 Err(Some(e)) => {
+3851 warn!("illegal packet: {}", e);
+3852 Err(e.into())
+3853 }
+3854 Err(None) => {
+3855 debug!("failed to authenticate packet");
+3856 self.authentication_failures += 1;
+3857 let integrity_limit = self.spaces[self.highest_space]
+3858 .crypto
+3859 .as_ref()
+3860 .unwrap()
+3861 .packet
+3862 .local
+3863 .integrity_limit();
+3864 if self.authentication_failures > integrity_limit {
+3865 Err(TransportError::AEAD_LIMIT_REACHED("integrity limit violated").into())
+3866 } else {
+3867 return;
+3868 }
+3869 }
+3870 Ok((packet, number)) => {
+3871 qlog.header(&packet.header, number, path_id);
+3872 let span = match number {
+3873 Some(pn) => trace_span!("recv", space = ?packet.header.space(), pn),
+3874 None => trace_span!("recv", space = ?packet.header.space()),
+3875 };
+3876 let _guard = span.enter();
+3877
+3878 let dedup = self.spaces[packet.header.space()]
+3879 .path_space_mut(path_id)
+3880 .map(|pns| &mut pns.dedup);
+3881 if number.zip(dedup).is_some_and(|(n, d)| d.insert(n)) {
+3882 debug!("discarding possible duplicate packet");
+3883 self.qlog.emit_packet_received(qlog, now);
+3884 return;
+3885 } else if self.state.is_handshake() && packet.header.is_short() {
+3886 // TODO: SHOULD buffer these to improve reordering tolerance.
+3887 trace!("dropping short packet during handshake");
+3888 self.qlog.emit_packet_received(qlog, now);
+3889 return;
+3890 } else {
+3891 if let Header::Initial(InitialHeader { ref token, .. }) = packet.header {
+3892 if let Some(hs) = self.state.as_handshake() {
+3893 if self.side.is_server() && token != &hs.expected_token {
+3894 // Clients must send the same retry token in every Initial. Initial
+3895 // packets can be spoofed, so we discard rather than killing the
+3896 // connection.
+3897 warn!("discarding Initial with invalid retry token");
+3898 self.qlog.emit_packet_received(qlog, now);
+3899 return;
+3900 }
+3901 }
+3902 }
+3903
+3904 if !self.state.is_closed() {
+3905 let spin = match packet.header {
+3906 Header::Short { spin, .. } => spin,
+3907 _ => false,
+3908 };
+3909
+3910 if self.side().is_server() && !self.abandoned_paths.contains(&path_id) {
+3911 // Only the client is allowed to open paths
+3912 self.ensure_path(path_id, network_path, now, number);
+3913 }
+3914 if self.paths.contains_key(&path_id) {
+3915 self.on_packet_authenticated(
+3916 now,
+3917 packet.header.space(),
+3918 path_id,
+3919 ecn,
+3920 number,
+3921 spin,
+3922 packet.header.is_1rtt(),
+3923 );
+3924 }
+3925 }
+3926
+3927 let res = self.process_decrypted_packet(
+3928 now,
+3929 network_path,
+3930 path_id,
+3931 number,
+3932 packet,
+3933 &mut qlog,
+3934 );
+3935
+3936 self.qlog.emit_packet_received(qlog, now);
+3937 res
+3938 }
+3939 }
+3940 };
+3941
+3942 // State transitions for error cases
+3943 if let Err(conn_err) = result {
+3944 match conn_err {
+3945 ConnectionError::ApplicationClosed(reason) => self.state.move_to_closed(reason),
+3946 ConnectionError::ConnectionClosed(reason) => self.state.move_to_closed(reason),
+3947 ConnectionError::Reset
+3948 | ConnectionError::TransportError(TransportError {
+3949 code: TransportErrorCode::AEAD_LIMIT_REACHED,
+3950 ..
+3951 }) => {
+3952 self.state.move_to_drained(Some(conn_err));
+3953 }
+3954 ConnectionError::TimedOut => {
+3955 unreachable!("timeouts aren't generated by packet processing");
+3956 }
+3957 ConnectionError::TransportError(err) => {
+3958 debug!("closing connection due to transport error: {}", err);
+3959 self.state.move_to_closed(err);
+3960 }
+3961 ConnectionError::VersionMismatch => {
+3962 self.state.move_to_draining(Some(conn_err));
+3963 }
+3964 ConnectionError::LocallyClosed => {
+3965 unreachable!("LocallyClosed isn't generated by packet processing");
+3966 }
+3967 ConnectionError::CidsExhausted => {
+3968 unreachable!("CidsExhausted isn't generated by packet processing");
+3969 }
+3970 };
+3971 }
3972
-3973 trace!("retrying with CID {}", rem_cid);
-3974 let client_hello = state.client_hello.take().unwrap();
-3975 self.retry_src_cid = Some(rem_cid);
-3976 self.rem_cids
-3977 .get_mut(&path_id)
-3978 .expect("PathId::ZERO not yet abandoned, is_valid_retry would have been false")
-3979 .update_initial_cid(rem_cid);
-3980 self.rem_handshake_cid = rem_cid;
-3981
-3982 let space = &mut self.spaces[SpaceId::Initial];
-3983 if let Some(info) = space.for_path(PathId::ZERO).take(0) {
-3984 self.on_packet_acked(now, PathId::ZERO, info);
-3985 };
+3973 if !was_closed && self.state.is_closed() {
+3974 self.close_common();
+3975 if !self.state.is_drained() {
+3976 self.set_close_timer(now);
+3977 }
+3978 }
+3979 if !was_drained && self.state.is_drained() {
+3980 self.endpoint_events.push_back(EndpointEventInner::Drained);
+3981 // Close timer may have been started previously, e.g. if we sent a close and got a
+3982 // stateless reset in response
+3983 self.timers
+3984 .stop(Timer::Conn(ConnTimer::Close), self.qlog.with_time(now));
+3985 }
3986
-3987 self.discard_space(now, SpaceId::Initial); // Make sure we clean up after
-3988 // any retransmitted Initials
-3989 self.spaces[SpaceId::Initial] = {
-3990 let mut space = PacketSpace::new(now, SpaceId::Initial, &mut self.rng);
-3991 space.crypto = Some(self.crypto.initial_keys(rem_cid, self.side.side()));
-3992 space.crypto_offset = client_hello.len() as u64;
-3993 space.for_path(path_id).next_packet_number = self.spaces[SpaceId::Initial]
-3994 .for_path(path_id)
-3995 .next_packet_number;
-3996 space.pending.crypto.push_back(frame::Crypto {
-3997 offset: 0,
-3998 data: client_hello,
-3999 });
-4000 space
-4001 };
-4002
-4003 // Retransmit all 0-RTT data
-4004 let zero_rtt = mem::take(
-4005 &mut self.spaces[SpaceId::Data]
-4006 .for_path(PathId::ZERO)
-4007 .sent_packets,
-4008 );
-4009 for (_, info) in zero_rtt.into_iter() {
-4010 self.paths
-4011 .get_mut(&PathId::ZERO)
-4012 .unwrap()
-4013 .remove_in_flight(&info);
-4014 self.spaces[SpaceId::Data].pending |= info.retransmits;
-4015 }
-4016 self.streams.retransmit_all_for_0rtt();
-4017
-4018 let token_len = packet.payload.len() - 16;
-4019 let ConnectionSide::Client { ref mut token, .. } = self.side else {
-4020 unreachable!("we already short-circuited if we're server");
-4021 };
-4022 *token = packet.payload.freeze().split_to(token_len);
-4023
-4024 self.state = State::handshake(state::Handshake {
-4025 expected_token: Bytes::new(),
-4026 rem_cid_set: false,
-4027 client_hello: None,
-4028 allow_server_migration: true,
-4029 });
-4030 Ok(())
-4031 }
-4032 Header::Long {
-4033 ty: LongType::Handshake,
-4034 src_cid: rem_cid,
-4035 dst_cid: loc_cid,
-4036 ..
-4037 } => {
-4038 debug_assert_eq!(path_id, PathId::ZERO);
-4039 if rem_cid != self.rem_handshake_cid {
-4040 debug!(
-4041 "discarding packet with mismatched remote CID: {} != {}",
-4042 self.rem_handshake_cid, rem_cid
-4043 );
-4044 return Ok(());
-4045 }
-4046 self.on_path_validated(path_id);
+3987 // Transmit CONNECTION_CLOSE if necessary
+3988 if matches!(self.state.as_type(), StateType::Closed) {
+3989 // If there is no PathData for this PathId the packet was for a brand new
+3990 // path. It was a valid packet however, so the remote is valid and we want to
+3991 // send CONNECTION_CLOSE.
+3992 let path_remote = self
+3993 .paths
+3994 .get(&path_id)
+3995 .map(|p| p.data.network_path)
+3996 .unwrap_or(network_path);
+3997 self.close = network_path == path_remote;
+3998 }
+3999 }
+4000
+4001 fn process_decrypted_packet(
+4002 &mut self,
+4003 now: Instant,
+4004 network_path: FourTuple,
+4005 path_id: PathId,
+4006 number: Option<u64>,
+4007 packet: Packet,
+4008 qlog: &mut QlogRecvPacket,
+4009 ) -> Result<(), ConnectionError> {
+4010 if !self.paths.contains_key(&path_id) {
+4011 // There is a chance this is a server side, first (for this path) packet, which would
+4012 // be a protocol violation. It's more likely, however, that this is a packet of a
+4013 // pruned path
+4014 trace!(%path_id, ?number, "discarding packet for unknown path");
+4015 return Ok(());
+4016 }
+4017 let state = match self.state.as_type() {
+4018 StateType::Established => {
+4019 match packet.header.space() {
+4020 SpaceId::Data => self.process_payload(
+4021 now,
+4022 network_path,
+4023 path_id,
+4024 number.unwrap(),
+4025 packet,
+4026 qlog,
+4027 )?,
+4028 _ if packet.header.has_frames() => {
+4029 self.process_early_payload(now, path_id, packet, qlog)?
+4030 }
+4031 _ => {
+4032 trace!("discarding unexpected pre-handshake packet");
+4033 }
+4034 }
+4035 return Ok(());
+4036 }
+4037 StateType::Closed => {
+4038 for result in frame::Iter::new(packet.payload.freeze())? {
+4039 let frame = match result {
+4040 Ok(frame) => frame,
+4041 Err(err) => {
+4042 debug!("frame decoding error: {err:?}");
+4043 continue;
+4044 }
+4045 };
+4046 qlog.frame(&frame);
4047
-4048 self.process_early_payload(now, path_id, packet, qlog)?;
-4049 if self.state.is_closed() {
-4050 return Ok(());
-4051 }
-4052
-4053 if self.crypto.is_handshaking() {
-4054 trace!("handshake ongoing");
-4055 return Ok(());
-4056 }
-4057
-4058 if self.side.is_client() {
-4059 // Client-only because server params were set from the client's Initial
-4060 let params = self.crypto.transport_parameters()?.ok_or_else(|| {
-4061 TransportError::new(
-4062 TransportErrorCode::crypto(0x6d),
-4063 "transport parameters missing".to_owned(),
-4064 )
-4065 })?;
-4066
-4067 if self.has_0rtt() {
-4068 if !self.crypto.early_data_accepted().unwrap() {
-4069 debug_assert!(self.side.is_client());
-4070 debug!("0-RTT rejected");
-4071 self.accepted_0rtt = false;
-4072 self.streams.zero_rtt_rejected();
+4048 if let Frame::Padding = frame {
+4049 continue;
+4050 };
+4051
+4052 self.stats.frame_rx.record(frame.ty());
+4053
+4054 if let Frame::Close(_error) = frame {
+4055 self.state.move_to_draining(None);
+4056 break;
+4057 }
+4058 }
+4059 return Ok(());
+4060 }
+4061 StateType::Draining | StateType::Drained => return Ok(()),
+4062 StateType::Handshake => self.state.as_handshake_mut().expect("checked"),
+4063 };
+4064
+4065 match packet.header {
+4066 Header::Retry {
+4067 src_cid: rem_cid, ..
+4068 } => {
+4069 debug_assert_eq!(path_id, PathId::ZERO);
+4070 if self.side.is_server() {
+4071 return Err(TransportError::PROTOCOL_VIOLATION("client sent Retry").into());
+4072 }
4073
-4074 // Discard already-queued frames
-4075 self.spaces[SpaceId::Data].pending = Retransmits::default();
-4076
-4077 // Discard 0-RTT packets
-4078 let sent_packets = mem::take(
-4079 &mut self.spaces[SpaceId::Data].for_path(path_id).sent_packets,
-4080 );
-4081 for (_, packet) in sent_packets.into_iter() {
-4082 self.paths
-4083 .get_mut(&path_id)
-4084 .unwrap()
-4085 .remove_in_flight(&packet);
-4086 }
-4087 } else {
-4088 self.accepted_0rtt = true;
-4089 params.validate_resumption_from(&self.peer_params)?;
-4090 }
-4091 }
-4092 if let Some(token) = params.stateless_reset_token {
-4093 // TODO(matheus23): Reset token for a remote, or for a 4-tuple?
-4094 let remote = self.path_data(path_id).network_path.remote;
-4095 self.endpoint_events
-4096 .push_back(EndpointEventInner::ResetToken(path_id, remote, token));
-4097 }
-4098 self.handle_peer_params(params, loc_cid, rem_cid, now)?;
-4099 self.issue_first_cids(now);
-4100 } else {
-4101 // Server-only
-4102 self.spaces[SpaceId::Data].pending.handshake_done = true;
-4103 self.discard_space(now, SpaceId::Handshake);
-4104 self.events.push_back(Event::HandshakeConfirmed);
-4105 trace!("handshake confirmed");
-4106 }
-4107
-4108 self.events.push_back(Event::Connected);
-4109 self.state.move_to_established();
-4110 trace!("established");
-4111
-4112 // Multipath can only be enabled after the state has reached Established.
-4113 // So this can not happen any earlier.
-4114 self.issue_first_path_cids(now);
-4115 Ok(())
-4116 }
-4117 Header::Initial(InitialHeader {
-4118 src_cid: rem_cid,
-4119 dst_cid: loc_cid,
-4120 ..
-4121 }) => {
-4122 debug_assert_eq!(path_id, PathId::ZERO);
-4123 if !state.rem_cid_set {
-4124 trace!("switching remote CID to {}", rem_cid);
-4125 let mut state = state.clone();
-4126 self.rem_cids
-4127 .get_mut(&path_id)
-4128 .expect("PathId::ZERO not yet abandoned")
-4129 .update_initial_cid(rem_cid);
-4130 self.rem_handshake_cid = rem_cid;
-4131 self.orig_rem_cid = rem_cid;
-4132 state.rem_cid_set = true;
-4133 self.state.move_to_handshake(state);
-4134 } else if rem_cid != self.rem_handshake_cid {
-4135 debug!(
-4136 "discarding packet with mismatched remote CID: {} != {}",
-4137 self.rem_handshake_cid, rem_cid
-4138 );
-4139 return Ok(());
-4140 }
-4141
-4142 let starting_space = self.highest_space;
-4143 self.process_early_payload(now, path_id, packet, qlog)?;
-4144
-4145 if self.side.is_server()
-4146 && starting_space == SpaceId::Initial
-4147 && self.highest_space != SpaceId::Initial
-4148 {
-4149 let params = self.crypto.transport_parameters()?.ok_or_else(|| {
-4150 TransportError::new(
-4151 TransportErrorCode::crypto(0x6d),
-4152 "transport parameters missing".to_owned(),
-4153 )
-4154 })?;
-4155 self.handle_peer_params(params, loc_cid, rem_cid, now)?;
-4156 self.issue_first_cids(now);
-4157 self.init_0rtt(now);
-4158 }
-4159 Ok(())
-4160 }
-4161 Header::Long {
-4162 ty: LongType::ZeroRtt,
-4163 ..
-4164 } => {
-4165 self.process_payload(now, network_path, path_id, number.unwrap(), packet, qlog)?;
-4166 Ok(())
-4167 }
-4168 Header::VersionNegotiate { .. } => {
-4169 if self.total_authed_packets > 1 {
-4170 return Ok(());
-4171 }
-4172 let supported = packet
-4173 .payload
-4174 .chunks(4)
-4175 .any(|x| match <[u8; 4]>::try_from(x) {
-4176 Ok(version) => self.version == u32::from_be_bytes(version),
-4177 Err(_) => false,
-4178 });
-4179 if supported {
-4180 return Ok(());
-4181 }
-4182 debug!("remote doesn't support our version");
-4183 Err(ConnectionError::VersionMismatch)
-4184 }
-4185 Header::Short { .. } => unreachable!(
-4186 "short packets received during handshake are discarded in handle_packet"
-4187 ),
-4188 }
-4189 }
-4190
-4191 /// Process an Initial or Handshake packet payload
-4192 fn process_early_payload(
-4193 &mut self,
-4194 now: Instant,
-4195 path_id: PathId,
-4196 packet: Packet,
-4197 #[allow(unused)] qlog: &mut QlogRecvPacket,
-4198 ) -> Result<(), TransportError> {
-4199 debug_assert_ne!(packet.header.space(), SpaceId::Data);
-4200 debug_assert_eq!(path_id, PathId::ZERO);
-4201 let payload_len = packet.payload.len();
-4202 let mut ack_eliciting = false;
-4203 for result in frame::Iter::new(packet.payload.freeze())? {
-4204 let frame = result?;
-4205 qlog.frame(&frame);
-4206 let span = match frame {
-4207 Frame::Padding => continue,
-4208 _ => Some(trace_span!("frame", ty = %frame.ty(), path = tracing::field::Empty)),
-4209 };
-4210
-4211 self.stats.frame_rx.record(frame.ty());
-4212
-4213 let _guard = span.as_ref().map(|x| x.enter());
-4214 ack_eliciting |= frame.is_ack_eliciting();
-4215
-4216 // Process frames
-4217 if frame.is_1rtt() && packet.header.space() != SpaceId::Data {
-4218 return Err(TransportError::PROTOCOL_VIOLATION(
-4219 "illegal frame type in handshake",
-4220 ));
-4221 }
-4222
-4223 match frame {
-4224 Frame::Padding | Frame::Ping => {}
-4225 Frame::Crypto(frame) => {
-4226 self.read_crypto(packet.header.space(), &frame, payload_len)?;
-4227 }
-4228 Frame::Ack(ack) => {
-4229 self.on_ack_received(now, packet.header.space(), ack)?;
-4230 }
-4231 Frame::PathAck(ack) => {
-4232 span.as_ref()
-4233 .map(|span| span.record("path", tracing::field::debug(&ack.path_id)));
-4234 self.on_path_ack_received(now, packet.header.space(), ack)?;
-4235 }
-4236 Frame::Close(reason) => {
-4237 self.state.move_to_draining(Some(reason.into()));
-4238 return Ok(());
-4239 }
-4240 _ => {
-4241 let mut err =
-4242 TransportError::PROTOCOL_VIOLATION("illegal frame type in handshake");
-4243 err.frame = frame::MaybeFrame::Known(frame.ty());
-4244 return Err(err);
-4245 }
-4246 }
-4247 }
-4248
-4249 if ack_eliciting {
-4250 // In the initial and handshake spaces, ACKs must be sent immediately
-4251 self.spaces[packet.header.space()]
-4252 .for_path(path_id)
-4253 .pending_acks
-4254 .set_immediate_ack_required();
-4255 }
-4256
-4257 self.write_crypto();
-4258 Ok(())
-4259 }
-4260
-4261 /// Processes the packet payload, always in the data space.
-4262 fn process_payload(
-4263 &mut self,
-4264 now: Instant,
-4265 network_path: FourTuple,
-4266 path_id: PathId,
-4267 number: u64,
-4268 packet: Packet,
-4269 #[allow(unused)] qlog: &mut QlogRecvPacket,
-4270 ) -> Result<(), TransportError> {
-4271 let payload = packet.payload.freeze();
-4272 let mut is_probing_packet = true;
-4273 let mut close = None;
-4274 let payload_len = payload.len();
-4275 let mut ack_eliciting = false;
-4276 // if this packet triggers a path migration and includes a observed address frame, it's
-4277 // stored here
-4278 let mut migration_observed_addr = None;
-4279 for result in frame::Iter::new(payload)? {
-4280 let frame = result?;
-4281 qlog.frame(&frame);
-4282 let span = match frame {
-4283 Frame::Padding => continue,
-4284 _ => trace_span!("frame", ty = %frame.ty(), path = tracing::field::Empty),
-4285 };
-4286
-4287 self.stats.frame_rx.record(frame.ty());
-4288 // Crypto, Stream and Datagram frames are special cased in order no pollute
-4289 // the log with payload data
-4290 match &frame {
-4291 Frame::Crypto(f) => {
-4292 trace!(offset = f.offset, len = f.data.len(), "got crypto frame");
-4293 }
-4294 Frame::Stream(f) => {
-4295 trace!(id = %f.id, offset = f.offset, len = f.data.len(), fin = f.fin, "got stream frame");
-4296 }
-4297 Frame::Datagram(f) => {
-4298 trace!(len = f.data.len(), "got datagram frame");
+4074 let is_valid_retry = self
+4075 .rem_cids
+4076 .get(&path_id)
+4077 .map(|cids| cids.active())
+4078 .map(|orig_dst_cid| {
+4079 self.crypto.is_valid_retry(
+4080 orig_dst_cid,
+4081 &packet.header_data,
+4082 &packet.payload,
+4083 )
+4084 })
+4085 .unwrap_or_default();
+4086 if self.total_authed_packets > 1
+4087 || packet.payload.len() <= 16 // token + 16 byte tag
+4088 || !is_valid_retry
+4089 {
+4090 trace!("discarding invalid Retry");
+4091 // - After the client has received and processed an Initial or Retry
+4092 // packet from the server, it MUST discard any subsequent Retry
+4093 // packets that it receives.
+4094 // - A client MUST discard a Retry packet with a zero-length Retry Token
+4095 // field.
+4096 // - Clients MUST discard Retry packets that have a Retry Integrity Tag
+4097 // that cannot be validated
+4098 return Ok(());
+4099 }
+4100
+4101 trace!("retrying with CID {}", rem_cid);
+4102 let client_hello = state.client_hello.take().unwrap();
+4103 self.retry_src_cid = Some(rem_cid);
+4104 self.rem_cids
+4105 .get_mut(&path_id)
+4106 .expect("PathId::ZERO not yet abandoned, is_valid_retry would have been false")
+4107 .update_initial_cid(rem_cid);
+4108 self.rem_handshake_cid = rem_cid;
+4109
+4110 let space = &mut self.spaces[SpaceId::Initial];
+4111 if let Some(info) = space.for_path(PathId::ZERO).take(0) {
+4112 self.on_packet_acked(now, PathId::ZERO, info);
+4113 };
+4114
+4115 self.discard_space(now, SpaceId::Initial); // Make sure we clean up after
+4116 // any retransmitted Initials
+4117 self.spaces[SpaceId::Initial] = {
+4118 let mut space = PacketSpace::new(now, SpaceId::Initial, &mut self.rng);
+4119 space.crypto = Some(self.crypto.initial_keys(rem_cid, self.side.side()));
+4120 space.crypto_offset = client_hello.len() as u64;
+4121 space.for_path(path_id).next_packet_number = self.spaces[SpaceId::Initial]
+4122 .for_path(path_id)
+4123 .next_packet_number;
+4124 space.pending.crypto.push_back(frame::Crypto {
+4125 offset: 0,
+4126 data: client_hello,
+4127 });
+4128 space
+4129 };
+4130
+4131 // Retransmit all 0-RTT data
+4132 let zero_rtt = mem::take(
+4133 &mut self.spaces[SpaceId::Data]
+4134 .for_path(PathId::ZERO)
+4135 .sent_packets,
+4136 );
+4137 for (_, info) in zero_rtt.into_iter() {
+4138 self.paths
+4139 .get_mut(&PathId::ZERO)
+4140 .unwrap()
+4141 .remove_in_flight(&info);
+4142 self.spaces[SpaceId::Data].pending |= info.retransmits;
+4143 }
+4144 self.streams.retransmit_all_for_0rtt();
+4145
+4146 let token_len = packet.payload.len() - 16;
+4147 let ConnectionSide::Client { ref mut token, .. } = self.side else {
+4148 unreachable!("we already short-circuited if we're server");
+4149 };
+4150 *token = packet.payload.freeze().split_to(token_len);
+4151
+4152 self.state = State::handshake(state::Handshake {
+4153 expected_token: Bytes::new(),
+4154 rem_cid_set: false,
+4155 client_hello: None,
+4156 allow_server_migration: true,
+4157 });
+4158 Ok(())
+4159 }
+4160 Header::Long {
+4161 ty: LongType::Handshake,
+4162 src_cid: rem_cid,
+4163 dst_cid: loc_cid,
+4164 ..
+4165 } => {
+4166 debug_assert_eq!(path_id, PathId::ZERO);
+4167 if rem_cid != self.rem_handshake_cid {
+4168 debug!(
+4169 "discarding packet with mismatched remote CID: {} != {}",
+4170 self.rem_handshake_cid, rem_cid
+4171 );
+4172 return Ok(());
+4173 }
+4174 self.on_path_validated(path_id);
+4175
+4176 self.process_early_payload(now, path_id, packet, qlog)?;
+4177 if self.state.is_closed() {
+4178 return Ok(());
+4179 }
+4180
+4181 if self.crypto.is_handshaking() {
+4182 trace!("handshake ongoing");
+4183 return Ok(());
+4184 }
+4185
+4186 if self.side.is_client() {
+4187 // Client-only because server params were set from the client's Initial
+4188 let params = self.crypto.transport_parameters()?.ok_or_else(|| {
+4189 TransportError::new(
+4190 TransportErrorCode::crypto(0x6d),
+4191 "transport parameters missing".to_owned(),
+4192 )
+4193 })?;
+4194
+4195 if self.has_0rtt() {
+4196 if !self.crypto.early_data_accepted().unwrap() {
+4197 debug_assert!(self.side.is_client());
+4198 debug!("0-RTT rejected");
+4199 self.accepted_0rtt = false;
+4200 self.streams.zero_rtt_rejected();
+4201
+4202 // Discard already-queued frames
+4203 self.spaces[SpaceId::Data].pending = Retransmits::default();
+4204
+4205 // Discard 0-RTT packets
+4206 let sent_packets = mem::take(
+4207 &mut self.spaces[SpaceId::Data].for_path(path_id).sent_packets,
+4208 );
+4209 for (_, packet) in sent_packets.into_iter() {
+4210 self.paths
+4211 .get_mut(&path_id)
+4212 .unwrap()
+4213 .remove_in_flight(&packet);
+4214 }
+4215 } else {
+4216 self.accepted_0rtt = true;
+4217 params.validate_resumption_from(&self.peer_params)?;
+4218 }
+4219 }
+4220 if let Some(token) = params.stateless_reset_token {
+4221 // TODO(matheus23): Reset token for a remote, or for a 4-tuple?
+4222 let remote = self.path_data(path_id).network_path.remote;
+4223 self.endpoint_events
+4224 .push_back(EndpointEventInner::ResetToken(path_id, remote, token));
+4225 }
+4226 self.handle_peer_params(params, loc_cid, rem_cid, now)?;
+4227 self.issue_first_cids(now);
+4228 } else {
+4229 // Server-only
+4230 self.spaces[SpaceId::Data].pending.handshake_done = true;
+4231 self.discard_space(now, SpaceId::Handshake);
+4232 self.events.push_back(Event::HandshakeConfirmed);
+4233 trace!("handshake confirmed");
+4234 }
+4235
+4236 self.events.push_back(Event::Connected);
+4237 self.state.move_to_established();
+4238 trace!("established");
+4239
+4240 // Multipath can only be enabled after the state has reached Established.
+4241 // So this can not happen any earlier.
+4242 self.issue_first_path_cids(now);
+4243 Ok(())
+4244 }
+4245 Header::Initial(InitialHeader {
+4246 src_cid: rem_cid,
+4247 dst_cid: loc_cid,
+4248 ..
+4249 }) => {
+4250 debug_assert_eq!(path_id, PathId::ZERO);
+4251 if !state.rem_cid_set {
+4252 trace!("switching remote CID to {}", rem_cid);
+4253 let mut state = state.clone();
+4254 self.rem_cids
+4255 .get_mut(&path_id)
+4256 .expect("PathId::ZERO not yet abandoned")
+4257 .update_initial_cid(rem_cid);
+4258 self.rem_handshake_cid = rem_cid;
+4259 self.orig_rem_cid = rem_cid;
+4260 state.rem_cid_set = true;
+4261 self.state.move_to_handshake(state);
+4262 } else if rem_cid != self.rem_handshake_cid {
+4263 debug!(
+4264 "discarding packet with mismatched remote CID: {} != {}",
+4265 self.rem_handshake_cid, rem_cid
+4266 );
+4267 return Ok(());
+4268 }
+4269
+4270 let starting_space = self.highest_space;
+4271 self.process_early_payload(now, path_id, packet, qlog)?;
+4272
+4273 if self.side.is_server()
+4274 && starting_space == SpaceId::Initial
+4275 && self.highest_space != SpaceId::Initial
+4276 {
+4277 let params = self.crypto.transport_parameters()?.ok_or_else(|| {
+4278 TransportError::new(
+4279 TransportErrorCode::crypto(0x6d),
+4280 "transport parameters missing".to_owned(),
+4281 )
+4282 })?;
+4283 self.handle_peer_params(params, loc_cid, rem_cid, now)?;
+4284 self.issue_first_cids(now);
+4285 self.init_0rtt(now);
+4286 }
+4287 Ok(())
+4288 }
+4289 Header::Long {
+4290 ty: LongType::ZeroRtt,
+4291 ..
+4292 } => {
+4293 self.process_payload(now, network_path, path_id, number.unwrap(), packet, qlog)?;
+4294 Ok(())
+4295 }
+4296 Header::VersionNegotiate { .. } => {
+4297 if self.total_authed_packets > 1 {
+4298 return Ok(());
4299 }
-4300 f => {
-4301 trace!("got frame {f}");
-4302 }
-4303 }
-4304
-4305 let _guard = span.enter();
-4306 if packet.header.is_0rtt() {
-4307 match frame {
-4308 Frame::Crypto(_) | Frame::Close(Close::Application(_)) => {
-4309 return Err(TransportError::PROTOCOL_VIOLATION(
-4310 "illegal frame type in 0-RTT",
-4311 ));
-4312 }
-4313 _ => {
-4314 if frame.is_1rtt() {
-4315 return Err(TransportError::PROTOCOL_VIOLATION(
-4316 "illegal frame type in 0-RTT",
-4317 ));
-4318 }
-4319 }
-4320 }
-4321 }
-4322 ack_eliciting |= frame.is_ack_eliciting();
-4323
-4324 // Check whether this could be a probing packet
-4325 match frame {
-4326 Frame::Padding
-4327 | Frame::PathChallenge(_)
-4328 | Frame::PathResponse(_)
-4329 | Frame::NewConnectionId(_)
-4330 | Frame::ObservedAddr(_) => {}
-4331 _ => {
-4332 is_probing_packet = false;
-4333 }
-4334 }
-4335
-4336 match frame {
-4337 Frame::Crypto(frame) => {
-4338 self.read_crypto(SpaceId::Data, &frame, payload_len)?;
-4339 }
-4340 Frame::Stream(frame) => {
-4341 if self.streams.received(frame, payload_len)?.should_transmit() {
-4342 self.spaces[SpaceId::Data].pending.max_data = true;
-4343 }
-4344 }
-4345 Frame::Ack(ack) => {
-4346 self.on_ack_received(now, SpaceId::Data, ack)?;
-4347 }
-4348 Frame::PathAck(ack) => {
-4349 span.record("path", tracing::field::debug(&ack.path_id));
-4350 self.on_path_ack_received(now, SpaceId::Data, ack)?;
-4351 }
+4300 let supported = packet
+4301 .payload
+4302 .chunks(4)
+4303 .any(|x| match <[u8; 4]>::try_from(x) {
+4304 Ok(version) => self.version == u32::from_be_bytes(version),
+4305 Err(_) => false,
+4306 });
+4307 if supported {
+4308 return Ok(());
+4309 }
+4310 debug!("remote doesn't support our version");
+4311 Err(ConnectionError::VersionMismatch)
+4312 }
+4313 Header::Short { .. } => unreachable!(
+4314 "short packets received during handshake are discarded in handle_packet"
+4315 ),
+4316 }
+4317 }
+4318
+4319 /// Process an Initial or Handshake packet payload
+4320 fn process_early_payload(
+4321 &mut self,
+4322 now: Instant,
+4323 path_id: PathId,
+4324 packet: Packet,
+4325 #[allow(unused)] qlog: &mut QlogRecvPacket,
+4326 ) -> Result<(), TransportError> {
+4327 debug_assert_ne!(packet.header.space(), SpaceId::Data);
+4328 debug_assert_eq!(path_id, PathId::ZERO);
+4329 let payload_len = packet.payload.len();
+4330 let mut ack_eliciting = false;
+4331 for result in frame::Iter::new(packet.payload.freeze())? {
+4332 let frame = result?;
+4333 qlog.frame(&frame);
+4334 let span = match frame {
+4335 Frame::Padding => continue,
+4336 _ => Some(trace_span!("frame", ty = %frame.ty(), path = tracing::field::Empty)),
+4337 };
+4338
+4339 self.stats.frame_rx.record(frame.ty());
+4340
+4341 let _guard = span.as_ref().map(|x| x.enter());
+4342 ack_eliciting |= frame.is_ack_eliciting();
+4343
+4344 // Process frames
+4345 if frame.is_1rtt() && packet.header.space() != SpaceId::Data {
+4346 return Err(TransportError::PROTOCOL_VIOLATION(
+4347 "illegal frame type in handshake",
+4348 ));
+4349 }
+4350
+4351 match frame {
4352 Frame::Padding | Frame::Ping => {}
-4353 Frame::Close(reason) => {
-4354 close = Some(reason);
+4353 Frame::Crypto(frame) => {
+4354 self.read_crypto(packet.header.space(), &frame, payload_len)?;
4355 }
-4356 Frame::PathChallenge(challenge) => {
-4357 let path = &mut self
-4358 .path_mut(path_id)
-4359 .expect("payload is processed only after the path becomes known");
-4360 path.path_responses.push(number, challenge.0, network_path);
-4361 // At this point, update_network_path_or_discard was already called, so
-4362 // we don't need to be lenient about `local_ip` possibly mis-matching.
-4363 if network_path == path.network_path {
-4364 // PATH_CHALLENGE on active path, possible off-path packet forwarding
-4365 // attack. Send a non-probing packet to recover the active path.
-4366 // TODO(flub): No longer true! We now path_challege also to validate
-4367 // the path if the path is new, without an RFC9000-style
-4368 // migration involved. This means we add in an extra
-4369 // IMMEDIATE_ACK on some challenges. It isn't really wrong to do
-4370 // so, but it still is something untidy. We should instead
-4371 // suppress this when we know the remote is still validating the
-4372 // path.
-4373 match self.peer_supports_ack_frequency() {
-4374 true => self.immediate_ack(path_id),
-4375 false => {
-4376 self.ping_path(path_id).ok();
-4377 }
-4378 }
-4379 }
-4380 }
-4381 Frame::PathResponse(response) => {
-4382 let path = self
-4383 .paths
-4384 .get_mut(&path_id)
-4385 .expect("payload is processed only after the path becomes known");
-4386
-4387 use PathTimer::*;
-4388 use paths::OnPathResponseReceived::*;
-4389 match path
-4390 .data
-4391 .on_path_response_received(now, response.0, network_path)
-4392 {
-4393 OnPath { was_open } => {
-4394 let qlog = self.qlog.with_time(now);
-4395
-4396 self.timers
-4397 .stop(Timer::PerPath(path_id, PathValidation), qlog.clone());
-4398 self.timers
-4399 .stop(Timer::PerPath(path_id, PathOpen), qlog.clone());
-4400
-4401 let next_challenge = path
-4402 .data
-4403 .earliest_expiring_challenge()
-4404 .map(|time| time + self.ack_frequency.max_ack_delay_for_pto());
-4405 self.timers.set_or_stop(
-4406 Timer::PerPath(path_id, PathChallengeLost),
-4407 next_challenge,
-4408 qlog,
-4409 );
-4410
-4411 if !was_open {
-4412 self.events
-4413 .push_back(Event::Path(PathEvent::Opened { id: path_id }));
-4414 if let Some(observed) = path.data.last_observed_addr_report.as_ref()
-4415 {
-4416 self.events.push_back(Event::Path(PathEvent::ObservedAddr {
-4417 id: path_id,
-4418 addr: observed.socket_addr(),
-4419 }));
-4420 }
-4421 }
-4422 if let Some((_, ref mut prev)) = path.prev {
-4423 prev.challenges_sent.clear();
-4424 prev.send_new_challenge = false;
-4425 }
-4426 }
-4427 OffPath => {
-4428 debug!("Response to off-path PathChallenge!");
-4429 let next_challenge = path
-4430 .data
-4431 .earliest_expiring_challenge()
-4432 .map(|time| time + self.ack_frequency.max_ack_delay_for_pto());
-4433 self.timers.set_or_stop(
-4434 Timer::PerPath(path_id, PathChallengeLost),
-4435 next_challenge,
-4436 self.qlog.with_time(now),
-4437 );
-4438 }
-4439 Invalid { expected } => {
-4440 debug!(%response, %network_path, %expected, "ignoring invalid PATH_RESPONSE")
-4441 }
-4442 Unknown => debug!(%response, "ignoring invalid PATH_RESPONSE"),
-4443 }
-4444 }
-4445 Frame::MaxData(frame::MaxData(bytes)) => {
-4446 self.streams.received_max_data(bytes);
-4447 }
-4448 Frame::MaxStreamData(frame::MaxStreamData { id, offset }) => {
-4449 self.streams.received_max_stream_data(id, offset)?;
-4450 }
-4451 Frame::MaxStreams(frame::MaxStreams { dir, count }) => {
-4452 self.streams.received_max_streams(dir, count)?;
-4453 }
-4454 Frame::ResetStream(frame) => {
-4455 if self.streams.received_reset(frame)?.should_transmit() {
-4456 self.spaces[SpaceId::Data].pending.max_data = true;
-4457 }
-4458 }
-4459 Frame::DataBlocked { offset } => {
-4460 debug!(offset, "peer claims to be blocked at connection level");
+4356 Frame::Ack(ack) => {
+4357 self.on_ack_received(now, packet.header.space(), ack)?;
+4358 }
+4359 Frame::PathAck(ack) => {
+4360 span.as_ref()
+4361 .map(|span| span.record("path", tracing::field::debug(&ack.path_id)));
+4362 self.on_path_ack_received(now, packet.header.space(), ack)?;
+4363 }
+4364 Frame::Close(reason) => {
+4365 self.state.move_to_draining(Some(reason.into()));
+4366 return Ok(());
+4367 }
+4368 _ => {
+4369 let mut err =
+4370 TransportError::PROTOCOL_VIOLATION("illegal frame type in handshake");
+4371 err.frame = frame::MaybeFrame::Known(frame.ty());
+4372 return Err(err);
+4373 }
+4374 }
+4375 }
+4376
+4377 if ack_eliciting {
+4378 // In the initial and handshake spaces, ACKs must be sent immediately
+4379 self.spaces[packet.header.space()]
+4380 .for_path(path_id)
+4381 .pending_acks
+4382 .set_immediate_ack_required();
+4383 }
+4384
+4385 self.write_crypto();
+4386 Ok(())
+4387 }
+4388
+4389 /// Processes the packet payload, always in the data space.
+4390 fn process_payload(
+4391 &mut self,
+4392 now: Instant,
+4393 network_path: FourTuple,
+4394 path_id: PathId,
+4395 number: u64,
+4396 packet: Packet,
+4397 #[allow(unused)] qlog: &mut QlogRecvPacket,
+4398 ) -> Result<(), TransportError> {
+4399 let payload = packet.payload.freeze();
+4400 let mut is_probing_packet = true;
+4401 let mut close = None;
+4402 let payload_len = payload.len();
+4403 let mut ack_eliciting = false;
+4404 // if this packet triggers a path migration and includes a observed address frame, it's
+4405 // stored here
+4406 let mut migration_observed_addr = None;
+4407 for result in frame::Iter::new(payload)? {
+4408 let frame = result?;
+4409 qlog.frame(&frame);
+4410 let span = match frame {
+4411 Frame::Padding => continue,
+4412 _ => trace_span!("frame", ty = %frame.ty(), path = tracing::field::Empty),
+4413 };
+4414
+4415 self.stats.frame_rx.record(frame.ty());
+4416 // Crypto, Stream and Datagram frames are special cased in order no pollute
+4417 // the log with payload data
+4418 match &frame {
+4419 Frame::Crypto(f) => {
+4420 trace!(offset = f.offset, len = f.data.len(), "got crypto frame");
+4421 }
+4422 Frame::Stream(f) => {
+4423 trace!(id = %f.id, offset = f.offset, len = f.data.len(), fin = f.fin, "got stream frame");
+4424 }
+4425 Frame::Datagram(f) => {
+4426 trace!(len = f.data.len(), "got datagram frame");
+4427 }
+4428 f => {
+4429 trace!("got frame {f}");
+4430 }
+4431 }
+4432
+4433 let _guard = span.enter();
+4434 if packet.header.is_0rtt() {
+4435 match frame {
+4436 Frame::Crypto(_) | Frame::Close(Close::Application(_)) => {
+4437 return Err(TransportError::PROTOCOL_VIOLATION(
+4438 "illegal frame type in 0-RTT",
+4439 ));
+4440 }
+4441 _ => {
+4442 if frame.is_1rtt() {
+4443 return Err(TransportError::PROTOCOL_VIOLATION(
+4444 "illegal frame type in 0-RTT",
+4445 ));
+4446 }
+4447 }
+4448 }
+4449 }
+4450 ack_eliciting |= frame.is_ack_eliciting();
+4451
+4452 // Check whether this could be a probing packet
+4453 match frame {
+4454 Frame::Padding
+4455 | Frame::PathChallenge(_)
+4456 | Frame::PathResponse(_)
+4457 | Frame::NewConnectionId(_)
+4458 | Frame::ObservedAddr(_) => {}
+4459 _ => {
+4460 is_probing_packet = false;
4461 }
-4462 Frame::StreamDataBlocked { id, offset } => {
-4463 if id.initiator() == self.side.side() && id.dir() == Dir::Uni {
-4464 debug!("got STREAM_DATA_BLOCKED on send-only {}", id);
-4465 return Err(TransportError::STREAM_STATE_ERROR(
-4466 "STREAM_DATA_BLOCKED on send-only stream",
-4467 ));
-4468 }
-4469 debug!(
-4470 stream = %id,
-4471 offset, "peer claims to be blocked at stream level"
-4472 );
-4473 }
-4474 Frame::StreamsBlocked { dir, limit } => {
-4475 if limit > MAX_STREAM_COUNT {
-4476 return Err(TransportError::FRAME_ENCODING_ERROR(
-4477 "unrepresentable stream limit",
-4478 ));
-4479 }
-4480 debug!(
-4481 "peer claims to be blocked opening more than {} {} streams",
-4482 limit, dir
-4483 );
-4484 }
-4485 Frame::StopSending(frame::StopSending { id, error_code }) => {
-4486 if id.initiator() != self.side.side() {
-4487 if id.dir() == Dir::Uni {
-4488 debug!("got STOP_SENDING on recv-only {}", id);
-4489 return Err(TransportError::STREAM_STATE_ERROR(
-4490 "STOP_SENDING on recv-only stream",
-4491 ));
-4492 }
-4493 } else if self.streams.is_local_unopened(id) {
-4494 return Err(TransportError::STREAM_STATE_ERROR(
-4495 "STOP_SENDING on unopened stream",
-4496 ));
-4497 }
-4498 self.streams.received_stop_sending(id, error_code);
-4499 }
-4500 Frame::RetireConnectionId(frame::RetireConnectionId { path_id, sequence }) => {
-4501 if let Some(ref path_id) = path_id {
-4502 span.record("path", tracing::field::debug(&path_id));
-4503 }
-4504 let path_id = path_id.unwrap_or_default();
-4505 match self.local_cid_state.get_mut(&path_id) {
-4506 None => error!(?path_id, "RETIRE_CONNECTION_ID for unknown path"),
-4507 Some(cid_state) => {
-4508 let allow_more_cids = cid_state
-4509 .on_cid_retirement(sequence, self.peer_params.issue_cids_limit())?;
-4510
-4511 // If the path has closed, we do not issue more CIDs for this path
-4512 // For details see https://www.ietf.org/archive/id/draft-ietf-quic-multipath-17.html#section-3.2.2
-4513 // > an endpoint SHOULD provide new connection IDs for that path, if still open, using PATH_NEW_CONNECTION_ID frames.
-4514 let has_path = !self.abandoned_paths.contains(&path_id);
-4515 let allow_more_cids = allow_more_cids && has_path;
-4516
-4517 self.endpoint_events
-4518 .push_back(EndpointEventInner::RetireConnectionId(
-4519 now,
-4520 path_id,
-4521 sequence,
-4522 allow_more_cids,
-4523 ));
-4524 }
-4525 }
-4526 }
-4527 Frame::NewConnectionId(frame) => {
-4528 let path_id = if let Some(path_id) = frame.path_id {
-4529 if !self.is_multipath_negotiated() {
-4530 return Err(TransportError::PROTOCOL_VIOLATION(
-4531 "received PATH_NEW_CONNECTION_ID frame when multipath was not negotiated",
-4532 ));
-4533 }
-4534 if path_id > self.local_max_path_id {
-4535 return Err(TransportError::PROTOCOL_VIOLATION(
-4536 "PATH_NEW_CONNECTION_ID contains path_id exceeding current max",
-4537 ));
-4538 }
-4539 path_id
-4540 } else {
-4541 PathId::ZERO
-4542 };
-4543
-4544 if self.abandoned_paths.contains(&path_id) {
-4545 trace!("ignoring issued CID for abandoned path");
-4546 continue;
-4547 }
-4548 if let Some(ref path_id) = frame.path_id {
-4549 span.record("path", tracing::field::debug(&path_id));
-4550 }
-4551 let rem_cids = self
-4552 .rem_cids
-4553 .entry(path_id)
-4554 .or_insert_with(|| CidQueue::new(frame.id));
-4555 if rem_cids.active().is_empty() {
-4556 return Err(TransportError::PROTOCOL_VIOLATION(
-4557 "NEW_CONNECTION_ID when CIDs aren't in use",
-4558 ));
-4559 }
-4560 if frame.retire_prior_to > frame.sequence {
-4561 return Err(TransportError::PROTOCOL_VIOLATION(
-4562 "NEW_CONNECTION_ID retiring unissued CIDs",
-4563 ));
-4564 }
-4565
-4566 use crate::cid_queue::InsertError;
-4567 match rem_cids.insert(frame) {
-4568 Ok(None) if self.path(path_id).is_none() => {
-4569 // if this gives us CIDs to open a new path and a nat traversal attempt
-4570 // is underway we could try to probe a pending remote
-4571 self.continue_nat_traversal_round(now);
-4572 }
-4573 Ok(None) => {}
-4574 Ok(Some((retired, reset_token))) => {
-4575 let pending_retired =
-4576 &mut self.spaces[SpaceId::Data].pending.retire_cids;
-4577 /// Ensure `pending_retired` cannot grow without bound. Limit is
-4578 /// somewhat arbitrary but very permissive.
-4579 const MAX_PENDING_RETIRED_CIDS: u64 = CidQueue::LEN as u64 * 10;
-4580 // We don't bother counting in-flight frames because those are bounded
-4581 // by congestion control.
-4582 if (pending_retired.len() as u64)
-4583 .saturating_add(retired.end.saturating_sub(retired.start))
-4584 > MAX_PENDING_RETIRED_CIDS
-4585 {
-4586 return Err(TransportError::CONNECTION_ID_LIMIT_ERROR(
-4587 "queued too many retired CIDs",
-4588 ));
-4589 }
-4590 pending_retired.extend(retired.map(|seq| (path_id, seq)));
-4591 // TODO(matheus23): Reset token for a remote or a full 4-tuple?
-4592 self.set_reset_token(path_id, network_path.remote, reset_token);
-4593 }
-4594 Err(InsertError::ExceedsLimit) => {
-4595 return Err(TransportError::CONNECTION_ID_LIMIT_ERROR(""));
-4596 }
-4597 Err(InsertError::Retired) => {
-4598 trace!("discarding already-retired");
-4599 // RETIRE_CONNECTION_ID might not have been previously sent if e.g. a
-4600 // range of connection IDs larger than the active connection ID limit
-4601 // was retired all at once via retire_prior_to.
-4602 self.spaces[SpaceId::Data]
-4603 .pending
-4604 .retire_cids
-4605 .push((path_id, frame.sequence));
-4606 continue;
-4607 }
-4608 };
-4609
-4610 if self.side.is_server()
-4611 && path_id == PathId::ZERO
-4612 && self
-4613 .rem_cids
-4614 .get(&PathId::ZERO)
-4615 .map(|cids| cids.active_seq() == 0)
-4616 .unwrap_or_default()
-4617 {
-4618 // We're a server still using the initial remote CID for the client, so
-4619 // let's switch immediately to enable clientside stateless resets.
-4620 self.update_rem_cid(PathId::ZERO);
-4621 }
-4622 }
-4623 Frame::NewToken(NewToken { token }) => {
-4624 let ConnectionSide::Client {
-4625 token_store,
-4626 server_name,
-4627 ..
-4628 } = &self.side
-4629 else {
-4630 return Err(TransportError::PROTOCOL_VIOLATION("client sent NEW_TOKEN"));
-4631 };
-4632 if token.is_empty() {
-4633 return Err(TransportError::FRAME_ENCODING_ERROR("empty token"));
-4634 }
-4635 trace!("got new token");
-4636 token_store.insert(server_name, token);
-4637 }
-4638 Frame::Datagram(datagram) => {
-4639 if self
-4640 .datagrams
-4641 .received(datagram, &self.config.datagram_receive_buffer_size)?
-4642 {
-4643 self.events.push_back(Event::DatagramReceived);
-4644 }
-4645 }
-4646 Frame::AckFrequency(ack_frequency) => {
-4647 // This frame can only be sent in the Data space
-4648
-4649 if !self.ack_frequency.ack_frequency_received(&ack_frequency)? {
-4650 // The AckFrequency frame is stale (we have already received a more
-4651 // recent one)
-4652 continue;
+4462 }
+4463
+4464 match frame {
+4465 Frame::Crypto(frame) => {
+4466 self.read_crypto(SpaceId::Data, &frame, payload_len)?;
+4467 }
+4468 Frame::Stream(frame) => {
+4469 if self.streams.received(frame, payload_len)?.should_transmit() {
+4470 self.spaces[SpaceId::Data].pending.max_data = true;
+4471 }
+4472 }
+4473 Frame::Ack(ack) => {
+4474 self.on_ack_received(now, SpaceId::Data, ack)?;
+4475 }
+4476 Frame::PathAck(ack) => {
+4477 span.record("path", tracing::field::debug(&ack.path_id));
+4478 self.on_path_ack_received(now, SpaceId::Data, ack)?;
+4479 }
+4480 Frame::Padding | Frame::Ping => {}
+4481 Frame::Close(reason) => {
+4482 close = Some(reason);
+4483 }
+4484 Frame::PathChallenge(challenge) => {
+4485 let path = &mut self
+4486 .path_mut(path_id)
+4487 .expect("payload is processed only after the path becomes known");
+4488 path.path_responses.push(number, challenge.0, network_path);
+4489 // At this point, update_network_path_or_discard was already called, so
+4490 // we don't need to be lenient about `local_ip` possibly mis-matching.
+4491 if network_path == path.network_path {
+4492 // PATH_CHALLENGE on active path, possible off-path packet forwarding
+4493 // attack. Send a non-probing packet to recover the active path.
+4494 // TODO(flub): No longer true! We now path_challege also to validate
+4495 // the path if the path is new, without an RFC9000-style
+4496 // migration involved. This means we add in an extra
+4497 // IMMEDIATE_ACK on some challenges. It isn't really wrong to do
+4498 // so, but it still is something untidy. We should instead
+4499 // suppress this when we know the remote is still validating the
+4500 // path.
+4501 match self.peer_supports_ack_frequency() {
+4502 true => self.immediate_ack(path_id),
+4503 false => {
+4504 self.ping_path(path_id).ok();
+4505 }
+4506 }
+4507 }
+4508 }
+4509 Frame::PathResponse(response) => {
+4510 let path = self
+4511 .paths
+4512 .get_mut(&path_id)
+4513 .expect("payload is processed only after the path becomes known");
+4514
+4515 use PathTimer::*;
+4516 use paths::OnPathResponseReceived::*;
+4517 match path
+4518 .data
+4519 .on_path_response_received(now, response.0, network_path)
+4520 {
+4521 OnPath { was_open } => {
+4522 let qlog = self.qlog.with_time(now);
+4523
+4524 self.timers
+4525 .stop(Timer::PerPath(path_id, PathValidation), qlog.clone());
+4526 self.timers
+4527 .stop(Timer::PerPath(path_id, PathOpen), qlog.clone());
+4528
+4529 let next_challenge = path
+4530 .data
+4531 .earliest_expiring_challenge()
+4532 .map(|time| time + self.ack_frequency.max_ack_delay_for_pto());
+4533 self.timers.set_or_stop(
+4534 Timer::PerPath(path_id, PathChallengeLost),
+4535 next_challenge,
+4536 qlog,
+4537 );
+4538
+4539 if !was_open {
+4540 self.events
+4541 .push_back(Event::Path(PathEvent::Opened { id: path_id }));
+4542 if let Some(observed) = path.data.last_observed_addr_report.as_ref()
+4543 {
+4544 self.events.push_back(Event::Path(PathEvent::ObservedAddr {
+4545 id: path_id,
+4546 addr: observed.socket_addr(),
+4547 }));
+4548 }
+4549 }
+4550 if let Some((_, ref mut prev)) = path.prev {
+4551 prev.challenges_sent.clear();
+4552 prev.send_new_challenge = false;
+4553 }
+4554 }
+4555 OffPath => {
+4556 debug!("Response to off-path PathChallenge!");
+4557 let next_challenge = path
+4558 .data
+4559 .earliest_expiring_challenge()
+4560 .map(|time| time + self.ack_frequency.max_ack_delay_for_pto());
+4561 self.timers.set_or_stop(
+4562 Timer::PerPath(path_id, PathChallengeLost),
+4563 next_challenge,
+4564 self.qlog.with_time(now),
+4565 );
+4566 }
+4567 Invalid { expected } => {
+4568 debug!(%response, %network_path, %expected, "ignoring invalid PATH_RESPONSE")
+4569 }
+4570 Unknown => debug!(%response, "ignoring invalid PATH_RESPONSE"),
+4571 }
+4572 }
+4573 Frame::MaxData(frame::MaxData(bytes)) => {
+4574 self.streams.received_max_data(bytes);
+4575 }
+4576 Frame::MaxStreamData(frame::MaxStreamData { id, offset }) => {
+4577 self.streams.received_max_stream_data(id, offset)?;
+4578 }
+4579 Frame::MaxStreams(frame::MaxStreams { dir, count }) => {
+4580 self.streams.received_max_streams(dir, count)?;
+4581 }
+4582 Frame::ResetStream(frame) => {
+4583 if self.streams.received_reset(frame)?.should_transmit() {
+4584 self.spaces[SpaceId::Data].pending.max_data = true;
+4585 }
+4586 }
+4587 Frame::DataBlocked { offset } => {
+4588 debug!(offset, "peer claims to be blocked at connection level");
+4589 }
+4590 Frame::StreamDataBlocked { id, offset } => {
+4591 if id.initiator() == self.side.side() && id.dir() == Dir::Uni {
+4592 debug!("got STREAM_DATA_BLOCKED on send-only {}", id);
+4593 return Err(TransportError::STREAM_STATE_ERROR(
+4594 "STREAM_DATA_BLOCKED on send-only stream",
+4595 ));
+4596 }
+4597 debug!(
+4598 stream = %id,
+4599 offset, "peer claims to be blocked at stream level"
+4600 );
+4601 }
+4602 Frame::StreamsBlocked { dir, limit } => {
+4603 if limit > MAX_STREAM_COUNT {
+4604 return Err(TransportError::FRAME_ENCODING_ERROR(
+4605 "unrepresentable stream limit",
+4606 ));
+4607 }
+4608 debug!(
+4609 "peer claims to be blocked opening more than {} {} streams",
+4610 limit, dir
+4611 );
+4612 }
+4613 Frame::StopSending(frame::StopSending { id, error_code }) => {
+4614 if id.initiator() != self.side.side() {
+4615 if id.dir() == Dir::Uni {
+4616 debug!("got STOP_SENDING on recv-only {}", id);
+4617 return Err(TransportError::STREAM_STATE_ERROR(
+4618 "STOP_SENDING on recv-only stream",
+4619 ));
+4620 }
+4621 } else if self.streams.is_local_unopened(id) {
+4622 return Err(TransportError::STREAM_STATE_ERROR(
+4623 "STOP_SENDING on unopened stream",
+4624 ));
+4625 }
+4626 self.streams.received_stop_sending(id, error_code);
+4627 }
+4628 Frame::RetireConnectionId(frame::RetireConnectionId { path_id, sequence }) => {
+4629 if let Some(ref path_id) = path_id {
+4630 span.record("path", tracing::field::debug(&path_id));
+4631 }
+4632 let path_id = path_id.unwrap_or_default();
+4633 match self.local_cid_state.get_mut(&path_id) {
+4634 None => error!(?path_id, "RETIRE_CONNECTION_ID for unknown path"),
+4635 Some(cid_state) => {
+4636 let allow_more_cids = cid_state
+4637 .on_cid_retirement(sequence, self.peer_params.issue_cids_limit())?;
+4638
+4639 // If the path has closed, we do not issue more CIDs for this path
+4640 // For details see https://www.ietf.org/archive/id/draft-ietf-quic-multipath-17.html#section-3.2.2
+4641 // > an endpoint SHOULD provide new connection IDs for that path, if still open, using PATH_NEW_CONNECTION_ID frames.
+4642 let has_path = !self.abandoned_paths.contains(&path_id);
+4643 let allow_more_cids = allow_more_cids && has_path;
+4644
+4645 self.endpoint_events
+4646 .push_back(EndpointEventInner::RetireConnectionId(
+4647 now,
+4648 path_id,
+4649 sequence,
+4650 allow_more_cids,
+4651 ));
+4652 }
4653 }
-4654
-4655 // Update the params for all of our paths
-4656 for (path_id, space) in self.spaces[SpaceId::Data].number_spaces.iter_mut() {
-4657 space.pending_acks.set_ack_frequency_params(&ack_frequency);
-4658
-4659 // Our `max_ack_delay` has been updated, so we may need to adjust
-4660 // its associated timeout
-4661 if let Some(timeout) = space
-4662 .pending_acks
-4663 .max_ack_delay_timeout(self.ack_frequency.max_ack_delay)
-4664 {
-4665 self.timers.set(
-4666 Timer::PerPath(*path_id, PathTimer::MaxAckDelay),
-4667 timeout,
-4668 self.qlog.with_time(now),
-4669 );
-4670 }
-4671 }
-4672 }
-4673 Frame::ImmediateAck => {
-4674 // This frame can only be sent in the Data space
-4675 for pns in self.spaces[SpaceId::Data].iter_paths_mut() {
-4676 pns.pending_acks.set_immediate_ack_required();
-4677 }
-4678 }
-4679 Frame::HandshakeDone => {
-4680 if self.side.is_server() {
-4681 return Err(TransportError::PROTOCOL_VIOLATION(
-4682 "client sent HANDSHAKE_DONE",
-4683 ));
-4684 }
-4685 if self.spaces[SpaceId::Handshake].crypto.is_some() {
-4686 self.discard_space(now, SpaceId::Handshake);
+4654 }
+4655 Frame::NewConnectionId(frame) => {
+4656 let path_id = if let Some(path_id) = frame.path_id {
+4657 if !self.is_multipath_negotiated() {
+4658 return Err(TransportError::PROTOCOL_VIOLATION(
+4659 "received PATH_NEW_CONNECTION_ID frame when multipath was not negotiated",
+4660 ));
+4661 }
+4662 if path_id > self.local_max_path_id {
+4663 return Err(TransportError::PROTOCOL_VIOLATION(
+4664 "PATH_NEW_CONNECTION_ID contains path_id exceeding current max",
+4665 ));
+4666 }
+4667 path_id
+4668 } else {
+4669 PathId::ZERO
+4670 };
+4671
+4672 if self.abandoned_paths.contains(&path_id) {
+4673 trace!("ignoring issued CID for abandoned path");
+4674 continue;
+4675 }
+4676 if let Some(ref path_id) = frame.path_id {
+4677 span.record("path", tracing::field::debug(&path_id));
+4678 }
+4679 let rem_cids = self
+4680 .rem_cids
+4681 .entry(path_id)
+4682 .or_insert_with(|| CidQueue::new(frame.id));
+4683 if rem_cids.active().is_empty() {
+4684 return Err(TransportError::PROTOCOL_VIOLATION(
+4685 "NEW_CONNECTION_ID when CIDs aren't in use",
+4686 ));
4687 }
-4688 self.events.push_back(Event::HandshakeConfirmed);
-4689 trace!("handshake confirmed");
-4690 }
-4691 Frame::ObservedAddr(observed) => {
-4692 // check if params allows the peer to send report and this node to receive it
-4693 trace!(seq_no = %observed.seq_no, ip = %observed.ip, port = observed.port);
-4694 if !self
-4695 .peer_params
-4696 .address_discovery_role
-4697 .should_report(&self.config.address_discovery_role)
-4698 {
-4699 return Err(TransportError::PROTOCOL_VIOLATION(
-4700 "received OBSERVED_ADDRESS frame when not negotiated",
-4701 ));
-4702 }
-4703 // must only be sent in data space
-4704 if packet.header.space() != SpaceId::Data {
-4705 return Err(TransportError::PROTOCOL_VIOLATION(
-4706 "OBSERVED_ADDRESS frame outside data space",
-4707 ));
-4708 }
-4709
-4710 let path = self.path_data_mut(path_id);
-4711 if network_path == path.network_path {
-4712 if let Some(updated) = path.update_observed_addr_report(observed) {
-4713 if path.open {
-4714 self.events.push_back(Event::Path(PathEvent::ObservedAddr {
-4715 id: path_id,
-4716 addr: updated,
-4717 }));
-4718 }
-4719 // otherwise the event is reported when the path is deemed open
-4720 }
-4721 } else {
-4722 // include in migration
-4723 migration_observed_addr = Some(observed)
-4724 }
-4725 }
-4726 Frame::PathAbandon(frame::PathAbandon {
-4727 path_id,
-4728 error_code,
-4729 }) => {
-4730 span.record("path", tracing::field::debug(&path_id));
-4731 // TODO(flub): don't really know which error code to use here.
-4732 match self.close_path(now, path_id, error_code.into()) {
-4733 Ok(()) => {
-4734 trace!("peer abandoned path");
+4688 if frame.retire_prior_to > frame.sequence {
+4689 return Err(TransportError::PROTOCOL_VIOLATION(
+4690 "NEW_CONNECTION_ID retiring unissued CIDs",
+4691 ));
+4692 }
+4693
+4694 use crate::cid_queue::InsertError;
+4695 match rem_cids.insert(frame) {
+4696 Ok(None) if self.path(path_id).is_none() => {
+4697 // if this gives us CIDs to open a new path and a nat traversal attempt
+4698 // is underway we could try to probe a pending remote
+4699 self.continue_nat_traversal_round(now);
+4700 }
+4701 Ok(None) => {}
+4702 Ok(Some((retired, reset_token))) => {
+4703 let pending_retired =
+4704 &mut self.spaces[SpaceId::Data].pending.retire_cids;
+4705 /// Ensure `pending_retired` cannot grow without bound. Limit is
+4706 /// somewhat arbitrary but very permissive.
+4707 const MAX_PENDING_RETIRED_CIDS: u64 = CidQueue::LEN as u64 * 10;
+4708 // We don't bother counting in-flight frames because those are bounded
+4709 // by congestion control.
+4710 if (pending_retired.len() as u64)
+4711 .saturating_add(retired.end.saturating_sub(retired.start))
+4712 > MAX_PENDING_RETIRED_CIDS
+4713 {
+4714 return Err(TransportError::CONNECTION_ID_LIMIT_ERROR(
+4715 "queued too many retired CIDs",
+4716 ));
+4717 }
+4718 pending_retired.extend(retired.map(|seq| (path_id, seq)));
+4719 // TODO(matheus23): Reset token for a remote or a full 4-tuple?
+4720 self.set_reset_token(path_id, network_path.remote, reset_token);
+4721 }
+4722 Err(InsertError::ExceedsLimit) => {
+4723 return Err(TransportError::CONNECTION_ID_LIMIT_ERROR(""));
+4724 }
+4725 Err(InsertError::Retired) => {
+4726 trace!("discarding already-retired");
+4727 // RETIRE_CONNECTION_ID might not have been previously sent if e.g. a
+4728 // range of connection IDs larger than the active connection ID limit
+4729 // was retired all at once via retire_prior_to.
+4730 self.spaces[SpaceId::Data]
+4731 .pending
+4732 .retire_cids
+4733 .push((path_id, frame.sequence));
+4734 continue;
4735 }
-4736 Err(ClosePathError::LastOpenPath) => {
-4737 trace!("peer abandoned last path, closing connection");
-4738 return Err(TransportError::NO_VIABLE_PATH(
-4739 "last path abandoned by peer",
-4740 ));
-4741 }
-4742 Err(ClosePathError::ClosedPath) => {
-4743 trace!("peer abandoned already closed path");
-4744 }
-4745 };
-4746 // If we receive a retransmit of PATH_ABANDON then we may already have
-4747 // abandoned this path locally. In that case the DiscardPath timer
-4748 // may already have fired and we no longer have any state for this path.
-4749 // Only set this timer if we still have path state.
-4750 if let Some(path) = self.paths.get_mut(&path_id) {
-4751 if !matches!(path.data.abandon_state, AbandonState::ReceivedPathAbandon) {
-4752 let ack_delay = self.ack_frequency.max_ack_delay_for_pto();
-4753 let pto = path.data.rtt.pto_base() + ack_delay;
-4754 self.timers.set(
-4755 Timer::PerPath(path_id, PathTimer::DiscardPath),
-4756 now + 3 * pto,
-4757 self.qlog.with_time(now),
-4758 );
-4759 // We received a PATH_ABANDON, we don't expect another one by a certain time.
-4760 path.data.abandon_state = AbandonState::ReceivedPathAbandon;
-4761 }
+4736 };
+4737
+4738 if self.side.is_server()
+4739 && path_id == PathId::ZERO
+4740 && self
+4741 .rem_cids
+4742 .get(&PathId::ZERO)
+4743 .map(|cids| cids.active_seq() == 0)
+4744 .unwrap_or_default()
+4745 {
+4746 // We're a server still using the initial remote CID for the client, so
+4747 // let's switch immediately to enable clientside stateless resets.
+4748 self.update_rem_cid(PathId::ZERO);
+4749 }
+4750 }
+4751 Frame::NewToken(NewToken { token }) => {
+4752 let ConnectionSide::Client {
+4753 token_store,
+4754 server_name,
+4755 ..
+4756 } = &self.side
+4757 else {
+4758 return Err(TransportError::PROTOCOL_VIOLATION("client sent NEW_TOKEN"));
+4759 };
+4760 if token.is_empty() {
+4761 return Err(TransportError::FRAME_ENCODING_ERROR("empty token"));
4762 }
-4763 }
-4764 Frame::PathStatusAvailable(info) => {
-4765 span.record("path", tracing::field::debug(&info.path_id));
-4766 if self.is_multipath_negotiated() {
-4767 self.on_path_status(
-4768 info.path_id,
-4769 PathStatus::Available,
-4770 info.status_seq_no,
-4771 );
-4772 } else {
-4773 return Err(TransportError::PROTOCOL_VIOLATION(
-4774 "received PATH_STATUS_AVAILABLE frame when multipath was not negotiated",
-4775 ));
-4776 }
-4777 }
-4778 Frame::PathStatusBackup(info) => {
-4779 span.record("path", tracing::field::debug(&info.path_id));
-4780 if self.is_multipath_negotiated() {
-4781 self.on_path_status(info.path_id, PathStatus::Backup, info.status_seq_no);
-4782 } else {
-4783 return Err(TransportError::PROTOCOL_VIOLATION(
-4784 "received PATH_STATUS_BACKUP frame when multipath was not negotiated",
-4785 ));
-4786 }
-4787 }
-4788 Frame::MaxPathId(frame::MaxPathId(path_id)) => {
-4789 span.record("path", tracing::field::debug(&path_id));
-4790 if !self.is_multipath_negotiated() {
-4791 return Err(TransportError::PROTOCOL_VIOLATION(
-4792 "received MAX_PATH_ID frame when multipath was not negotiated",
-4793 ));
-4794 }
-4795 // frames that do not increase the path id are ignored
-4796 if path_id > self.remote_max_path_id {
-4797 self.remote_max_path_id = path_id;
-4798 self.issue_first_path_cids(now);
-4799 while let Some(true) = self.continue_nat_traversal_round(now) {}
-4800 }
-4801 }
-4802 Frame::PathsBlocked(frame::PathsBlocked(max_path_id)) => {
-4803 // Receipt of a value of Maximum Path Identifier or Path Identifier that is higher than the local maximum value MUST
-4804 // be treated as a connection error of type PROTOCOL_VIOLATION.
-4805 // Ref <https://www.ietf.org/archive/id/draft-ietf-quic-multipath-14.html#name-paths_blocked-and-path_cids>
-4806 if self.is_multipath_negotiated() {
-4807 if max_path_id > self.local_max_path_id {
-4808 return Err(TransportError::PROTOCOL_VIOLATION(
-4809 "PATHS_BLOCKED maximum path identifier was larger than local maximum",
-4810 ));
-4811 }
-4812 debug!("received PATHS_BLOCKED({:?})", max_path_id);
-4813 // TODO(@divma): ensure max concurrent paths
-4814 } else {
-4815 return Err(TransportError::PROTOCOL_VIOLATION(
-4816 "received PATHS_BLOCKED frame when not multipath was not negotiated",
-4817 ));
-4818 }
-4819 }
-4820 Frame::PathCidsBlocked(frame::PathCidsBlocked { path_id, next_seq }) => {
-4821 // Nothing to do. This is recorded in the frame stats, but otherwise we
-4822 // always issue all CIDs we're allowed to issue, so either this is an
-4823 // impatient peer or a bug on our side.
-4824
-4825 // Receipt of a value of Maximum Path Identifier or Path Identifier that is higher than the local maximum value MUST
-4826 // be treated as a connection error of type PROTOCOL_VIOLATION.
-4827 // Ref <https://www.ietf.org/archive/id/draft-ietf-quic-multipath-14.html#name-paths_blocked-and-path_cids>
-4828 if self.is_multipath_negotiated() {
-4829 if path_id > self.local_max_path_id {
-4830 return Err(TransportError::PROTOCOL_VIOLATION(
-4831 "PATH_CIDS_BLOCKED path identifier was larger than local maximum",
-4832 ));
-4833 }
-4834 if next_seq.0
-4835 > self
-4836 .local_cid_state
-4837 .get(&path_id)
-4838 .map(|cid_state| cid_state.active_seq().1 + 1)
-4839 .unwrap_or_default()
-4840 {
-4841 return Err(TransportError::PROTOCOL_VIOLATION(
-4842 "PATH_CIDS_BLOCKED next sequence number larger than in local state",
-4843 ));
-4844 }
-4845 debug!(%path_id, %next_seq, "received PATH_CIDS_BLOCKED");
-4846 } else {
-4847 return Err(TransportError::PROTOCOL_VIOLATION(
-4848 "received PATH_CIDS_BLOCKED frame when not multipath was not negotiated",
-4849 ));
-4850 }
-4851 }
-4852 Frame::AddAddress(addr) => {
-4853 let client_state = match self.iroh_hp.client_side_mut() {
-4854 Ok(state) => state,
-4855 Err(err) => {
-4856 return Err(TransportError::PROTOCOL_VIOLATION(format!(
-4857 "Nat traversal(ADD_ADDRESS): {err}"
-4858 )));
-4859 }
-4860 };
-4861
-4862 if !client_state.check_remote_address(&addr) {
-4863 // if the address is not valid we flag it, but update anyway
-4864 warn!(?addr, "server sent illegal ADD_ADDRESS frame");
-4865 }
-4866
-4867 match client_state.add_remote_address(addr) {
-4868 Ok(maybe_added) => {
-4869 if let Some(added) = maybe_added {
-4870 self.events.push_back(Event::NatTraversal(
-4871 iroh_hp::Event::AddressAdded(added),
-4872 ));
-4873 }
-4874 }
-4875 Err(e) => {
-4876 warn!(%e, "failed to add remote address")
-4877 }
-4878 }
-4879 }
-4880 Frame::RemoveAddress(addr) => {
-4881 let client_state = match self.iroh_hp.client_side_mut() {
-4882 Ok(state) => state,
-4883 Err(err) => {
-4884 return Err(TransportError::PROTOCOL_VIOLATION(format!(
-4885 "Nat traversal(REMOVE_ADDRESS): {err}"
-4886 )));
-4887 }
-4888 };
-4889 if let Some(removed_addr) = client_state.remove_remote_address(addr) {
-4890 self.events
-4891 .push_back(Event::NatTraversal(iroh_hp::Event::AddressRemoved(
-4892 removed_addr,
-4893 )));
-4894 }
-4895 }
-4896 Frame::ReachOut(reach_out) => {
-4897 let server_state = match self.iroh_hp.server_side_mut() {
-4898 Ok(state) => state,
-4899 Err(err) => {
-4900 return Err(TransportError::PROTOCOL_VIOLATION(format!(
-4901 "Nat traversal(REACH_OUT): {err}"
-4902 )));
-4903 }
-4904 };
-4905
-4906 if let Err(err) = server_state.handle_reach_out(reach_out) {
-4907 return Err(TransportError::PROTOCOL_VIOLATION(format!(
-4908 "Nat traversal(REACH_OUT): {err}"
-4909 )));
-4910 }
-4911 }
-4912 }
-4913 }
-4914
-4915 let space = self.spaces[SpaceId::Data].for_path(path_id);
-4916 if space
-4917 .pending_acks
-4918 .packet_received(now, number, ack_eliciting, &space.dedup)
-4919 {
-4920 if self.abandoned_paths.contains(&path_id) {
-4921 // § 3.4.3 QUIC-MULTIPATH: promptly send ACKs for packets received from
-4922 // abandoned paths.
-4923 space.pending_acks.set_immediate_ack_required();
-4924 } else {
-4925 self.timers.set(
-4926 Timer::PerPath(path_id, PathTimer::MaxAckDelay),
-4927 now + self.ack_frequency.max_ack_delay,
-4928 self.qlog.with_time(now),
-4929 );
-4930 }
-4931 }
-4932
-4933 // Issue stream ID credit due to ACKs of outgoing finish/resets and incoming finish/resets
-4934 // on stopped streams. Incoming finishes/resets on open streams are not handled here as they
-4935 // are only freed, and hence only issue credit, once the application has been notified
-4936 // during a read on the stream.
-4937 let pending = &mut self.spaces[SpaceId::Data].pending;
-4938 self.streams.queue_max_stream_id(pending);
-4939
-4940 if let Some(reason) = close {
-4941 self.state.move_to_draining(Some(reason.into()));
-4942 self.close = true;
-4943 }
-4944
-4945 if Some(number) == self.spaces[SpaceId::Data].for_path(path_id).rx_packet
-4946 && !is_probing_packet
-4947 && network_path != self.path_data(path_id).network_path
-4948 {
-4949 let ConnectionSide::Server { ref server_config } = self.side else {
-4950 panic!("packets from unknown remote should be dropped by clients");
-4951 };
-4952 debug_assert!(
-4953 server_config.migration,
-4954 "migration-initiating packets should have been dropped immediately"
-4955 );
-4956 self.migrate(path_id, now, network_path, migration_observed_addr);
-4957 // Break linkability, if possible
-4958 self.update_rem_cid(path_id);
-4959 self.spin = false;
-4960 }
-4961
-4962 Ok(())
-4963 }
-4964
-4965 fn migrate(
-4966 &mut self,
-4967 path_id: PathId,
-4968 now: Instant,
-4969 network_path: FourTuple,
-4970 observed_addr: Option<ObservedAddr>,
-4971 ) {
-4972 trace!(%network_path, %path_id, "migration initiated");
-4973 self.path_generation_counter = self.path_generation_counter.wrapping_add(1);
-4974 // TODO(@divma): conditions for path migration in multipath are very specific, check them
-4975 // again to prevent path migrations that should actually create a new path
-4976
-4977 // Reset rtt/congestion state for new path unless it looks like a NAT rebinding.
-4978 // Note that the congestion window will not grow until validation terminates. Helps mitigate
-4979 // amplification attacks performed by spoofing source addresses.
-4980 let prev_pto = self.pto(SpaceId::Data, path_id);
-4981 let known_path = self.paths.get_mut(&path_id).expect("known path");
-4982 let path = &mut known_path.data;
-4983 let mut new_path = if network_path.remote.is_ipv4()
-4984 && network_path.remote.ip() == path.network_path.remote.ip()
-4985 {
-4986 PathData::from_previous(network_path, path, self.path_generation_counter, now)
-4987 } else {
-4988 let peer_max_udp_payload_size =
-4989 u16::try_from(self.peer_params.max_udp_payload_size.into_inner())
-4990 .unwrap_or(u16::MAX);
-4991 PathData::new(
-4992 network_path,
-4993 self.allow_mtud,
-4994 Some(peer_max_udp_payload_size),
-4995 self.path_generation_counter,
-4996 now,
-4997 &self.config,
-4998 )
-4999 };
-5000 new_path.last_observed_addr_report = path.last_observed_addr_report.clone();
-5001 if let Some(report) = observed_addr {
-5002 if let Some(updated) = new_path.update_observed_addr_report(report) {
-5003 tracing::info!("adding observed addr event from migration");
-5004 self.events.push_back(Event::Path(PathEvent::ObservedAddr {
-5005 id: path_id,
-5006 addr: updated,
-5007 }));
-5008 }
-5009 }
-5010 new_path.send_new_challenge = true;
-5011
-5012 let mut prev = mem::replace(path, new_path);
-5013 // Don't clobber the original path if the previous one hasn't been validated yet
-5014 if !prev.is_validating_path() {
-5015 prev.send_new_challenge = true;
-5016 // We haven't updated the remote CID yet, this captures the remote CID we were using on
-5017 // the previous path.
-5018
-5019 known_path.prev = Some((self.rem_cids.get(&path_id).unwrap().active(), prev));
-5020 }
-5021
-5022 // We need to re-assign the correct remote to this path in qlog
-5023 self.qlog.emit_tuple_assigned(path_id, network_path, now);
-5024
-5025 self.timers.set(
-5026 Timer::PerPath(path_id, PathTimer::PathValidation),
-5027 now + 3 * cmp::max(self.pto(SpaceId::Data, path_id), prev_pto),
-5028 self.qlog.with_time(now),
-5029 );
-5030 }
-5031
-5032 /// Handle a change in the local address, i.e. an active migration
-5033 pub fn local_address_changed(&mut self) {
-5034 // TODO(flub): if multipath is enabled this needs to create a new path entirely.
-5035 self.update_rem_cid(PathId::ZERO);
-5036 self.ping();
-5037 }
-5038
-5039 /// Switch to a previously unused remote connection ID, if possible
-5040 fn update_rem_cid(&mut self, path_id: PathId) {
-5041 let Some((reset_token, retired)) =
-5042 self.rem_cids.get_mut(&path_id).and_then(|cids| cids.next())
-5043 else {
-5044 return;
-5045 };
-5046
-5047 // Retire the current remote CID and any CIDs we had to skip.
-5048 self.spaces[SpaceId::Data]
-5049 .pending
-5050 .retire_cids
-5051 .extend(retired.map(|seq| (path_id, seq)));
-5052 let remote = self.path_data(path_id).network_path.remote;
-5053 self.set_reset_token(path_id, remote, reset_token);
-5054 }
-5055
-5056 /// Sends this reset token to the endpoint
-5057 ///
-5058 /// The endpoint needs to know the reset tokens issued by the peer, so that if the peer
-5059 /// sends a reset token it knows to route it to this connection. See RFC 9000 section
-5060 /// 10.3. Stateless Reset.
-5061 ///
-5062 /// Reset tokens are different for each path, the endpoint identifies paths by peer
-5063 /// socket address however, not by path ID.
-5064 fn set_reset_token(&mut self, path_id: PathId, remote: SocketAddr, reset_token: ResetToken) {
-5065 self.endpoint_events
-5066 .push_back(EndpointEventInner::ResetToken(path_id, remote, reset_token));
+4763 trace!("got new token");
+4764 token_store.insert(server_name, token);
+4765 }
+4766 Frame::Datagram(datagram) => {
+4767 if self
+4768 .datagrams
+4769 .received(datagram, &self.config.datagram_receive_buffer_size)?
+4770 {
+4771 self.events.push_back(Event::DatagramReceived);
+4772 }
+4773 }
+4774 Frame::AckFrequency(ack_frequency) => {
+4775 // This frame can only be sent in the Data space
+4776
+4777 if !self.ack_frequency.ack_frequency_received(&ack_frequency)? {
+4778 // The AckFrequency frame is stale (we have already received a more
+4779 // recent one)
+4780 continue;
+4781 }
+4782
+4783 // Update the params for all of our paths
+4784 for (path_id, space) in self.spaces[SpaceId::Data].number_spaces.iter_mut() {
+4785 space.pending_acks.set_ack_frequency_params(&ack_frequency);
+4786
+4787 // Our `max_ack_delay` has been updated, so we may need to adjust
+4788 // its associated timeout
+4789 if let Some(timeout) = space
+4790 .pending_acks
+4791 .max_ack_delay_timeout(self.ack_frequency.max_ack_delay)
+4792 {
+4793 self.timers.set(
+4794 Timer::PerPath(*path_id, PathTimer::MaxAckDelay),
+4795 timeout,
+4796 self.qlog.with_time(now),
+4797 );
+4798 }
+4799 }
+4800 }
+4801 Frame::ImmediateAck => {
+4802 // This frame can only be sent in the Data space
+4803 for pns in self.spaces[SpaceId::Data].iter_paths_mut() {
+4804 pns.pending_acks.set_immediate_ack_required();
+4805 }
+4806 }
+4807 Frame::HandshakeDone => {
+4808 if self.side.is_server() {
+4809 return Err(TransportError::PROTOCOL_VIOLATION(
+4810 "client sent HANDSHAKE_DONE",
+4811 ));
+4812 }
+4813 if self.spaces[SpaceId::Handshake].crypto.is_some() {
+4814 self.discard_space(now, SpaceId::Handshake);
+4815 }
+4816 self.events.push_back(Event::HandshakeConfirmed);
+4817 trace!("handshake confirmed");
+4818 }
+4819 Frame::ObservedAddr(observed) => {
+4820 // check if params allows the peer to send report and this node to receive it
+4821 trace!(seq_no = %observed.seq_no, ip = %observed.ip, port = observed.port);
+4822 if !self
+4823 .peer_params
+4824 .address_discovery_role
+4825 .should_report(&self.config.address_discovery_role)
+4826 {
+4827 return Err(TransportError::PROTOCOL_VIOLATION(
+4828 "received OBSERVED_ADDRESS frame when not negotiated",
+4829 ));
+4830 }
+4831 // must only be sent in data space
+4832 if packet.header.space() != SpaceId::Data {
+4833 return Err(TransportError::PROTOCOL_VIOLATION(
+4834 "OBSERVED_ADDRESS frame outside data space",
+4835 ));
+4836 }
+4837
+4838 let path = self.path_data_mut(path_id);
+4839 if network_path == path.network_path {
+4840 if let Some(updated) = path.update_observed_addr_report(observed) {
+4841 if path.open {
+4842 self.events.push_back(Event::Path(PathEvent::ObservedAddr {
+4843 id: path_id,
+4844 addr: updated,
+4845 }));
+4846 }
+4847 // otherwise the event is reported when the path is deemed open
+4848 }
+4849 } else {
+4850 // include in migration
+4851 migration_observed_addr = Some(observed)
+4852 }
+4853 }
+4854 Frame::PathAbandon(frame::PathAbandon {
+4855 path_id,
+4856 error_code,
+4857 }) => {
+4858 span.record("path", tracing::field::debug(&path_id));
+4859 // TODO(flub): don't really know which error code to use here.
+4860 match self.close_path(now, path_id, error_code.into()) {
+4861 Ok(()) => {
+4862 trace!("peer abandoned path");
+4863 }
+4864 Err(ClosePathError::LastOpenPath) => {
+4865 trace!("peer abandoned last path, closing connection");
+4866 return Err(TransportError::NO_VIABLE_PATH(
+4867 "last path abandoned by peer",
+4868 ));
+4869 }
+4870 Err(ClosePathError::ClosedPath) => {
+4871 trace!("peer abandoned already closed path");
+4872 }
+4873 };
+4874 // If we receive a retransmit of PATH_ABANDON then we may already have
+4875 // abandoned this path locally. In that case the DiscardPath timer
+4876 // may already have fired and we no longer have any state for this path.
+4877 // Only set this timer if we still have path state.
+4878 if let Some(path) = self.paths.get_mut(&path_id) {
+4879 if !matches!(path.data.abandon_state, AbandonState::ReceivedPathAbandon) {
+4880 let ack_delay = self.ack_frequency.max_ack_delay_for_pto();
+4881 let pto = path.data.rtt.pto_base() + ack_delay;
+4882 self.timers.set(
+4883 Timer::PerPath(path_id, PathTimer::DiscardPath),
+4884 now + 3 * pto,
+4885 self.qlog.with_time(now),
+4886 );
+4887 // We received a PATH_ABANDON, we don't expect another one by a certain time.
+4888 path.data.abandon_state = AbandonState::ReceivedPathAbandon;
+4889 }
+4890 }
+4891 }
+4892 Frame::PathStatusAvailable(info) => {
+4893 span.record("path", tracing::field::debug(&info.path_id));
+4894 if self.is_multipath_negotiated() {
+4895 self.on_path_status(
+4896 info.path_id,
+4897 PathStatus::Available,
+4898 info.status_seq_no,
+4899 );
+4900 } else {
+4901 return Err(TransportError::PROTOCOL_VIOLATION(
+4902 "received PATH_STATUS_AVAILABLE frame when multipath was not negotiated",
+4903 ));
+4904 }
+4905 }
+4906 Frame::PathStatusBackup(info) => {
+4907 span.record("path", tracing::field::debug(&info.path_id));
+4908 if self.is_multipath_negotiated() {
+4909 self.on_path_status(info.path_id, PathStatus::Backup, info.status_seq_no);
+4910 } else {
+4911 return Err(TransportError::PROTOCOL_VIOLATION(
+4912 "received PATH_STATUS_BACKUP frame when multipath was not negotiated",
+4913 ));
+4914 }
+4915 }
+4916 Frame::MaxPathId(frame::MaxPathId(path_id)) => {
+4917 span.record("path", tracing::field::debug(&path_id));
+4918 if !self.is_multipath_negotiated() {
+4919 return Err(TransportError::PROTOCOL_VIOLATION(
+4920 "received MAX_PATH_ID frame when multipath was not negotiated",
+4921 ));
+4922 }
+4923 // frames that do not increase the path id are ignored
+4924 if path_id > self.remote_max_path_id {
+4925 self.remote_max_path_id = path_id;
+4926 self.issue_first_path_cids(now);
+4927 while let Some(true) = self.continue_nat_traversal_round(now) {}
+4928 }
+4929 }
+4930 Frame::PathsBlocked(frame::PathsBlocked(max_path_id)) => {
+4931 // Receipt of a value of Maximum Path Identifier or Path Identifier that is higher than the local maximum value MUST
+4932 // be treated as a connection error of type PROTOCOL_VIOLATION.
+4933 // Ref <https://www.ietf.org/archive/id/draft-ietf-quic-multipath-14.html#name-paths_blocked-and-path_cids>
+4934 if self.is_multipath_negotiated() {
+4935 if max_path_id > self.local_max_path_id {
+4936 return Err(TransportError::PROTOCOL_VIOLATION(
+4937 "PATHS_BLOCKED maximum path identifier was larger than local maximum",
+4938 ));
+4939 }
+4940 debug!("received PATHS_BLOCKED({:?})", max_path_id);
+4941 // TODO(@divma): ensure max concurrent paths
+4942 } else {
+4943 return Err(TransportError::PROTOCOL_VIOLATION(
+4944 "received PATHS_BLOCKED frame when not multipath was not negotiated",
+4945 ));
+4946 }
+4947 }
+4948 Frame::PathCidsBlocked(frame::PathCidsBlocked { path_id, next_seq }) => {
+4949 // Nothing to do. This is recorded in the frame stats, but otherwise we
+4950 // always issue all CIDs we're allowed to issue, so either this is an
+4951 // impatient peer or a bug on our side.
+4952
+4953 // Receipt of a value of Maximum Path Identifier or Path Identifier that is higher than the local maximum value MUST
+4954 // be treated as a connection error of type PROTOCOL_VIOLATION.
+4955 // Ref <https://www.ietf.org/archive/id/draft-ietf-quic-multipath-14.html#name-paths_blocked-and-path_cids>
+4956 if self.is_multipath_negotiated() {
+4957 if path_id > self.local_max_path_id {
+4958 return Err(TransportError::PROTOCOL_VIOLATION(
+4959 "PATH_CIDS_BLOCKED path identifier was larger than local maximum",
+4960 ));
+4961 }
+4962 if next_seq.0
+4963 > self
+4964 .local_cid_state
+4965 .get(&path_id)
+4966 .map(|cid_state| cid_state.active_seq().1 + 1)
+4967 .unwrap_or_default()
+4968 {
+4969 return Err(TransportError::PROTOCOL_VIOLATION(
+4970 "PATH_CIDS_BLOCKED next sequence number larger than in local state",
+4971 ));
+4972 }
+4973 debug!(%path_id, %next_seq, "received PATH_CIDS_BLOCKED");
+4974 } else {
+4975 return Err(TransportError::PROTOCOL_VIOLATION(
+4976 "received PATH_CIDS_BLOCKED frame when not multipath was not negotiated",
+4977 ));
+4978 }
+4979 }
+4980 Frame::AddAddress(addr) => {
+4981 let client_state = match self.iroh_hp.client_side_mut() {
+4982 Ok(state) => state,
+4983 Err(err) => {
+4984 return Err(TransportError::PROTOCOL_VIOLATION(format!(
+4985 "Nat traversal(ADD_ADDRESS): {err}"
+4986 )));
+4987 }
+4988 };
+4989
+4990 if !client_state.check_remote_address(&addr) {
+4991 // if the address is not valid we flag it, but update anyway
+4992 warn!(?addr, "server sent illegal ADD_ADDRESS frame");
+4993 }
+4994
+4995 match client_state.add_remote_address(addr) {
+4996 Ok(maybe_added) => {
+4997 if let Some(added) = maybe_added {
+4998 self.events.push_back(Event::NatTraversal(
+4999 iroh_hp::Event::AddressAdded(added),
+5000 ));
+5001 }
+5002 }
+5003 Err(e) => {
+5004 warn!(%e, "failed to add remote address")
+5005 }
+5006 }
+5007 }
+5008 Frame::RemoveAddress(addr) => {
+5009 let client_state = match self.iroh_hp.client_side_mut() {
+5010 Ok(state) => state,
+5011 Err(err) => {
+5012 return Err(TransportError::PROTOCOL_VIOLATION(format!(
+5013 "Nat traversal(REMOVE_ADDRESS): {err}"
+5014 )));
+5015 }
+5016 };
+5017 if let Some(removed_addr) = client_state.remove_remote_address(addr) {
+5018 self.events
+5019 .push_back(Event::NatTraversal(iroh_hp::Event::AddressRemoved(
+5020 removed_addr,
+5021 )));
+5022 }
+5023 }
+5024 Frame::ReachOut(reach_out) => {
+5025 let server_state = match self.iroh_hp.server_side_mut() {
+5026 Ok(state) => state,
+5027 Err(err) => {
+5028 return Err(TransportError::PROTOCOL_VIOLATION(format!(
+5029 "Nat traversal(REACH_OUT): {err}"
+5030 )));
+5031 }
+5032 };
+5033
+5034 if let Err(err) = server_state.handle_reach_out(reach_out) {
+5035 return Err(TransportError::PROTOCOL_VIOLATION(format!(
+5036 "Nat traversal(REACH_OUT): {err}"
+5037 )));
+5038 }
+5039 }
+5040 }
+5041 }
+5042
+5043 let space = self.spaces[SpaceId::Data].for_path(path_id);
+5044 if space
+5045 .pending_acks
+5046 .packet_received(now, number, ack_eliciting, &space.dedup)
+5047 {
+5048 if self.abandoned_paths.contains(&path_id) {
+5049 // § 3.4.3 QUIC-MULTIPATH: promptly send ACKs for packets received from
+5050 // abandoned paths.
+5051 space.pending_acks.set_immediate_ack_required();
+5052 } else {
+5053 self.timers.set(
+5054 Timer::PerPath(path_id, PathTimer::MaxAckDelay),
+5055 now + self.ack_frequency.max_ack_delay,
+5056 self.qlog.with_time(now),
+5057 );
+5058 }
+5059 }
+5060
+5061 // Issue stream ID credit due to ACKs of outgoing finish/resets and incoming finish/resets
+5062 // on stopped streams. Incoming finishes/resets on open streams are not handled here as they
+5063 // are only freed, and hence only issue credit, once the application has been notified
+5064 // during a read on the stream.
+5065 let pending = &mut self.spaces[SpaceId::Data].pending;
+5066 self.streams.queue_max_stream_id(pending);
5067
-5068 // During the handshake the server sends a reset token in the transport
-5069 // parameters. When we are the client and we receive the reset token during the
-5070 // handshake we want this to affect our peer transport parameters.
-5071 // TODO(flub): Pretty sure this is pointless, the entire params is overwritten
-5072 // shortly after this was called. And then the params don't have this anymore.
-5073 if path_id == PathId::ZERO {
-5074 self.peer_params.stateless_reset_token = Some(reset_token);
-5075 }
-5076 }
-5077
-5078 /// Issue an initial set of connection IDs to the peer upon connection
-5079 fn issue_first_cids(&mut self, now: Instant) {
-5080 if self
-5081 .local_cid_state
-5082 .get(&PathId::ZERO)
-5083 .expect("PathId::ZERO exists when the connection is created")
-5084 .cid_len()
-5085 == 0
-5086 {
-5087 return;
+5068 if let Some(reason) = close {
+5069 self.state.move_to_draining(Some(reason.into()));
+5070 self.close = true;
+5071 }
+5072
+5073 if Some(number) == self.spaces[SpaceId::Data].for_path(path_id).rx_packet
+5074 && !is_probing_packet
+5075 && network_path != self.path_data(path_id).network_path
+5076 {
+5077 let ConnectionSide::Server { ref server_config } = self.side else {
+5078 panic!("packets from unknown remote should be dropped by clients");
+5079 };
+5080 debug_assert!(
+5081 server_config.migration,
+5082 "migration-initiating packets should have been dropped immediately"
+5083 );
+5084 self.migrate(path_id, now, network_path, migration_observed_addr);
+5085 // Break linkability, if possible
+5086 self.update_rem_cid(path_id);
+5087 self.spin = false;
5088 }
5089
-5090 // Subtract 1 to account for the CID we supplied while handshaking
-5091 let mut n = self.peer_params.issue_cids_limit() - 1;
-5092 if let ConnectionSide::Server { server_config } = &self.side {
-5093 if server_config.has_preferred_address() {
-5094 // We also sent a CID in the transport parameters
-5095 n -= 1;
-5096 }
-5097 }
-5098 self.endpoint_events
-5099 .push_back(EndpointEventInner::NeedIdentifiers(PathId::ZERO, now, n));
-5100 }
-5101
-5102 /// Issues an initial set of CIDs for paths that have not yet had any CIDs issued
-5103 ///
-5104 /// Later CIDs are issued when CIDs expire or are retired by the peer.
-5105 fn issue_first_path_cids(&mut self, now: Instant) {
-5106 if let Some(max_path_id) = self.max_path_id() {
-5107 let mut path_id = self.max_path_id_with_cids.next();
-5108 while path_id <= max_path_id {
-5109 self.endpoint_events
-5110 .push_back(EndpointEventInner::NeedIdentifiers(
-5111 path_id,
-5112 now,
-5113 self.peer_params.issue_cids_limit(),
-5114 ));
-5115 path_id = path_id.next();
-5116 }
-5117 self.max_path_id_with_cids = max_path_id;
-5118 }
-5119 }
-5120
-5121 /// Populates a packet with frames
-5122 ///
-5123 /// This tries to fit as many frames as possible into the packet.
-5124 ///
-5125 /// *path_exclusive_only* means to only build frames which can only be sent on this
-5126 /// *path. This is used in multipath for backup paths while there is still an active
-5127 /// *path.
-5128 fn populate_packet<'a, 'b>(
-5129 &mut self,
-5130 now: Instant,
-5131 space_id: SpaceId,
-5132 path_id: PathId,
-5133 path_exclusive_only: bool,
-5134 builder: &mut PacketBuilder<'a, 'b>,
-5135 ) {
-5136 let pn = builder.exact_number;
-5137 let is_multipath_negotiated = self.is_multipath_negotiated();
-5138 let stats = &mut self.stats.frame_tx;
-5139 let space = &mut self.spaces[space_id];
-5140 let path = &mut self.paths.get_mut(&path_id).expect("known path").data;
-5141 let is_0rtt = space_id == SpaceId::Data && space.crypto.is_none();
-5142 space
-5143 .for_path(path_id)
-5144 .pending_acks
-5145 .maybe_ack_non_eliciting();
+5090 Ok(())
+5091 }
+5092
+5093 fn migrate(
+5094 &mut self,
+5095 path_id: PathId,
+5096 now: Instant,
+5097 network_path: FourTuple,
+5098 observed_addr: Option<ObservedAddr>,
+5099 ) {
+5100 trace!(%network_path, %path_id, "migration initiated");
+5101 self.path_generation_counter = self.path_generation_counter.wrapping_add(1);
+5102 // TODO(@divma): conditions for path migration in multipath are very specific, check them
+5103 // again to prevent path migrations that should actually create a new path
+5104
+5105 // Reset rtt/congestion state for new path unless it looks like a NAT rebinding.
+5106 // Note that the congestion window will not grow until validation terminates. Helps mitigate
+5107 // amplification attacks performed by spoofing source addresses.
+5108 let prev_pto = self.pto(SpaceId::Data, path_id);
+5109 let known_path = self.paths.get_mut(&path_id).expect("known path");
+5110 let path = &mut known_path.data;
+5111 let mut new_path = if network_path.remote.is_ipv4()
+5112 && network_path.remote.ip() == path.network_path.remote.ip()
+5113 {
+5114 PathData::from_previous(network_path, path, self.path_generation_counter, now)
+5115 } else {
+5116 let peer_max_udp_payload_size =
+5117 u16::try_from(self.peer_params.max_udp_payload_size.into_inner())
+5118 .unwrap_or(u16::MAX);
+5119 PathData::new(
+5120 network_path,
+5121 self.allow_mtud,
+5122 Some(peer_max_udp_payload_size),
+5123 self.path_generation_counter,
+5124 now,
+5125 &self.config,
+5126 )
+5127 };
+5128 new_path.last_observed_addr_report = path.last_observed_addr_report.clone();
+5129 if let Some(report) = observed_addr {
+5130 if let Some(updated) = new_path.update_observed_addr_report(report) {
+5131 tracing::info!("adding observed addr event from migration");
+5132 self.events.push_back(Event::Path(PathEvent::ObservedAddr {
+5133 id: path_id,
+5134 addr: updated,
+5135 }));
+5136 }
+5137 }
+5138 new_path.send_new_challenge = true;
+5139
+5140 let mut prev = mem::replace(path, new_path);
+5141 // Don't clobber the original path if the previous one hasn't been validated yet
+5142 if !prev.is_validating_path() {
+5143 prev.send_new_challenge = true;
+5144 // We haven't updated the remote CID yet, this captures the remote CID we were using on
+5145 // the previous path.
5146
-5147 // HANDSHAKE_DONE
-5148 if !is_0rtt && mem::replace(&mut space.pending.handshake_done, false) {
-5149 builder.write_frame(frame::HandshakeDone, stats);
-5150 }
-5151
-5152 // REACH_OUT
-5153 // TODO(@divma): path explusive considerations
-5154 if let Some((round, addresses)) = space.pending.reach_out.as_mut() {
-5155 while let Some(local_addr) = addresses.pop() {
-5156 let reach_out = frame::ReachOut::new(*round, local_addr);
-5157 if builder.frame_space_remaining() > reach_out.size() {
-5158 builder.write_frame(reach_out, stats);
-5159 } else {
-5160 addresses.push(local_addr);
-5161 break;
-5162 }
-5163 }
-5164 if addresses.is_empty() {
-5165 space.pending.reach_out = None;
-5166 }
-5167 }
-5168
-5169 // OBSERVED_ADDR
-5170 if !path_exclusive_only
-5171 && space_id == SpaceId::Data
-5172 && self
-5173 .config
-5174 .address_discovery_role
-5175 .should_report(&self.peer_params.address_discovery_role)
-5176 && (!path.observed_addr_sent || space.pending.observed_addr)
-5177 {
-5178 let frame =
-5179 frame::ObservedAddr::new(path.network_path.remote, self.next_observed_addr_seq_no);
-5180 if builder.frame_space_remaining() > frame.size() {
-5181 builder.write_frame(frame, stats);
-5182
-5183 self.next_observed_addr_seq_no = self.next_observed_addr_seq_no.saturating_add(1u8);
-5184 path.observed_addr_sent = true;
-5185
-5186 space.pending.observed_addr = false;
-5187 }
-5188 }
-5189
-5190 // PING
-5191 if mem::replace(&mut space.for_path(path_id).ping_pending, false) {
-5192 builder.write_frame(frame::Ping, stats);
-5193 }
-5194
-5195 // IMMEDIATE_ACK
-5196 if mem::replace(&mut space.for_path(path_id).immediate_ack_pending, false) {
-5197 debug_assert_eq!(
-5198 space_id,
-5199 SpaceId::Data,
-5200 "immediate acks must be sent in the data space"
-5201 );
-5202 builder.write_frame(frame::ImmediateAck, stats);
+5147 known_path.prev = Some((self.rem_cids.get(&path_id).unwrap().active(), prev));
+5148 }
+5149
+5150 // We need to re-assign the correct remote to this path in qlog
+5151 self.qlog.emit_tuple_assigned(path_id, network_path, now);
+5152
+5153 self.timers.set(
+5154 Timer::PerPath(path_id, PathTimer::PathValidation),
+5155 now + 3 * cmp::max(self.pto(SpaceId::Data, path_id), prev_pto),
+5156 self.qlog.with_time(now),
+5157 );
+5158 }
+5159
+5160 /// Handle a change in the local address, i.e. an active migration
+5161 pub fn local_address_changed(&mut self) {
+5162 // TODO(flub): if multipath is enabled this needs to create a new path entirely.
+5163 self.update_rem_cid(PathId::ZERO);
+5164 self.ping();
+5165 }
+5166
+5167 /// Switch to a previously unused remote connection ID, if possible
+5168 fn update_rem_cid(&mut self, path_id: PathId) {
+5169 let Some((reset_token, retired)) =
+5170 self.rem_cids.get_mut(&path_id).and_then(|cids| cids.next())
+5171 else {
+5172 return;
+5173 };
+5174
+5175 // Retire the current remote CID and any CIDs we had to skip.
+5176 self.spaces[SpaceId::Data]
+5177 .pending
+5178 .retire_cids
+5179 .extend(retired.map(|seq| (path_id, seq)));
+5180 let remote = self.path_data(path_id).network_path.remote;
+5181 self.set_reset_token(path_id, remote, reset_token);
+5182 }
+5183
+5184 /// Sends this reset token to the endpoint
+5185 ///
+5186 /// The endpoint needs to know the reset tokens issued by the peer, so that if the peer
+5187 /// sends a reset token it knows to route it to this connection. See RFC 9000 section
+5188 /// 10.3. Stateless Reset.
+5189 ///
+5190 /// Reset tokens are different for each path, the endpoint identifies paths by peer
+5191 /// socket address however, not by path ID.
+5192 fn set_reset_token(&mut self, path_id: PathId, remote: SocketAddr, reset_token: ResetToken) {
+5193 self.endpoint_events
+5194 .push_back(EndpointEventInner::ResetToken(path_id, remote, reset_token));
+5195
+5196 // During the handshake the server sends a reset token in the transport
+5197 // parameters. When we are the client and we receive the reset token during the
+5198 // handshake we want this to affect our peer transport parameters.
+5199 // TODO(flub): Pretty sure this is pointless, the entire params is overwritten
+5200 // shortly after this was called. And then the params don't have this anymore.
+5201 if path_id == PathId::ZERO {
+5202 self.peer_params.stateless_reset_token = Some(reset_token);
5203 }
-5204
-5205 // ACK
-5206 // TODO(flub): Should this send acks for this path anyway?
-5207
-5208 if !path_exclusive_only {
-5209 for path_id in space
-5210 .number_spaces
-5211 .iter_mut()
-5212 .filter(|(_, pns)| pns.pending_acks.can_send())
-5213 .map(|(&path_id, _)| path_id)
-5214 .collect::<Vec<_>>()
-5215 {
-5216 Self::populate_acks(
-5217 now,
-5218 self.receiving_ecn,
-5219 path_id,
-5220 space_id,
-5221 space,
-5222 is_multipath_negotiated,
-5223 builder,
-5224 stats,
-5225 );
-5226 }
-5227 }
-5228
-5229 // ACK_FREQUENCY
-5230 if !path_exclusive_only && mem::replace(&mut space.pending.ack_frequency, false) {
-5231 let sequence_number = self.ack_frequency.next_sequence_number();
-5232
-5233 // Safe to unwrap because this is always provided when ACK frequency is enabled
-5234 let config = self.config.ack_frequency_config.as_ref().unwrap();
-5235
-5236 // Ensure the delay is within bounds to avoid a PROTOCOL_VIOLATION error
-5237 let max_ack_delay = self.ack_frequency.candidate_max_ack_delay(
-5238 path.rtt.get(),
-5239 config,
-5240 &self.peer_params,
-5241 );
-5242
-5243 let frame = frame::AckFrequency {
-5244 sequence: sequence_number,
-5245 ack_eliciting_threshold: config.ack_eliciting_threshold,
-5246 request_max_ack_delay: max_ack_delay.as_micros().try_into().unwrap_or(VarInt::MAX),
-5247 reordering_threshold: config.reordering_threshold,
-5248 };
-5249 builder.write_frame(frame, stats);
-5250
-5251 self.ack_frequency
-5252 .ack_frequency_sent(path_id, pn, max_ack_delay);
-5253 }
-5254
-5255 // PATH_CHALLENGE
-5256 if builder.frame_space_remaining() > frame::PathChallenge::SIZE_BOUND
-5257 && space_id == SpaceId::Data
-5258 && path.send_new_challenge
-5259 && !self.state.is_closed()
-5260 // we don't want to send new challenges if we are already closing
-5261 {
-5262 path.send_new_challenge = false;
-5263
-5264 // Generate a new challenge every time we send a new PATH_CHALLENGE
-5265 let token = self.rng.random();
-5266 let info = paths::SentChallengeInfo {
-5267 sent_instant: now,
-5268 network_path: path.network_path,
-5269 };
-5270 path.challenges_sent.insert(token, info);
-5271 let challenge = frame::PathChallenge(token);
-5272 trace!(frame = %challenge);
-5273 builder.write_frame(challenge, stats);
-5274 builder.require_padding();
-5275 let pto = self.ack_frequency.max_ack_delay_for_pto() + path.rtt.pto_base();
-5276 self.timers.set(
-5277 Timer::PerPath(path_id, PathTimer::PathChallengeLost),
-5278 now + pto,
-5279 self.qlog.with_time(now),
-5280 );
-5281
-5282 if is_multipath_negotiated && !path.validated && path.send_new_challenge {
-5283 // queue informing the path status along with the challenge
-5284 space.pending.path_status.insert(path_id);
-5285 }
-5286
-5287 // Always include an OBSERVED_ADDR frame with a PATH_CHALLENGE, regardless
-5288 // of whether one has already been sent on this path.
-5289 if space_id == SpaceId::Data
-5290 && self
-5291 .config
-5292 .address_discovery_role
-5293 .should_report(&self.peer_params.address_discovery_role)
-5294 {
-5295 let frame = frame::ObservedAddr::new(
-5296 path.network_path.remote,
-5297 self.next_observed_addr_seq_no,
-5298 );
-5299 if builder.frame_space_remaining() > frame.size() {
-5300 builder.write_frame(frame, stats);
-5301
-5302 self.next_observed_addr_seq_no =
-5303 self.next_observed_addr_seq_no.saturating_add(1u8);
-5304 path.observed_addr_sent = true;
-5305
-5306 space.pending.observed_addr = false;
-5307 }
-5308 }
-5309 }
+5204 }
+5205
+5206 /// Issue an initial set of connection IDs to the peer upon connection
+5207 fn issue_first_cids(&mut self, now: Instant) {
+5208 if self
+5209 .local_cid_state
+5210 .get(&PathId::ZERO)
+5211 .expect("PathId::ZERO exists when the connection is created")
+5212 .cid_len()
+5213 == 0
+5214 {
+5215 return;
+5216 }
+5217
+5218 // Subtract 1 to account for the CID we supplied while handshaking
+5219 let mut n = self.peer_params.issue_cids_limit() - 1;
+5220 if let ConnectionSide::Server { server_config } = &self.side {
+5221 if server_config.has_preferred_address() {
+5222 // We also sent a CID in the transport parameters
+5223 n -= 1;
+5224 }
+5225 }
+5226 self.endpoint_events
+5227 .push_back(EndpointEventInner::NeedIdentifiers(PathId::ZERO, now, n));
+5228 }
+5229
+5230 /// Issues an initial set of CIDs for paths that have not yet had any CIDs issued
+5231 ///
+5232 /// Later CIDs are issued when CIDs expire or are retired by the peer.
+5233 fn issue_first_path_cids(&mut self, now: Instant) {
+5234 if let Some(max_path_id) = self.max_path_id() {
+5235 let mut path_id = self.max_path_id_with_cids.next();
+5236 while path_id <= max_path_id {
+5237 self.endpoint_events
+5238 .push_back(EndpointEventInner::NeedIdentifiers(
+5239 path_id,
+5240 now,
+5241 self.peer_params.issue_cids_limit(),
+5242 ));
+5243 path_id = path_id.next();
+5244 }
+5245 self.max_path_id_with_cids = max_path_id;
+5246 }
+5247 }
+5248
+5249 /// Populates a packet with frames
+5250 ///
+5251 /// This tries to fit as many frames as possible into the packet.
+5252 ///
+5253 /// *path_exclusive_only* means to only build frames which can only be sent on this
+5254 /// *path. This is used in multipath for backup paths while there is still an active
+5255 /// *path.
+5256 fn populate_packet<'a, 'b>(
+5257 &mut self,
+5258 now: Instant,
+5259 space_id: SpaceId,
+5260 path_id: PathId,
+5261 path_exclusive_only: bool,
+5262 builder: &mut PacketBuilder<'a, 'b>,
+5263 ) {
+5264 let pn = builder.exact_number;
+5265 let is_multipath_negotiated = self.is_multipath_negotiated();
+5266 let stats = &mut self.stats.frame_tx;
+5267 let space = &mut self.spaces[space_id];
+5268 let path = &mut self.paths.get_mut(&path_id).expect("known path").data;
+5269 let is_0rtt = space_id == SpaceId::Data && space.crypto.is_none();
+5270 space
+5271 .for_path(path_id)
+5272 .pending_acks
+5273 .maybe_ack_non_eliciting();
+5274
+5275 // HANDSHAKE_DONE
+5276 if !is_0rtt && mem::replace(&mut space.pending.handshake_done, false) {
+5277 builder.write_frame(frame::HandshakeDone, stats);
+5278 }
+5279
+5280 // REACH_OUT
+5281 // TODO(@divma): path explusive considerations
+5282 if let Some((round, addresses)) = space.pending.reach_out.as_mut() {
+5283 while let Some(local_addr) = addresses.pop() {
+5284 let reach_out = frame::ReachOut::new(*round, local_addr);
+5285 if builder.frame_space_remaining() > reach_out.size() {
+5286 builder.write_frame(reach_out, stats);
+5287 } else {
+5288 addresses.push(local_addr);
+5289 break;
+5290 }
+5291 }
+5292 if addresses.is_empty() {
+5293 space.pending.reach_out = None;
+5294 }
+5295 }
+5296
+5297 // OBSERVED_ADDR
+5298 if !path_exclusive_only
+5299 && space_id == SpaceId::Data
+5300 && self
+5301 .config
+5302 .address_discovery_role
+5303 .should_report(&self.peer_params.address_discovery_role)
+5304 && (!path.observed_addr_sent || space.pending.observed_addr)
+5305 {
+5306 let frame =
+5307 frame::ObservedAddr::new(path.network_path.remote, self.next_observed_addr_seq_no);
+5308 if builder.frame_space_remaining() > frame.size() {
+5309 builder.write_frame(frame, stats);
5310
-5311 // PATH_RESPONSE
-5312 if builder.frame_space_remaining() > frame::PathResponse::SIZE_BOUND
-5313 && space_id == SpaceId::Data
-5314 {
-5315 if let Some(token) = path.path_responses.pop_on_path(path.network_path) {
-5316 let response = frame::PathResponse(token);
-5317 trace!(frame = %response);
-5318 builder.write_frame(response, stats);
-5319 builder.require_padding();
-5320
-5321 // NOTE: this is technically not required but might be useful to ride the
-5322 // request/response nature of path challenges to refresh an observation
-5323 // Since PATH_RESPONSE is a probing frame, this is allowed by the spec.
-5324 if space_id == SpaceId::Data
-5325 && self
-5326 .config
-5327 .address_discovery_role
-5328 .should_report(&self.peer_params.address_discovery_role)
-5329 {
-5330 let frame = frame::ObservedAddr::new(
-5331 path.network_path.remote,
-5332 self.next_observed_addr_seq_no,
-5333 );
-5334 if builder.frame_space_remaining() > frame.size() {
-5335 builder.write_frame(frame, stats);
-5336
-5337 self.next_observed_addr_seq_no =
-5338 self.next_observed_addr_seq_no.saturating_add(1u8);
-5339 path.observed_addr_sent = true;
-5340
-5341 space.pending.observed_addr = false;
-5342 }
-5343 }
-5344 }
-5345 }
-5346
-5347 // CRYPTO
-5348 while !path_exclusive_only
-5349 && builder.frame_space_remaining() > frame::Crypto::SIZE_BOUND
-5350 && !is_0rtt
-5351 {
-5352 let mut frame = match space.pending.crypto.pop_front() {
-5353 Some(x) => x,
-5354 None => break,
-5355 };
+5311 self.next_observed_addr_seq_no = self.next_observed_addr_seq_no.saturating_add(1u8);
+5312 path.observed_addr_sent = true;
+5313
+5314 space.pending.observed_addr = false;
+5315 }
+5316 }
+5317
+5318 // PING
+5319 if mem::replace(&mut space.for_path(path_id).ping_pending, false) {
+5320 builder.write_frame(frame::Ping, stats);
+5321 }
+5322
+5323 // IMMEDIATE_ACK
+5324 if mem::replace(&mut space.for_path(path_id).immediate_ack_pending, false) {
+5325 debug_assert_eq!(
+5326 space_id,
+5327 SpaceId::Data,
+5328 "immediate acks must be sent in the data space"
+5329 );
+5330 builder.write_frame(frame::ImmediateAck, stats);
+5331 }
+5332
+5333 // ACK
+5334 // TODO(flub): Should this send acks for this path anyway?
+5335
+5336 if !path_exclusive_only {
+5337 for path_id in space
+5338 .number_spaces
+5339 .iter_mut()
+5340 .filter(|(_, pns)| pns.pending_acks.can_send())
+5341 .map(|(&path_id, _)| path_id)
+5342 .collect::<Vec<_>>()
+5343 {
+5344 Self::populate_acks(
+5345 now,
+5346 self.receiving_ecn,
+5347 path_id,
+5348 space_id,
+5349 space,
+5350 is_multipath_negotiated,
+5351 builder,
+5352 stats,
+5353 );
+5354 }
+5355 }
5356
-5357 // Calculate the maximum amount of crypto data we can store in the buffer.
-5358 // Since the offset is known, we can reserve the exact size required to encode it.
-5359 // For length we reserve 2bytes which allows to encode up to 2^14,
-5360 // which is more than what fits into normally sized QUIC frames.
-5361 let max_crypto_data_size = builder.frame_space_remaining()
-5362 - 1 // Frame Type
-5363 - VarInt::size(unsafe { VarInt::from_u64_unchecked(frame.offset) })
-5364 - 2; // Maximum encoded length for frame size, given we send less than 2^14 bytes
-5365
-5366 let len = frame
-5367 .data
-5368 .len()
-5369 .min(2usize.pow(14) - 1)
-5370 .min(max_crypto_data_size);
-5371
-5372 let data = frame.data.split_to(len);
-5373 let offset = frame.offset;
-5374 let truncated = frame::Crypto { offset, data };
-5375 builder.write_frame(truncated, stats);
-5376
-5377 if !frame.data.is_empty() {
-5378 frame.offset += len as u64;
-5379 space.pending.crypto.push_front(frame);
-5380 }
+5357 // ACK_FREQUENCY
+5358 if !path_exclusive_only && mem::replace(&mut space.pending.ack_frequency, false) {
+5359 let sequence_number = self.ack_frequency.next_sequence_number();
+5360
+5361 // Safe to unwrap because this is always provided when ACK frequency is enabled
+5362 let config = self.config.ack_frequency_config.as_ref().unwrap();
+5363
+5364 // Ensure the delay is within bounds to avoid a PROTOCOL_VIOLATION error
+5365 let max_ack_delay = self.ack_frequency.candidate_max_ack_delay(
+5366 path.rtt.get(),
+5367 config,
+5368 &self.peer_params,
+5369 );
+5370
+5371 let frame = frame::AckFrequency {
+5372 sequence: sequence_number,
+5373 ack_eliciting_threshold: config.ack_eliciting_threshold,
+5374 request_max_ack_delay: max_ack_delay.as_micros().try_into().unwrap_or(VarInt::MAX),
+5375 reordering_threshold: config.reordering_threshold,
+5376 };
+5377 builder.write_frame(frame, stats);
+5378
+5379 self.ack_frequency
+5380 .ack_frequency_sent(path_id, pn, max_ack_delay);
5381 }
5382
-5383 // TODO(flub): maybe this is much higher priority?
-5384 // PATH_ABANDON
-5385 while !path_exclusive_only
-5386 && space_id == SpaceId::Data
-5387 && frame::PathAbandon::SIZE_BOUND <= builder.frame_space_remaining()
-5388 {
-5389 let Some((abandoned_path_id, error_code)) = space.pending.path_abandon.pop_first()
-5390 else {
-5391 break;
-5392 };
-5393 let frame = frame::PathAbandon {
-5394 path_id: abandoned_path_id,
-5395 error_code,
-5396 };
-5397 builder.write_frame(frame, stats);
-5398
-5399 let ack_delay = self.ack_frequency.max_ack_delay_for_pto();
-5400 // We can't access path here anymore due to borrowing issues.
-5401 let send_pto = self.paths.get(&path_id).unwrap().data.rtt.pto_base() + ack_delay;
-5402 if let Some(abandoned_path) = self.paths.get_mut(&abandoned_path_id) {
-5403 // We only want to set the deadline on the *first* PATH_ABANDON we send.
-5404 // Retransmits shouldn't run this code again
-5405 if matches!(
-5406 abandoned_path.data.abandon_state,
-5407 AbandonState::NotAbandoned
-5408 ) {
-5409 // The peer MUST respond with a corresponding PATH_ABANDON frame.
-5410 // The other peer has 3 * PTO to do that.
-5411 // This uses the PTO of the path we send on!
-5412 // If the PATH_ABANDON comes in within the deadline we're giving here, then this
-5413 // state will be set to `AbandonState::ReceivedPathAbandon`, essentially clearing
-5414 // the deadline. If we receive a frame after the deadline, we error out with a
-5415 // protocol violation.
-5416 // Receiving other frames before the deadline is fine, as those might be packets
-5417 // that were still in-flight.
-5418 abandoned_path.data.abandon_state = AbandonState::ExpectingPathAbandon {
-5419 deadline: now + 3 * send_pto,
-5420 };
-5421
-5422 // At some point, we need to forget about the path.
-5423 // If we do so too early, then we'll have discarded the CIDs of that path and won't
-5424 // handle incoming packets on that path correctly.
-5425 // To give this path enough time, we assume that the peer will have received our
-5426 // PATH_ABANDON within 3 * PTO of the path we sent the abandon on,
-5427 // and then we give the path 3 * PTO time to make it very unlikely that there will
-5428 // still be packets incoming on the path at that point.
-5429 // This timer will actually get reset to a value that's likely to be even earlier
-5430 // once we actually receive the PATH_ABANDON frame itself.
-5431 let abandoned_pto =
-5432 self.paths.get(&path_id).unwrap().data.rtt.pto_base() + ack_delay;
-5433 self.timers.set(
-5434 Timer::PerPath(abandoned_path_id, PathTimer::DiscardPath),
-5435 now + 3 * send_pto + 3 * abandoned_pto,
-5436 self.qlog.with_time(now),
-5437 );
-5438 }
-5439 } else {
-5440 warn!("sent PATH_ABANDON after path was already discarded");
-5441 }
-5442 }
-5443
-5444 // PATH_STATUS_AVAILABLE & PATH_STATUS_BACKUP
-5445 while !path_exclusive_only
-5446 && space_id == SpaceId::Data
-5447 && frame::PathStatusAvailable::SIZE_BOUND <= builder.frame_space_remaining()
-5448 {
-5449 let Some(path_id) = space.pending.path_status.pop_first() else {
-5450 break;
-5451 };
-5452 let Some(path) = self.paths.get(&path_id).map(|path_state| &path_state.data) else {
-5453 trace!(%path_id, "discarding queued path status for unknown path");
-5454 continue;
-5455 };
-5456
-5457 let seq = path.status.seq();
-5458 match path.local_status() {
-5459 PathStatus::Available => {
-5460 let frame = frame::PathStatusAvailable {
-5461 path_id,
-5462 status_seq_no: seq,
-5463 };
-5464 builder.write_frame(frame, stats);
-5465 }
-5466 PathStatus::Backup => {
-5467 let frame = frame::PathStatusBackup {
-5468 path_id,
-5469 status_seq_no: seq,
-5470 };
-5471 builder.write_frame(frame, stats);
-5472 }
-5473 }
-5474 }
-5475
-5476 // MAX_PATH_ID
-5477 if space_id == SpaceId::Data
-5478 && space.pending.max_path_id
-5479 && frame::MaxPathId::SIZE_BOUND <= builder.frame_space_remaining()
-5480 {
-5481 let frame = frame::MaxPathId(self.local_max_path_id);
-5482 builder.write_frame(frame, stats);
-5483 space.pending.max_path_id = false;
-5484 }
-5485
-5486 // PATHS_BLOCKED
-5487 if space_id == SpaceId::Data
-5488 && space.pending.paths_blocked
-5489 && frame::PathsBlocked::SIZE_BOUND <= builder.frame_space_remaining()
-5490 {
-5491 let frame = frame::PathsBlocked(self.remote_max_path_id);
-5492 builder.write_frame(frame, stats);
-5493 space.pending.paths_blocked = false;
-5494 }
-5495
-5496 // PATH_CIDS_BLOCKED
-5497 while space_id == SpaceId::Data
-5498 && frame::PathCidsBlocked::SIZE_BOUND <= builder.frame_space_remaining()
-5499 {
-5500 let Some(path_id) = space.pending.path_cids_blocked.pop_first() else {
-5501 break;
-5502 };
-5503 let next_seq = match self.rem_cids.get(&path_id) {
-5504 Some(cid_queue) => VarInt(cid_queue.active_seq() + 1),
-5505 None => VarInt(0),
-5506 };
-5507 let frame = frame::PathCidsBlocked { path_id, next_seq };
-5508 builder.write_frame(frame, stats);
+5383 // PATH_CHALLENGE
+5384 if builder.frame_space_remaining() > frame::PathChallenge::SIZE_BOUND
+5385 && space_id == SpaceId::Data
+5386 && path.send_new_challenge
+5387 && !self.state.is_closed()
+5388 // we don't want to send new challenges if we are already closing
+5389 {
+5390 path.send_new_challenge = false;
+5391
+5392 // Generate a new challenge every time we send a new PATH_CHALLENGE
+5393 let token = self.rng.random();
+5394 let info = paths::SentChallengeInfo {
+5395 sent_instant: now,
+5396 network_path: path.network_path,
+5397 };
+5398 path.challenges_sent.insert(token, info);
+5399 let challenge = frame::PathChallenge(token);
+5400 trace!(frame = %challenge);
+5401 builder.write_frame(challenge, stats);
+5402 builder.require_padding();
+5403 let pto = self.ack_frequency.max_ack_delay_for_pto() + path.rtt.pto_base();
+5404 self.timers.set(
+5405 Timer::PerPath(path_id, PathTimer::PathChallengeLost),
+5406 now + pto,
+5407 self.qlog.with_time(now),
+5408 );
+5409
+5410 if is_multipath_negotiated && !path.validated && path.send_new_challenge {
+5411 // queue informing the path status along with the challenge
+5412 space.pending.path_status.insert(path_id);
+5413 }
+5414
+5415 // Always include an OBSERVED_ADDR frame with a PATH_CHALLENGE, regardless
+5416 // of whether one has already been sent on this path.
+5417 if space_id == SpaceId::Data
+5418 && self
+5419 .config
+5420 .address_discovery_role
+5421 .should_report(&self.peer_params.address_discovery_role)
+5422 {
+5423 let frame = frame::ObservedAddr::new(
+5424 path.network_path.remote,
+5425 self.next_observed_addr_seq_no,
+5426 );
+5427 if builder.frame_space_remaining() > frame.size() {
+5428 builder.write_frame(frame, stats);
+5429
+5430 self.next_observed_addr_seq_no =
+5431 self.next_observed_addr_seq_no.saturating_add(1u8);
+5432 path.observed_addr_sent = true;
+5433
+5434 space.pending.observed_addr = false;
+5435 }
+5436 }
+5437 }
+5438
+5439 // PATH_RESPONSE
+5440 if builder.frame_space_remaining() > frame::PathResponse::SIZE_BOUND
+5441 && space_id == SpaceId::Data
+5442 {
+5443 if let Some(token) = path.path_responses.pop_on_path(path.network_path) {
+5444 let response = frame::PathResponse(token);
+5445 trace!(frame = %response);
+5446 builder.write_frame(response, stats);
+5447 builder.require_padding();
+5448
+5449 // NOTE: this is technically not required but might be useful to ride the
+5450 // request/response nature of path challenges to refresh an observation
+5451 // Since PATH_RESPONSE is a probing frame, this is allowed by the spec.
+5452 if space_id == SpaceId::Data
+5453 && self
+5454 .config
+5455 .address_discovery_role
+5456 .should_report(&self.peer_params.address_discovery_role)
+5457 {
+5458 let frame = frame::ObservedAddr::new(
+5459 path.network_path.remote,
+5460 self.next_observed_addr_seq_no,
+5461 );
+5462 if builder.frame_space_remaining() > frame.size() {
+5463 builder.write_frame(frame, stats);
+5464
+5465 self.next_observed_addr_seq_no =
+5466 self.next_observed_addr_seq_no.saturating_add(1u8);
+5467 path.observed_addr_sent = true;
+5468
+5469 space.pending.observed_addr = false;
+5470 }
+5471 }
+5472 }
+5473 }
+5474
+5475 // CRYPTO
+5476 while !path_exclusive_only
+5477 && builder.frame_space_remaining() > frame::Crypto::SIZE_BOUND
+5478 && !is_0rtt
+5479 {
+5480 let mut frame = match space.pending.crypto.pop_front() {
+5481 Some(x) => x,
+5482 None => break,
+5483 };
+5484
+5485 // Calculate the maximum amount of crypto data we can store in the buffer.
+5486 // Since the offset is known, we can reserve the exact size required to encode it.
+5487 // For length we reserve 2bytes which allows to encode up to 2^14,
+5488 // which is more than what fits into normally sized QUIC frames.
+5489 let max_crypto_data_size = builder.frame_space_remaining()
+5490 - 1 // Frame Type
+5491 - VarInt::size(unsafe { VarInt::from_u64_unchecked(frame.offset) })
+5492 - 2; // Maximum encoded length for frame size, given we send less than 2^14 bytes
+5493
+5494 let len = frame
+5495 .data
+5496 .len()
+5497 .min(2usize.pow(14) - 1)
+5498 .min(max_crypto_data_size);
+5499
+5500 let data = frame.data.split_to(len);
+5501 let offset = frame.offset;
+5502 let truncated = frame::Crypto { offset, data };
+5503 builder.write_frame(truncated, stats);
+5504
+5505 if !frame.data.is_empty() {
+5506 frame.offset += len as u64;
+5507 space.pending.crypto.push_front(frame);
+5508 }
5509 }
5510
-5511 // RESET_STREAM, STOP_SENDING, MAX_DATA, MAX_STREAM_DATA, MAX_STREAMS
-5512 if space_id == SpaceId::Data {
-5513 self.streams
-5514 .write_control_frames(builder, &mut space.pending, stats);
-5515 }
-5516
-5517 // NEW_CONNECTION_ID
-5518 let cid_len = self
-5519 .local_cid_state
-5520 .values()
-5521 .map(|cid_state| cid_state.cid_len())
-5522 .max()
-5523 .expect("some local CID state must exist");
-5524 let new_cid_size_bound =
-5525 frame::NewConnectionId::size_bound(is_multipath_negotiated, cid_len);
-5526 while !path_exclusive_only && builder.frame_space_remaining() > new_cid_size_bound {
-5527 let issued = match space.pending.new_cids.pop() {
-5528 Some(x) => x,
-5529 None => break,
-5530 };
-5531 let retire_prior_to = self
-5532 .local_cid_state
-5533 .get(&issued.path_id)
-5534 .map(|cid_state| cid_state.retire_prior_to())
-5535 .unwrap_or_else(|| panic!("missing local CID state for path={}", issued.path_id));
-5536
-5537 let cid_path_id = match is_multipath_negotiated {
-5538 true => Some(issued.path_id),
-5539 false => {
-5540 debug_assert_eq!(issued.path_id, PathId::ZERO);
-5541 None
-5542 }
-5543 };
-5544 let frame = frame::NewConnectionId {
-5545 path_id: cid_path_id,
-5546 sequence: issued.sequence,
-5547 retire_prior_to,
-5548 id: issued.id,
-5549 reset_token: issued.reset_token,
-5550 };
-5551 builder.write_frame(frame, stats);
-5552 }
-5553
-5554 // RETIRE_CONNECTION_ID
-5555 let retire_cid_bound = frame::RetireConnectionId::size_bound(is_multipath_negotiated);
-5556 while !path_exclusive_only && builder.frame_space_remaining() > retire_cid_bound {
-5557 let (path_id, sequence) = match space.pending.retire_cids.pop() {
-5558 Some((PathId::ZERO, seq)) if !is_multipath_negotiated => (None, seq),
-5559 Some((path_id, seq)) => (Some(path_id), seq),
-5560 None => break,
-5561 };
-5562 let frame = frame::RetireConnectionId { path_id, sequence };
-5563 builder.write_frame(frame, stats);
-5564 }
-5565
-5566 // DATAGRAM
-5567 let mut sent_datagrams = false;
-5568 while !path_exclusive_only
-5569 && builder.frame_space_remaining() > Datagram::SIZE_BOUND
-5570 && space_id == SpaceId::Data
-5571 {
-5572 match self.datagrams.write(builder, stats) {
-5573 true => {
-5574 sent_datagrams = true;
-5575 }
-5576 false => break,
-5577 }
-5578 }
-5579 if self.datagrams.send_blocked && sent_datagrams {
-5580 self.events.push_back(Event::DatagramsUnblocked);
-5581 self.datagrams.send_blocked = false;
-5582 }
-5583
-5584 let path = &mut self.paths.get_mut(&path_id).expect("known path").data;
-5585
-5586 // NEW_TOKEN
-5587 if !path_exclusive_only {
-5588 while let Some(network_path) = space.pending.new_tokens.pop() {
-5589 debug_assert_eq!(space_id, SpaceId::Data);
-5590 let ConnectionSide::Server { server_config } = &self.side else {
-5591 panic!("NEW_TOKEN frames should not be enqueued by clients");
-5592 };
-5593
-5594 if !network_path.is_probably_same_path(&path.network_path) {
-5595 // NEW_TOKEN frames contain tokens bound to a client's IP address, and are only
-5596 // useful if used from the same IP address. Thus, we abandon enqueued NEW_TOKEN
-5597 // frames upon an path change. Instead, when the new path becomes validated,
-5598 // NEW_TOKEN frames may be enqueued for the new path instead.
-5599 continue;
+5511 // TODO(flub): maybe this is much higher priority?
+5512 // PATH_ABANDON
+5513 while !path_exclusive_only
+5514 && space_id == SpaceId::Data
+5515 && frame::PathAbandon::SIZE_BOUND <= builder.frame_space_remaining()
+5516 {
+5517 let Some((abandoned_path_id, error_code)) = space.pending.path_abandon.pop_first()
+5518 else {
+5519 break;
+5520 };
+5521 let frame = frame::PathAbandon {
+5522 path_id: abandoned_path_id,
+5523 error_code,
+5524 };
+5525 builder.write_frame(frame, stats);
+5526
+5527 let ack_delay = self.ack_frequency.max_ack_delay_for_pto();
+5528 // We can't access path here anymore due to borrowing issues.
+5529 let send_pto = self.paths.get(&path_id).unwrap().data.rtt.pto_base() + ack_delay;
+5530 if let Some(abandoned_path) = self.paths.get_mut(&abandoned_path_id) {
+5531 // We only want to set the deadline on the *first* PATH_ABANDON we send.
+5532 // Retransmits shouldn't run this code again
+5533 if matches!(
+5534 abandoned_path.data.abandon_state,
+5535 AbandonState::NotAbandoned
+5536 ) {
+5537 // The peer MUST respond with a corresponding PATH_ABANDON frame.
+5538 // The other peer has 3 * PTO to do that.
+5539 // This uses the PTO of the path we send on!
+5540 // If the PATH_ABANDON comes in within the deadline we're giving here, then this
+5541 // state will be set to `AbandonState::ReceivedPathAbandon`, essentially clearing
+5542 // the deadline. If we receive a frame after the deadline, we error out with a
+5543 // protocol violation.
+5544 // Receiving other frames before the deadline is fine, as those might be packets
+5545 // that were still in-flight.
+5546 abandoned_path.data.abandon_state = AbandonState::ExpectingPathAbandon {
+5547 deadline: now + 3 * send_pto,
+5548 };
+5549
+5550 // At some point, we need to forget about the path.
+5551 // If we do so too early, then we'll have discarded the CIDs of that path and won't
+5552 // handle incoming packets on that path correctly.
+5553 // To give this path enough time, we assume that the peer will have received our
+5554 // PATH_ABANDON within 3 * PTO of the path we sent the abandon on,
+5555 // and then we give the path 3 * PTO time to make it very unlikely that there will
+5556 // still be packets incoming on the path at that point.
+5557 // This timer will actually get reset to a value that's likely to be even earlier
+5558 // once we actually receive the PATH_ABANDON frame itself.
+5559 let abandoned_pto =
+5560 self.paths.get(&path_id).unwrap().data.rtt.pto_base() + ack_delay;
+5561 self.timers.set(
+5562 Timer::PerPath(abandoned_path_id, PathTimer::DiscardPath),
+5563 now + 3 * send_pto + 3 * abandoned_pto,
+5564 self.qlog.with_time(now),
+5565 );
+5566 }
+5567 } else {
+5568 warn!("sent PATH_ABANDON after path was already discarded");
+5569 }
+5570 }
+5571
+5572 // PATH_STATUS_AVAILABLE & PATH_STATUS_BACKUP
+5573 while !path_exclusive_only
+5574 && space_id == SpaceId::Data
+5575 && frame::PathStatusAvailable::SIZE_BOUND <= builder.frame_space_remaining()
+5576 {
+5577 let Some(path_id) = space.pending.path_status.pop_first() else {
+5578 break;
+5579 };
+5580 let Some(path) = self.paths.get(&path_id).map(|path_state| &path_state.data) else {
+5581 trace!(%path_id, "discarding queued path status for unknown path");
+5582 continue;
+5583 };
+5584
+5585 let seq = path.status.seq();
+5586 match path.local_status() {
+5587 PathStatus::Available => {
+5588 let frame = frame::PathStatusAvailable {
+5589 path_id,
+5590 status_seq_no: seq,
+5591 };
+5592 builder.write_frame(frame, stats);
+5593 }
+5594 PathStatus::Backup => {
+5595 let frame = frame::PathStatusBackup {
+5596 path_id,
+5597 status_seq_no: seq,
+5598 };
+5599 builder.write_frame(frame, stats);
5600 }
-5601
-5602 let token = Token::new(
-5603 TokenPayload::Validation {
-5604 ip: network_path.remote.ip(),
-5605 issued: server_config.time_source.now(),
-5606 },
-5607 &mut self.rng,
-5608 );
-5609 let new_token = NewToken {
-5610 token: token.encode(&*server_config.token_key).into(),
-5611 };
-5612
-5613 if builder.frame_space_remaining() < new_token.size() {
-5614 space.pending.new_tokens.push(network_path);
-5615 break;
-5616 }
-5617
-5618 builder.write_frame(new_token, stats);
-5619 builder.retransmits_mut().new_tokens.push(network_path);
-5620 }
-5621 }
-5622
-5623 // STREAM
-5624 if !path_exclusive_only && space_id == SpaceId::Data {
-5625 self.streams
-5626 .write_stream_frames(builder, self.config.send_fairness, stats);
-5627 }
-5628
-5629 // ADD_ADDRESS
-5630 // TODO(@divma): check if we need to do path exclusive filters
-5631 while space_id == SpaceId::Data
-5632 && frame::AddAddress::SIZE_BOUND <= builder.frame_space_remaining()
-5633 {
-5634 if let Some(added_address) = space.pending.add_address.pop_last() {
-5635 builder.write_frame(added_address, stats);
-5636 } else {
-5637 break;
-5638 }
-5639 }
-5640
-5641 // REMOVE_ADDRESS
-5642 while space_id == SpaceId::Data
-5643 && frame::RemoveAddress::SIZE_BOUND <= builder.frame_space_remaining()
-5644 {
-5645 if let Some(removed_address) = space.pending.remove_address.pop_last() {
-5646 builder.write_frame(removed_address, stats);
-5647 } else {
-5648 break;
-5649 }
-5650 }
-5651 }
-5652
-5653 /// Write pending ACKs into a buffer
-5654 fn populate_acks<'a, 'b>(
-5655 now: Instant,
-5656 receiving_ecn: bool,
-5657 path_id: PathId,
-5658 space_id: SpaceId,
-5659 space: &mut PacketSpace,
-5660 is_multipath_negotiated: bool,
-5661 builder: &mut PacketBuilder<'a, 'b>,
-5662 stats: &mut FrameStats,
-5663 ) {
-5664 // 0-RTT packets must never carry acks (which would have to be of handshake packets)
-5665 debug_assert!(space.crypto.is_some(), "tried to send ACK in 0-RTT");
-5666
-5667 debug_assert!(
-5668 is_multipath_negotiated || path_id == PathId::ZERO,
-5669 "Only PathId::ZERO allowed without multipath (have {path_id:?})"
-5670 );
-5671 if is_multipath_negotiated {
-5672 debug_assert!(
-5673 space_id == SpaceId::Data || path_id == PathId::ZERO,
-5674 "path acks must be sent in 1RTT space (have {space_id:?})"
-5675 );
-5676 }
-5677
-5678 let pns = space.for_path(path_id);
-5679 let ranges = pns.pending_acks.ranges();
-5680 debug_assert!(!ranges.is_empty(), "can not send empty ACK range");
-5681 let ecn = if receiving_ecn {
-5682 Some(&pns.ecn_counters)
-5683 } else {
-5684 None
-5685 };
-5686
-5687 let delay_micros = pns.pending_acks.ack_delay(now).as_micros() as u64;
-5688 // TODO: This should come from `TransportConfig` if that gets configurable.
-5689 let ack_delay_exp = TransportParameters::default().ack_delay_exponent;
-5690 let delay = delay_micros >> ack_delay_exp.into_inner();
-5691
-5692 if is_multipath_negotiated && space_id == SpaceId::Data {
-5693 if !ranges.is_empty() {
-5694 let frame = frame::PathAck::encoder(path_id, delay, ranges, ecn);
-5695 builder.write_frame(frame, stats);
-5696 }
-5697 } else {
-5698 builder.write_frame(frame::Ack::encoder(delay, ranges, ecn), stats);
-5699 }
-5700 }
-5701
-5702 fn close_common(&mut self) {
-5703 trace!("connection closed");
-5704 self.timers.reset();
-5705 }
-5706
-5707 fn set_close_timer(&mut self, now: Instant) {
-5708 // QUIC-MULTIPATH § 2.6 Connection Closure: draining for 3*PTO with PTO the max of
-5709 // the PTO for all paths.
-5710 let pto_max = self.pto_max_path(self.highest_space, true);
-5711 self.timers.set(
-5712 Timer::Conn(ConnTimer::Close),
-5713 now + 3 * pto_max,
-5714 self.qlog.with_time(now),
-5715 );
-5716 }
-5717
-5718 /// Handle transport parameters received from the peer
-5719 ///
-5720 /// *rem_cid* and *loc_cid* are the source and destination CIDs respectively of the
-5721 /// *packet into which the transport parameters arrived.
-5722 fn handle_peer_params(
-5723 &mut self,
-5724 params: TransportParameters,
-5725 loc_cid: ConnectionId,
-5726 rem_cid: ConnectionId,
-5727 now: Instant,
-5728 ) -> Result<(), TransportError> {
-5729 if Some(self.orig_rem_cid) != params.initial_src_cid
-5730 || (self.side.is_client()
-5731 && (Some(self.initial_dst_cid) != params.original_dst_cid
-5732 || self.retry_src_cid != params.retry_src_cid))
-5733 {
-5734 return Err(TransportError::TRANSPORT_PARAMETER_ERROR(
-5735 "CID authentication failure",
-5736 ));
-5737 }
-5738 if params.initial_max_path_id.is_some() && (loc_cid.is_empty() || rem_cid.is_empty()) {
-5739 return Err(TransportError::PROTOCOL_VIOLATION(
-5740 "multipath must not use zero-length CIDs",
-5741 ));
-5742 }
-5743
-5744 self.set_peer_params(params);
-5745 self.qlog.emit_peer_transport_params_received(self, now);
-5746
-5747 Ok(())
-5748 }
-5749
-5750 fn set_peer_params(&mut self, params: TransportParameters) {
-5751 self.streams.set_params(¶ms);
-5752 self.idle_timeout =
-5753 negotiate_max_idle_timeout(self.config.max_idle_timeout, Some(params.max_idle_timeout));
-5754 trace!("negotiated max idle timeout {:?}", self.idle_timeout);
-5755
-5756 if let Some(ref info) = params.preferred_address {
-5757 // During the handshake PathId::ZERO exists.
-5758 self.rem_cids.get_mut(&PathId::ZERO).expect("not yet abandoned").insert(frame::NewConnectionId {
-5759 path_id: None,
-5760 sequence: 1,
-5761 id: info.connection_id,
-5762 reset_token: info.stateless_reset_token,
-5763 retire_prior_to: 0,
-5764 })
-5765 .expect(
-5766 "preferred address CID is the first received, and hence is guaranteed to be legal",
-5767 );
-5768 let remote = self.path_data(PathId::ZERO).network_path.remote;
-5769 self.set_reset_token(PathId::ZERO, remote, info.stateless_reset_token);
-5770 }
-5771 self.ack_frequency.peer_max_ack_delay = get_max_ack_delay(¶ms);
-5772
-5773 let mut multipath_enabled = None;
-5774 if let (Some(local_max_path_id), Some(remote_max_path_id)) = (
-5775 self.config.get_initial_max_path_id(),
-5776 params.initial_max_path_id,
-5777 ) {
-5778 // multipath is enabled, register the local and remote maximums
-5779 self.local_max_path_id = local_max_path_id;
-5780 self.remote_max_path_id = remote_max_path_id;
-5781 let initial_max_path_id = local_max_path_id.min(remote_max_path_id);
-5782 debug!(%initial_max_path_id, "multipath negotiated");
-5783 multipath_enabled = Some(initial_max_path_id);
-5784 }
-5785
-5786 if let Some((max_locally_allowed_remote_addresses, max_remotely_allowed_remote_addresses)) =
-5787 self.config
-5788 .max_remote_nat_traversal_addresses
-5789 .zip(params.max_remote_nat_traversal_addresses)
-5790 {
-5791 if let Some(max_initial_paths) =
-5792 multipath_enabled.map(|path_id| path_id.saturating_add(1u8))
-5793 {
-5794 let max_local_addresses = max_remotely_allowed_remote_addresses.get();
-5795 let max_remote_addresses = max_locally_allowed_remote_addresses.get();
-5796 self.iroh_hp =
-5797 iroh_hp::State::new(max_remote_addresses, max_local_addresses, self.side());
-5798 debug!(
-5799 %max_remote_addresses, %max_local_addresses,
-5800 "iroh hole punching negotiated"
-5801 );
-5802
-5803 match self.side() {
-5804 Side::Client => {
-5805 if max_initial_paths.as_u32() < max_remote_addresses as u32 + 1 {
-5806 // in this case the client might try to open `max_remote_addresses` new
-5807 // paths, but the current multipath configuration will not allow it
-5808 warn!(%max_initial_paths, %max_remote_addresses, "local client configuration might cause nat traversal issues")
-5809 } else if max_local_addresses as u64
-5810 > params.active_connection_id_limit.into_inner()
-5811 {
-5812 // the server allows us to send at most `params.active_connection_id_limit`
-5813 // but they might need at least `max_local_addresses` to effectively send
-5814 // `PATH_CHALLENGE` frames to each advertised local address
-5815 warn!(%max_local_addresses, remote_cid_limit=%params.active_connection_id_limit.into_inner(), "remote server configuration might cause nat traversal issues")
-5816 }
-5817 }
-5818 Side::Server => {
-5819 if (max_initial_paths.as_u32() as u64) < crate::LOC_CID_COUNT {
-5820 warn!(%max_initial_paths, local_cid_limit=%crate::LOC_CID_COUNT, "local server configuration might cause nat traversal issues")
-5821 }
-5822 }
-5823 }
-5824 } else {
-5825 debug!("iroh nat traversal enabled for both endpoints, but multipath is missing")
-5826 }
+5601 }
+5602 }
+5603
+5604 // MAX_PATH_ID
+5605 if space_id == SpaceId::Data
+5606 && space.pending.max_path_id
+5607 && frame::MaxPathId::SIZE_BOUND <= builder.frame_space_remaining()
+5608 {
+5609 let frame = frame::MaxPathId(self.local_max_path_id);
+5610 builder.write_frame(frame, stats);
+5611 space.pending.max_path_id = false;
+5612 }
+5613
+5614 // PATHS_BLOCKED
+5615 if space_id == SpaceId::Data
+5616 && space.pending.paths_blocked
+5617 && frame::PathsBlocked::SIZE_BOUND <= builder.frame_space_remaining()
+5618 {
+5619 let frame = frame::PathsBlocked(self.remote_max_path_id);
+5620 builder.write_frame(frame, stats);
+5621 space.pending.paths_blocked = false;
+5622 }
+5623
+5624 // PATH_CIDS_BLOCKED
+5625 while space_id == SpaceId::Data
+5626 && frame::PathCidsBlocked::SIZE_BOUND <= builder.frame_space_remaining()
+5627 {
+5628 let Some(path_id) = space.pending.path_cids_blocked.pop_first() else {
+5629 break;
+5630 };
+5631 let next_seq = match self.rem_cids.get(&path_id) {
+5632 Some(cid_queue) => VarInt(cid_queue.active_seq() + 1),
+5633 None => VarInt(0),
+5634 };
+5635 let frame = frame::PathCidsBlocked { path_id, next_seq };
+5636 builder.write_frame(frame, stats);
+5637 }
+5638
+5639 // RESET_STREAM, STOP_SENDING, MAX_DATA, MAX_STREAM_DATA, MAX_STREAMS
+5640 if space_id == SpaceId::Data {
+5641 self.streams
+5642 .write_control_frames(builder, &mut space.pending, stats);
+5643 }
+5644
+5645 // NEW_CONNECTION_ID
+5646 let cid_len = self
+5647 .local_cid_state
+5648 .values()
+5649 .map(|cid_state| cid_state.cid_len())
+5650 .max()
+5651 .expect("some local CID state must exist");
+5652 let new_cid_size_bound =
+5653 frame::NewConnectionId::size_bound(is_multipath_negotiated, cid_len);
+5654 while !path_exclusive_only && builder.frame_space_remaining() > new_cid_size_bound {
+5655 let issued = match space.pending.new_cids.pop() {
+5656 Some(x) => x,
+5657 None => break,
+5658 };
+5659 let retire_prior_to = self
+5660 .local_cid_state
+5661 .get(&issued.path_id)
+5662 .map(|cid_state| cid_state.retire_prior_to())
+5663 .unwrap_or_else(|| panic!("missing local CID state for path={}", issued.path_id));
+5664
+5665 let cid_path_id = match is_multipath_negotiated {
+5666 true => Some(issued.path_id),
+5667 false => {
+5668 debug_assert_eq!(issued.path_id, PathId::ZERO);
+5669 None
+5670 }
+5671 };
+5672 let frame = frame::NewConnectionId {
+5673 path_id: cid_path_id,
+5674 sequence: issued.sequence,
+5675 retire_prior_to,
+5676 id: issued.id,
+5677 reset_token: issued.reset_token,
+5678 };
+5679 builder.write_frame(frame, stats);
+5680 }
+5681
+5682 // RETIRE_CONNECTION_ID
+5683 let retire_cid_bound = frame::RetireConnectionId::size_bound(is_multipath_negotiated);
+5684 while !path_exclusive_only && builder.frame_space_remaining() > retire_cid_bound {
+5685 let (path_id, sequence) = match space.pending.retire_cids.pop() {
+5686 Some((PathId::ZERO, seq)) if !is_multipath_negotiated => (None, seq),
+5687 Some((path_id, seq)) => (Some(path_id), seq),
+5688 None => break,
+5689 };
+5690 let frame = frame::RetireConnectionId { path_id, sequence };
+5691 builder.write_frame(frame, stats);
+5692 }
+5693
+5694 // DATAGRAM
+5695 let mut sent_datagrams = false;
+5696 while !path_exclusive_only
+5697 && builder.frame_space_remaining() > Datagram::SIZE_BOUND
+5698 && space_id == SpaceId::Data
+5699 {
+5700 match self.datagrams.write(builder, stats) {
+5701 true => {
+5702 sent_datagrams = true;
+5703 }
+5704 false => break,
+5705 }
+5706 }
+5707 if self.datagrams.send_blocked && sent_datagrams {
+5708 self.events.push_back(Event::DatagramsUnblocked);
+5709 self.datagrams.send_blocked = false;
+5710 }
+5711
+5712 let path = &mut self.paths.get_mut(&path_id).expect("known path").data;
+5713
+5714 // NEW_TOKEN
+5715 if !path_exclusive_only {
+5716 while let Some(network_path) = space.pending.new_tokens.pop() {
+5717 debug_assert_eq!(space_id, SpaceId::Data);
+5718 let ConnectionSide::Server { server_config } = &self.side else {
+5719 panic!("NEW_TOKEN frames should not be enqueued by clients");
+5720 };
+5721
+5722 if !network_path.is_probably_same_path(&path.network_path) {
+5723 // NEW_TOKEN frames contain tokens bound to a client's IP address, and are only
+5724 // useful if used from the same IP address. Thus, we abandon enqueued NEW_TOKEN
+5725 // frames upon an path change. Instead, when the new path becomes validated,
+5726 // NEW_TOKEN frames may be enqueued for the new path instead.
+5727 continue;
+5728 }
+5729
+5730 let token = Token::new(
+5731 TokenPayload::Validation {
+5732 ip: network_path.remote.ip(),
+5733 issued: server_config.time_source.now(),
+5734 },
+5735 &mut self.rng,
+5736 );
+5737 let new_token = NewToken {
+5738 token: token.encode(&*server_config.token_key).into(),
+5739 };
+5740
+5741 if builder.frame_space_remaining() < new_token.size() {
+5742 space.pending.new_tokens.push(network_path);
+5743 break;
+5744 }
+5745
+5746 builder.write_frame(new_token, stats);
+5747 builder.retransmits_mut().new_tokens.push(network_path);
+5748 }
+5749 }
+5750
+5751 // STREAM
+5752 if !path_exclusive_only && space_id == SpaceId::Data {
+5753 self.streams
+5754 .write_stream_frames(builder, self.config.send_fairness, stats);
+5755 }
+5756
+5757 // ADD_ADDRESS
+5758 // TODO(@divma): check if we need to do path exclusive filters
+5759 while space_id == SpaceId::Data
+5760 && frame::AddAddress::SIZE_BOUND <= builder.frame_space_remaining()
+5761 {
+5762 if let Some(added_address) = space.pending.add_address.pop_last() {
+5763 builder.write_frame(added_address, stats);
+5764 } else {
+5765 break;
+5766 }
+5767 }
+5768
+5769 // REMOVE_ADDRESS
+5770 while space_id == SpaceId::Data
+5771 && frame::RemoveAddress::SIZE_BOUND <= builder.frame_space_remaining()
+5772 {
+5773 if let Some(removed_address) = space.pending.remove_address.pop_last() {
+5774 builder.write_frame(removed_address, stats);
+5775 } else {
+5776 break;
+5777 }
+5778 }
+5779 }
+5780
+5781 /// Write pending ACKs into a buffer
+5782 fn populate_acks<'a, 'b>(
+5783 now: Instant,
+5784 receiving_ecn: bool,
+5785 path_id: PathId,
+5786 space_id: SpaceId,
+5787 space: &mut PacketSpace,
+5788 is_multipath_negotiated: bool,
+5789 builder: &mut PacketBuilder<'a, 'b>,
+5790 stats: &mut FrameStats,
+5791 ) {
+5792 // 0-RTT packets must never carry acks (which would have to be of handshake packets)
+5793 debug_assert!(space.crypto.is_some(), "tried to send ACK in 0-RTT");
+5794
+5795 debug_assert!(
+5796 is_multipath_negotiated || path_id == PathId::ZERO,
+5797 "Only PathId::ZERO allowed without multipath (have {path_id:?})"
+5798 );
+5799 if is_multipath_negotiated {
+5800 debug_assert!(
+5801 space_id == SpaceId::Data || path_id == PathId::ZERO,
+5802 "path acks must be sent in 1RTT space (have {space_id:?})"
+5803 );
+5804 }
+5805
+5806 let pns = space.for_path(path_id);
+5807 let ranges = pns.pending_acks.ranges();
+5808 debug_assert!(!ranges.is_empty(), "can not send empty ACK range");
+5809 let ecn = if receiving_ecn {
+5810 Some(&pns.ecn_counters)
+5811 } else {
+5812 None
+5813 };
+5814
+5815 let delay_micros = pns.pending_acks.ack_delay(now).as_micros() as u64;
+5816 // TODO: This should come from `TransportConfig` if that gets configurable.
+5817 let ack_delay_exp = TransportParameters::default().ack_delay_exponent;
+5818 let delay = delay_micros >> ack_delay_exp.into_inner();
+5819
+5820 if is_multipath_negotiated && space_id == SpaceId::Data {
+5821 if !ranges.is_empty() {
+5822 let frame = frame::PathAck::encoder(path_id, delay, ranges, ecn);
+5823 builder.write_frame(frame, stats);
+5824 }
+5825 } else {
+5826 builder.write_frame(frame::Ack::encoder(delay, ranges, ecn), stats);
5827 }
-5828
-5829 self.peer_params = params;
-5830 let peer_max_udp_payload_size =
-5831 u16::try_from(self.peer_params.max_udp_payload_size.into_inner()).unwrap_or(u16::MAX);
-5832 self.path_data_mut(PathId::ZERO)
-5833 .mtud
-5834 .on_peer_max_udp_payload_size_received(peer_max_udp_payload_size);
-5835 }
-5836
-5837 /// Decrypts a packet, returning the packet number on success
-5838 fn decrypt_packet(
-5839 &mut self,
-5840 now: Instant,
-5841 path_id: PathId,
-5842 packet: &mut Packet,
-5843 ) -> Result<Option<u64>, Option<TransportError>> {
-5844 let result = packet_crypto::decrypt_packet_body(
-5845 packet,
-5846 path_id,
-5847 &self.spaces,
-5848 self.zero_rtt_crypto.as_ref(),
-5849 self.key_phase,
-5850 self.prev_crypto.as_ref(),
-5851 self.next_crypto.as_ref(),
-5852 )?;
-5853
-5854 let result = match result {
-5855 Some(r) => r,
-5856 None => return Ok(None),
-5857 };
-5858
-5859 if result.outgoing_key_update_acked {
-5860 if let Some(prev) = self.prev_crypto.as_mut() {
-5861 prev.end_packet = Some((result.number, now));
-5862 self.set_key_discard_timer(now, packet.header.space());
-5863 }
-5864 }
-5865
-5866 if result.incoming_key_update {
-5867 trace!("key update authenticated");
-5868 self.update_keys(Some((result.number, now)), true);
-5869 self.set_key_discard_timer(now, packet.header.space());
+5828 }
+5829
+5830 fn close_common(&mut self) {
+5831 trace!("connection closed");
+5832 self.timers.reset();
+5833 }
+5834
+5835 fn set_close_timer(&mut self, now: Instant) {
+5836 // QUIC-MULTIPATH § 2.6 Connection Closure: draining for 3*PTO with PTO the max of
+5837 // the PTO for all paths.
+5838 let pto_max = self.pto_max_path(self.highest_space, true);
+5839 self.timers.set(
+5840 Timer::Conn(ConnTimer::Close),
+5841 now + 3 * pto_max,
+5842 self.qlog.with_time(now),
+5843 );
+5844 }
+5845
+5846 /// Handle transport parameters received from the peer
+5847 ///
+5848 /// *rem_cid* and *loc_cid* are the source and destination CIDs respectively of the
+5849 /// *packet into which the transport parameters arrived.
+5850 fn handle_peer_params(
+5851 &mut self,
+5852 params: TransportParameters,
+5853 loc_cid: ConnectionId,
+5854 rem_cid: ConnectionId,
+5855 now: Instant,
+5856 ) -> Result<(), TransportError> {
+5857 if Some(self.orig_rem_cid) != params.initial_src_cid
+5858 || (self.side.is_client()
+5859 && (Some(self.initial_dst_cid) != params.original_dst_cid
+5860 || self.retry_src_cid != params.retry_src_cid))
+5861 {
+5862 return Err(TransportError::TRANSPORT_PARAMETER_ERROR(
+5863 "CID authentication failure",
+5864 ));
+5865 }
+5866 if params.initial_max_path_id.is_some() && (loc_cid.is_empty() || rem_cid.is_empty()) {
+5867 return Err(TransportError::PROTOCOL_VIOLATION(
+5868 "multipath must not use zero-length CIDs",
+5869 ));
5870 }
5871
-5872 Ok(Some(result.number))
-5873 }
+5872 self.set_peer_params(params);
+5873 self.qlog.emit_peer_transport_params_received(self, now);
5874
-5875 fn update_keys(&mut self, end_packet: Option<(u64, Instant)>, remote: bool) {
-5876 trace!("executing key update");
-5877 // Generate keys for the key phase after the one we're switching to, store them in
-5878 // `next_crypto`, make the contents of `next_crypto` current, and move the current keys into
-5879 // `prev_crypto`.
-5880 let new = self
-5881 .crypto
-5882 .next_1rtt_keys()
-5883 .expect("only called for `Data` packets");
-5884 self.key_phase_size = new
-5885 .local
-5886 .confidentiality_limit()
-5887 .saturating_sub(KEY_UPDATE_MARGIN);
-5888 let old = mem::replace(
-5889 &mut self.spaces[SpaceId::Data]
-5890 .crypto
-5891 .as_mut()
-5892 .unwrap() // safe because update_keys() can only be triggered by short packets
-5893 .packet,
-5894 mem::replace(self.next_crypto.as_mut().unwrap(), new),
-5895 );
-5896 self.spaces[SpaceId::Data]
-5897 .iter_paths_mut()
-5898 .for_each(|s| s.sent_with_keys = 0);
-5899 self.prev_crypto = Some(PrevCrypto {
-5900 crypto: old,
-5901 end_packet,
-5902 update_unacked: remote,
-5903 });
-5904 self.key_phase = !self.key_phase;
-5905 }
-5906
-5907 fn peer_supports_ack_frequency(&self) -> bool {
-5908 self.peer_params.min_ack_delay.is_some()
-5909 }
-5910
-5911 /// Send an IMMEDIATE_ACK frame to the remote endpoint
-5912 ///
-5913 /// According to the spec, this will result in an error if the remote endpoint does not support
-5914 /// the Acknowledgement Frequency extension
-5915 pub(crate) fn immediate_ack(&mut self, path_id: PathId) {
-5916 debug_assert_eq!(
-5917 self.highest_space,
-5918 SpaceId::Data,
-5919 "immediate ack must be written in the data space"
-5920 );
-5921 self.spaces[self.highest_space]
-5922 .for_path(path_id)
-5923 .immediate_ack_pending = true;
-5924 }
-5925
-5926 /// Decodes a packet, returning its decrypted payload, so it can be inspected in tests
-5927 #[cfg(test)]
-5928 pub(crate) fn decode_packet(&self, event: &ConnectionEvent) -> Option<Vec<u8>> {
-5929 let (path_id, first_decode, remaining) = match &event.0 {
-5930 ConnectionEventInner::Datagram(DatagramConnectionEvent {
-5931 path_id,
-5932 first_decode,
-5933 remaining,
-5934 ..
-5935 }) => (path_id, first_decode, remaining),
-5936 _ => return None,
-5937 };
-5938
-5939 if remaining.is_some() {
-5940 panic!("Packets should never be coalesced in tests");
-5941 }
-5942
-5943 let decrypted_header = packet_crypto::unprotect_header(
-5944 first_decode.clone(),
-5945 &self.spaces,
-5946 self.zero_rtt_crypto.as_ref(),
-5947 self.peer_params.stateless_reset_token,
-5948 )?;
-5949
-5950 let mut packet = decrypted_header.packet?;
-5951 packet_crypto::decrypt_packet_body(
-5952 &mut packet,
-5953 *path_id,
-5954 &self.spaces,
-5955 self.zero_rtt_crypto.as_ref(),
-5956 self.key_phase,
-5957 self.prev_crypto.as_ref(),
-5958 self.next_crypto.as_ref(),
-5959 )
-5960 .ok()?;
-5961
-5962 Some(packet.payload.to_vec())
+5875 Ok(())
+5876 }
+5877
+5878 fn set_peer_params(&mut self, params: TransportParameters) {
+5879 self.streams.set_params(¶ms);
+5880 self.idle_timeout =
+5881 negotiate_max_idle_timeout(self.config.max_idle_timeout, Some(params.max_idle_timeout));
+5882 trace!("negotiated max idle timeout {:?}", self.idle_timeout);
+5883
+5884 if let Some(ref info) = params.preferred_address {
+5885 // During the handshake PathId::ZERO exists.
+5886 self.rem_cids.get_mut(&PathId::ZERO).expect("not yet abandoned").insert(frame::NewConnectionId {
+5887 path_id: None,
+5888 sequence: 1,
+5889 id: info.connection_id,
+5890 reset_token: info.stateless_reset_token,
+5891 retire_prior_to: 0,
+5892 })
+5893 .expect(
+5894 "preferred address CID is the first received, and hence is guaranteed to be legal",
+5895 );
+5896 let remote = self.path_data(PathId::ZERO).network_path.remote;
+5897 self.set_reset_token(PathId::ZERO, remote, info.stateless_reset_token);
+5898 }
+5899 self.ack_frequency.peer_max_ack_delay = get_max_ack_delay(¶ms);
+5900
+5901 let mut multipath_enabled = None;
+5902 if let (Some(local_max_path_id), Some(remote_max_path_id)) = (
+5903 self.config.get_initial_max_path_id(),
+5904 params.initial_max_path_id,
+5905 ) {
+5906 // multipath is enabled, register the local and remote maximums
+5907 self.local_max_path_id = local_max_path_id;
+5908 self.remote_max_path_id = remote_max_path_id;
+5909 let initial_max_path_id = local_max_path_id.min(remote_max_path_id);
+5910 debug!(%initial_max_path_id, "multipath negotiated");
+5911 multipath_enabled = Some(initial_max_path_id);
+5912 }
+5913
+5914 if let Some((max_locally_allowed_remote_addresses, max_remotely_allowed_remote_addresses)) =
+5915 self.config
+5916 .max_remote_nat_traversal_addresses
+5917 .zip(params.max_remote_nat_traversal_addresses)
+5918 {
+5919 if let Some(max_initial_paths) =
+5920 multipath_enabled.map(|path_id| path_id.saturating_add(1u8))
+5921 {
+5922 let max_local_addresses = max_remotely_allowed_remote_addresses.get();
+5923 let max_remote_addresses = max_locally_allowed_remote_addresses.get();
+5924 self.iroh_hp =
+5925 iroh_hp::State::new(max_remote_addresses, max_local_addresses, self.side());
+5926 debug!(
+5927 %max_remote_addresses, %max_local_addresses,
+5928 "iroh hole punching negotiated"
+5929 );
+5930
+5931 match self.side() {
+5932 Side::Client => {
+5933 if max_initial_paths.as_u32() < max_remote_addresses as u32 + 1 {
+5934 // in this case the client might try to open `max_remote_addresses` new
+5935 // paths, but the current multipath configuration will not allow it
+5936 warn!(%max_initial_paths, %max_remote_addresses, "local client configuration might cause nat traversal issues")
+5937 } else if max_local_addresses as u64
+5938 > params.active_connection_id_limit.into_inner()
+5939 {
+5940 // the server allows us to send at most `params.active_connection_id_limit`
+5941 // but they might need at least `max_local_addresses` to effectively send
+5942 // `PATH_CHALLENGE` frames to each advertised local address
+5943 warn!(%max_local_addresses, remote_cid_limit=%params.active_connection_id_limit.into_inner(), "remote server configuration might cause nat traversal issues")
+5944 }
+5945 }
+5946 Side::Server => {
+5947 if (max_initial_paths.as_u32() as u64) < crate::LOC_CID_COUNT {
+5948 warn!(%max_initial_paths, local_cid_limit=%crate::LOC_CID_COUNT, "local server configuration might cause nat traversal issues")
+5949 }
+5950 }
+5951 }
+5952 } else {
+5953 debug!("iroh nat traversal enabled for both endpoints, but multipath is missing")
+5954 }
+5955 }
+5956
+5957 self.peer_params = params;
+5958 let peer_max_udp_payload_size =
+5959 u16::try_from(self.peer_params.max_udp_payload_size.into_inner()).unwrap_or(u16::MAX);
+5960 self.path_data_mut(PathId::ZERO)
+5961 .mtud
+5962 .on_peer_max_udp_payload_size_received(peer_max_udp_payload_size);
5963 }
5964
-5965 /// The number of bytes of packets containing retransmittable frames that have not been
-5966 /// acknowledged or declared lost.
-5967 #[cfg(test)]
-5968 pub(crate) fn bytes_in_flight(&self) -> u64 {
-5969 // TODO(@divma): consider including for multipath?
-5970 self.path_data(PathId::ZERO).in_flight.bytes
-5971 }
-5972
-5973 /// Number of bytes worth of non-ack-only packets that may be sent
-5974 #[cfg(test)]
-5975 pub(crate) fn congestion_window(&self) -> u64 {
-5976 let path = self.path_data(PathId::ZERO);
-5977 path.congestion
-5978 .window()
-5979 .saturating_sub(path.in_flight.bytes)
-5980 }
+5965 /// Decrypts a packet, returning the packet number on success
+5966 fn decrypt_packet(
+5967 &mut self,
+5968 now: Instant,
+5969 path_id: PathId,
+5970 packet: &mut Packet,
+5971 ) -> Result<Option<u64>, Option<TransportError>> {
+5972 let result = packet_crypto::decrypt_packet_body(
+5973 packet,
+5974 path_id,
+5975 &self.spaces,
+5976 self.zero_rtt_crypto.as_ref(),
+5977 self.key_phase,
+5978 self.prev_crypto.as_ref(),
+5979 self.next_crypto.as_ref(),
+5980 )?;
5981
-5982 /// Whether no timers but keepalive, idle, rtt, pushnewcid, and key discard are running
-5983 #[cfg(test)]
-5984 pub(crate) fn is_idle(&self) -> bool {
-5985 let current_timers = self.timers.values();
-5986 current_timers
-5987 .into_iter()
-5988 .filter(|(timer, _)| {
-5989 !matches!(
-5990 timer,
-5991 Timer::Conn(ConnTimer::KeepAlive)
-5992 | Timer::PerPath(_, PathTimer::PathKeepAlive)
-5993 | Timer::Conn(ConnTimer::PushNewCid)
-5994 | Timer::Conn(ConnTimer::KeyDiscard)
-5995 )
-5996 })
-5997 .min_by_key(|(_, time)| *time)
-5998 .is_none_or(|(timer, _)| timer == Timer::Conn(ConnTimer::Idle))
-5999 }
-6000
-6001 /// Whether explicit congestion notification is in use on outgoing packets.
-6002 #[cfg(test)]
-6003 pub(crate) fn using_ecn(&self) -> bool {
-6004 self.path_data(PathId::ZERO).sending_ecn
-6005 }
-6006
-6007 /// The number of received bytes in the current path
-6008 #[cfg(test)]
-6009 pub(crate) fn total_recvd(&self) -> u64 {
-6010 self.path_data(PathId::ZERO).total_recvd
-6011 }
-6012
-6013 #[cfg(test)]
-6014 pub(crate) fn active_local_cid_seq(&self) -> (u64, u64) {
-6015 self.local_cid_state
-6016 .get(&PathId::ZERO)
-6017 .unwrap()
-6018 .active_seq()
-6019 }
-6020
-6021 #[cfg(test)]
-6022 #[track_caller]
-6023 pub(crate) fn active_local_path_cid_seq(&self, path_id: u32) -> (u64, u64) {
-6024 self.local_cid_state
-6025 .get(&PathId(path_id))
-6026 .unwrap()
-6027 .active_seq()
-6028 }
-6029
-6030 /// Instruct the peer to replace previously issued CIDs by sending a NEW_CONNECTION_ID frame
-6031 /// with updated `retire_prior_to` field set to `v`
-6032 #[cfg(test)]
-6033 pub(crate) fn rotate_local_cid(&mut self, v: u64, now: Instant) {
-6034 let n = self
-6035 .local_cid_state
-6036 .get_mut(&PathId::ZERO)
-6037 .unwrap()
-6038 .assign_retire_seq(v);
-6039 self.endpoint_events
-6040 .push_back(EndpointEventInner::NeedIdentifiers(PathId::ZERO, now, n));
-6041 }
-6042
-6043 /// Check the current active remote CID sequence for `PathId::ZERO`
-6044 #[cfg(test)]
-6045 pub(crate) fn active_rem_cid_seq(&self) -> u64 {
-6046 self.rem_cids.get(&PathId::ZERO).unwrap().active_seq()
-6047 }
-6048
-6049 /// Returns the detected maximum udp payload size for the current path
-6050 #[cfg(test)]
-6051 pub(crate) fn path_mtu(&self, path_id: PathId) -> u16 {
-6052 self.path_data(path_id).current_mtu()
-6053 }
-6054
-6055 /// Triggers path validation on all paths
-6056 #[cfg(test)]
-6057 pub(crate) fn trigger_path_validation(&mut self) {
-6058 for path in self.paths.values_mut() {
-6059 path.data.send_new_challenge = true;
-6060 }
-6061 }
-6062
-6063 /// Whether we have 1-RTT data to send
-6064 ///
-6065 /// This checks for frames that can only be sent in the data space (1-RTT):
-6066 /// - Pending PATH_CHALLENGE frames on the active and previous path if just migrated.
-6067 /// - Pending PATH_RESPONSE frames.
-6068 /// - Pending data to send in STREAM frames.
-6069 /// - Pending DATAGRAM frames to send.
-6070 ///
-6071 /// See also [`PacketSpace::can_send`] which keeps track of all other frame types that
-6072 /// may need to be sent.
-6073 fn can_send_1rtt(&self, path_id: PathId, max_size: usize) -> SendableFrames {
-6074 let path_exclusive = self.paths.get(&path_id).is_some_and(|path| {
-6075 path.data.send_new_challenge
-6076 || path
-6077 .prev
-6078 .as_ref()
-6079 .is_some_and(|(_, path)| path.send_new_challenge)
-6080 || !path.data.path_responses.is_empty()
-6081 });
-6082 let other = self.streams.can_send_stream_data()
-6083 || self
-6084 .datagrams
-6085 .outgoing
-6086 .front()
-6087 .is_some_and(|x| x.size(true) <= max_size);
-6088 SendableFrames {
-6089 acks: false,
-6090 other,
-6091 close: false,
-6092 path_exclusive,
-6093 }
-6094 }
-6095
-6096 /// Terminate the connection instantly, without sending a close packet
-6097 fn kill(&mut self, reason: ConnectionError) {
-6098 self.close_common();
-6099 self.state.move_to_drained(Some(reason));
-6100 self.endpoint_events.push_back(EndpointEventInner::Drained);
-6101 }
-6102
-6103 /// Storage size required for the largest packet that can be transmitted on all currently
-6104 /// available paths
-6105 ///
-6106 /// Buffers passed to [`Connection::poll_transmit`] should be at least this large.
-6107 ///
-6108 /// When multipath is enabled, this value is the minimum MTU across all available paths.
-6109 pub fn current_mtu(&self) -> u16 {
-6110 self.paths
-6111 .iter()
-6112 .filter(|&(path_id, _path_state)| !self.abandoned_paths.contains(path_id))
-6113 .map(|(_path_id, path_state)| path_state.data.current_mtu())
-6114 .min()
-6115 .expect("There is always at least one available path")
-6116 }
-6117
-6118 /// Size of non-frame data for a 1-RTT packet
-6119 ///
-6120 /// Quantifies space consumed by the QUIC header and AEAD tag. All other bytes in a packet are
-6121 /// frames. Changes if the length of the remote connection ID changes, which is expected to be
-6122 /// rare. If `pn` is specified, may additionally change unpredictably due to variations in
-6123 /// latency and packet loss.
-6124 fn predict_1rtt_overhead(&mut self, pn: u64, path: PathId) -> usize {
-6125 let pn_len = PacketNumber::new(
-6126 pn,
-6127 self.spaces[SpaceId::Data]
-6128 .for_path(path)
-6129 .largest_acked_packet
-6130 .unwrap_or(0),
-6131 )
-6132 .len();
-6133
-6134 // 1 byte for flags
-6135 1 + self
-6136 .rem_cids
-6137 .get(&path)
-6138 .map(|cids| cids.active().len())
-6139 .unwrap_or(20) // Max CID len in QUIC v1
-6140 + pn_len
-6141 + self.tag_len_1rtt()
-6142 }
-6143
-6144 fn predict_1rtt_overhead_no_pn(&self) -> usize {
-6145 let pn_len = 4;
-6146
-6147 let cid_len = self
-6148 .rem_cids
-6149 .values()
-6150 .map(|cids| cids.active().len())
-6151 .max()
-6152 .unwrap_or(20); // Max CID len in QUIC v1
-6153
-6154 // 1 byte for flags
-6155 1 + cid_len + pn_len + self.tag_len_1rtt()
+5982 let result = match result {
+5983 Some(r) => r,
+5984 None => return Ok(None),
+5985 };
+5986
+5987 if result.outgoing_key_update_acked {
+5988 if let Some(prev) = self.prev_crypto.as_mut() {
+5989 prev.end_packet = Some((result.number, now));
+5990 self.set_key_discard_timer(now, packet.header.space());
+5991 }
+5992 }
+5993
+5994 if result.incoming_key_update {
+5995 trace!("key update authenticated");
+5996 self.update_keys(Some((result.number, now)), true);
+5997 self.set_key_discard_timer(now, packet.header.space());
+5998 }
+5999
+6000 Ok(Some(result.number))
+6001 }
+6002
+6003 fn update_keys(&mut self, end_packet: Option<(u64, Instant)>, remote: bool) {
+6004 trace!("executing key update");
+6005 // Generate keys for the key phase after the one we're switching to, store them in
+6006 // `next_crypto`, make the contents of `next_crypto` current, and move the current keys into
+6007 // `prev_crypto`.
+6008 let new = self
+6009 .crypto
+6010 .next_1rtt_keys()
+6011 .expect("only called for `Data` packets");
+6012 self.key_phase_size = new
+6013 .local
+6014 .confidentiality_limit()
+6015 .saturating_sub(KEY_UPDATE_MARGIN);
+6016 let old = mem::replace(
+6017 &mut self.spaces[SpaceId::Data]
+6018 .crypto
+6019 .as_mut()
+6020 .unwrap() // safe because update_keys() can only be triggered by short packets
+6021 .packet,
+6022 mem::replace(self.next_crypto.as_mut().unwrap(), new),
+6023 );
+6024 self.spaces[SpaceId::Data]
+6025 .iter_paths_mut()
+6026 .for_each(|s| s.sent_with_keys = 0);
+6027 self.prev_crypto = Some(PrevCrypto {
+6028 crypto: old,
+6029 end_packet,
+6030 update_unacked: remote,
+6031 });
+6032 self.key_phase = !self.key_phase;
+6033 }
+6034
+6035 fn peer_supports_ack_frequency(&self) -> bool {
+6036 self.peer_params.min_ack_delay.is_some()
+6037 }
+6038
+6039 /// Send an IMMEDIATE_ACK frame to the remote endpoint
+6040 ///
+6041 /// According to the spec, this will result in an error if the remote endpoint does not support
+6042 /// the Acknowledgement Frequency extension
+6043 pub(crate) fn immediate_ack(&mut self, path_id: PathId) {
+6044 debug_assert_eq!(
+6045 self.highest_space,
+6046 SpaceId::Data,
+6047 "immediate ack must be written in the data space"
+6048 );
+6049 self.spaces[self.highest_space]
+6050 .for_path(path_id)
+6051 .immediate_ack_pending = true;
+6052 }
+6053
+6054 /// Decodes a packet, returning its decrypted payload, so it can be inspected in tests
+6055 #[cfg(test)]
+6056 pub(crate) fn decode_packet(&self, event: &ConnectionEvent) -> Option<Vec<u8>> {
+6057 let (path_id, first_decode, remaining) = match &event.0 {
+6058 ConnectionEventInner::Datagram(DatagramConnectionEvent {
+6059 path_id,
+6060 first_decode,
+6061 remaining,
+6062 ..
+6063 }) => (path_id, first_decode, remaining),
+6064 _ => return None,
+6065 };
+6066
+6067 if remaining.is_some() {
+6068 panic!("Packets should never be coalesced in tests");
+6069 }
+6070
+6071 let decrypted_header = packet_crypto::unprotect_header(
+6072 first_decode.clone(),
+6073 &self.spaces,
+6074 self.zero_rtt_crypto.as_ref(),
+6075 self.peer_params.stateless_reset_token,
+6076 )?;
+6077
+6078 let mut packet = decrypted_header.packet?;
+6079 packet_crypto::decrypt_packet_body(
+6080 &mut packet,
+6081 *path_id,
+6082 &self.spaces,
+6083 self.zero_rtt_crypto.as_ref(),
+6084 self.key_phase,
+6085 self.prev_crypto.as_ref(),
+6086 self.next_crypto.as_ref(),
+6087 )
+6088 .ok()?;
+6089
+6090 Some(packet.payload.to_vec())
+6091 }
+6092
+6093 /// The number of bytes of packets containing retransmittable frames that have not been
+6094 /// acknowledged or declared lost.
+6095 #[cfg(test)]
+6096 pub(crate) fn bytes_in_flight(&self) -> u64 {
+6097 // TODO(@divma): consider including for multipath?
+6098 self.path_data(PathId::ZERO).in_flight.bytes
+6099 }
+6100
+6101 /// Number of bytes worth of non-ack-only packets that may be sent
+6102 #[cfg(test)]
+6103 pub(crate) fn congestion_window(&self) -> u64 {
+6104 let path = self.path_data(PathId::ZERO);
+6105 path.congestion
+6106 .window()
+6107 .saturating_sub(path.in_flight.bytes)
+6108 }
+6109
+6110 /// Whether no timers but keepalive, idle, rtt, pushnewcid, and key discard are running
+6111 #[cfg(test)]
+6112 pub(crate) fn is_idle(&self) -> bool {
+6113 let current_timers = self.timers.values();
+6114 current_timers
+6115 .into_iter()
+6116 .filter(|(timer, _)| {
+6117 !matches!(
+6118 timer,
+6119 Timer::Conn(ConnTimer::KeepAlive)
+6120 | Timer::PerPath(_, PathTimer::PathKeepAlive)
+6121 | Timer::Conn(ConnTimer::PushNewCid)
+6122 | Timer::Conn(ConnTimer::KeyDiscard)
+6123 )
+6124 })
+6125 .min_by_key(|(_, time)| *time)
+6126 .is_none_or(|(timer, _)| timer == Timer::Conn(ConnTimer::Idle))
+6127 }
+6128
+6129 /// Whether explicit congestion notification is in use on outgoing packets.
+6130 #[cfg(test)]
+6131 pub(crate) fn using_ecn(&self) -> bool {
+6132 self.path_data(PathId::ZERO).sending_ecn
+6133 }
+6134
+6135 /// The number of received bytes in the current path
+6136 #[cfg(test)]
+6137 pub(crate) fn total_recvd(&self) -> u64 {
+6138 self.path_data(PathId::ZERO).total_recvd
+6139 }
+6140
+6141 #[cfg(test)]
+6142 pub(crate) fn active_local_cid_seq(&self) -> (u64, u64) {
+6143 self.local_cid_state
+6144 .get(&PathId::ZERO)
+6145 .unwrap()
+6146 .active_seq()
+6147 }
+6148
+6149 #[cfg(test)]
+6150 #[track_caller]
+6151 pub(crate) fn active_local_path_cid_seq(&self, path_id: u32) -> (u64, u64) {
+6152 self.local_cid_state
+6153 .get(&PathId(path_id))
+6154 .unwrap()
+6155 .active_seq()
6156 }
6157
-6158 fn tag_len_1rtt(&self) -> usize {
-6159 let key = match self.spaces[SpaceId::Data].crypto.as_ref() {
-6160 Some(crypto) => Some(&*crypto.packet.local),
-6161 None => self.zero_rtt_crypto.as_ref().map(|x| &*x.packet),
-6162 };
-6163 // If neither Data nor 0-RTT keys are available, make a reasonable tag length guess. As of
-6164 // this writing, all QUIC cipher suites use 16-byte tags. We could return `None` instead,
-6165 // but that would needlessly prevent sending datagrams during 0-RTT.
-6166 key.map_or(16, |x| x.tag_len())
-6167 }
-6168
-6169 /// Mark the path as validated, and enqueue NEW_TOKEN frames to be sent as appropriate
-6170 fn on_path_validated(&mut self, path_id: PathId) {
-6171 self.path_data_mut(path_id).validated = true;
-6172 let ConnectionSide::Server { server_config } = &self.side else {
-6173 return;
-6174 };
-6175 let network_path = self.path_data(path_id).network_path;
-6176 let new_tokens = &mut self.spaces[SpaceId::Data as usize].pending.new_tokens;
-6177 new_tokens.clear();
-6178 for _ in 0..server_config.validation_token.sent {
-6179 new_tokens.push(network_path);
-6180 }
+6158 /// Instruct the peer to replace previously issued CIDs by sending a NEW_CONNECTION_ID frame
+6159 /// with updated `retire_prior_to` field set to `v`
+6160 #[cfg(test)]
+6161 pub(crate) fn rotate_local_cid(&mut self, v: u64, now: Instant) {
+6162 let n = self
+6163 .local_cid_state
+6164 .get_mut(&PathId::ZERO)
+6165 .unwrap()
+6166 .assign_retire_seq(v);
+6167 self.endpoint_events
+6168 .push_back(EndpointEventInner::NeedIdentifiers(PathId::ZERO, now, n));
+6169 }
+6170
+6171 /// Check the current active remote CID sequence for `PathId::ZERO`
+6172 #[cfg(test)]
+6173 pub(crate) fn active_rem_cid_seq(&self) -> u64 {
+6174 self.rem_cids.get(&PathId::ZERO).unwrap().active_seq()
+6175 }
+6176
+6177 /// Returns the detected maximum udp payload size for the current path
+6178 #[cfg(test)]
+6179 pub(crate) fn path_mtu(&self, path_id: PathId) -> u16 {
+6180 self.path_data(path_id).current_mtu()
6181 }
6182
-6183 /// Handle new path status information: PATH_STATUS_AVAILABLE, PATH_STATUS_BACKUP
-6184 fn on_path_status(&mut self, path_id: PathId, status: PathStatus, status_seq_no: VarInt) {
-6185 if let Some(path) = self.paths.get_mut(&path_id) {
-6186 path.data.status.remote_update(status, status_seq_no);
-6187 } else {
-6188 debug!("PATH_STATUS_AVAILABLE received unknown path {:?}", path_id);
-6189 }
-6190 self.events.push_back(
-6191 PathEvent::RemoteStatus {
-6192 id: path_id,
-6193 status,
-6194 }
-6195 .into(),
-6196 );
-6197 }
-6198
-6199 /// Returns the maximum [`PathId`] to be used for sending in this connection.
-6200 ///
-6201 /// This is calculated as minimum between the local and remote's maximums when multipath is
-6202 /// enabled, or `None` when disabled.
-6203 ///
-6204 /// For data that's received, we should use [`Self::local_max_path_id`] instead.
-6205 /// The reasoning is that the remote might already have updated to its own newer
-6206 /// [`Self::max_path_id`] after sending out a `MAX_PATH_ID` frame, but it got re-ordered.
-6207 fn max_path_id(&self) -> Option<PathId> {
-6208 if self.is_multipath_negotiated() {
-6209 Some(self.remote_max_path_id.min(self.local_max_path_id))
-6210 } else {
-6211 None
-6212 }
-6213 }
-6214
-6215 /// Add addresses the local endpoint considers are reachable for nat traversal
-6216 pub fn add_nat_traversal_address(&mut self, address: SocketAddr) -> Result<(), iroh_hp::Error> {
-6217 if let Some(added) = self.iroh_hp.add_local_address(address)? {
-6218 self.spaces[SpaceId::Data].pending.add_address.insert(added);
-6219 };
-6220 Ok(())
-6221 }
-6222
-6223 /// Removes an address the endpoing no longer considers reachable for nat traversal
-6224 ///
-6225 /// Addresses not present in the set will be silently ignored.
-6226 pub fn remove_nat_traversal_address(
-6227 &mut self,
-6228 address: SocketAddr,
-6229 ) -> Result<(), iroh_hp::Error> {
-6230 if let Some(removed) = self.iroh_hp.remove_local_address(address)? {
-6231 self.spaces[SpaceId::Data]
-6232 .pending
-6233 .remove_address
-6234 .insert(removed);
-6235 }
-6236 Ok(())
-6237 }
-6238
-6239 /// Get the current local nat traversal addresses
-6240 pub fn get_local_nat_traversal_addresses(&self) -> Result<Vec<SocketAddr>, iroh_hp::Error> {
-6241 self.iroh_hp.get_local_nat_traversal_addresses()
-6242 }
-6243
-6244 /// Get the currently advertised nat traversal addresses by the server
-6245 pub fn get_remote_nat_traversal_addresses(&self) -> Result<Vec<SocketAddr>, iroh_hp::Error> {
-6246 Ok(self
-6247 .iroh_hp
-6248 .client_side()?
-6249 .get_remote_nat_traversal_addresses())
-6250 }
-6251
-6252 /// Attempts to open a path for nat traversal.
-6253 ///
-6254 /// `ipv6` indicates if the path should be opened using an IPV6 remote. If the address is
-6255 /// ignored, it will return `None`.
-6256 ///
-6257 /// On success returns the [`PathId`] and remote address of the path, as well as whether the path
-6258 /// existed for the adjusted remote.
-6259 fn open_nat_traversal_path(
-6260 &mut self,
-6261 now: Instant,
-6262 (ip, port): (IpAddr, u16),
-6263 ipv6: bool,
-6264 ) -> Result<Option<(PathId, SocketAddr, bool)>, PathError> {
-6265 // If this endpoint is an IPv6 endpoint we use IPv6 addresses for all remotes.
-6266 let remote = match ip {
-6267 IpAddr::V4(addr) if ipv6 => SocketAddr::new(addr.to_ipv6_mapped().into(), port),
-6268 IpAddr::V4(addr) => SocketAddr::new(addr.into(), port),
-6269 IpAddr::V6(_) if ipv6 => SocketAddr::new(ip, port),
-6270 IpAddr::V6(_) => {
-6271 trace!("not using IPv6 nat candidate for IPv4 socket");
-6272 return Ok(None);
-6273 }
-6274 };
-6275 // TODO(matheus23): Probe the correct 4-tuple, instead of only a remote address?
-6276 // By specifying None, we do two things: 1. open_path_ensure won't generate two
-6277 // paths to the same remote and 2. we let the OS choose which interface to use for
-6278 // sending on that path.
-6279 let network_path = FourTuple {
-6280 remote,
-6281 local_ip: None,
-6282 };
-6283 match self.open_path_ensure(network_path, PathStatus::Backup, now) {
-6284 Ok((path_id, path_was_known)) => {
-6285 if path_was_known {
-6286 trace!(%path_id, %remote, "nat traversal: path existed for remote");
-6287 }
-6288 Ok(Some((path_id, remote, path_was_known)))
-6289 }
-6290 Err(e) => {
-6291 debug!(%remote, %e, "nat traversal: failed to probe remote");
-6292 Err(e)
-6293 }
-6294 }
+6183 /// Triggers path validation on all paths
+6184 #[cfg(test)]
+6185 pub(crate) fn trigger_path_validation(&mut self) {
+6186 for path in self.paths.values_mut() {
+6187 path.data.send_new_challenge = true;
+6188 }
+6189 }
+6190
+6191 /// Whether we have 1-RTT data to send
+6192 ///
+6193 /// This checks for frames that can only be sent in the data space (1-RTT):
+6194 /// - Pending PATH_CHALLENGE frames on the active and previous path if just migrated.
+6195 /// - Pending PATH_RESPONSE frames.
+6196 /// - Pending data to send in STREAM frames.
+6197 /// - Pending DATAGRAM frames to send.
+6198 ///
+6199 /// See also [`PacketSpace::can_send`] which keeps track of all other frame types that
+6200 /// may need to be sent.
+6201 fn can_send_1rtt(&self, path_id: PathId, max_size: usize) -> SendableFrames {
+6202 let path_exclusive = self.paths.get(&path_id).is_some_and(|path| {
+6203 path.data.send_new_challenge
+6204 || path
+6205 .prev
+6206 .as_ref()
+6207 .is_some_and(|(_, path)| path.send_new_challenge)
+6208 || !path.data.path_responses.is_empty()
+6209 });
+6210 let other = self.streams.can_send_stream_data()
+6211 || self
+6212 .datagrams
+6213 .outgoing
+6214 .front()
+6215 .is_some_and(|x| x.size(true) <= max_size);
+6216 SendableFrames {
+6217 acks: false,
+6218 other,
+6219 close: false,
+6220 path_exclusive,
+6221 }
+6222 }
+6223
+6224 /// Terminate the connection instantly, without sending a close packet
+6225 fn kill(&mut self, reason: ConnectionError) {
+6226 self.close_common();
+6227 self.state.move_to_drained(Some(reason));
+6228 self.endpoint_events.push_back(EndpointEventInner::Drained);
+6229 }
+6230
+6231 /// Storage size required for the largest packet that can be transmitted on all currently
+6232 /// available paths
+6233 ///
+6234 /// Buffers passed to [`Connection::poll_transmit`] should be at least this large.
+6235 ///
+6236 /// When multipath is enabled, this value is the minimum MTU across all available paths.
+6237 pub fn current_mtu(&self) -> u16 {
+6238 self.paths
+6239 .iter()
+6240 .filter(|&(path_id, _path_state)| !self.abandoned_paths.contains(path_id))
+6241 .map(|(_path_id, path_state)| path_state.data.current_mtu())
+6242 .min()
+6243 .expect("There is always at least one available path")
+6244 }
+6245
+6246 /// Size of non-frame data for a 1-RTT packet
+6247 ///
+6248 /// Quantifies space consumed by the QUIC header and AEAD tag. All other bytes in a packet are
+6249 /// frames. Changes if the length of the remote connection ID changes, which is expected to be
+6250 /// rare. If `pn` is specified, may additionally change unpredictably due to variations in
+6251 /// latency and packet loss.
+6252 fn predict_1rtt_overhead(&mut self, pn: u64, path: PathId) -> usize {
+6253 let pn_len = PacketNumber::new(
+6254 pn,
+6255 self.spaces[SpaceId::Data]
+6256 .for_path(path)
+6257 .largest_acked_packet
+6258 .unwrap_or(0),
+6259 )
+6260 .len();
+6261
+6262 // 1 byte for flags
+6263 1 + self
+6264 .rem_cids
+6265 .get(&path)
+6266 .map(|cids| cids.active().len())
+6267 .unwrap_or(20) // Max CID len in QUIC v1
+6268 + pn_len
+6269 + self.tag_len_1rtt()
+6270 }
+6271
+6272 fn predict_1rtt_overhead_no_pn(&self) -> usize {
+6273 let pn_len = 4;
+6274
+6275 let cid_len = self
+6276 .rem_cids
+6277 .values()
+6278 .map(|cids| cids.active().len())
+6279 .max()
+6280 .unwrap_or(20); // Max CID len in QUIC v1
+6281
+6282 // 1 byte for flags
+6283 1 + cid_len + pn_len + self.tag_len_1rtt()
+6284 }
+6285
+6286 fn tag_len_1rtt(&self) -> usize {
+6287 let key = match self.spaces[SpaceId::Data].crypto.as_ref() {
+6288 Some(crypto) => Some(&*crypto.packet.local),
+6289 None => self.zero_rtt_crypto.as_ref().map(|x| &*x.packet),
+6290 };
+6291 // If neither Data nor 0-RTT keys are available, make a reasonable tag length guess. As of
+6292 // this writing, all QUIC cipher suites use 16-byte tags. We could return `None` instead,
+6293 // but that would needlessly prevent sending datagrams during 0-RTT.
+6294 key.map_or(16, |x| x.tag_len())
6295 }
6296
-6297 /// Initiates a new nat traversal round
-6298 ///
-6299 /// A nat traversal round involves advertising the client's local addresses in `REACH_OUT`
-6300 /// frames, and initiating probing of the known remote addresses. When a new round is
-6301 /// initiated, the previous one is cancelled, and paths that have not been opened are closed.
-6302 ///
-6303 /// Returns the server addresses that are now being probed.
-6304 /// If addresses fail due to spurious errors, these might succeed later and not be returned in
-6305 /// this set.
-6306 pub fn initiate_nat_traversal_round(
-6307 &mut self,
-6308 now: Instant,
-6309 ) -> Result<Vec<SocketAddr>, iroh_hp::Error> {
-6310 if self.state.is_closed() {
-6311 return Err(iroh_hp::Error::Closed);
-6312 }
-6313
-6314 let client_state = self.iroh_hp.client_side_mut()?;
-6315 let iroh_hp::NatTraversalRound {
-6316 new_round,
-6317 reach_out_at,
-6318 addresses_to_probe,
-6319 prev_round_path_ids,
-6320 } = client_state.initiate_nat_traversal_round()?;
-6321
-6322 self.spaces[SpaceId::Data].pending.reach_out = Some((new_round, reach_out_at));
-6323
-6324 for path_id in prev_round_path_ids {
-6325 // TODO(@divma): this sounds reasonable but we need if this actually works for the
-6326 // purposes of the protocol
-6327 let validated = self
-6328 .path(path_id)
-6329 .map(|path| path.validated)
-6330 .unwrap_or(false);
-6331
-6332 if !validated {
-6333 let _ = self.close_path(
-6334 now,
-6335 path_id,
-6336 TransportErrorCode::APPLICATION_ABANDON_PATH.into(),
-6337 );
-6338 }
-6339 }
-6340
-6341 let mut err = None;
+6297 /// Mark the path as validated, and enqueue NEW_TOKEN frames to be sent as appropriate
+6298 fn on_path_validated(&mut self, path_id: PathId) {
+6299 self.path_data_mut(path_id).validated = true;
+6300 let ConnectionSide::Server { server_config } = &self.side else {
+6301 return;
+6302 };
+6303 let network_path = self.path_data(path_id).network_path;
+6304 let new_tokens = &mut self.spaces[SpaceId::Data as usize].pending.new_tokens;
+6305 new_tokens.clear();
+6306 for _ in 0..server_config.validation_token.sent {
+6307 new_tokens.push(network_path);
+6308 }
+6309 }
+6310
+6311 /// Handle new path status information: PATH_STATUS_AVAILABLE, PATH_STATUS_BACKUP
+6312 fn on_path_status(&mut self, path_id: PathId, status: PathStatus, status_seq_no: VarInt) {
+6313 if let Some(path) = self.paths.get_mut(&path_id) {
+6314 path.data.status.remote_update(status, status_seq_no);
+6315 } else {
+6316 debug!("PATH_STATUS_AVAILABLE received unknown path {:?}", path_id);
+6317 }
+6318 self.events.push_back(
+6319 PathEvent::RemoteStatus {
+6320 id: path_id,
+6321 status,
+6322 }
+6323 .into(),
+6324 );
+6325 }
+6326
+6327 /// Returns the maximum [`PathId`] to be used for sending in this connection.
+6328 ///
+6329 /// This is calculated as minimum between the local and remote's maximums when multipath is
+6330 /// enabled, or `None` when disabled.
+6331 ///
+6332 /// For data that's received, we should use [`Self::local_max_path_id`] instead.
+6333 /// The reasoning is that the remote might already have updated to its own newer
+6334 /// [`Self::max_path_id`] after sending out a `MAX_PATH_ID` frame, but it got re-ordered.
+6335 fn max_path_id(&self) -> Option<PathId> {
+6336 if self.is_multipath_negotiated() {
+6337 Some(self.remote_max_path_id.min(self.local_max_path_id))
+6338 } else {
+6339 None
+6340 }
+6341 }
6342
-6343 let mut path_ids = Vec::with_capacity(addresses_to_probe.len());
-6344 let mut probed_addresses = Vec::with_capacity(addresses_to_probe.len());
-6345 let ipv6 = self
-6346 .paths
-6347 .values()
-6348 .any(|p| p.data.network_path.remote.is_ipv6());
-6349
-6350 for (id, address) in addresses_to_probe {
-6351 match self.open_nat_traversal_path(now, address, ipv6) {
-6352 Ok(None) => {}
-6353 Ok(Some((path_id, remote, path_was_known))) => {
-6354 if !path_was_known {
-6355 path_ids.push(path_id);
-6356 probed_addresses.push(remote);
-6357 }
-6358 }
-6359 Err(e) => {
-6360 self.iroh_hp
-6361 .client_side_mut()
-6362 .expect("validated")
-6363 .report_in_continuation(id, e);
-6364 err.get_or_insert(e);
-6365 }
-6366 }
-6367 }
-6368
-6369 if let Some(err) = err {
-6370 // We failed to probe any addresses, bail out
-6371 if probed_addresses.is_empty() {
-6372 return Err(iroh_hp::Error::Multipath(err));
-6373 }
-6374 }
-6375
-6376 self.iroh_hp
-6377 .client_side_mut()
-6378 .expect("connection side validated")
-6379 .set_round_path_ids(path_ids);
-6380
-6381 Ok(probed_addresses)
-6382 }
-6383
-6384 /// Attempts to continue a nat traversal round by trying to open paths for pending client probes.
-6385 ///
-6386 /// If there was nothing to do, it returns `None`. Otherwise it returns whether the path was
-6387 /// successfully open.
-6388 fn continue_nat_traversal_round(&mut self, now: Instant) -> Option<bool> {
-6389 let client_state = self.iroh_hp.client_side_mut().ok()?;
-6390 let (id, address) = client_state.continue_nat_traversal_round()?;
-6391 let ipv6 = self
-6392 .paths
-6393 .values()
-6394 .any(|p| p.data.network_path.remote.is_ipv6());
-6395 let open_result = self.open_nat_traversal_path(now, address, ipv6);
-6396 let client_state = self.iroh_hp.client_side_mut().expect("validated");
-6397 match open_result {
-6398 Ok(None) => Some(true),
-6399 Ok(Some((path_id, _remote, path_was_known))) => {
-6400 if !path_was_known {
-6401 client_state.add_round_path_id(path_id);
-6402 }
-6403 Some(true)
-6404 }
-6405 Err(e) => {
-6406 client_state.report_in_continuation(id, e);
-6407 Some(false)
-6408 }
-6409 }
-6410 }
-6411}
-6412
-6413impl fmt::Debug for Connection {
-6414 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
-6415 f.debug_struct("Connection")
-6416 .field("handshake_cid", &self.handshake_cid)
-6417 .finish()
-6418 }
-6419}
-6420
-6421#[derive(Debug, Copy, Clone, PartialEq, Eq)]
-6422enum PathBlocked {
-6423 No,
-6424 AntiAmplification,
-6425 Congestion,
-6426 Pacing,
-6427}
-6428
-6429/// Fields of `Connection` specific to it being client-side or server-side
-6430enum ConnectionSide {
-6431 Client {
-6432 /// Sent in every outgoing Initial packet. Always empty after Initial keys are discarded
-6433 token: Bytes,
-6434 token_store: Arc<dyn TokenStore>,
-6435 server_name: String,
-6436 },
-6437 Server {
-6438 server_config: Arc<ServerConfig>,
-6439 },
-6440}
+6343 /// Add addresses the local endpoint considers are reachable for nat traversal
+6344 pub fn add_nat_traversal_address(&mut self, address: SocketAddr) -> Result<(), iroh_hp::Error> {
+6345 if let Some(added) = self.iroh_hp.add_local_address(address)? {
+6346 self.spaces[SpaceId::Data].pending.add_address.insert(added);
+6347 };
+6348 Ok(())
+6349 }
+6350
+6351 /// Removes an address the endpoing no longer considers reachable for nat traversal
+6352 ///
+6353 /// Addresses not present in the set will be silently ignored.
+6354 pub fn remove_nat_traversal_address(
+6355 &mut self,
+6356 address: SocketAddr,
+6357 ) -> Result<(), iroh_hp::Error> {
+6358 if let Some(removed) = self.iroh_hp.remove_local_address(address)? {
+6359 self.spaces[SpaceId::Data]
+6360 .pending
+6361 .remove_address
+6362 .insert(removed);
+6363 }
+6364 Ok(())
+6365 }
+6366
+6367 /// Get the current local nat traversal addresses
+6368 pub fn get_local_nat_traversal_addresses(&self) -> Result<Vec<SocketAddr>, iroh_hp::Error> {
+6369 self.iroh_hp.get_local_nat_traversal_addresses()
+6370 }
+6371
+6372 /// Get the currently advertised nat traversal addresses by the server
+6373 pub fn get_remote_nat_traversal_addresses(&self) -> Result<Vec<SocketAddr>, iroh_hp::Error> {
+6374 Ok(self
+6375 .iroh_hp
+6376 .client_side()?
+6377 .get_remote_nat_traversal_addresses())
+6378 }
+6379
+6380 /// Attempts to open a path for nat traversal.
+6381 ///
+6382 /// `ipv6` indicates if the path should be opened using an IPV6 remote. If the address is
+6383 /// ignored, it will return `None`.
+6384 ///
+6385 /// On success returns the [`PathId`] and remote address of the path, as well as whether the path
+6386 /// existed for the adjusted remote.
+6387 fn open_nat_traversal_path(
+6388 &mut self,
+6389 now: Instant,
+6390 (ip, port): (IpAddr, u16),
+6391 ipv6: bool,
+6392 ) -> Result<Option<(PathId, SocketAddr, bool)>, PathError> {
+6393 // If this endpoint is an IPv6 endpoint we use IPv6 addresses for all remotes.
+6394 let remote = match ip {
+6395 IpAddr::V4(addr) if ipv6 => SocketAddr::new(addr.to_ipv6_mapped().into(), port),
+6396 IpAddr::V4(addr) => SocketAddr::new(addr.into(), port),
+6397 IpAddr::V6(_) if ipv6 => SocketAddr::new(ip, port),
+6398 IpAddr::V6(_) => {
+6399 trace!("not using IPv6 nat candidate for IPv4 socket");
+6400 return Ok(None);
+6401 }
+6402 };
+6403 // TODO(matheus23): Probe the correct 4-tuple, instead of only a remote address?
+6404 // By specifying None, we do two things: 1. open_path_ensure won't generate two
+6405 // paths to the same remote and 2. we let the OS choose which interface to use for
+6406 // sending on that path.
+6407 let network_path = FourTuple {
+6408 remote,
+6409 local_ip: None,
+6410 };
+6411 match self.open_path_ensure(network_path, PathStatus::Backup, now) {
+6412 Ok((path_id, path_was_known)) => {
+6413 if path_was_known {
+6414 trace!(%path_id, %remote, "nat traversal: path existed for remote");
+6415 }
+6416 Ok(Some((path_id, remote, path_was_known)))
+6417 }
+6418 Err(e) => {
+6419 debug!(%remote, %e, "nat traversal: failed to probe remote");
+6420 Err(e)
+6421 }
+6422 }
+6423 }
+6424
+6425 /// Initiates a new nat traversal round
+6426 ///
+6427 /// A nat traversal round involves advertising the client's local addresses in `REACH_OUT`
+6428 /// frames, and initiating probing of the known remote addresses. When a new round is
+6429 /// initiated, the previous one is cancelled, and paths that have not been opened are closed.
+6430 ///
+6431 /// Returns the server addresses that are now being probed.
+6432 /// If addresses fail due to spurious errors, these might succeed later and not be returned in
+6433 /// this set.
+6434 pub fn initiate_nat_traversal_round(
+6435 &mut self,
+6436 now: Instant,
+6437 ) -> Result<Vec<SocketAddr>, iroh_hp::Error> {
+6438 if self.state.is_closed() {
+6439 return Err(iroh_hp::Error::Closed);
+6440 }
6441
-6442impl ConnectionSide {
-6443 fn remote_may_migrate(&self, state: &State) -> bool {
-6444 match self {
-6445 Self::Server { server_config } => server_config.migration,
-6446 Self::Client { .. } => {
-6447 if let Some(hs) = state.as_handshake() {
-6448 hs.allow_server_migration
-6449 } else {
-6450 false
-6451 }
-6452 }
-6453 }
-6454 }
-6455
-6456 fn is_client(&self) -> bool {
-6457 self.side().is_client()
-6458 }
+6442 let client_state = self.iroh_hp.client_side_mut()?;
+6443 let iroh_hp::NatTraversalRound {
+6444 new_round,
+6445 reach_out_at,
+6446 addresses_to_probe,
+6447 prev_round_path_ids,
+6448 } = client_state.initiate_nat_traversal_round()?;
+6449
+6450 self.spaces[SpaceId::Data].pending.reach_out = Some((new_round, reach_out_at));
+6451
+6452 for path_id in prev_round_path_ids {
+6453 // TODO(@divma): this sounds reasonable but we need if this actually works for the
+6454 // purposes of the protocol
+6455 let validated = self
+6456 .path(path_id)
+6457 .map(|path| path.validated)
+6458 .unwrap_or(false);
6459
-6460 fn is_server(&self) -> bool {
-6461 self.side().is_server()
-6462 }
-6463
-6464 fn side(&self) -> Side {
-6465 match *self {
-6466 Self::Client { .. } => Side::Client,
-6467 Self::Server { .. } => Side::Server,
-6468 }
-6469 }
-6470}
-6471
-6472impl From<SideArgs> for ConnectionSide {
-6473 fn from(side: SideArgs) -> Self {
-6474 match side {
-6475 SideArgs::Client {
-6476 token_store,
-6477 server_name,
-6478 } => Self::Client {
-6479 token: token_store.take(&server_name).unwrap_or_default(),
-6480 token_store,
-6481 server_name,
-6482 },
-6483 SideArgs::Server {
-6484 server_config,
-6485 pref_addr_cid: _,
-6486 path_validated: _,
-6487 } => Self::Server { server_config },
-6488 }
-6489 }
-6490}
-6491
-6492/// Parameters to `Connection::new` specific to it being client-side or server-side
-6493pub(crate) enum SideArgs {
-6494 Client {
-6495 token_store: Arc<dyn TokenStore>,
-6496 server_name: String,
-6497 },
-6498 Server {
-6499 server_config: Arc<ServerConfig>,
-6500 pref_addr_cid: Option<ConnectionId>,
-6501 path_validated: bool,
-6502 },
-6503}
-6504
-6505impl SideArgs {
-6506 pub(crate) fn pref_addr_cid(&self) -> Option<ConnectionId> {
-6507 match *self {
-6508 Self::Client { .. } => None,
-6509 Self::Server { pref_addr_cid, .. } => pref_addr_cid,
-6510 }
-6511 }
-6512
-6513 pub(crate) fn path_validated(&self) -> bool {
-6514 match *self {
-6515 Self::Client { .. } => true,
-6516 Self::Server { path_validated, .. } => path_validated,
-6517 }
-6518 }
-6519
-6520 pub(crate) fn side(&self) -> Side {
-6521 match *self {
-6522 Self::Client { .. } => Side::Client,
-6523 Self::Server { .. } => Side::Server,
-6524 }
-6525 }
-6526}
-6527
-6528/// Reasons why a connection might be lost
-6529#[derive(Debug, Error, Clone, PartialEq, Eq)]
-6530pub enum ConnectionError {
-6531 /// The peer doesn't implement any supported version
-6532 #[error("peer doesn't implement any supported version")]
-6533 VersionMismatch,
-6534 /// The peer violated the QUIC specification as understood by this implementation
-6535 #[error(transparent)]
-6536 TransportError(#[from] TransportError),
-6537 /// The peer's QUIC stack aborted the connection automatically
-6538 #[error("aborted by peer: {0}")]
-6539 ConnectionClosed(frame::ConnectionClose),
-6540 /// The peer closed the connection
-6541 #[error("closed by peer: {0}")]
-6542 ApplicationClosed(frame::ApplicationClose),
-6543 /// The peer is unable to continue processing this connection, usually due to having restarted
-6544 #[error("reset by peer")]
-6545 Reset,
-6546 /// Communication with the peer has lapsed for longer than the negotiated idle timeout
-6547 ///
-6548 /// If neither side is sending keep-alives, a connection will time out after a long enough idle
-6549 /// period even if the peer is still reachable. See also [`TransportConfig::max_idle_timeout()`]
-6550 /// and [`TransportConfig::keep_alive_interval()`].
-6551 #[error("timed out")]
-6552 TimedOut,
-6553 /// The local application closed the connection
-6554 #[error("closed")]
-6555 LocallyClosed,
-6556 /// The connection could not be created because not enough of the CID space is available
-6557 ///
-6558 /// Try using longer connection IDs.
-6559 #[error("CIDs exhausted")]
-6560 CidsExhausted,
-6561}
-6562
-6563impl From<Close> for ConnectionError {
-6564 fn from(x: Close) -> Self {
-6565 match x {
-6566 Close::Connection(reason) => Self::ConnectionClosed(reason),
-6567 Close::Application(reason) => Self::ApplicationClosed(reason),
-6568 }
-6569 }
-6570}
-6571
-6572// For compatibility with API consumers
-6573impl From<ConnectionError> for io::Error {
-6574 fn from(x: ConnectionError) -> Self {
-6575 use ConnectionError::*;
-6576 let kind = match x {
-6577 TimedOut => io::ErrorKind::TimedOut,
-6578 Reset => io::ErrorKind::ConnectionReset,
-6579 ApplicationClosed(_) | ConnectionClosed(_) => io::ErrorKind::ConnectionAborted,
-6580 TransportError(_) | VersionMismatch | LocallyClosed | CidsExhausted => {
-6581 io::ErrorKind::Other
-6582 }
-6583 };
-6584 Self::new(kind, x)
-6585 }
-6586}
+6460 if !validated {
+6461 let _ = self.close_path(
+6462 now,
+6463 path_id,
+6464 TransportErrorCode::APPLICATION_ABANDON_PATH.into(),
+6465 );
+6466 }
+6467 }
+6468
+6469 let mut err = None;
+6470
+6471 let mut path_ids = Vec::with_capacity(addresses_to_probe.len());
+6472 let mut probed_addresses = Vec::with_capacity(addresses_to_probe.len());
+6473 let ipv6 = self
+6474 .paths
+6475 .values()
+6476 .any(|p| p.data.network_path.remote.is_ipv6());
+6477
+6478 for (id, address) in addresses_to_probe {
+6479 match self.open_nat_traversal_path(now, address, ipv6) {
+6480 Ok(None) => {}
+6481 Ok(Some((path_id, remote, path_was_known))) => {
+6482 if !path_was_known {
+6483 path_ids.push(path_id);
+6484 probed_addresses.push(remote);
+6485 }
+6486 }
+6487 Err(e) => {
+6488 self.iroh_hp
+6489 .client_side_mut()
+6490 .expect("validated")
+6491 .report_in_continuation(id, e);
+6492 err.get_or_insert(e);
+6493 }
+6494 }
+6495 }
+6496
+6497 if let Some(err) = err {
+6498 // We failed to probe any addresses, bail out
+6499 if probed_addresses.is_empty() {
+6500 return Err(iroh_hp::Error::Multipath(err));
+6501 }
+6502 }
+6503
+6504 self.iroh_hp
+6505 .client_side_mut()
+6506 .expect("connection side validated")
+6507 .set_round_path_ids(path_ids);
+6508
+6509 Ok(probed_addresses)
+6510 }
+6511
+6512 /// Attempts to continue a nat traversal round by trying to open paths for pending client probes.
+6513 ///
+6514 /// If there was nothing to do, it returns `None`. Otherwise it returns whether the path was
+6515 /// successfully open.
+6516 fn continue_nat_traversal_round(&mut self, now: Instant) -> Option<bool> {
+6517 let client_state = self.iroh_hp.client_side_mut().ok()?;
+6518 let (id, address) = client_state.continue_nat_traversal_round()?;
+6519 let ipv6 = self
+6520 .paths
+6521 .values()
+6522 .any(|p| p.data.network_path.remote.is_ipv6());
+6523 let open_result = self.open_nat_traversal_path(now, address, ipv6);
+6524 let client_state = self.iroh_hp.client_side_mut().expect("validated");
+6525 match open_result {
+6526 Ok(None) => Some(true),
+6527 Ok(Some((path_id, _remote, path_was_known))) => {
+6528 if !path_was_known {
+6529 client_state.add_round_path_id(path_id);
+6530 }
+6531 Some(true)
+6532 }
+6533 Err(e) => {
+6534 client_state.report_in_continuation(id, e);
+6535 Some(false)
+6536 }
+6537 }
+6538 }
+6539}
+6540
+6541impl fmt::Debug for Connection {
+6542 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+6543 f.debug_struct("Connection")
+6544 .field("handshake_cid", &self.handshake_cid)
+6545 .finish()
+6546 }
+6547}
+6548
+6549#[derive(Debug, Copy, Clone, PartialEq, Eq)]
+6550enum PathBlocked {
+6551 No,
+6552 AntiAmplification,
+6553 Congestion,
+6554 Pacing,
+6555}
+6556
+6557/// Fields of `Connection` specific to it being client-side or server-side
+6558enum ConnectionSide {
+6559 Client {
+6560 /// Sent in every outgoing Initial packet. Always empty after Initial keys are discarded
+6561 token: Bytes,
+6562 token_store: Arc<dyn TokenStore>,
+6563 server_name: String,
+6564 },
+6565 Server {
+6566 server_config: Arc<ServerConfig>,
+6567 },
+6568}
+6569
+6570impl ConnectionSide {
+6571 fn remote_may_migrate(&self, state: &State) -> bool {
+6572 match self {
+6573 Self::Server { server_config } => server_config.migration,
+6574 Self::Client { .. } => {
+6575 if let Some(hs) = state.as_handshake() {
+6576 hs.allow_server_migration
+6577 } else {
+6578 false
+6579 }
+6580 }
+6581 }
+6582 }
+6583
+6584 fn is_client(&self) -> bool {
+6585 self.side().is_client()
+6586 }
6587
-6588/// Errors that might trigger a path being closed
-6589// TODO(@divma): maybe needs to be reworked based on what we want to do with the public API
-6590#[derive(Debug, Error, PartialEq, Eq, Clone, Copy)]
-6591pub enum PathError {
-6592 /// The extension was not negotiated with the peer
-6593 #[error("multipath extension not negotiated")]
-6594 MultipathNotNegotiated,
-6595 /// Paths can only be opened client-side
-6596 #[error("the server side may not open a path")]
-6597 ServerSideNotAllowed,
-6598 /// Current limits do not allow us to open more paths
-6599 #[error("maximum number of concurrent paths reached")]
-6600 MaxPathIdReached,
-6601 /// No remote CIDs available to open a new path
-6602 #[error("remoted CIDs exhausted")]
-6603 RemoteCidsExhausted,
-6604 /// Path could not be validated and will be abandoned
-6605 #[error("path validation failed")]
-6606 ValidationFailed,
-6607 /// The remote address for the path is not supported by the endpoint
-6608 #[error("invalid remote address")]
-6609 InvalidRemoteAddress(SocketAddr),
-6610}
-6611
-6612/// Errors triggered when abandoning a path
-6613#[derive(Debug, Error, Clone, Eq, PartialEq)]
-6614pub enum ClosePathError {
-6615 /// The path is already closed or was never opened
-6616 #[error("closed path")]
-6617 ClosedPath,
-6618 /// This is the last path, which can not be abandoned
-6619 #[error("last open path")]
-6620 LastOpenPath,
-6621}
-6622
-6623/// Error when the multipath extension was not negotiated, but attempted to be used.
-6624#[derive(Debug, Error, Clone, Copy)]
-6625#[error("Multipath extension not negotiated")]
-6626pub struct MultipathNotNegotiated {
-6627 _private: (),
-6628}
-6629
-6630/// Events of interest to the application
-6631#[derive(Debug)]
-6632pub enum Event {
-6633 /// The connection's handshake data is ready
-6634 HandshakeDataReady,
-6635 /// The connection was successfully established
-6636 Connected,
-6637 /// The TLS handshake was confirmed
-6638 HandshakeConfirmed,
-6639 /// The connection was lost
-6640 ///
-6641 /// Emitted if the peer closes the connection or an error is encountered.
-6642 ConnectionLost {
-6643 /// Reason that the connection was closed
-6644 reason: ConnectionError,
-6645 },
-6646 /// Stream events
-6647 Stream(StreamEvent),
-6648 /// One or more application datagrams have been received
-6649 DatagramReceived,
-6650 /// One or more application datagrams have been sent after blocking
-6651 DatagramsUnblocked,
-6652 /// (Multi)Path events
-6653 Path(PathEvent),
-6654 /// Iroh's nat traversal events
-6655 NatTraversal(iroh_hp::Event),
-6656}
-6657
-6658impl From<PathEvent> for Event {
-6659 fn from(source: PathEvent) -> Self {
-6660 Self::Path(source)
-6661 }
-6662}
-6663
-6664fn get_max_ack_delay(params: &TransportParameters) -> Duration {
-6665 Duration::from_micros(params.max_ack_delay.0 * 1000)
-6666}
-6667
-6668// Prevents overflow and improves behavior in extreme circumstances
-6669const MAX_BACKOFF_EXPONENT: u32 = 16;
-6670
-6671/// Minimal remaining size to allow packet coalescing, excluding cryptographic tag
-6672///
-6673/// This must be at least as large as the header for a well-formed empty packet to be coalesced,
-6674/// plus some space for frames. We only care about handshake headers because short header packets
-6675/// necessarily have smaller headers, and initial packets are only ever the first packet in a
-6676/// datagram (because we coalesce in ascending packet space order and the only reason to split a
-6677/// packet is when packet space changes).
-6678const MIN_PACKET_SPACE: usize = MAX_HANDSHAKE_OR_0RTT_HEADER_SIZE + 32;
-6679
-6680/// Largest amount of space that could be occupied by a Handshake or 0-RTT packet's header
-6681///
-6682/// Excludes packet-type-specific fields such as packet number or Initial token
-6683// https://www.rfc-editor.org/rfc/rfc9000.html#name-0-rtt: flags + version + dcid len + dcid +
-6684// scid len + scid + length + pn
-6685const MAX_HANDSHAKE_OR_0RTT_HEADER_SIZE: usize =
-6686 1 + 4 + 1 + MAX_CID_SIZE + 1 + MAX_CID_SIZE + VarInt::from_u32(u16::MAX as u32).size() + 4;
-6687
-6688/// Perform key updates this many packets before the AEAD confidentiality limit.
-6689///
-6690/// Chosen arbitrarily, intended to be large enough to prevent spurious connection loss.
-6691const KEY_UPDATE_MARGIN: u64 = 10_000;
-6692
-6693#[derive(Default)]
-6694struct SentFrames {
-6695 retransmits: ThinRetransmits,
-6696 /// The packet number of the largest acknowledged packet for each path
-6697 largest_acked: FxHashMap<PathId, u64>,
-6698 stream_frames: StreamMetaVec,
-6699 /// Whether the packet contains non-retransmittable frames (like datagrams)
-6700 non_retransmits: bool,
-6701 /// If the datagram containing these frames should be padded to the min MTU
-6702 requires_padding: bool,
-6703}
-6704
-6705impl SentFrames {
-6706 /// Returns whether the packet contains only ACKs
-6707 fn is_ack_only(&self, streams: &StreamsState) -> bool {
-6708 !self.largest_acked.is_empty()
-6709 && !self.non_retransmits
-6710 && self.stream_frames.is_empty()
-6711 && self.retransmits.is_empty(streams)
-6712 }
-6713
-6714 fn retransmits_mut(&mut self) -> &mut Retransmits {
-6715 self.retransmits.get_or_create()
-6716 }
-6717
-6718 fn record_sent_frame(&mut self, frame: frame::EncodableFrame<'_>) {
-6719 use frame::EncodableFrame::*;
-6720 match frame {
-6721 PathAck(path_ack_encoder) => {
-6722 if let Some(max) = path_ack_encoder.ranges.max() {
-6723 self.largest_acked.insert(path_ack_encoder.path_id, max);
-6724 }
-6725 }
-6726 Ack(ack_encoder) => {
-6727 if let Some(max) = ack_encoder.ranges.max() {
-6728 self.largest_acked.insert(PathId::ZERO, max);
-6729 }
-6730 }
-6731 Close(_) => { /* non retransmittable, but after this we don't really care */ }
-6732 PathResponse(_) => self.non_retransmits = true,
-6733 HandshakeDone(_) => self.retransmits_mut().handshake_done = true,
-6734 ReachOut(frame::ReachOut { round, ip, port }) => self
-6735 .retransmits_mut()
-6736 .reach_out
-6737 .get_or_insert_with(|| (round, Vec::new()))
-6738 .1
-6739 .push((ip, port)),
-6740 ObservedAddr(_) => self.retransmits_mut().observed_addr = true,
-6741 Ping(_) => self.non_retransmits = true,
-6742 ImmediateAck(_) => self.non_retransmits = true,
-6743 AckFrequency(_) => self.retransmits_mut().ack_frequency = true,
-6744 PathChallenge(_) => self.non_retransmits = true,
-6745 Crypto(crypto) => self.retransmits_mut().crypto.push_back(crypto),
-6746 PathAbandon(path_abandon) => {
-6747 self.retransmits_mut()
-6748 .path_abandon
-6749 .entry(path_abandon.path_id)
-6750 .or_insert(path_abandon.error_code);
-6751 }
-6752 PathStatusAvailable(frame::PathStatusAvailable { path_id, .. })
-6753 | PathStatusBackup(frame::PathStatusBackup { path_id, .. }) => {
-6754 self.retransmits_mut().path_status.insert(path_id);
-6755 }
-6756 MaxPathId(_) => self.retransmits_mut().max_path_id = true,
-6757 PathsBlocked(_) => self.retransmits_mut().paths_blocked = true,
-6758 PathCidsBlocked(path_cids_blocked) => {
-6759 self.retransmits_mut()
-6760 .path_cids_blocked
-6761 .insert(path_cids_blocked.path_id);
-6762 }
-6763 ResetStream(reset) => self
-6764 .retransmits_mut()
-6765 .reset_stream
-6766 .push((reset.id, reset.error_code)),
-6767 StopSending(stop_sending) => self.retransmits_mut().stop_sending.push(stop_sending),
-6768 NewConnectionId(new_cid) => self.retransmits_mut().new_cids.push(new_cid.issued()),
-6769 RetireConnectionId(retire_cid) => self
-6770 .retransmits_mut()
-6771 .retire_cids
-6772 .push((retire_cid.path_id.unwrap_or_default(), retire_cid.sequence)),
-6773 Datagram(_) => self.non_retransmits = true,
-6774 NewToken(_) => {}
-6775 AddAddress(add_address) => {
-6776 self.retransmits_mut().add_address.insert(add_address);
-6777 }
-6778 RemoveAddress(remove_address) => {
-6779 self.retransmits_mut().remove_address.insert(remove_address);
-6780 }
-6781 StreamMeta(stream_meta_encoder) => self.stream_frames.push(stream_meta_encoder.meta),
-6782 MaxData(_) => self.retransmits_mut().max_data = true,
-6783 MaxStreamData(max) => {
-6784 self.retransmits_mut().max_stream_data.insert(max.id);
-6785 }
-6786 MaxStreams(max_streams) => {
-6787 self.retransmits_mut().max_stream_id[max_streams.dir as usize] = true
-6788 }
-6789 }
-6790 }
-6791}
-6792
-6793/// Compute the negotiated idle timeout based on local and remote max_idle_timeout transport parameters.
-6794///
-6795/// According to the definition of max_idle_timeout, a value of `0` means the timeout is disabled; see <https://www.rfc-editor.org/rfc/rfc9000#section-18.2-4.4.1.>
-6796///
-6797/// According to the negotiation procedure, either the minimum of the timeouts or one specified is used as the negotiated value; see <https://www.rfc-editor.org/rfc/rfc9000#section-10.1-2.>
-6798///
-6799/// Returns the negotiated idle timeout as a `Duration`, or `None` when both endpoints have opted out of idle timeout.
-6800fn negotiate_max_idle_timeout(x: Option<VarInt>, y: Option<VarInt>) -> Option<Duration> {
-6801 match (x, y) {
-6802 (Some(VarInt(0)) | None, Some(VarInt(0)) | None) => None,
-6803 (Some(VarInt(0)) | None, Some(y)) => Some(Duration::from_millis(y.0)),
-6804 (Some(x), Some(VarInt(0)) | None) => Some(Duration::from_millis(x.0)),
-6805 (Some(x), Some(y)) => Some(Duration::from_millis(cmp::min(x, y).0)),
-6806 }
-6807}
-6808
-6809#[cfg(test)]
-6810mod tests {
-6811 use super::*;
-6812
-6813 #[test]
-6814 fn negotiate_max_idle_timeout_commutative() {
-6815 let test_params = [
-6816 (None, None, None),
-6817 (None, Some(VarInt(0)), None),
-6818 (None, Some(VarInt(2)), Some(Duration::from_millis(2))),
-6819 (Some(VarInt(0)), Some(VarInt(0)), None),
-6820 (
-6821 Some(VarInt(2)),
-6822 Some(VarInt(0)),
-6823 Some(Duration::from_millis(2)),
-6824 ),
-6825 (
-6826 Some(VarInt(1)),
-6827 Some(VarInt(4)),
-6828 Some(Duration::from_millis(1)),
-6829 ),
-6830 ];
-6831
-6832 for (left, right, result) in test_params {
-6833 assert_eq!(negotiate_max_idle_timeout(left, right), result);
-6834 assert_eq!(negotiate_max_idle_timeout(right, left), result);
-6835 }
-6836 }
-6837}
\ No newline at end of file
+6588 fn is_server(&self) -> bool {
+6589 self.side().is_server()
+6590 }
+6591
+6592 fn side(&self) -> Side {
+6593 match *self {
+6594 Self::Client { .. } => Side::Client,
+6595 Self::Server { .. } => Side::Server,
+6596 }
+6597 }
+6598}
+6599
+6600impl From<SideArgs> for ConnectionSide {
+6601 fn from(side: SideArgs) -> Self {
+6602 match side {
+6603 SideArgs::Client {
+6604 token_store,
+6605 server_name,
+6606 } => Self::Client {
+6607 token: token_store.take(&server_name).unwrap_or_default(),
+6608 token_store,
+6609 server_name,
+6610 },
+6611 SideArgs::Server {
+6612 server_config,
+6613 pref_addr_cid: _,
+6614 path_validated: _,
+6615 } => Self::Server { server_config },
+6616 }
+6617 }
+6618}
+6619
+6620/// Parameters to `Connection::new` specific to it being client-side or server-side
+6621pub(crate) enum SideArgs {
+6622 Client {
+6623 token_store: Arc<dyn TokenStore>,
+6624 server_name: String,
+6625 },
+6626 Server {
+6627 server_config: Arc<ServerConfig>,
+6628 pref_addr_cid: Option<ConnectionId>,
+6629 path_validated: bool,
+6630 },
+6631}
+6632
+6633impl SideArgs {
+6634 pub(crate) fn pref_addr_cid(&self) -> Option<ConnectionId> {
+6635 match *self {
+6636 Self::Client { .. } => None,
+6637 Self::Server { pref_addr_cid, .. } => pref_addr_cid,
+6638 }
+6639 }
+6640
+6641 pub(crate) fn path_validated(&self) -> bool {
+6642 match *self {
+6643 Self::Client { .. } => true,
+6644 Self::Server { path_validated, .. } => path_validated,
+6645 }
+6646 }
+6647
+6648 pub(crate) fn side(&self) -> Side {
+6649 match *self {
+6650 Self::Client { .. } => Side::Client,
+6651 Self::Server { .. } => Side::Server,
+6652 }
+6653 }
+6654}
+6655
+6656/// Reasons why a connection might be lost
+6657#[derive(Debug, Error, Clone, PartialEq, Eq)]
+6658pub enum ConnectionError {
+6659 /// The peer doesn't implement any supported version
+6660 #[error("peer doesn't implement any supported version")]
+6661 VersionMismatch,
+6662 /// The peer violated the QUIC specification as understood by this implementation
+6663 #[error(transparent)]
+6664 TransportError(#[from] TransportError),
+6665 /// The peer's QUIC stack aborted the connection automatically
+6666 #[error("aborted by peer: {0}")]
+6667 ConnectionClosed(frame::ConnectionClose),
+6668 /// The peer closed the connection
+6669 #[error("closed by peer: {0}")]
+6670 ApplicationClosed(frame::ApplicationClose),
+6671 /// The peer is unable to continue processing this connection, usually due to having restarted
+6672 #[error("reset by peer")]
+6673 Reset,
+6674 /// Communication with the peer has lapsed for longer than the negotiated idle timeout
+6675 ///
+6676 /// If neither side is sending keep-alives, a connection will time out after a long enough idle
+6677 /// period even if the peer is still reachable. See also [`TransportConfig::max_idle_timeout()`]
+6678 /// and [`TransportConfig::keep_alive_interval()`].
+6679 #[error("timed out")]
+6680 TimedOut,
+6681 /// The local application closed the connection
+6682 #[error("closed")]
+6683 LocallyClosed,
+6684 /// The connection could not be created because not enough of the CID space is available
+6685 ///
+6686 /// Try using longer connection IDs.
+6687 #[error("CIDs exhausted")]
+6688 CidsExhausted,
+6689}
+6690
+6691impl From<Close> for ConnectionError {
+6692 fn from(x: Close) -> Self {
+6693 match x {
+6694 Close::Connection(reason) => Self::ConnectionClosed(reason),
+6695 Close::Application(reason) => Self::ApplicationClosed(reason),
+6696 }
+6697 }
+6698}
+6699
+6700// For compatibility with API consumers
+6701impl From<ConnectionError> for io::Error {
+6702 fn from(x: ConnectionError) -> Self {
+6703 use ConnectionError::*;
+6704 let kind = match x {
+6705 TimedOut => io::ErrorKind::TimedOut,
+6706 Reset => io::ErrorKind::ConnectionReset,
+6707 ApplicationClosed(_) | ConnectionClosed(_) => io::ErrorKind::ConnectionAborted,
+6708 TransportError(_) | VersionMismatch | LocallyClosed | CidsExhausted => {
+6709 io::ErrorKind::Other
+6710 }
+6711 };
+6712 Self::new(kind, x)
+6713 }
+6714}
+6715
+6716/// Errors that might trigger a path being closed
+6717// TODO(@divma): maybe needs to be reworked based on what we want to do with the public API
+6718#[derive(Debug, Error, PartialEq, Eq, Clone, Copy)]
+6719pub enum PathError {
+6720 /// The extension was not negotiated with the peer
+6721 #[error("multipath extension not negotiated")]
+6722 MultipathNotNegotiated,
+6723 /// Paths can only be opened client-side
+6724 #[error("the server side may not open a path")]
+6725 ServerSideNotAllowed,
+6726 /// Current limits do not allow us to open more paths
+6727 #[error("maximum number of concurrent paths reached")]
+6728 MaxPathIdReached,
+6729 /// No remote CIDs available to open a new path
+6730 #[error("remoted CIDs exhausted")]
+6731 RemoteCidsExhausted,
+6732 /// Path could not be validated and will be abandoned
+6733 #[error("path validation failed")]
+6734 ValidationFailed,
+6735 /// The remote address for the path is not supported by the endpoint
+6736 #[error("invalid remote address")]
+6737 InvalidRemoteAddress(SocketAddr),
+6738}
+6739
+6740/// Errors triggered when abandoning a path
+6741#[derive(Debug, Error, Clone, Eq, PartialEq)]
+6742pub enum ClosePathError {
+6743 /// The path is already closed or was never opened
+6744 #[error("closed path")]
+6745 ClosedPath,
+6746 /// This is the last path, which can not be abandoned
+6747 #[error("last open path")]
+6748 LastOpenPath,
+6749}
+6750
+6751/// Error when the multipath extension was not negotiated, but attempted to be used.
+6752#[derive(Debug, Error, Clone, Copy)]
+6753#[error("Multipath extension not negotiated")]
+6754pub struct MultipathNotNegotiated {
+6755 _private: (),
+6756}
+6757
+6758/// Events of interest to the application
+6759#[derive(Debug)]
+6760pub enum Event {
+6761 /// The connection's handshake data is ready
+6762 HandshakeDataReady,
+6763 /// The connection was successfully established
+6764 Connected,
+6765 /// The TLS handshake was confirmed
+6766 HandshakeConfirmed,
+6767 /// The connection was lost
+6768 ///
+6769 /// Emitted if the peer closes the connection or an error is encountered.
+6770 ConnectionLost {
+6771 /// Reason that the connection was closed
+6772 reason: ConnectionError,
+6773 },
+6774 /// Stream events
+6775 Stream(StreamEvent),
+6776 /// One or more application datagrams have been received
+6777 DatagramReceived,
+6778 /// One or more application datagrams have been sent after blocking
+6779 DatagramsUnblocked,
+6780 /// (Multi)Path events
+6781 Path(PathEvent),
+6782 /// Iroh's nat traversal events
+6783 NatTraversal(iroh_hp::Event),
+6784}
+6785
+6786impl From<PathEvent> for Event {
+6787 fn from(source: PathEvent) -> Self {
+6788 Self::Path(source)
+6789 }
+6790}
+6791
+6792fn get_max_ack_delay(params: &TransportParameters) -> Duration {
+6793 Duration::from_micros(params.max_ack_delay.0 * 1000)
+6794}
+6795
+6796// Prevents overflow and improves behavior in extreme circumstances
+6797const MAX_BACKOFF_EXPONENT: u32 = 16;
+6798
+6799/// Minimal remaining size to allow packet coalescing, excluding cryptographic tag
+6800///
+6801/// This must be at least as large as the header for a well-formed empty packet to be coalesced,
+6802/// plus some space for frames. We only care about handshake headers because short header packets
+6803/// necessarily have smaller headers, and initial packets are only ever the first packet in a
+6804/// datagram (because we coalesce in ascending packet space order and the only reason to split a
+6805/// packet is when packet space changes).
+6806const MIN_PACKET_SPACE: usize = MAX_HANDSHAKE_OR_0RTT_HEADER_SIZE + 32;
+6807
+6808/// Largest amount of space that could be occupied by a Handshake or 0-RTT packet's header
+6809///
+6810/// Excludes packet-type-specific fields such as packet number or Initial token
+6811// https://www.rfc-editor.org/rfc/rfc9000.html#name-0-rtt: flags + version + dcid len + dcid +
+6812// scid len + scid + length + pn
+6813const MAX_HANDSHAKE_OR_0RTT_HEADER_SIZE: usize =
+6814 1 + 4 + 1 + MAX_CID_SIZE + 1 + MAX_CID_SIZE + VarInt::from_u32(u16::MAX as u32).size() + 4;
+6815
+6816/// Perform key updates this many packets before the AEAD confidentiality limit.
+6817///
+6818/// Chosen arbitrarily, intended to be large enough to prevent spurious connection loss.
+6819const KEY_UPDATE_MARGIN: u64 = 10_000;
+6820
+6821#[derive(Default)]
+6822struct SentFrames {
+6823 retransmits: ThinRetransmits,
+6824 /// The packet number of the largest acknowledged packet for each path
+6825 largest_acked: FxHashMap<PathId, u64>,
+6826 stream_frames: StreamMetaVec,
+6827 /// Whether the packet contains non-retransmittable frames (like datagrams)
+6828 non_retransmits: bool,
+6829 /// If the datagram containing these frames should be padded to the min MTU
+6830 requires_padding: bool,
+6831}
+6832
+6833impl SentFrames {
+6834 /// Returns whether the packet contains only ACKs
+6835 fn is_ack_only(&self, streams: &StreamsState) -> bool {
+6836 !self.largest_acked.is_empty()
+6837 && !self.non_retransmits
+6838 && self.stream_frames.is_empty()
+6839 && self.retransmits.is_empty(streams)
+6840 }
+6841
+6842 fn retransmits_mut(&mut self) -> &mut Retransmits {
+6843 self.retransmits.get_or_create()
+6844 }
+6845
+6846 fn record_sent_frame(&mut self, frame: frame::EncodableFrame<'_>) {
+6847 use frame::EncodableFrame::*;
+6848 match frame {
+6849 PathAck(path_ack_encoder) => {
+6850 if let Some(max) = path_ack_encoder.ranges.max() {
+6851 self.largest_acked.insert(path_ack_encoder.path_id, max);
+6852 }
+6853 }
+6854 Ack(ack_encoder) => {
+6855 if let Some(max) = ack_encoder.ranges.max() {
+6856 self.largest_acked.insert(PathId::ZERO, max);
+6857 }
+6858 }
+6859 Close(_) => { /* non retransmittable, but after this we don't really care */ }
+6860 PathResponse(_) => self.non_retransmits = true,
+6861 HandshakeDone(_) => self.retransmits_mut().handshake_done = true,
+6862 ReachOut(frame::ReachOut { round, ip, port }) => self
+6863 .retransmits_mut()
+6864 .reach_out
+6865 .get_or_insert_with(|| (round, Vec::new()))
+6866 .1
+6867 .push((ip, port)),
+6868 ObservedAddr(_) => self.retransmits_mut().observed_addr = true,
+6869 Ping(_) => self.non_retransmits = true,
+6870 ImmediateAck(_) => self.non_retransmits = true,
+6871 AckFrequency(_) => self.retransmits_mut().ack_frequency = true,
+6872 PathChallenge(_) => self.non_retransmits = true,
+6873 Crypto(crypto) => self.retransmits_mut().crypto.push_back(crypto),
+6874 PathAbandon(path_abandon) => {
+6875 self.retransmits_mut()
+6876 .path_abandon
+6877 .entry(path_abandon.path_id)
+6878 .or_insert(path_abandon.error_code);
+6879 }
+6880 PathStatusAvailable(frame::PathStatusAvailable { path_id, .. })
+6881 | PathStatusBackup(frame::PathStatusBackup { path_id, .. }) => {
+6882 self.retransmits_mut().path_status.insert(path_id);
+6883 }
+6884 MaxPathId(_) => self.retransmits_mut().max_path_id = true,
+6885 PathsBlocked(_) => self.retransmits_mut().paths_blocked = true,
+6886 PathCidsBlocked(path_cids_blocked) => {
+6887 self.retransmits_mut()
+6888 .path_cids_blocked
+6889 .insert(path_cids_blocked.path_id);
+6890 }
+6891 ResetStream(reset) => self
+6892 .retransmits_mut()
+6893 .reset_stream
+6894 .push((reset.id, reset.error_code)),
+6895 StopSending(stop_sending) => self.retransmits_mut().stop_sending.push(stop_sending),
+6896 NewConnectionId(new_cid) => self.retransmits_mut().new_cids.push(new_cid.issued()),
+6897 RetireConnectionId(retire_cid) => self
+6898 .retransmits_mut()
+6899 .retire_cids
+6900 .push((retire_cid.path_id.unwrap_or_default(), retire_cid.sequence)),
+6901 Datagram(_) => self.non_retransmits = true,
+6902 NewToken(_) => {}
+6903 AddAddress(add_address) => {
+6904 self.retransmits_mut().add_address.insert(add_address);
+6905 }
+6906 RemoveAddress(remove_address) => {
+6907 self.retransmits_mut().remove_address.insert(remove_address);
+6908 }
+6909 StreamMeta(stream_meta_encoder) => self.stream_frames.push(stream_meta_encoder.meta),
+6910 MaxData(_) => self.retransmits_mut().max_data = true,
+6911 MaxStreamData(max) => {
+6912 self.retransmits_mut().max_stream_data.insert(max.id);
+6913 }
+6914 MaxStreams(max_streams) => {
+6915 self.retransmits_mut().max_stream_id[max_streams.dir as usize] = true
+6916 }
+6917 }
+6918 }
+6919}
+6920
+6921/// Compute the negotiated idle timeout based on local and remote max_idle_timeout transport parameters.
+6922///
+6923/// According to the definition of max_idle_timeout, a value of `0` means the timeout is disabled; see <https://www.rfc-editor.org/rfc/rfc9000#section-18.2-4.4.1.>
+6924///
+6925/// According to the negotiation procedure, either the minimum of the timeouts or one specified is used as the negotiated value; see <https://www.rfc-editor.org/rfc/rfc9000#section-10.1-2.>
+6926///
+6927/// Returns the negotiated idle timeout as a `Duration`, or `None` when both endpoints have opted out of idle timeout.
+6928fn negotiate_max_idle_timeout(x: Option<VarInt>, y: Option<VarInt>) -> Option<Duration> {
+6929 match (x, y) {
+6930 (Some(VarInt(0)) | None, Some(VarInt(0)) | None) => None,
+6931 (Some(VarInt(0)) | None, Some(y)) => Some(Duration::from_millis(y.0)),
+6932 (Some(x), Some(VarInt(0)) | None) => Some(Duration::from_millis(x.0)),
+6933 (Some(x), Some(y)) => Some(Duration::from_millis(cmp::min(x, y).0)),
+6934 }
+6935}
+6936
+6937#[cfg(test)]
+6938mod tests {
+6939 use super::*;
+6940
+6941 #[test]
+6942 fn negotiate_max_idle_timeout_commutative() {
+6943 let test_params = [
+6944 (None, None, None),
+6945 (None, Some(VarInt(0)), None),
+6946 (None, Some(VarInt(2)), Some(Duration::from_millis(2))),
+6947 (Some(VarInt(0)), Some(VarInt(0)), None),
+6948 (
+6949 Some(VarInt(2)),
+6950 Some(VarInt(0)),
+6951 Some(Duration::from_millis(2)),
+6952 ),
+6953 (
+6954 Some(VarInt(1)),
+6955 Some(VarInt(4)),
+6956 Some(Duration::from_millis(1)),
+6957 ),
+6958 ];
+6959
+6960 for (left, right, result) in test_params {
+6961 assert_eq!(negotiate_max_idle_timeout(left, right), result);
+6962 assert_eq!(negotiate_max_idle_timeout(right, left), result);
+6963 }
+6964 }
+6965}
\ No newline at end of file
diff --git a/pr/255/docs/src/iroh_quinn_proto/connection/packet_builder.rs.html b/pr/255/docs/src/iroh_quinn_proto/connection/packet_builder.rs.html
index cb07743e9..923cf2568 100644
--- a/pr/255/docs/src/iroh_quinn_proto/connection/packet_builder.rs.html
+++ b/pr/255/docs/src/iroh_quinn_proto/connection/packet_builder.rs.html
@@ -282,7 +282,7 @@
282 conn: &mut Connection,
283 path_id: PathId,
284 pad_datagram: PadDatagram,
-285 ) {
+285 ) -> u64 {
286 match pad_datagram {
287 PadDatagram::No => (),
288 PadDatagram::ToSize(size) => self.pad_to(size),
@@ -329,121 +329,123 @@
329 conn.set_loss_detection_timer(now, path_id);
330 conn.path_data_mut(path_id).pacing.on_transmit(size);
331 }
-332 }
-333
-334 /// Encrypt packet, returning the length of the packet and whether padding was added
-335 pub(super) fn finish(
-336 mut self,
-337 conn: &mut Connection,
-338 now: Instant,
-339 ) -> (usize, bool, SentFrames) {
-340 debug_assert!(
-341 self.buf.len() <= self.buf.datagram_max_offset() - self.tag_len,
-342 "packet exceeds maximum size"
-343 );
-344 let pad = self.buf.len() < self.min_size;
-345 if pad {
-346 let padding = self.min_size - self.buf.len();
-347 trace!("PADDING * {}", padding);
-348 self.buf.put_bytes(0, padding);
-349 self.qlog.frame_padding(padding);
-350 }
-351
-352 let space = &conn.spaces[self.space];
-353 let (header_crypto, packet_crypto) = if let Some(ref crypto) = space.crypto {
-354 (&*crypto.header.local, &*crypto.packet.local)
-355 } else if self.space == SpaceId::Data {
-356 let zero_rtt = conn.zero_rtt_crypto.as_ref().unwrap();
-357 (&*zero_rtt.header, &*zero_rtt.packet)
-358 } else {
-359 unreachable!("tried to send {:?} packet without keys", self.space);
-360 };
-361
-362 debug_assert_eq!(
-363 packet_crypto.tag_len(),
-364 self.tag_len,
-365 "Mismatching crypto tag len"
-366 );
-367
-368 self.buf.put_bytes(0, packet_crypto.tag_len());
-369 let encode_start = self.partial_encode.start;
-370 let packet_buf = &mut self.buf.as_mut_slice()[encode_start..];
-371 // for packet protection, PathId::ZERO and no path are equivalent.
-372 self.partial_encode.finish(
-373 packet_buf,
-374 header_crypto,
-375 Some((self.exact_number, self.path, packet_crypto)),
-376 );
-377
-378 let packet_len = self.buf.len() - encode_start;
-379 trace!(size = %packet_len, short_header = %self.short_header, "wrote packet");
-380 self.qlog.finalize(packet_len);
-381 conn.qlog.emit_packet_sent(self.qlog, now);
-382 (packet_len, pad, self.sent_frames)
-383 }
-384
-385 /// The number of additional bytes the current packet would take up if it was finished now
-386 ///
-387 /// This will include any padding which is required to make the size large enough to be
-388 /// encrypted correctly.
-389 pub(super) fn predict_packet_end(&self) -> usize {
-390 self.buf.len().max(self.min_size) + self.tag_len - self.buf.len()
-391 }
-392
-393 /// Returns the remaining space in the packet that can be taken up by QUIC frames
-394 ///
-395 /// This leaves space in the datagram for the cryptographic tag that needs to be written
-396 /// when the packet is finished.
-397 pub(super) fn frame_space_remaining(&self) -> usize {
-398 let max_offset = self.buf.datagram_max_offset() - self.tag_len;
-399 max_offset.saturating_sub(self.buf.len())
-400 }
-401
-402 pub(crate) fn require_padding(&mut self) {
-403 self.sent_frames.requires_padding = true;
-404 }
-405
-406 pub(crate) fn retransmits_mut(&mut self) -> &mut Retransmits {
-407 self.sent_frames.retransmits_mut()
-408 }
-409}
-410
-411#[derive(Debug, Copy, Clone)]
-412pub(super) enum PadDatagram {
-413 /// Do not pad the datagram
-414 No,
-415 /// To a specific size
-416 ToSize(u16),
-417 /// Pad to the current MTU/segment size
-418 ///
-419 /// For the first datagram in a transmit the MTU is the same as the
-420 /// [`TransmitBuf::segment_size`].
-421 ToSegmentSize,
-422 /// Pad to [`MIN_INITIAL_SIZE`], the minimal QUIC MTU of 1200 bytes
-423 ToMinMtu,
-424}
-425
-426impl std::ops::BitOrAssign for PadDatagram {
-427 fn bitor_assign(&mut self, rhs: Self) {
-428 *self = *self | rhs;
-429 }
-430}
-431
-432impl std::ops::BitOr for PadDatagram {
-433 type Output = Self;
-434
-435 fn bitor(self, rhs: Self) -> Self::Output {
-436 match (self, rhs) {
-437 (Self::No, rhs) => rhs,
-438 (Self::ToSize(size), Self::No) => Self::ToSize(size),
-439 (Self::ToSize(a), Self::ToSize(b)) => Self::ToSize(a.max(b)),
-440 (Self::ToSize(_), Self::ToSegmentSize) => Self::ToSegmentSize,
-441 (Self::ToSize(_), Self::ToMinMtu) => Self::ToMinMtu,
-442 (Self::ToSegmentSize, Self::No) => Self::ToSegmentSize,
-443 (Self::ToSegmentSize, Self::ToSize(_)) => Self::ToSegmentSize,
-444 (Self::ToSegmentSize, Self::ToSegmentSize) => Self::ToSegmentSize,
-445 (Self::ToSegmentSize, Self::ToMinMtu) => Self::ToMinMtu,
-446 (Self::ToMinMtu, _) => Self::ToMinMtu,
-447 }
-448 }
-449}
impl<T> CloneToUninit for Twhere
diff --git a/pr/255/docs/iroh_quinn_proto/struct.TransportError.html b/pr/255/docs/iroh_quinn_proto/struct.TransportError.html
index c4be43306..b6dc15d42 100644
--- a/pr/255/docs/iroh_quinn_proto/struct.TransportError.html
+++ b/pr/255/docs/iroh_quinn_proto/struct.TransportError.html
@@ -11,7 +11,7 @@
§reason: StringHuman-readable explanation of the reason
§crypto: Option<Arc<dyn Error + Send + Sync>>An underlying crypto (e.g. TLS) layer error
Implementations§
Trait Implementations§
Source§impl Error for Error
1.30.0 · Source§fn source(&self) -> Option<&(dyn Error + 'static)>
Returns the lower-level source of this error, if any. Read more1.0.0 · Source§fn description(&self) -> &str
👎Deprecated since 1.42.0: use the Display impl or to_string()Source§impl From<Error> for ConnectionClose
Source§fn from(x: TransportError) -> Self
Converts to this type from the input type.Source§impl From<Error> for ConnectionError
Source§fn from(source: TransportError) -> Self
Converts to this type from the input type.Source§impl From<Error> for TransportError
Trait Implementations§
Source§impl Error for Error
1.30.0 · Source§fn source(&self) -> Option<&(dyn Error + 'static)>
Returns the lower-level source of this error, if any. Read more1.0.0 · Source§fn description(&self) -> &str
👎Deprecated since 1.42.0: use the Display impl or to_string()Source§impl From<Error> for ConnectionClose
Source§fn from(x: TransportError) -> Self
Converts to this type from the input type.Source§impl From<Error> for ConnectionError
Source§fn from(source: TransportError) -> Self
Converts to this type from the input type.Source§impl From<Error> for TransportError
Source§impl Eq for Error
Auto Trait Implementations§
§impl Freeze for Error
§impl !RefUnwindSafe for Error
§impl Send for Error
§impl Sync for Error
§impl Unpin for Error
§impl !UnwindSafe for Error
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
diff --git a/pr/255/docs/src/iroh_quinn_proto/connection/mod.rs.html b/pr/255/docs/src/iroh_quinn_proto/connection/mod.rs.html
index 81ec7afd0..e1b5a58ea 100644
--- a/pr/255/docs/src/iroh_quinn_proto/connection/mod.rs.html
+++ b/pr/255/docs/src/iroh_quinn_proto/connection/mod.rs.html
@@ -343,826 +343,826 @@
343/// Return value for `poll_transmit_path_space`.
344#[derive(Debug)]
345enum PollPathSpaceStatus {
-346 /// Nothing to send on this space, continue to the next one.
-347 NextSpace,
-348 /// We have something to send, and data has been accumulated
-349 /// on the passed in `transmit`.
-350 Send { last_packet_number: Option<u64> },
-351 /// A transmit is ready to be sent out.
-352 SendTransmit {
-353 /// The transmit to send.
-354 transmit: Transmit,
-355 },
-356 /// Nothing to send currently.
-357 NothingToSend {
-358 /// Set to `true` if we consider the current inability to send something because of congestion control
-359 congestion_blocked: bool,
-360 },
-361}
-362
-363impl Connection {
-364 pub(crate) fn new(
-365 endpoint_config: Arc<EndpointConfig>,
-366 config: Arc<TransportConfig>,
-367 init_cid: ConnectionId,
-368 loc_cid: ConnectionId,
-369 rem_cid: ConnectionId,
-370 network_path: FourTuple,
-371 crypto: Box<dyn crypto::Session>,
-372 cid_gen: &dyn ConnectionIdGenerator,
-373 now: Instant,
-374 version: u32,
-375 allow_mtud: bool,
-376 rng_seed: [u8; 32],
-377 side_args: SideArgs,
-378 qlog: QlogSink,
-379 ) -> Self {
-380 let pref_addr_cid = side_args.pref_addr_cid();
-381 let path_validated = side_args.path_validated();
-382 let connection_side = ConnectionSide::from(side_args);
-383 let side = connection_side.side();
-384 let mut rng = StdRng::from_seed(rng_seed);
-385 let initial_space = {
-386 let mut space = PacketSpace::new(now, SpaceId::Initial, &mut rng);
-387 space.crypto = Some(crypto.initial_keys(init_cid, side));
-388 space
-389 };
-390 let handshake_space = PacketSpace::new(now, SpaceId::Handshake, &mut rng);
-391 #[cfg(test)]
-392 let data_space = match config.deterministic_packet_numbers {
-393 true => PacketSpace::new_deterministic(now, SpaceId::Data),
-394 false => PacketSpace::new(now, SpaceId::Data, &mut rng),
-395 };
-396 #[cfg(not(test))]
-397 let data_space = PacketSpace::new(now, SpaceId::Data, &mut rng);
-398 let state = State::handshake(state::Handshake {
-399 rem_cid_set: side.is_server(),
-400 expected_token: Bytes::new(),
-401 client_hello: None,
-402 allow_server_migration: side.is_client(),
-403 });
-404 let local_cid_state = FxHashMap::from_iter([(
-405 PathId::ZERO,
-406 CidState::new(
-407 cid_gen.cid_len(),
-408 cid_gen.cid_lifetime(),
-409 now,
-410 if pref_addr_cid.is_some() { 2 } else { 1 },
-411 ),
-412 )]);
-413
-414 let mut path = PathData::new(network_path, allow_mtud, None, 0, now, &config);
-415 // TODO(@divma): consider if we want to delay this until the path is validated
-416 path.open = true;
-417 let mut this = Self {
-418 endpoint_config,
-419 crypto,
-420 handshake_cid: loc_cid,
-421 rem_handshake_cid: rem_cid,
-422 local_cid_state,
-423 paths: BTreeMap::from_iter([(
-424 PathId::ZERO,
-425 PathState {
-426 data: path,
-427 prev: None,
-428 },
-429 )]),
-430 path_generation_counter: 0,
-431 allow_mtud,
-432 state,
-433 side: connection_side,
-434 zero_rtt_enabled: false,
-435 zero_rtt_crypto: None,
-436 key_phase: false,
-437 // A small initial key phase size ensures peers that don't handle key updates correctly
-438 // fail sooner rather than later. It's okay for both peers to do this, as the first one
-439 // to perform an update will reset the other's key phase size in `update_keys`, and a
-440 // simultaneous key update by both is just like a regular key update with a really fast
-441 // response. Inspired by quic-go's similar behavior of performing the first key update
-442 // at the 100th short-header packet.
-443 key_phase_size: rng.random_range(10..1000),
-444 peer_params: TransportParameters::default(),
-445 orig_rem_cid: rem_cid,
-446 initial_dst_cid: init_cid,
-447 retry_src_cid: None,
-448 events: VecDeque::new(),
-449 endpoint_events: VecDeque::new(),
-450 spin_enabled: config.allow_spin && rng.random_ratio(7, 8),
-451 spin: false,
-452 spaces: [initial_space, handshake_space, data_space],
-453 highest_space: SpaceId::Initial,
-454 prev_crypto: None,
-455 next_crypto: None,
-456 accepted_0rtt: false,
-457 permit_idle_reset: true,
-458 idle_timeout: match config.max_idle_timeout {
-459 None | Some(VarInt(0)) => None,
-460 Some(dur) => Some(Duration::from_millis(dur.0)),
-461 },
-462 timers: TimerTable::default(),
-463 authentication_failures: 0,
-464 close: false,
-465
-466 ack_frequency: AckFrequencyState::new(get_max_ack_delay(
-467 &TransportParameters::default(),
-468 )),
-469
-470 app_limited: false,
-471 receiving_ecn: false,
-472 total_authed_packets: 0,
+346 /// We have something to send, and data has been accumulated
+347 /// on the passed in `transmit`.
+348 Send { last_packet_number: Option<u64> },
+349 /// A transmit is ready to be sent out.
+350 SendTransmit {
+351 /// The transmit to send.
+352 transmit: Transmit,
+353 },
+354 /// Nothing to send currently on this space/path.
+355 NothingToSend {
+356 /// Set to `true` if we consider the current inability to send something because of congestion control
+357 congestion_blocked: bool,
+358 },
+359}
+360
+361impl Connection {
+362 pub(crate) fn new(
+363 endpoint_config: Arc<EndpointConfig>,
+364 config: Arc<TransportConfig>,
+365 init_cid: ConnectionId,
+366 loc_cid: ConnectionId,
+367 rem_cid: ConnectionId,
+368 network_path: FourTuple,
+369 crypto: Box<dyn crypto::Session>,
+370 cid_gen: &dyn ConnectionIdGenerator,
+371 now: Instant,
+372 version: u32,
+373 allow_mtud: bool,
+374 rng_seed: [u8; 32],
+375 side_args: SideArgs,
+376 qlog: QlogSink,
+377 ) -> Self {
+378 let pref_addr_cid = side_args.pref_addr_cid();
+379 let path_validated = side_args.path_validated();
+380 let connection_side = ConnectionSide::from(side_args);
+381 let side = connection_side.side();
+382 let mut rng = StdRng::from_seed(rng_seed);
+383 let initial_space = {
+384 let mut space = PacketSpace::new(now, SpaceId::Initial, &mut rng);
+385 space.crypto = Some(crypto.initial_keys(init_cid, side));
+386 space
+387 };
+388 let handshake_space = PacketSpace::new(now, SpaceId::Handshake, &mut rng);
+389 #[cfg(test)]
+390 let data_space = match config.deterministic_packet_numbers {
+391 true => PacketSpace::new_deterministic(now, SpaceId::Data),
+392 false => PacketSpace::new(now, SpaceId::Data, &mut rng),
+393 };
+394 #[cfg(not(test))]
+395 let data_space = PacketSpace::new(now, SpaceId::Data, &mut rng);
+396 let state = State::handshake(state::Handshake {
+397 rem_cid_set: side.is_server(),
+398 expected_token: Bytes::new(),
+399 client_hello: None,
+400 allow_server_migration: side.is_client(),
+401 });
+402 let local_cid_state = FxHashMap::from_iter([(
+403 PathId::ZERO,
+404 CidState::new(
+405 cid_gen.cid_len(),
+406 cid_gen.cid_lifetime(),
+407 now,
+408 if pref_addr_cid.is_some() { 2 } else { 1 },
+409 ),
+410 )]);
+411
+412 let mut path = PathData::new(network_path, allow_mtud, None, 0, now, &config);
+413 // TODO(@divma): consider if we want to delay this until the path is validated
+414 path.open = true;
+415 let mut this = Self {
+416 endpoint_config,
+417 crypto,
+418 handshake_cid: loc_cid,
+419 rem_handshake_cid: rem_cid,
+420 local_cid_state,
+421 paths: BTreeMap::from_iter([(
+422 PathId::ZERO,
+423 PathState {
+424 data: path,
+425 prev: None,
+426 },
+427 )]),
+428 path_generation_counter: 0,
+429 allow_mtud,
+430 state,
+431 side: connection_side,
+432 zero_rtt_enabled: false,
+433 zero_rtt_crypto: None,
+434 key_phase: false,
+435 // A small initial key phase size ensures peers that don't handle key updates correctly
+436 // fail sooner rather than later. It's okay for both peers to do this, as the first one
+437 // to perform an update will reset the other's key phase size in `update_keys`, and a
+438 // simultaneous key update by both is just like a regular key update with a really fast
+439 // response. Inspired by quic-go's similar behavior of performing the first key update
+440 // at the 100th short-header packet.
+441 key_phase_size: rng.random_range(10..1000),
+442 peer_params: TransportParameters::default(),
+443 orig_rem_cid: rem_cid,
+444 initial_dst_cid: init_cid,
+445 retry_src_cid: None,
+446 events: VecDeque::new(),
+447 endpoint_events: VecDeque::new(),
+448 spin_enabled: config.allow_spin && rng.random_ratio(7, 8),
+449 spin: false,
+450 spaces: [initial_space, handshake_space, data_space],
+451 highest_space: SpaceId::Initial,
+452 prev_crypto: None,
+453 next_crypto: None,
+454 accepted_0rtt: false,
+455 permit_idle_reset: true,
+456 idle_timeout: match config.max_idle_timeout {
+457 None | Some(VarInt(0)) => None,
+458 Some(dur) => Some(Duration::from_millis(dur.0)),
+459 },
+460 timers: TimerTable::default(),
+461 authentication_failures: 0,
+462 close: false,
+463
+464 ack_frequency: AckFrequencyState::new(get_max_ack_delay(
+465 &TransportParameters::default(),
+466 )),
+467
+468 app_limited: false,
+469 receiving_ecn: false,
+470 total_authed_packets: 0,
+471
+472 next_observed_addr_seq_no: 0u32.into(),
473
-474 next_observed_addr_seq_no: 0u32.into(),
-475
-476 streams: StreamsState::new(
-477 side,
-478 config.max_concurrent_uni_streams,
-479 config.max_concurrent_bidi_streams,
-480 config.send_window,
-481 config.receive_window,
-482 config.stream_receive_window,
-483 ),
-484 datagrams: DatagramState::default(),
-485 config,
-486 rem_cids: FxHashMap::from_iter([(PathId::ZERO, CidQueue::new(rem_cid))]),
-487 rng,
-488 stats: ConnectionStats::default(),
-489 path_stats: Default::default(),
-490 version,
-491
-492 // peer params are not yet known, so multipath is not enabled
-493 max_concurrent_paths: NonZeroU32::MIN,
-494 local_max_path_id: PathId::ZERO,
-495 remote_max_path_id: PathId::ZERO,
-496 max_path_id_with_cids: PathId::ZERO,
-497 abandoned_paths: Default::default(),
-498
-499 // iroh's nat traversal
-500 iroh_hp: Default::default(),
-501 qlog,
-502 };
-503 if path_validated {
-504 this.on_path_validated(PathId::ZERO);
-505 }
-506 if side.is_client() {
-507 // Kick off the connection
-508 this.write_crypto();
-509 this.init_0rtt(now);
-510 }
-511 this.qlog
-512 .emit_tuple_assigned(PathId::ZERO, network_path, now);
-513 this
-514 }
-515
-516 /// Returns the next time at which `handle_timeout` should be called
-517 ///
-518 /// The value returned may change after:
-519 /// - the application performed some I/O on the connection
-520 /// - a call was made to `handle_event`
-521 /// - a call to `poll_transmit` returned `Some`
-522 /// - a call was made to `handle_timeout`
-523 #[must_use]
-524 pub fn poll_timeout(&mut self) -> Option<Instant> {
-525 self.timers.peek()
-526 }
-527
-528 /// Returns application-facing events
-529 ///
-530 /// Connections should be polled for events after:
-531 /// - a call was made to `handle_event`
-532 /// - a call was made to `handle_timeout`
-533 #[must_use]
-534 pub fn poll(&mut self) -> Option<Event> {
-535 if let Some(x) = self.events.pop_front() {
-536 return Some(x);
-537 }
-538
-539 if let Some(event) = self.streams.poll() {
-540 return Some(Event::Stream(event));
-541 }
-542
-543 if let Some(reason) = self.state.take_error() {
-544 return Some(Event::ConnectionLost { reason });
-545 }
-546
-547 None
-548 }
-549
-550 /// Return endpoint-facing events
-551 #[must_use]
-552 pub fn poll_endpoint_events(&mut self) -> Option<EndpointEvent> {
-553 self.endpoint_events.pop_front().map(EndpointEvent)
-554 }
-555
-556 /// Provide control over streams
-557 #[must_use]
-558 pub fn streams(&mut self) -> Streams<'_> {
-559 Streams {
-560 state: &mut self.streams,
-561 conn_state: &self.state,
-562 }
-563 }
-564
-565 /// Provide control over streams
-566 #[must_use]
-567 pub fn recv_stream(&mut self, id: StreamId) -> RecvStream<'_> {
-568 assert!(id.dir() == Dir::Bi || id.initiator() != self.side.side());
-569 RecvStream {
-570 id,
-571 state: &mut self.streams,
-572 pending: &mut self.spaces[SpaceId::Data].pending,
-573 }
-574 }
-575
-576 /// Provide control over streams
-577 #[must_use]
-578 pub fn send_stream(&mut self, id: StreamId) -> SendStream<'_> {
-579 assert!(id.dir() == Dir::Bi || id.initiator() == self.side.side());
-580 SendStream {
-581 id,
-582 state: &mut self.streams,
-583 pending: &mut self.spaces[SpaceId::Data].pending,
-584 conn_state: &self.state,
-585 }
-586 }
-587
-588 /// Opens a new path only if no path on the same network path currently exists.
-589 ///
-590 /// This comparison will use [`FourTuple::is_probably_same_path`] on the given `network_path`
-591 /// and pass it existing path's network paths.
-592 ///
-593 /// This means that you can pass `local_ip: None` to make the comparison only compare
-594 /// remote addresses.
-595 ///
-596 /// This avoids having to guess which local interface will be used to communicate with the
-597 /// remote, should it not be known yet. We assume that if we already have a path to the remote,
-598 /// the OS is likely to use the same interface to talk to said remote.
-599 ///
-600 /// See also [`open_path`]. Returns `(path_id, true)` if the path already existed. `(path_id,
-601 /// false)` if was opened.
-602 ///
-603 /// [`open_path`]: Connection::open_path
-604 pub fn open_path_ensure(
-605 &mut self,
-606 network_path: FourTuple,
-607 initial_status: PathStatus,
-608 now: Instant,
-609 ) -> Result<(PathId, bool), PathError> {
-610 let existing_open_path = self.paths.iter().find(|(id, path)| {
-611 network_path.is_probably_same_path(&path.data.network_path)
-612 && !self.abandoned_paths.contains(*id)
-613 });
-614 match existing_open_path {
-615 Some((path_id, _state)) => Ok((*path_id, true)),
-616 None => Ok((self.open_path(network_path, initial_status, now)?, false)),
-617 }
-618 }
-619
-620 /// Opens a new path
-621 ///
-622 /// Further errors might occur and they will be emitted in [`PathEvent::LocallyClosed`] events.
-623 /// When the path is opened it will be reported as an [`PathEvent::Opened`].
-624 pub fn open_path(
-625 &mut self,
-626 network_path: FourTuple,
-627 initial_status: PathStatus,
-628 now: Instant,
-629 ) -> Result<PathId, PathError> {
-630 if !self.is_multipath_negotiated() {
-631 return Err(PathError::MultipathNotNegotiated);
-632 }
-633 if self.side().is_server() {
-634 return Err(PathError::ServerSideNotAllowed);
-635 }
-636
-637 let max_abandoned = self.abandoned_paths.iter().max().copied();
-638 let max_used = self.paths.keys().last().copied();
-639 let path_id = max_abandoned
-640 .max(max_used)
-641 .unwrap_or(PathId::ZERO)
-642 .saturating_add(1u8);
-643
-644 if Some(path_id) > self.max_path_id() {
-645 return Err(PathError::MaxPathIdReached);
-646 }
-647 if path_id > self.remote_max_path_id {
-648 self.spaces[SpaceId::Data].pending.paths_blocked = true;
-649 return Err(PathError::MaxPathIdReached);
-650 }
-651 if self.rem_cids.get(&path_id).map(CidQueue::active).is_none() {
-652 self.spaces[SpaceId::Data]
-653 .pending
-654 .path_cids_blocked
-655 .insert(path_id);
-656 return Err(PathError::RemoteCidsExhausted);
-657 }
-658
-659 let path = self.ensure_path(path_id, network_path, now, None);
-660 path.status.local_update(initial_status);
-661
-662 Ok(path_id)
-663 }
-664
-665 /// Closes a path by sending a PATH_ABANDON frame
-666 ///
-667 /// This will not allow closing the last path. It does allow closing paths which have
-668 /// not yet been opened, as e.g. is the case when receiving a PATH_ABANDON from the peer
-669 /// for a path that was never opened locally.
-670 pub fn close_path(
-671 &mut self,
-672 now: Instant,
-673 path_id: PathId,
-674 error_code: VarInt,
-675 ) -> Result<(), ClosePathError> {
-676 if self.abandoned_paths.contains(&path_id)
-677 || Some(path_id) > self.max_path_id()
-678 || !self.paths.contains_key(&path_id)
-679 {
-680 return Err(ClosePathError::ClosedPath);
-681 }
-682 if self
-683 .paths
-684 .iter()
-685 // Would there be any remaining, non-abandoned, validated paths
-686 .any(|(id, path)| {
-687 *id != path_id && !self.abandoned_paths.contains(id) && path.data.validated
-688 })
-689 .not()
-690 {
-691 return Err(ClosePathError::LastOpenPath);
-692 }
-693
-694 // Send PATH_ABANDON
-695 self.spaces[SpaceId::Data]
-696 .pending
-697 .path_abandon
-698 .insert(path_id, error_code.into());
-699
-700 // Remove pending NEW CIDs for this path
-701 let pending_space = &mut self.spaces[SpaceId::Data].pending;
-702 pending_space.new_cids.retain(|cid| cid.path_id != path_id);
-703 pending_space.path_cids_blocked.retain(|&id| id != path_id);
-704 pending_space.path_status.retain(|&id| id != path_id);
-705
-706 // Cleanup retransmits across ALL paths (CIDs for path_id may have been transmitted on other paths)
-707 for space in self.spaces[SpaceId::Data].iter_paths_mut() {
-708 for sent_packet in space.sent_packets.values_mut() {
-709 if let Some(retransmits) = sent_packet.retransmits.get_mut() {
-710 retransmits.new_cids.retain(|cid| cid.path_id != path_id);
-711 retransmits.path_cids_blocked.retain(|&id| id != path_id);
-712 retransmits.path_status.retain(|&id| id != path_id);
-713 }
-714 }
-715 }
-716
-717 // Consider remotely issued CIDs as retired.
-718 // Technically we don't have to do this just yet. We only need to do this *after*
-719 // the ABANDON_PATH frame is sent, allowing us to still send it on the
-720 // to-be-abandoned path. However it is recommended to send it on another path, and
-721 // we do not allow abandoning the last path anyway.
-722 self.rem_cids.remove(&path_id);
-723 self.endpoint_events
-724 .push_back(EndpointEventInner::RetireResetToken(path_id));
+474 streams: StreamsState::new(
+475 side,
+476 config.max_concurrent_uni_streams,
+477 config.max_concurrent_bidi_streams,
+478 config.send_window,
+479 config.receive_window,
+480 config.stream_receive_window,
+481 ),
+482 datagrams: DatagramState::default(),
+483 config,
+484 rem_cids: FxHashMap::from_iter([(PathId::ZERO, CidQueue::new(rem_cid))]),
+485 rng,
+486 stats: ConnectionStats::default(),
+487 path_stats: Default::default(),
+488 version,
+489
+490 // peer params are not yet known, so multipath is not enabled
+491 max_concurrent_paths: NonZeroU32::MIN,
+492 local_max_path_id: PathId::ZERO,
+493 remote_max_path_id: PathId::ZERO,
+494 max_path_id_with_cids: PathId::ZERO,
+495 abandoned_paths: Default::default(),
+496
+497 // iroh's nat traversal
+498 iroh_hp: Default::default(),
+499 qlog,
+500 };
+501 if path_validated {
+502 this.on_path_validated(PathId::ZERO);
+503 }
+504 if side.is_client() {
+505 // Kick off the connection
+506 this.write_crypto();
+507 this.init_0rtt(now);
+508 }
+509 this.qlog
+510 .emit_tuple_assigned(PathId::ZERO, network_path, now);
+511 this
+512 }
+513
+514 /// Returns the next time at which `handle_timeout` should be called
+515 ///
+516 /// The value returned may change after:
+517 /// - the application performed some I/O on the connection
+518 /// - a call was made to `handle_event`
+519 /// - a call to `poll_transmit` returned `Some`
+520 /// - a call was made to `handle_timeout`
+521 #[must_use]
+522 pub fn poll_timeout(&mut self) -> Option<Instant> {
+523 self.timers.peek()
+524 }
+525
+526 /// Returns application-facing events
+527 ///
+528 /// Connections should be polled for events after:
+529 /// - a call was made to `handle_event`
+530 /// - a call was made to `handle_timeout`
+531 #[must_use]
+532 pub fn poll(&mut self) -> Option<Event> {
+533 if let Some(x) = self.events.pop_front() {
+534 return Some(x);
+535 }
+536
+537 if let Some(event) = self.streams.poll() {
+538 return Some(Event::Stream(event));
+539 }
+540
+541 if let Some(reason) = self.state.take_error() {
+542 return Some(Event::ConnectionLost { reason });
+543 }
+544
+545 None
+546 }
+547
+548 /// Return endpoint-facing events
+549 #[must_use]
+550 pub fn poll_endpoint_events(&mut self) -> Option<EndpointEvent> {
+551 self.endpoint_events.pop_front().map(EndpointEvent)
+552 }
+553
+554 /// Provide control over streams
+555 #[must_use]
+556 pub fn streams(&mut self) -> Streams<'_> {
+557 Streams {
+558 state: &mut self.streams,
+559 conn_state: &self.state,
+560 }
+561 }
+562
+563 /// Provide control over streams
+564 #[must_use]
+565 pub fn recv_stream(&mut self, id: StreamId) -> RecvStream<'_> {
+566 assert!(id.dir() == Dir::Bi || id.initiator() != self.side.side());
+567 RecvStream {
+568 id,
+569 state: &mut self.streams,
+570 pending: &mut self.spaces[SpaceId::Data].pending,
+571 }
+572 }
+573
+574 /// Provide control over streams
+575 #[must_use]
+576 pub fn send_stream(&mut self, id: StreamId) -> SendStream<'_> {
+577 assert!(id.dir() == Dir::Bi || id.initiator() == self.side.side());
+578 SendStream {
+579 id,
+580 state: &mut self.streams,
+581 pending: &mut self.spaces[SpaceId::Data].pending,
+582 conn_state: &self.state,
+583 }
+584 }
+585
+586 /// Opens a new path only if no path on the same network path currently exists.
+587 ///
+588 /// This comparison will use [`FourTuple::is_probably_same_path`] on the given `network_path`
+589 /// and pass it existing path's network paths.
+590 ///
+591 /// This means that you can pass `local_ip: None` to make the comparison only compare
+592 /// remote addresses.
+593 ///
+594 /// This avoids having to guess which local interface will be used to communicate with the
+595 /// remote, should it not be known yet. We assume that if we already have a path to the remote,
+596 /// the OS is likely to use the same interface to talk to said remote.
+597 ///
+598 /// See also [`open_path`]. Returns `(path_id, true)` if the path already existed. `(path_id,
+599 /// false)` if was opened.
+600 ///
+601 /// [`open_path`]: Connection::open_path
+602 pub fn open_path_ensure(
+603 &mut self,
+604 network_path: FourTuple,
+605 initial_status: PathStatus,
+606 now: Instant,
+607 ) -> Result<(PathId, bool), PathError> {
+608 let existing_open_path = self.paths.iter().find(|(id, path)| {
+609 network_path.is_probably_same_path(&path.data.network_path)
+610 && !self.abandoned_paths.contains(*id)
+611 });
+612 match existing_open_path {
+613 Some((path_id, _state)) => Ok((*path_id, true)),
+614 None => Ok((self.open_path(network_path, initial_status, now)?, false)),
+615 }
+616 }
+617
+618 /// Opens a new path
+619 ///
+620 /// Further errors might occur and they will be emitted in [`PathEvent::LocallyClosed`] events.
+621 /// When the path is opened it will be reported as an [`PathEvent::Opened`].
+622 pub fn open_path(
+623 &mut self,
+624 network_path: FourTuple,
+625 initial_status: PathStatus,
+626 now: Instant,
+627 ) -> Result<PathId, PathError> {
+628 if !self.is_multipath_negotiated() {
+629 return Err(PathError::MultipathNotNegotiated);
+630 }
+631 if self.side().is_server() {
+632 return Err(PathError::ServerSideNotAllowed);
+633 }
+634
+635 let max_abandoned = self.abandoned_paths.iter().max().copied();
+636 let max_used = self.paths.keys().last().copied();
+637 let path_id = max_abandoned
+638 .max(max_used)
+639 .unwrap_or(PathId::ZERO)
+640 .saturating_add(1u8);
+641
+642 if Some(path_id) > self.max_path_id() {
+643 return Err(PathError::MaxPathIdReached);
+644 }
+645 if path_id > self.remote_max_path_id {
+646 self.spaces[SpaceId::Data].pending.paths_blocked = true;
+647 return Err(PathError::MaxPathIdReached);
+648 }
+649 if self.rem_cids.get(&path_id).map(CidQueue::active).is_none() {
+650 self.spaces[SpaceId::Data]
+651 .pending
+652 .path_cids_blocked
+653 .insert(path_id);
+654 return Err(PathError::RemoteCidsExhausted);
+655 }
+656
+657 let path = self.ensure_path(path_id, network_path, now, None);
+658 path.status.local_update(initial_status);
+659
+660 Ok(path_id)
+661 }
+662
+663 /// Closes a path by sending a PATH_ABANDON frame
+664 ///
+665 /// This will not allow closing the last path. It does allow closing paths which have
+666 /// not yet been opened, as e.g. is the case when receiving a PATH_ABANDON from the peer
+667 /// for a path that was never opened locally.
+668 pub fn close_path(
+669 &mut self,
+670 now: Instant,
+671 path_id: PathId,
+672 error_code: VarInt,
+673 ) -> Result<(), ClosePathError> {
+674 if self.abandoned_paths.contains(&path_id)
+675 || Some(path_id) > self.max_path_id()
+676 || !self.paths.contains_key(&path_id)
+677 {
+678 return Err(ClosePathError::ClosedPath);
+679 }
+680 if self
+681 .paths
+682 .iter()
+683 // Would there be any remaining, non-abandoned, validated paths
+684 .any(|(id, path)| {
+685 *id != path_id && !self.abandoned_paths.contains(id) && path.data.validated
+686 })
+687 .not()
+688 {
+689 return Err(ClosePathError::LastOpenPath);
+690 }
+691
+692 // Send PATH_ABANDON
+693 self.spaces[SpaceId::Data]
+694 .pending
+695 .path_abandon
+696 .insert(path_id, error_code.into());
+697
+698 // Remove pending NEW CIDs for this path
+699 let pending_space = &mut self.spaces[SpaceId::Data].pending;
+700 pending_space.new_cids.retain(|cid| cid.path_id != path_id);
+701 pending_space.path_cids_blocked.retain(|&id| id != path_id);
+702 pending_space.path_status.retain(|&id| id != path_id);
+703
+704 // Cleanup retransmits across ALL paths (CIDs for path_id may have been transmitted on other paths)
+705 for space in self.spaces[SpaceId::Data].iter_paths_mut() {
+706 for sent_packet in space.sent_packets.values_mut() {
+707 if let Some(retransmits) = sent_packet.retransmits.get_mut() {
+708 retransmits.new_cids.retain(|cid| cid.path_id != path_id);
+709 retransmits.path_cids_blocked.retain(|&id| id != path_id);
+710 retransmits.path_status.retain(|&id| id != path_id);
+711 }
+712 }
+713 }
+714
+715 // Consider remotely issued CIDs as retired.
+716 // Technically we don't have to do this just yet. We only need to do this *after*
+717 // the ABANDON_PATH frame is sent, allowing us to still send it on the
+718 // to-be-abandoned path. However it is recommended to send it on another path, and
+719 // we do not allow abandoning the last path anyway.
+720 self.rem_cids.remove(&path_id);
+721 self.endpoint_events
+722 .push_back(EndpointEventInner::RetireResetToken(path_id));
+723
+724 self.abandoned_paths.insert(path_id);
725
-726 self.abandoned_paths.insert(path_id);
+726 self.set_max_path_id(now, self.local_max_path_id.saturating_add(1u8));
727
-728 self.set_max_path_id(now, self.local_max_path_id.saturating_add(1u8));
-729
-730 Ok(())
-731 }
-732
-733 /// Gets the [`PathData`] for a known [`PathId`].
-734 ///
-735 /// Will panic if the path_id does not reference any known path.
-736 #[track_caller]
-737 fn path_data(&self, path_id: PathId) -> &PathData {
-738 if let Some(data) = self.paths.get(&path_id) {
-739 &data.data
-740 } else {
-741 panic!(
-742 "unknown path: {path_id}, currently known paths: {:?}",
-743 self.paths.keys().collect::<Vec<_>>()
-744 );
-745 }
-746 }
-747
-748 /// Gets a reference to the [`PathData`] for a [`PathId`]
-749 fn path(&self, path_id: PathId) -> Option<&PathData> {
-750 self.paths.get(&path_id).map(|path_state| &path_state.data)
-751 }
-752
-753 /// Gets a mutable reference to the [`PathData`] for a [`PathId`]
-754 fn path_mut(&mut self, path_id: PathId) -> Option<&mut PathData> {
-755 self.paths
-756 .get_mut(&path_id)
-757 .map(|path_state| &mut path_state.data)
-758 }
-759
-760 /// Returns all known paths.
-761 ///
-762 /// There is no guarantee any of these paths are open or usable.
-763 pub fn paths(&self) -> Vec<PathId> {
-764 self.paths.keys().copied().collect()
-765 }
-766
-767 /// Gets the local [`PathStatus`] for a known [`PathId`]
-768 pub fn path_status(&self, path_id: PathId) -> Result<PathStatus, ClosedPath> {
-769 self.path(path_id)
-770 .map(PathData::local_status)
-771 .ok_or(ClosedPath { _private: () })
-772 }
-773
-774 /// Returns the path's network path represented as a 4-tuple.
-775 pub fn network_path(&self, path_id: PathId) -> Result<FourTuple, ClosedPath> {
-776 self.path(path_id)
-777 .map(|path| path.network_path)
-778 .ok_or(ClosedPath { _private: () })
-779 }
-780
-781 /// Sets the [`PathStatus`] for a known [`PathId`]
-782 ///
-783 /// Returns the previous path status on success.
-784 pub fn set_path_status(
-785 &mut self,
-786 path_id: PathId,
-787 status: PathStatus,
-788 ) -> Result<PathStatus, SetPathStatusError> {
-789 if !self.is_multipath_negotiated() {
-790 return Err(SetPathStatusError::MultipathNotNegotiated);
-791 }
-792 let path = self
-793 .path_mut(path_id)
-794 .ok_or(SetPathStatusError::ClosedPath)?;
-795 let prev = match path.status.local_update(status) {
-796 Some(prev) => {
-797 self.spaces[SpaceId::Data]
-798 .pending
-799 .path_status
-800 .insert(path_id);
-801 prev
-802 }
-803 None => path.local_status(),
-804 };
-805 Ok(prev)
-806 }
-807
-808 /// Returns the remote path status
-809 // TODO(flub): Probably should also be some kind of path event? Not even sure if I like
-810 // this as an API, but for now it allows me to write a test easily.
-811 // TODO(flub): Technically this should be a Result<Option<PathSTatus>>?
-812 pub fn remote_path_status(&self, path_id: PathId) -> Option<PathStatus> {
-813 self.path(path_id).and_then(|path| path.remote_status())
-814 }
-815
-816 /// Sets the max_idle_timeout for a specific path
+728 Ok(())
+729 }
+730
+731 /// Gets the [`PathData`] for a known [`PathId`].
+732 ///
+733 /// Will panic if the path_id does not reference any known path.
+734 #[track_caller]
+735 fn path_data(&self, path_id: PathId) -> &PathData {
+736 if let Some(data) = self.paths.get(&path_id) {
+737 &data.data
+738 } else {
+739 panic!(
+740 "unknown path: {path_id}, currently known paths: {:?}",
+741 self.paths.keys().collect::<Vec<_>>()
+742 );
+743 }
+744 }
+745
+746 /// Gets a reference to the [`PathData`] for a [`PathId`]
+747 fn path(&self, path_id: PathId) -> Option<&PathData> {
+748 self.paths.get(&path_id).map(|path_state| &path_state.data)
+749 }
+750
+751 /// Gets a mutable reference to the [`PathData`] for a [`PathId`]
+752 fn path_mut(&mut self, path_id: PathId) -> Option<&mut PathData> {
+753 self.paths
+754 .get_mut(&path_id)
+755 .map(|path_state| &mut path_state.data)
+756 }
+757
+758 /// Returns all known paths.
+759 ///
+760 /// There is no guarantee any of these paths are open or usable.
+761 pub fn paths(&self) -> Vec<PathId> {
+762 self.paths.keys().copied().collect()
+763 }
+764
+765 /// Gets the local [`PathStatus`] for a known [`PathId`]
+766 pub fn path_status(&self, path_id: PathId) -> Result<PathStatus, ClosedPath> {
+767 self.path(path_id)
+768 .map(PathData::local_status)
+769 .ok_or(ClosedPath { _private: () })
+770 }
+771
+772 /// Returns the path's network path represented as a 4-tuple.
+773 pub fn network_path(&self, path_id: PathId) -> Result<FourTuple, ClosedPath> {
+774 self.path(path_id)
+775 .map(|path| path.network_path)
+776 .ok_or(ClosedPath { _private: () })
+777 }
+778
+779 /// Sets the [`PathStatus`] for a known [`PathId`]
+780 ///
+781 /// Returns the previous path status on success.
+782 pub fn set_path_status(
+783 &mut self,
+784 path_id: PathId,
+785 status: PathStatus,
+786 ) -> Result<PathStatus, SetPathStatusError> {
+787 if !self.is_multipath_negotiated() {
+788 return Err(SetPathStatusError::MultipathNotNegotiated);
+789 }
+790 let path = self
+791 .path_mut(path_id)
+792 .ok_or(SetPathStatusError::ClosedPath)?;
+793 let prev = match path.status.local_update(status) {
+794 Some(prev) => {
+795 self.spaces[SpaceId::Data]
+796 .pending
+797 .path_status
+798 .insert(path_id);
+799 prev
+800 }
+801 None => path.local_status(),
+802 };
+803 Ok(prev)
+804 }
+805
+806 /// Returns the remote path status
+807 // TODO(flub): Probably should also be some kind of path event? Not even sure if I like
+808 // this as an API, but for now it allows me to write a test easily.
+809 // TODO(flub): Technically this should be a Result<Option<PathSTatus>>?
+810 pub fn remote_path_status(&self, path_id: PathId) -> Option<PathStatus> {
+811 self.path(path_id).and_then(|path| path.remote_status())
+812 }
+813
+814 /// Sets the max_idle_timeout for a specific path
+815 ///
+816 /// See [`TransportConfig::default_path_max_idle_timeout`] for details.
817 ///
-818 /// See [`TransportConfig::default_path_max_idle_timeout`] for details.
-819 ///
-820 /// Returns the previous value of the setting.
-821 pub fn set_path_max_idle_timeout(
-822 &mut self,
-823 path_id: PathId,
-824 timeout: Option<Duration>,
-825 ) -> Result<Option<Duration>, ClosedPath> {
-826 let path = self
-827 .paths
-828 .get_mut(&path_id)
-829 .ok_or(ClosedPath { _private: () })?;
-830 Ok(std::mem::replace(&mut path.data.idle_timeout, timeout))
-831 }
-832
-833 /// Sets the keep_alive_interval for a specific path
+818 /// Returns the previous value of the setting.
+819 pub fn set_path_max_idle_timeout(
+820 &mut self,
+821 path_id: PathId,
+822 timeout: Option<Duration>,
+823 ) -> Result<Option<Duration>, ClosedPath> {
+824 let path = self
+825 .paths
+826 .get_mut(&path_id)
+827 .ok_or(ClosedPath { _private: () })?;
+828 Ok(std::mem::replace(&mut path.data.idle_timeout, timeout))
+829 }
+830
+831 /// Sets the keep_alive_interval for a specific path
+832 ///
+833 /// See [`TransportConfig::default_path_keep_alive_interval`] for details.
834 ///
-835 /// See [`TransportConfig::default_path_keep_alive_interval`] for details.
-836 ///
-837 /// Returns the previous value of the setting.
-838 pub fn set_path_keep_alive_interval(
-839 &mut self,
-840 path_id: PathId,
-841 interval: Option<Duration>,
-842 ) -> Result<Option<Duration>, ClosedPath> {
-843 let path = self
-844 .paths
-845 .get_mut(&path_id)
-846 .ok_or(ClosedPath { _private: () })?;
-847 Ok(std::mem::replace(&mut path.data.keep_alive, interval))
-848 }
-849
-850 /// Gets the [`PathData`] for a known [`PathId`].
-851 ///
-852 /// Will panic if the path_id does not reference any known path.
-853 #[track_caller]
-854 fn path_data_mut(&mut self, path_id: PathId) -> &mut PathData {
-855 &mut self.paths.get_mut(&path_id).expect("known path").data
-856 }
-857
-858 /// Find an open, validated path that's on the same network path as the given network path.
-859 ///
-860 /// Returns the first path matching, even if there's multiple.
-861 fn find_validated_path_on_network_path(
-862 &self,
-863 network_path: FourTuple,
-864 ) -> Option<(&PathId, &PathState)> {
-865 self.paths.iter().find(|(path_id, path_state)| {
-866 path_state.data.validated
-867 // Would this use the same network path, if network_path were used to send right now?
-868 && network_path.is_probably_same_path(&path_state.data.network_path)
-869 && !self.abandoned_paths.contains(path_id)
-870 })
-871 // TODO(@divma): we might want to ensure the path has been recently active to consider the
-872 // address validated
-873 // matheus23: Perhaps looking at !self.abandoned_paths.contains(path_id) is enough, given keep-alives?
-874 }
-875
-876 fn ensure_path(
-877 &mut self,
-878 path_id: PathId,
-879 network_path: FourTuple,
-880 now: Instant,
-881 pn: Option<u64>,
-882 ) -> &mut PathData {
-883 let valid_path = self.find_validated_path_on_network_path(network_path);
-884 let validated = valid_path.is_some();
-885 let initial_rtt = valid_path.map(|(_, path)| path.data.rtt.conservative());
-886 let vacant_entry = match self.paths.entry(path_id) {
-887 btree_map::Entry::Vacant(vacant_entry) => vacant_entry,
-888 btree_map::Entry::Occupied(occupied_entry) => {
-889 return &mut occupied_entry.into_mut().data;
-890 }
-891 };
-892
-893 debug!(%validated, %path_id, %network_path, "path added");
-894 let peer_max_udp_payload_size =
-895 u16::try_from(self.peer_params.max_udp_payload_size.into_inner()).unwrap_or(u16::MAX);
-896 self.path_generation_counter = self.path_generation_counter.wrapping_add(1);
-897 let mut data = PathData::new(
-898 network_path,
-899 self.allow_mtud,
-900 Some(peer_max_udp_payload_size),
-901 self.path_generation_counter,
-902 now,
-903 &self.config,
-904 );
-905
-906 data.validated = validated;
-907 if let Some(initial_rtt) = initial_rtt {
-908 data.rtt.reset_initial_rtt(initial_rtt);
-909 }
-910
-911 let pto = self.ack_frequency.max_ack_delay_for_pto() + data.rtt.pto_base();
-912 self.timers.set(
-913 Timer::PerPath(path_id, PathTimer::PathOpen),
-914 now + 3 * pto,
-915 self.qlog.with_time(now),
-916 );
-917
-918 // for the path to be opened we need to send a packet on the path. Sending a challenge
-919 // guarantees this
-920 data.send_new_challenge = true;
+835 /// Returns the previous value of the setting.
+836 pub fn set_path_keep_alive_interval(
+837 &mut self,
+838 path_id: PathId,
+839 interval: Option<Duration>,
+840 ) -> Result<Option<Duration>, ClosedPath> {
+841 let path = self
+842 .paths
+843 .get_mut(&path_id)
+844 .ok_or(ClosedPath { _private: () })?;
+845 Ok(std::mem::replace(&mut path.data.keep_alive, interval))
+846 }
+847
+848 /// Gets the [`PathData`] for a known [`PathId`].
+849 ///
+850 /// Will panic if the path_id does not reference any known path.
+851 #[track_caller]
+852 fn path_data_mut(&mut self, path_id: PathId) -> &mut PathData {
+853 &mut self.paths.get_mut(&path_id).expect("known path").data
+854 }
+855
+856 /// Find an open, validated path that's on the same network path as the given network path.
+857 ///
+858 /// Returns the first path matching, even if there's multiple.
+859 fn find_validated_path_on_network_path(
+860 &self,
+861 network_path: FourTuple,
+862 ) -> Option<(&PathId, &PathState)> {
+863 self.paths.iter().find(|(path_id, path_state)| {
+864 path_state.data.validated
+865 // Would this use the same network path, if network_path were used to send right now?
+866 && network_path.is_probably_same_path(&path_state.data.network_path)
+867 && !self.abandoned_paths.contains(path_id)
+868 })
+869 // TODO(@divma): we might want to ensure the path has been recently active to consider the
+870 // address validated
+871 // matheus23: Perhaps looking at !self.abandoned_paths.contains(path_id) is enough, given keep-alives?
+872 }
+873
+874 fn ensure_path(
+875 &mut self,
+876 path_id: PathId,
+877 network_path: FourTuple,
+878 now: Instant,
+879 pn: Option<u64>,
+880 ) -> &mut PathData {
+881 let valid_path = self.find_validated_path_on_network_path(network_path);
+882 let validated = valid_path.is_some();
+883 let initial_rtt = valid_path.map(|(_, path)| path.data.rtt.conservative());
+884 let vacant_entry = match self.paths.entry(path_id) {
+885 btree_map::Entry::Vacant(vacant_entry) => vacant_entry,
+886 btree_map::Entry::Occupied(occupied_entry) => {
+887 return &mut occupied_entry.into_mut().data;
+888 }
+889 };
+890
+891 debug!(%validated, %path_id, %network_path, "path added");
+892 let peer_max_udp_payload_size =
+893 u16::try_from(self.peer_params.max_udp_payload_size.into_inner()).unwrap_or(u16::MAX);
+894 self.path_generation_counter = self.path_generation_counter.wrapping_add(1);
+895 let mut data = PathData::new(
+896 network_path,
+897 self.allow_mtud,
+898 Some(peer_max_udp_payload_size),
+899 self.path_generation_counter,
+900 now,
+901 &self.config,
+902 );
+903
+904 data.validated = validated;
+905 if let Some(initial_rtt) = initial_rtt {
+906 data.rtt.reset_initial_rtt(initial_rtt);
+907 }
+908
+909 let pto = self.ack_frequency.max_ack_delay_for_pto() + data.rtt.pto_base();
+910 self.timers.set(
+911 Timer::PerPath(path_id, PathTimer::PathOpen),
+912 now + 3 * pto,
+913 self.qlog.with_time(now),
+914 );
+915
+916 // for the path to be opened we need to send a packet on the path. Sending a challenge
+917 // guarantees this
+918 data.send_new_challenge = true;
+919
+920 let path = vacant_entry.insert(PathState { data, prev: None });
921
-922 let path = vacant_entry.insert(PathState { data, prev: None });
-923
-924 let mut pn_space = spaces::PacketNumberSpace::new(now, SpaceId::Data, &mut self.rng);
-925 if let Some(pn) = pn {
-926 pn_space.dedup.insert(pn);
-927 }
-928 self.spaces[SpaceId::Data]
-929 .number_spaces
-930 .insert(path_id, pn_space);
-931 self.qlog.emit_tuple_assigned(path_id, network_path, now);
-932 &mut path.data
-933 }
-934
-935 /// Returns packets to transmit
-936 ///
-937 /// Connections should be polled for transmit after:
-938 /// - the application performed some I/O on the connection
-939 /// - a call was made to `handle_event`
-940 /// - a call was made to `handle_timeout`
-941 ///
-942 /// `max_datagrams` specifies how many datagrams can be returned inside a
-943 /// single Transmit using GSO. This must be at least 1.
-944 #[must_use]
-945 pub fn poll_transmit(
-946 &mut self,
-947 now: Instant,
-948 max_datagrams: NonZeroUsize,
-949 buf: &mut Vec<u8>,
-950 ) -> Option<Transmit> {
-951 if let Some(probing) = self
-952 .iroh_hp
-953 .server_side_mut()
-954 .ok()
-955 .and_then(iroh_hp::ServerState::next_probe)
-956 {
-957 let destination = probing.remote();
-958 trace!(%destination, "RAND_DATA packet");
-959 let token: u64 = self.rng.random();
-960 buf.put_u64(token);
-961 probing.finish(token);
-962 return Some(Transmit {
-963 destination,
-964 ecn: None,
-965 size: 8,
-966 segment_size: None,
-967 src_ip: None,
-968 });
-969 }
-970
-971 let max_datagrams = match self.config.enable_segmentation_offload {
-972 false => NonZeroUsize::MIN,
-973 true => max_datagrams,
-974 };
-975
-976 // Each call to poll_transmit can only send datagrams to one destination, because
-977 // all datagrams in a GSO batch are for the same destination. Therefore only
-978 // datagrams for one Path ID are produced for each poll_transmit call.
-979
-980 // TODO(flub): this is wishful thinking and not actually implemented, but perhaps it
-981 // should be:
-982
-983 // First, if we have to send a close, select a path for that.
-984 // Next, all paths that have a PATH_CHALLENGE or PATH_RESPONSE pending.
-985
-986 // For all, open, validated and AVAILABLE paths:
-987 // - Is the path congestion blocked or pacing blocked?
-988 // - call maybe_queue_ to ensure a tail-loss probe would be sent?
-989 // - do we need to send a close message?
-990 // - call can_send
-991 // Once there's nothing more to send on the AVAILABLE paths, do the same for BACKUP paths
-992
-993 // Check whether we need to send a close message
-994 let close = match self.state.as_type() {
-995 StateType::Drained => {
-996 self.app_limited = true;
-997 return None;
-998 }
-999 StateType::Draining | StateType::Closed => {
-1000 // self.close is only reset once the associated packet had been
-1001 // encoded successfully
-1002 if !self.close {
-1003 self.app_limited = true;
-1004 return None;
-1005 }
-1006 true
-1007 }
-1008 _ => false,
-1009 };
-1010
-1011 // Check whether we need to send an ACK_FREQUENCY frame
-1012 if let Some(config) = &self.config.ack_frequency_config {
-1013 let rtt = self
-1014 .paths
-1015 .values()
-1016 .map(|p| p.data.rtt.get())
-1017 .min()
-1018 .expect("one path exists");
-1019 self.spaces[SpaceId::Data].pending.ack_frequency = self
-1020 .ack_frequency
-1021 .should_send_ack_frequency(rtt, config, &self.peer_params)
-1022 && self.highest_space == SpaceId::Data
-1023 && self.peer_supports_ack_frequency();
-1024 }
-1025
-1026 // If there is any open, validated and available path we only want to send frames to
-1027 // any backup path that must be sent on that backup path exclusively.
-1028 let have_available_path = self.paths.iter().any(|(id, path)| {
-1029 path.data.validated
-1030 && path.data.local_status() == PathStatus::Available
-1031 && self.rem_cids.contains_key(id)
-1032 });
-1033
-1034 // TODO: how to avoid the allocation? Cannot use a for loop because of borrowing
-1035 let path_ids: Vec<_> = self.paths.keys().copied().collect();
-1036
-1037 let mut transmit = TransmitBuf::new(
-1038 buf,
-1039 max_datagrams,
-1040 self.path_data(path_ids[0]).current_mtu().into(),
-1041 );
+922 let mut pn_space = spaces::PacketNumberSpace::new(now, SpaceId::Data, &mut self.rng);
+923 if let Some(pn) = pn {
+924 pn_space.dedup.insert(pn);
+925 }
+926 self.spaces[SpaceId::Data]
+927 .number_spaces
+928 .insert(path_id, pn_space);
+929 self.qlog.emit_tuple_assigned(path_id, network_path, now);
+930 &mut path.data
+931 }
+932
+933 /// Returns packets to transmit
+934 ///
+935 /// Connections should be polled for transmit after:
+936 /// - the application performed some I/O on the connection
+937 /// - a call was made to `handle_event`
+938 /// - a call was made to `handle_timeout`
+939 ///
+940 /// `max_datagrams` specifies how many datagrams can be returned inside a
+941 /// single Transmit using GSO. This must be at least 1.
+942 #[must_use]
+943 pub fn poll_transmit(
+944 &mut self,
+945 now: Instant,
+946 max_datagrams: NonZeroUsize,
+947 buf: &mut Vec<u8>,
+948 ) -> Option<Transmit> {
+949 if let Some(probing) = self
+950 .iroh_hp
+951 .server_side_mut()
+952 .ok()
+953 .and_then(iroh_hp::ServerState::next_probe)
+954 {
+955 let destination = probing.remote();
+956 trace!(%destination, "RAND_DATA packet");
+957 let token: u64 = self.rng.random();
+958 buf.put_u64(token);
+959 probing.finish(token);
+960 return Some(Transmit {
+961 destination,
+962 ecn: None,
+963 size: 8,
+964 segment_size: None,
+965 src_ip: None,
+966 });
+967 }
+968
+969 let max_datagrams = match self.config.enable_segmentation_offload {
+970 false => NonZeroUsize::MIN,
+971 true => max_datagrams,
+972 };
+973
+974 // Each call to poll_transmit can only send datagrams to one destination, because
+975 // all datagrams in a GSO batch are for the same destination. Therefore only
+976 // datagrams for one Path ID are produced for each poll_transmit call.
+977
+978 // TODO(flub): this is wishful thinking and not actually implemented, but perhaps it
+979 // should be:
+980
+981 // First, if we have to send a close, select a path for that.
+982 // Next, all paths that have a PATH_CHALLENGE or PATH_RESPONSE pending.
+983
+984 // For all, open, validated and AVAILABLE paths:
+985 // - Is the path congestion blocked or pacing blocked?
+986 // - call maybe_queue_ to ensure a tail-loss probe would be sent?
+987 // - do we need to send a close message?
+988 // - call can_send
+989 // Once there's nothing more to send on the AVAILABLE paths, do the same for BACKUP paths
+990
+991 // Check whether we need to send a close message
+992 let close = match self.state.as_type() {
+993 StateType::Drained => {
+994 self.app_limited = true;
+995 return None;
+996 }
+997 StateType::Draining | StateType::Closed => {
+998 // self.close is only reset once the associated packet had been
+999 // encoded successfully
+1000 if !self.close {
+1001 self.app_limited = true;
+1002 return None;
+1003 }
+1004 true
+1005 }
+1006 _ => false,
+1007 };
+1008
+1009 // Check whether we need to send an ACK_FREQUENCY frame
+1010 if let Some(config) = &self.config.ack_frequency_config {
+1011 let rtt = self
+1012 .paths
+1013 .values()
+1014 .map(|p| p.data.rtt.get())
+1015 .min()
+1016 .expect("one path exists");
+1017 self.spaces[SpaceId::Data].pending.ack_frequency = self
+1018 .ack_frequency
+1019 .should_send_ack_frequency(rtt, config, &self.peer_params)
+1020 && self.highest_space == SpaceId::Data
+1021 && self.peer_supports_ack_frequency();
+1022 }
+1023
+1024 // If there is any open, validated and available path we only want to send frames to
+1025 // any backup path that must be sent on that backup path exclusively.
+1026 let have_available_path = self.paths.iter().any(|(id, path)| {
+1027 path.data.validated
+1028 && path.data.local_status() == PathStatus::Available
+1029 && self.rem_cids.contains_key(id)
+1030 });
+1031
+1032 // TODO: how to avoid the allocation? Cannot use a for loop because of borrowing
+1033 let path_ids: Vec<_> = self.paths.keys().copied().collect();
+1034
+1035 let mut transmit = TransmitBuf::new(
+1036 buf,
+1037 max_datagrams,
+1038 self.path_data(path_ids[0]).current_mtu().into(),
+1039 );
+1040
+1041 let mut congestion_blocked = false;
1042
-1043 let mut congestion_blocked = false;
-1044
-1045 for &path_id in &path_ids {
-1046 // Update per path state
-1047 transmit.set_segment_size(self.path_data(path_id).current_mtu().into());
-1048
-1049 if let Some(challenge) = self.send_prev_path_challenge(now, &mut transmit, path_id) {
-1050 return Some(challenge);
-1051 }
-1052
-1053 match self.poll_transmit_path(now, &mut transmit, path_id, have_available_path, close) {
-1054 PollPathStatus::SendTransmit { transmit } => {
-1055 return Some(transmit);
-1056 }
-1057 PollPathStatus::Send => {
-1058 let transmit = self.build_transmit(path_id, transmit);
-1059 return Some(transmit);
-1060 }
-1061 PollPathStatus::NothingToSend {
-1062 congestion_blocked: cb,
-1063 } => {
-1064 // TODO: congestion blocked should be per path? how to deal with this better
-1065
-1066 // move to next path
-1067 if cb {
-1068 // note congestion block
-1069 congestion_blocked = true;
-1070 }
-1071 }
-1072 }
-1073 // Nothing more to send.
-1074 trace!(%path_id, "nothing to send on path");
-1075 }
-1076
-1077 // We didn't produce any application data packet
-1078 debug_assert!(
-1079 transmit.is_empty(),
-1080 "there was data in the transmit, but it was not sent"
-1081 );
-1082 self.app_limited = !congestion_blocked;
-1083
-1084 if self.state.is_established() {
-1085 // Try MTU probing now
-1086 for path_id in path_ids {
-1087 // Update per path state
-1088 transmit.set_segment_size(self.path_data(path_id).current_mtu().into());
-1089 self.poll_transmit_mtu_probe(now, &mut transmit, path_id);
-1090 if !transmit.is_empty() {
-1091 let transmit = self.build_transmit(path_id, transmit);
-1092 return Some(transmit);
-1093 }
-1094 }
-1095 }
-1096
-1097 None
-1098 }
-1099
-1100 fn build_transmit(&mut self, path_id: PathId, transmit: TransmitBuf<'_>) -> Transmit {
-1101 debug_assert!(
-1102 !transmit.is_empty(),
-1103 "must not be called with an empty transmit buffer"
-1104 );
-1105
-1106 let network_path = self.path_data(path_id).network_path;
-1107 trace!(
-1108 segment_size = transmit.segment_size(),
-1109 last_datagram_len = transmit.len() % transmit.segment_size(),
-1110 %network_path,
-1111 "sending {} bytes in {} datagrams",
-1112 transmit.len(),
-1113 transmit.num_datagrams()
-1114 );
-1115 self.path_data_mut(path_id)
-1116 .inc_total_sent(transmit.len() as u64);
-1117
-1118 self.stats
-1119 .udp_tx
-1120 .on_sent(transmit.num_datagrams() as u64, transmit.len());
-1121
-1122 Transmit {
-1123 destination: network_path.remote,
-1124 size: transmit.len(),
-1125 ecn: if self.path_data(path_id).sending_ecn {
-1126 Some(EcnCodepoint::Ect0)
-1127 } else {
-1128 None
-1129 },
-1130 segment_size: match transmit.num_datagrams() {
-1131 1 => None,
-1132 _ => Some(transmit.segment_size()),
-1133 },
-1134 src_ip: network_path.local_ip,
-1135 }
-1136 }
-1137
-1138 /// poll_transmit logic for a specific path
-1139 #[must_use]
-1140 fn poll_transmit_path(
-1141 &mut self,
-1142 now: Instant,
-1143 transmit: &mut TransmitBuf<'_>,
-1144 path_id: PathId,
-1145 have_available_path: bool,
-1146 close: bool,
-1147 ) -> PollPathStatus {
-1148 // Check if there is at least one active CID to use for sending
-1149 let Some(remote_cid) = self.rem_cids.get(&path_id).map(CidQueue::active) else {
-1150 self.on_remote_cids_exhausted(now, path_id);
-1151
-1152 return PollPathStatus::NothingToSend {
-1153 congestion_blocked: false,
-1154 };
-1155 };
-1156
-1157 // Whether this packet can be coalesced with another one in the same datagram.
-1158 let mut coalesce = true;
-1159
-1160 // Whether the last packet in the datagram must be padded so the datagram takes up
-1161 // to at least MIN_INITIAL_SIZE, or to the maximum segment size if this is smaller.
-1162 let mut pad_datagram = PadDatagram::No;
-1163
-1164 // The packet number of the last built packet.
-1165 let mut last_packet_number = None;
+1043 for &path_id in &path_ids {
+1044 // Update per path state
+1045 transmit.set_segment_size(self.path_data(path_id).current_mtu().into());
+1046
+1047 if let Some(challenge) = self.send_prev_path_challenge(now, &mut transmit, path_id) {
+1048 return Some(challenge);
+1049 }
+1050
+1051 match self.poll_transmit_path(now, &mut transmit, path_id, have_available_path, close) {
+1052 PollPathStatus::SendTransmit { transmit } => {
+1053 return Some(transmit);
+1054 }
+1055 PollPathStatus::Send => {
+1056 let transmit = self.build_transmit(path_id, transmit);
+1057 return Some(transmit);
+1058 }
+1059 PollPathStatus::NothingToSend {
+1060 congestion_blocked: cb,
+1061 } => {
+1062 // TODO: congestion blocked should be per path? how to deal with this better
+1063
+1064 // move to next path
+1065 if cb {
+1066 // note congestion block
+1067 congestion_blocked = true;
+1068 }
+1069 }
+1070 }
+1071 // Nothing more to send.
+1072 trace!(%path_id, "nothing to send on path");
+1073 }
+1074
+1075 // We didn't produce any application data packet
+1076 debug_assert!(
+1077 transmit.is_empty(),
+1078 "there was data in the transmit, but it was not sent"
+1079 );
+1080 self.app_limited = !congestion_blocked;
+1081
+1082 if self.state.is_established() {
+1083 // Try MTU probing now
+1084 for path_id in path_ids {
+1085 // Update per path state
+1086 transmit.set_segment_size(self.path_data(path_id).current_mtu().into());
+1087 self.poll_transmit_mtu_probe(now, &mut transmit, path_id);
+1088 if !transmit.is_empty() {
+1089 let transmit = self.build_transmit(path_id, transmit);
+1090 return Some(transmit);
+1091 }
+1092 }
+1093 }
+1094
+1095 None
+1096 }
+1097
+1098 fn build_transmit(&mut self, path_id: PathId, transmit: TransmitBuf<'_>) -> Transmit {
+1099 debug_assert!(
+1100 !transmit.is_empty(),
+1101 "must not be called with an empty transmit buffer"
+1102 );
+1103
+1104 let network_path = self.path_data(path_id).network_path;
+1105 trace!(
+1106 segment_size = transmit.segment_size(),
+1107 last_datagram_len = transmit.len() % transmit.segment_size(),
+1108 %network_path,
+1109 "sending {} bytes in {} datagrams",
+1110 transmit.len(),
+1111 transmit.num_datagrams()
+1112 );
+1113 self.path_data_mut(path_id)
+1114 .inc_total_sent(transmit.len() as u64);
+1115
+1116 self.stats
+1117 .udp_tx
+1118 .on_sent(transmit.num_datagrams() as u64, transmit.len());
+1119
+1120 Transmit {
+1121 destination: network_path.remote,
+1122 size: transmit.len(),
+1123 ecn: if self.path_data(path_id).sending_ecn {
+1124 Some(EcnCodepoint::Ect0)
+1125 } else {
+1126 None
+1127 },
+1128 segment_size: match transmit.num_datagrams() {
+1129 1 => None,
+1130 _ => Some(transmit.segment_size()),
+1131 },
+1132 src_ip: network_path.local_ip,
+1133 }
+1134 }
+1135
+1136 /// poll_transmit logic for a specific path
+1137 #[must_use]
+1138 fn poll_transmit_path(
+1139 &mut self,
+1140 now: Instant,
+1141 transmit: &mut TransmitBuf<'_>,
+1142 path_id: PathId,
+1143 have_available_path: bool,
+1144 close: bool,
+1145 ) -> PollPathStatus {
+1146 // Check if there is at least one active CID to use for sending
+1147 let Some(remote_cid) = self.rem_cids.get(&path_id).map(CidQueue::active) else {
+1148 self.on_remote_cids_exhausted(now, path_id);
+1149
+1150 return PollPathStatus::NothingToSend {
+1151 congestion_blocked: false,
+1152 };
+1153 };
+1154
+1155 // Whether the last packet in the datagram must be padded so the datagram takes up
+1156 // to at least MIN_INITIAL_SIZE, or to the maximum segment size if this is smaller.
+1157 let mut pad_datagram = PadDatagram::No;
+1158
+1159 // The packet number of the last built packet.
+1160 let mut last_packet_number = None;
+1161
+1162 // Whether this packet can be coalesced with another one in the same datagram.
+1163 let mut coalesce = true;
+1164
+1165 let mut congestion_blocked = false;
1166
1167 // Iterate over the available spaces
1168 for space_id in SpaceId::iter() {
@@ -1170,5668 +1170,5796 @@
1170 if path_id != PathId::ZERO && space_id != SpaceId::Data {
1171 continue;
1172 }
-1173
-1174 let res = self.poll_transmit_path_space(
-1175 now,
-1176 transmit,
-1177 path_id,
-1178 space_id,
-1179 remote_cid,
-1180 have_available_path,
-1181 close,
-1182 &mut coalesce,
-1183 &mut pad_datagram,
-1184 );
-1185 match res {
-1186 PollPathSpaceStatus::Send {
-1187 last_packet_number: lp,
-1188 } => {
-1189 last_packet_number = lp;
-1190 break;
-1191 }
-1192 PollPathSpaceStatus::SendTransmit { transmit } => {
-1193 return PollPathStatus::SendTransmit { transmit };
-1194 }
-1195 PollPathSpaceStatus::NothingToSend { congestion_blocked } => {
-1196 return PollPathStatus::NothingToSend { congestion_blocked };
-1197 }
-1198 PollPathSpaceStatus::NextSpace => {
-1199 // moving onto the next space
-1200 }
-1201 }
-1202 }
-1203
-1204 if let Some(last_packet_number) = last_packet_number {
-1205 // Note that when sending in multiple packet spaces the last packet number will
-1206 // be the one from the highest packet space.
-1207 self.path_data_mut(path_id).congestion.on_sent(
-1208 now,
-1209 transmit.len() as u64,
-1210 last_packet_number,
-1211 );
-1212 }
-1213
-1214 self.qlog.emit_recovery_metrics(
-1215 path_id,
-1216 &mut self.paths.get_mut(&path_id).unwrap().data,
-1217 now,
-1218 );
-1219
-1220 if transmit.is_empty() {
-1221 PollPathStatus::NothingToSend {
-1222 congestion_blocked: false,
+1173 // We only send up to the highest space
+1174 if space_id >= self.highest_space {
+1175 break;
+1176 }
+1177
+1178 // Check if we can send in this space for lower spaces.
+1179 let sendable_frames = if space_id < SpaceId::Data {
+1180 let sendable_frames = self.space_can_send_non_data(space_id, path_id, close);
+1181 let path_should_send =
+1182 self.path_should_send(path_id, space_id, have_available_path, &sendable_frames);
+1183
+1184 if !path_should_send {
+1185 if self.spaces[space_id].crypto.is_some() {
+1186 trace!(?space_id, %path_id, "nothing to send in space");
+1187 }
+1188 continue;
+1189 }
+1190 Some(sendable_frames)
+1191 } else {
+1192 None
+1193 };
+1194
+1195 let res = self.poll_transmit_path_space(
+1196 now,
+1197 transmit,
+1198 path_id,
+1199 space_id,
+1200 remote_cid,
+1201 have_available_path,
+1202 close,
+1203 sendable_frames,
+1204 &mut coalesce,
+1205 &mut pad_datagram,
+1206 );
+1207 match res {
+1208 PollPathSpaceStatus::Send {
+1209 last_packet_number: lp,
+1210 } => {
+1211 last_packet_number = lp;
+1212 break;
+1213 }
+1214 PollPathSpaceStatus::SendTransmit { transmit } => {
+1215 return PollPathStatus::SendTransmit { transmit };
+1216 }
+1217 PollPathSpaceStatus::NothingToSend {
+1218 congestion_blocked: cb,
+1219 } => {
+1220 congestion_blocked |= cb;
+1221 continue;
+1222 }
1223 }
-1224 } else {
-1225 PollPathStatus::Send
-1226 }
-1227 }
-1228
-1229 /// poll_transmit logic for a path_id - space_id combination
-1230 #[must_use]
-1231 fn poll_transmit_path_space(
-1232 &mut self,
-1233 now: Instant,
-1234 transmit: &mut TransmitBuf<'_>,
-1235 path_id: PathId,
-1236 space_id: SpaceId,
-1237 remote_cid: ConnectionId,
-1238 have_available_path: bool,
-1239 close: bool,
-1240 coalesce: &mut bool,
-1241 pad_datagram: &mut PadDatagram,
-1242 ) -> PollPathSpaceStatus {
-1243 let mut last_packet_number = None;
-1244
-1245 loop {
-1246 // Determine if anything can be sent in this packet number space (SpaceId + PathId).
-1247 let max_packet_size = if transmit.datagram_remaining_mut() > 0 {
-1248 // We are trying to coalesce another packet into this datagram.
-1249 transmit.datagram_remaining_mut()
-1250 } else {
-1251 // A new datagram needs to be started.
-1252 transmit.segment_size()
-1253 };
-1254
-1255 let can_send = self.space_can_send(space_id, path_id, max_packet_size, close);
-1256
-1257 let path_should_send = {
-1258 let path_exclusive_only = space_id == SpaceId::Data
-1259 && have_available_path
-1260 && self.path_data(path_id).local_status() == PathStatus::Backup;
-1261 let path_should_send = if path_exclusive_only {
-1262 can_send.path_exclusive
-1263 } else {
-1264 !can_send.is_empty()
-1265 };
-1266 let needs_loss_probe = self.spaces[space_id].for_path(path_id).loss_probes > 0;
-1267 path_should_send || needs_loss_probe
-1268 };
-1269
-1270 if !path_should_send && space_id < SpaceId::Data {
-1271 if self.spaces[space_id].crypto.is_some() {
-1272 trace!(?space_id, %path_id, "nothing to send in space");
-1273 }
-1274 return PollPathSpaceStatus::NextSpace;
-1275 }
-1276
-1277 let send_blocked = if path_should_send && transmit.datagram_remaining_mut() == 0 {
-1278 // Only check congestion control if a new datagram is needed.
-1279 self.path_congestion_check(space_id, path_id, transmit, &can_send, now)
-1280 } else {
-1281 PathBlocked::No
-1282 };
-1283
-1284 // Whether congestion control stopped the next packet from being sent. Further
-1285 // packets could still be built, as e.g. tail-loss probes are not congestion
-1286 // limited.
-1287 let congestion_blocked = if send_blocked != PathBlocked::No {
-1288 trace!(?space_id, %path_id, ?send_blocked, "congestion blocked");
-1289 true
-1290 } else {
-1291 false
-1292 };
-1293 if send_blocked != PathBlocked::No && space_id < SpaceId::Data {
-1294 // Higher spaces might still have tail-loss probes to send, which are not
-1295 // congestion blocked.
-1296 return PollPathSpaceStatus::NextSpace;
-1297 }
-1298 if !path_should_send || send_blocked != PathBlocked::No {
-1299 // Nothing more to send on this path, check the next path if possible.
-1300
-1301 // If there are any datagrams in the transmit, packets for another path can
-1302 // not be built.
-1303 if transmit.num_datagrams() > 0 {
-1304 return PollPathSpaceStatus::Send { last_packet_number };
-1305 }
-1306
-1307 return PollPathSpaceStatus::NothingToSend { congestion_blocked };
-1308 }
-1309
-1310 if transmit.datagram_remaining_mut() == 0 {
-1311 if transmit.num_datagrams() >= transmit.max_datagrams().get() {
-1312 // No more datagrams allowed
-1313 return PollPathSpaceStatus::Send { last_packet_number };
-1314 }
-1315
-1316 match self.spaces[space_id].for_path(path_id).loss_probes {
-1317 0 => transmit.start_new_datagram(),
-1318 _ => {
-1319 // We need something to send for a tail-loss probe.
-1320 let request_immediate_ack =
-1321 space_id == SpaceId::Data && self.peer_supports_ack_frequency();
-1322 self.spaces[space_id].maybe_queue_probe(
-1323 path_id,
-1324 request_immediate_ack,
-1325 &self.streams,
-1326 );
-1327
-1328 self.spaces[space_id].for_path(path_id).loss_probes -= 1;
-1329
-1330 // Clamp the datagram to at most the minimum MTU to ensure that loss
-1331 // probes can get through and enable recovery even if the path MTU
-1332 // has shrank unexpectedly.
-1333 transmit.start_new_datagram_with_size(std::cmp::min(
-1334 usize::from(INITIAL_MTU),
-1335 transmit.segment_size(),
-1336 ));
-1337 }
-1338 }
-1339 trace!(count = transmit.num_datagrams(), "new datagram started");
-1340 *coalesce = true;
-1341 *pad_datagram = PadDatagram::No;
-1342 }
-1343
-1344 // If coalescing another packet into the existing datagram, there should
-1345 // still be enough space for a whole packet.
-1346 if transmit.datagram_start_offset() < transmit.len() {
-1347 debug_assert!(transmit.datagram_remaining_mut() >= MIN_PACKET_SPACE);
-1348 }
-1349
-1350 //
-1351 // From here on, we've determined that a packet will definitely be sent.
-1352 //
+1224 }
+1225
+1226 if let Some(last_packet_number) = last_packet_number {
+1227 // Note that when sending in multiple packet spaces the last packet number will
+1228 // be the one from the highest packet space.
+1229 self.path_data_mut(path_id).congestion.on_sent(
+1230 now,
+1231 transmit.len() as u64,
+1232 last_packet_number,
+1233 );
+1234 }
+1235
+1236 self.qlog.emit_recovery_metrics(
+1237 path_id,
+1238 &mut self.paths.get_mut(&path_id).unwrap().data,
+1239 now,
+1240 );
+1241
+1242 if transmit.is_empty() {
+1243 PollPathStatus::NothingToSend { congestion_blocked }
+1244 } else {
+1245 PollPathStatus::Send
+1246 }
+1247 }
+1248
+1249 /// Determine if anything can be sent in this packet number space (SpaceId + PathId).
+1250 /// poll_transmit logic for a path_id - space_id combination
+1251 #[must_use]
+1252 fn poll_transmit_path_space(
+1253 &mut self,
+1254 now: Instant,
+1255 transmit: &mut TransmitBuf<'_>,
+1256 path_id: PathId,
+1257 space_id: SpaceId,
+1258 remote_cid: ConnectionId,
+1259 have_available_path: bool,
+1260 close: bool,
+1261 sendable_frames: Option<SendableFrames>,
+1262 coalesce: &mut bool,
+1263 pad_datagram: &mut PadDatagram,
+1264 ) -> PollPathSpaceStatus {
+1265 let mut last_packet_number = None;
+1266
+1267 // Build datagrams until the transmit is full.
+1268 while transmit.num_datagrams() < transmit.max_datagrams().get() {
+1269 let max_packet_size = if transmit.datagram_remaining_mut() > 0 {
+1270 // We are trying to coalesce another packet into this datagram.
+1271 debug_assert!(*coalesce, "coalescing must be set");
+1272 transmit.datagram_remaining_mut()
+1273 } else {
+1274 // A new datagram needs to be started.
+1275 transmit.segment_size()
+1276 };
+1277
+1278 let sendable_frames = sendable_frames
+1279 .unwrap_or_else(|| self.space_can_send(space_id, path_id, max_packet_size, close));
+1280 let path_should_send =
+1281 self.path_should_send(path_id, space_id, have_available_path, &sendable_frames);
+1282
+1283 let send_blocked = if path_should_send && transmit.datagram_remaining_mut() == 0 {
+1284 // Only check congestion control if a new datagram is needed.
+1285 self.path_congestion_check(space_id, path_id, transmit, &sendable_frames, now)
+1286 } else {
+1287 PathBlocked::No
+1288 };
+1289
+1290 // Whether congestion control stopped the next packet from being sent. Further
+1291 // packets could still be built, as e.g. tail-loss probes are not congestion limited.
+1292 let congestion_blocked = if send_blocked != PathBlocked::No {
+1293 trace!(?space_id, %path_id, ?send_blocked, "congestion blocked");
+1294 true
+1295 } else {
+1296 false
+1297 };
+1298 if send_blocked != PathBlocked::No && space_id < SpaceId::Data {
+1299 // Higher spaces might still have tail-loss probes to send, which are not
+1300 // congestion blocked.
+1301 return PollPathSpaceStatus::NothingToSend { congestion_blocked };
+1302 }
+1303 if !path_should_send || send_blocked != PathBlocked::No {
+1304 // Nothing more to send on this path, check the next path if possible.
+1305
+1306 // If there are any datagrams in the transmit, packets for another path can
+1307 // not be built.
+1308 if transmit.num_datagrams() > 0 {
+1309 return PollPathSpaceStatus::Send { last_packet_number };
+1310 }
+1311
+1312 return PollPathSpaceStatus::NothingToSend { congestion_blocked };
+1313 }
+1314
+1315 // If coalescing another packet into the existing datagram, there should
+1316 // still be enough space for a whole packet.
+1317 if *coalesce && transmit.datagram_start_offset() < transmit.len() {
+1318 debug_assert!(transmit.datagram_remaining_mut() >= MIN_PACKET_SPACE);
+1319 }
+1320
+1321 if transmit.datagram_remaining_mut() == 0 {
+1322 // If the datagram is full, start a new datagram.
+1323 // Only needed if we are building a new datagram
+1324 match self.spaces[space_id].for_path(path_id).loss_probes {
+1325 0 => transmit.start_new_datagram(),
+1326 _ => {
+1327 // We need something to send for a tail-loss probe.
+1328 let request_immediate_ack =
+1329 space_id == SpaceId::Data && self.peer_supports_ack_frequency();
+1330 self.spaces[space_id].maybe_queue_probe(
+1331 path_id,
+1332 request_immediate_ack,
+1333 &self.streams,
+1334 );
+1335
+1336 self.spaces[space_id].for_path(path_id).loss_probes -= 1;
+1337
+1338 // Clamp the datagram to at most the minimum MTU to ensure that loss
+1339 // probes can get through and enable recovery even if the path MTU
+1340 // has shrank unexpectedly.
+1341 transmit.start_new_datagram_with_size(std::cmp::min(
+1342 usize::from(INITIAL_MTU),
+1343 transmit.segment_size(),
+1344 ));
+1345 }
+1346 }
+1347 trace!(count = transmit.num_datagrams(), "new datagram started");
+1348 *coalesce = true;
+1349 *pad_datagram = PadDatagram::No;
+1350 }
+1351
+1352 let is_close = sendable_frames.close;
1353
-1354 if self.spaces[SpaceId::Initial].crypto.is_some()
-1355 && space_id == SpaceId::Handshake
-1356 && self.side.is_client()
-1357 {
-1358 // A client stops both sending and processing Initial packets when it
-1359 // sends its first Handshake packet.
-1360 self.discard_space(now, SpaceId::Initial);
-1361 }
-1362 if let Some(ref mut prev) = self.prev_crypto {
-1363 prev.update_unacked = false;
-1364 }
-1365
-1366 let Some(mut builder) = PacketBuilder::new(
-1367 now,
-1368 space_id,
-1369 path_id,
-1370 remote_cid,
+1354 // Send an off-path PATH_RESPONSE. Prioritized over on-path data to ensure that
+1355 // path validation can occur while the link is saturated.
+1356 if !is_close && space_id == SpaceId::Data {
+1357 if let Some(transmit) = self.build_off_path_challenge(
+1358 now,
+1359 path_id,
+1360 space_id,
+1361 remote_cid,
+1362 sendable_frames.other,
+1363 transmit,
+1364 ) {
+1365 return PollPathSpaceStatus::SendTransmit { transmit };
+1366 }
+1367 }
+1368
+1369 let n = self.poll_transmit_path_space_datagram(
+1370 now,
1371 transmit,
-1372 can_send.other,
-1373 self,
-1374 ) else {
-1375 return PollPathSpaceStatus::NothingToSend { congestion_blocked };
-1376 };
-1377 last_packet_number = Some(builder.exact_number);
-1378 *coalesce = *coalesce && !builder.short_header;
-1379
-1380 if space_id == SpaceId::Initial && (self.side.is_client() || can_send.other) {
-1381 // https://www.rfc-editor.org/rfc/rfc9000.html#section-14.1
-1382 *pad_datagram |= PadDatagram::ToMinMtu;
-1383 }
-1384 if space_id == SpaceId::Data && self.config.pad_to_mtu {
-1385 *pad_datagram |= PadDatagram::ToSegmentSize;
+1372 path_id,
+1373 space_id,
+1374 remote_cid,
+1375 have_available_path,
+1376 close,
+1377 sendable_frames,
+1378 coalesce,
+1379 pad_datagram,
+1380 );
+1381 if let Some(n) = n {
+1382 last_packet_number.replace(n);
+1383 } else {
+1384 // error, unable to send
+1385 return PollPathSpaceStatus::NothingToSend { congestion_blocked };
1386 }
1387
-1388 if can_send.close {
-1389 trace!("sending CONNECTION_CLOSE");
-1390 // Encode ACKs before the ConnectionClose message, to give the receiver
-1391 // a better approximate on what data has been processed. This is
-1392 // especially important with ack delay, since the peer might not
-1393 // have gotten any other ACK for the data earlier on.
-1394 let is_multipath_negotiated = self.is_multipath_negotiated();
-1395 for path_id in self.spaces[space_id]
-1396 .number_spaces
-1397 .iter()
-1398 .filter(|(_, pns)| !pns.pending_acks.ranges().is_empty())
-1399 .map(|(&path_id, _)| path_id)
-1400 .collect::<Vec<_>>()
-1401 {
-1402 Self::populate_acks(
-1403 now,
-1404 self.receiving_ecn,
-1405 path_id,
-1406 space_id,
-1407 &mut self.spaces[space_id],
-1408 is_multipath_negotiated,
-1409 &mut builder,
-1410 &mut self.stats.frame_tx,
-1411 );
-1412 }
-1413
-1414 // Since there only 64 ACK frames there will always be enough space
-1415 // to encode the ConnectionClose frame too. However we still have the
-1416 // check here to prevent crashes if something changes.
-1417 debug_assert!(
-1418 builder.frame_space_remaining() > frame::ConnectionClose::SIZE_BOUND,
-1419 "ACKs should leave space for ConnectionClose"
-1420 );
-1421 let stats = &mut self.stats.frame_tx;
-1422 if frame::ConnectionClose::SIZE_BOUND < builder.frame_space_remaining() {
-1423 let max_frame_size = builder.frame_space_remaining();
-1424 let close: Close = match self.state.as_type() {
-1425 StateType::Closed => {
-1426 let reason: Close =
-1427 self.state.as_closed().expect("checked").clone().into();
-1428 if space_id == SpaceId::Data || reason.is_transport_layer() {
-1429 reason
-1430 } else {
-1431 TransportError::APPLICATION_ERROR("").into()
-1432 }
-1433 }
-1434 StateType::Draining => TransportError::NO_ERROR("").into(),
-1435 _ => unreachable!(
-1436 "tried to make a close packet when the connection wasn't closed"
-1437 ),
-1438 };
-1439 builder.write_frame(close.encoder(max_frame_size), stats);
-1440 }
-1441 builder.finish_and_track(now, self, path_id, *pad_datagram);
-1442 if space_id == self.highest_space {
-1443 // Don't send another close packet. Even with multipath we only send
-1444 // CONNECTION_CLOSE on a single path since we expect our paths to work.
-1445 self.close = false;
-1446 // `CONNECTION_CLOSE` is the final packet
-1447 return PollPathSpaceStatus::Send { last_packet_number };
-1448 } else {
-1449 // Send a close frame in every possible space for robustness, per
-1450 // RFC9000 "Immediate Close during the Handshake". Don't bother trying
-1451 // to send anything else.
-1452 return PollPathSpaceStatus::NextSpace;
-1453 }
-1454 }
-1455
-1456 // Send an off-path PATH_RESPONSE. Prioritized over on-path data to ensure that
-1457 // path validation can occur while the link is saturated.
-1458 if space_id == SpaceId::Data && builder.buf.num_datagrams() == 1 {
-1459 let path = self.path_data_mut(path_id);
-1460 if let Some((token, network_path)) =
-1461 path.path_responses.pop_off_path(path.network_path)
-1462 {
-1463 // TODO(flub): We need to use the right CID! We shouldn't use the same
-1464 // CID as the current active one for the path. Though see also
-1465 // https://github.com/quinn-rs/quinn/issues/2184
-1466 //
-1467 let stats = &mut self.stats.frame_tx;
-1468 let frame = frame::PathResponse(token);
-1469 builder.write_frame_with_log_msg(frame, stats, Some("(off-path)"));
-1470 builder.finish_and_track(now, self, path_id, PadDatagram::ToMinMtu);
-1471 self.stats.udp_tx.on_sent(1, transmit.len());
-1472 return PollPathSpaceStatus::SendTransmit {
-1473 transmit: Transmit {
-1474 destination: network_path.remote,
-1475 size: transmit.len(),
-1476 ecn: None,
-1477 segment_size: None,
-1478 src_ip: network_path.local_ip,
-1479 },
-1480 };
-1481 }
-1482 }
-1483
-1484 let path_exclusive_only =
-1485 have_available_path && self.path_data(path_id).local_status() == PathStatus::Backup;
-1486 self.populate_packet(now, space_id, path_id, path_exclusive_only, &mut builder);
-1487
-1488 // ACK-only packets should only be sent when explicitly allowed. If we write them due to
-1489 // any other reason, there is a bug which leads to one component announcing write
-1490 // readiness while not writing any data. This degrades performance. The condition is
-1491 // only checked if the full MTU is available and when potentially large fixed-size
-1492 // frames aren't queued, so that lack of space in the datagram isn't the reason for just
-1493 // writing ACKs.
-1494 debug_assert!(
-1495 !(builder.sent_frames().is_ack_only(&self.streams)
-1496 && !can_send.acks
-1497 && can_send.other
-1498 && builder.buf.segment_size()
-1499 == self.path_data(path_id).current_mtu() as usize
-1500 && self.datagrams.outgoing.is_empty()),
-1501 "SendableFrames was {can_send:?}, but only ACKs have been written"
-1502 );
-1503 if builder.sent_frames().requires_padding {
-1504 *pad_datagram |= PadDatagram::ToMinMtu;
-1505 }
-1506
-1507 for (path_id, _pn) in builder.sent_frames().largest_acked.iter() {
-1508 self.spaces[space_id]
-1509 .for_path(*path_id)
-1510 .pending_acks
-1511 .acks_sent();
-1512 self.timers.stop(
-1513 Timer::PerPath(*path_id, PathTimer::MaxAckDelay),
-1514 self.qlog.with_time(now),
-1515 );
-1516 }
-1517
-1518 // Now we need to finish the packet. Before we do so we need to know if we will
-1519 // be coalescing the next packet into this one, or will be ending the datagram
-1520 // as well. Because if this is the last packet in the datagram more padding
-1521 // might be needed because of the packet type, or to fill the GSO segment size.
-1522
-1523 // Are we allowed to coalesce AND is there enough space for another *packet* in
-1524 // this datagram AND is there another packet to send in this or the next space?
-1525 if *coalesce
-1526 && builder
-1527 .buf
-1528 .datagram_remaining_mut()
-1529 .saturating_sub(builder.predict_packet_end())
-1530 > MIN_PACKET_SPACE
-1531 && self.has_next_send_space(space_id, path_id, builder.buf, close)
-1532 {
-1533 // We can append/coalesce the next packet into the current
-1534 // datagram. Finish the current packet without adding extra padding.
-1535 builder.finish_and_track(now, self, path_id, PadDatagram::No);
-1536 } else {
-1537 // We need a new datagram for the next packet. Finish the current
-1538 // packet with padding.
-1539 if builder.buf.num_datagrams() > 1 && matches!(pad_datagram, PadDatagram::No) {
-1540 // If too many padding bytes would be required to continue the
-1541 // GSO batch after this packet, end the GSO batch here. Ensures
-1542 // that fixed-size frames with heterogeneous sizes
-1543 // (e.g. application datagrams) won't inadvertently waste large
-1544 // amounts of bandwidth. The exact threshold is a bit arbitrary
-1545 // and might benefit from further tuning, though there's no
-1546 // universally optimal value.
-1547 const MAX_PADDING: usize = 32;
-1548 if builder.buf.datagram_remaining_mut()
-1549 > builder.predict_packet_end() + MAX_PADDING
-1550 {
-1551 trace!(
-1552 "GSO truncated by demand for {} padding bytes",
-1553 builder.buf.datagram_remaining_mut() - builder.predict_packet_end()
-1554 );
-1555 builder.finish_and_track(now, self, path_id, PadDatagram::No);
-1556 return PollPathSpaceStatus::Send { last_packet_number };
-1557 }
-1558
-1559 // Pad the current datagram to GSO segment size so it can be
-1560 // included in the GSO batch.
-1561 builder.finish_and_track(now, self, path_id, PadDatagram::ToSegmentSize);
-1562 } else {
-1563 builder.finish_and_track(now, self, path_id, *pad_datagram);
-1564 }
-1565 if transmit.num_datagrams() == 1 {
-1566 transmit.clip_datagram_size();
-1567 }
-1568 }
-1569 }
-1570 }
-1571
-1572 fn on_remote_cids_exhausted(&mut self, now: Instant, path_id: PathId) {
-1573 if self.abandoned_paths.contains(&path_id) {
-1574 trace!(%path_id, "remote CIDs retired for abandoned path");
-1575 return;
-1576 }
-1577
-1578 let error = PathError::RemoteCidsExhausted;
-1579 debug!(?error, %path_id, "no active CID for path");
-1580 self.events
-1581 .push_back(Event::Path(PathEvent::LocallyClosed { id: path_id, error }));
-1582 // Locally we should have refused to open this path, the remote should
-1583 // have given us CIDs for this path before opening it. So we can always
-1584 // abandon this here.
-1585 self.close_path(
-1586 now,
-1587 path_id,
-1588 TransportErrorCode::NO_CID_AVAILABLE_FOR_PATH.into(),
-1589 )
-1590 .ok();
-1591 self.spaces[SpaceId::Data]
-1592 .pending
-1593 .path_cids_blocked
-1594 .insert(path_id);
-1595 }
-1596
-1597 fn poll_transmit_mtu_probe(
-1598 &mut self,
-1599 now: Instant,
-1600 transmit: &mut TransmitBuf<'_>,
-1601 path_id: PathId,
-1602 ) {
-1603 let Some((active_cid, probe_size)) = self.get_mtu_probe_data(now, path_id) else {
-1604 return;
-1605 };
-1606
-1607 // We are definitely sending a DPLPMTUD probe.
-1608 debug_assert_eq!(transmit.num_datagrams(), 0);
-1609 transmit.start_new_datagram_with_size(probe_size as usize);
-1610
-1611 let Some(mut builder) = PacketBuilder::new(
-1612 now,
-1613 SpaceId::Data,
-1614 path_id,
-1615 active_cid,
-1616 transmit,
-1617 true,
-1618 self,
-1619 ) else {
-1620 return;
-1621 };
-1622
-1623 // We implement MTU probes as ping packets padded up to the probe size
-1624 trace!(?probe_size, "writing MTUD probe");
-1625 builder.write_frame(frame::Ping, &mut self.stats.frame_tx);
-1626
-1627 // If supported by the peer, we want no delays to the probe's ACK
-1628 if self.peer_supports_ack_frequency() {
-1629 builder.write_frame(frame::ImmediateAck, &mut self.stats.frame_tx);
-1630 }
-1631
-1632 builder.finish_and_track(now, self, path_id, PadDatagram::ToSize(probe_size));
-1633
-1634 self.path_stats
-1635 .entry(path_id)
-1636 .or_default()
-1637 .sent_plpmtud_probes += 1;
-1638 }
-1639
-1640 fn get_mtu_probe_data(&mut self, now: Instant, path_id: PathId) -> Option<(ConnectionId, u16)> {
-1641 // We MTU probe all paths for which all of the following is true:
-1642 // - We have an active destination CID for the path.
-1643 // - The remote address *and* path are validated.
-1644 // - The path is not abandoned.
-1645 // - The MTU Discovery subsystem wants to probe the path.
-1646 let active_cid = self.rem_cids.get(&path_id).map(CidQueue::active)?;
-1647 let is_eligible = self.path_data(path_id).validated
-1648 && !self.path_data(path_id).is_validating_path()
-1649 && !self.abandoned_paths.contains(&path_id);
-1650
-1651 if !is_eligible {
-1652 return None;
+1388 if is_close {
+1389 if space_id < self.highest_space {
+1390 // Send a close frame in every possible space for robustness, per
+1391 // RFC9000 "Immediate Close during the Handshake". Don't bother trying
+1392 // to send anything else.
+1393 continue;
+1394 } else {
+1395 // Don't send another close packet, we are in the highest space
+1396
+1397 // Even with multipath we only send
+1398 // CONNECTION_CLOSE on a single path since we expect our paths to work.
+1399 self.close = false;
+1400 // `CONNECTION_CLOSE` is the final packet
+1401 break;
+1402 }
+1403 }
+1404
+1405 // Clip the first datagram
+1406 if transmit.num_datagrams() == 1 {
+1407 transmit.clip_datagram_size();
+1408 }
+1409 }
+1410
+1411 if last_packet_number.is_some() {
+1412 // No more datagrams allowed
+1413 PollPathSpaceStatus::Send { last_packet_number }
+1414 } else {
+1415 PollPathSpaceStatus::NothingToSend {
+1416 congestion_blocked: false,
+1417 }
+1418 }
+1419 }
+1420
+1421 /// Tries to build a single datagram
+1422 fn poll_transmit_path_space_datagram(
+1423 &mut self,
+1424 now: Instant,
+1425 transmit: &mut TransmitBuf<'_>,
+1426 path_id: PathId,
+1427 space_id: SpaceId,
+1428 remote_cid: ConnectionId,
+1429 have_available_path: bool,
+1430 close: bool,
+1431 sendable_frames: SendableFrames,
+1432 coalesce: &mut bool,
+1433 pad_datagram: &mut PadDatagram,
+1434 ) -> Option<u64> {
+1435 if self.spaces[SpaceId::Initial].crypto.is_some()
+1436 && space_id == SpaceId::Handshake
+1437 && self.side.is_client()
+1438 {
+1439 // A client stops both sending and processing Initial packets when it
+1440 // sends its first Handshake packet.
+1441 self.discard_space(now, SpaceId::Initial);
+1442 }
+1443 if let Some(ref mut prev) = self.prev_crypto {
+1444 prev.update_unacked = false;
+1445 }
+1446
+1447 let mut builder = PacketBuilder::new(
+1448 now,
+1449 space_id,
+1450 path_id,
+1451 remote_cid,
+1452 transmit,
+1453 sendable_frames.other,
+1454 self,
+1455 )?;
+1456 *coalesce = *coalesce && !builder.short_header;
+1457
+1458 if space_id == SpaceId::Initial && (self.side.is_client() || sendable_frames.other) {
+1459 // https://www.rfc-editor.org/rfc/rfc9000.html#section-14.1
+1460 *pad_datagram |= PadDatagram::ToMinMtu;
+1461 }
+1462 if space_id == SpaceId::Data && self.config.pad_to_mtu {
+1463 *pad_datagram |= PadDatagram::ToSegmentSize;
+1464 }
+1465
+1466 if sendable_frames.close {
+1467 trace!("sending CONNECTION_CLOSE");
+1468 // Encode ACKs before the ConnectionClose message, to give the receiver
+1469 // a better approximate on what data has been processed. This is
+1470 // especially important with ack delay, since the peer might not
+1471 // have gotten any other ACK for the data earlier on.
+1472 let is_multipath_negotiated = self.is_multipath_negotiated();
+1473 for path_id in self.spaces[space_id]
+1474 .number_spaces
+1475 .iter()
+1476 .filter(|(_, pns)| !pns.pending_acks.ranges().is_empty())
+1477 .map(|(&path_id, _)| path_id)
+1478 .collect::<Vec<_>>()
+1479 {
+1480 Self::populate_acks(
+1481 now,
+1482 self.receiving_ecn,
+1483 path_id,
+1484 space_id,
+1485 &mut self.spaces[space_id],
+1486 is_multipath_negotiated,
+1487 &mut builder,
+1488 &mut self.stats.frame_tx,
+1489 );
+1490 }
+1491
+1492 // Since there only 64 ACK frames there will always be enough space
+1493 // to encode the ConnectionClose frame too. However we still have the
+1494 // check here to prevent crashes if something changes.
+1495 debug_assert!(
+1496 builder.frame_space_remaining() > frame::ConnectionClose::SIZE_BOUND,
+1497 "ACKs should leave space for ConnectionClose"
+1498 );
+1499 let stats = &mut self.stats.frame_tx;
+1500 if frame::ConnectionClose::SIZE_BOUND < builder.frame_space_remaining() {
+1501 let max_frame_size = builder.frame_space_remaining();
+1502 let close: Close = match self.state.as_type() {
+1503 StateType::Closed => {
+1504 let reason: Close = self.state.as_closed().expect("checked").clone().into();
+1505 if space_id == SpaceId::Data || reason.is_transport_layer() {
+1506 reason
+1507 } else {
+1508 TransportError::APPLICATION_ERROR("").into()
+1509 }
+1510 }
+1511 StateType::Draining => TransportError::NO_ERROR("").into(),
+1512 _ => unreachable!(
+1513 "tried to make a close packet when the connection wasn't closed"
+1514 ),
+1515 };
+1516 builder.write_frame(close.encoder(max_frame_size), stats);
+1517 }
+1518 return Some(builder.finish_and_track(now, self, path_id, *pad_datagram));
+1519 }
+1520
+1521 let path_exclusive_only =
+1522 have_available_path && self.path_data(path_id).local_status() == PathStatus::Backup;
+1523 self.populate_packet(now, space_id, path_id, path_exclusive_only, &mut builder);
+1524
+1525 // ACK-only packets should only be sent when explicitly allowed. If we write them due to
+1526 // any other reason, there is a bug which leads to one component announcing write
+1527 // readiness while not writing any data. This degrades performance. The condition is
+1528 // only checked if the full MTU is available and when potentially large fixed-size
+1529 // frames aren't queued, so that lack of space in the datagram isn't the reason for just
+1530 // writing ACKs.
+1531 debug_assert!(
+1532 !(builder.sent_frames().is_ack_only(&self.streams)
+1533 && !sendable_frames.acks
+1534 && sendable_frames.other
+1535 && builder.buf.segment_size() == self.path_data(path_id).current_mtu() as usize
+1536 && self.datagrams.outgoing.is_empty()),
+1537 "SendableFrames was {sendable_frames:?}, but only ACKs have been written"
+1538 );
+1539 if builder.sent_frames().requires_padding {
+1540 *pad_datagram |= PadDatagram::ToMinMtu;
+1541 }
+1542
+1543 for (path_id, _pn) in builder.sent_frames().largest_acked.iter() {
+1544 self.spaces[space_id]
+1545 .for_path(*path_id)
+1546 .pending_acks
+1547 .acks_sent();
+1548 self.timers.stop(
+1549 Timer::PerPath(*path_id, PathTimer::MaxAckDelay),
+1550 self.qlog.with_time(now),
+1551 );
+1552 }
+1553
+1554 // Now we need to finish the packet. Before we do so we need to know if we will
+1555 // be coalescing the next packet into this one, or will be ending the datagram
+1556 // as well. Because if this is the last packet in the datagram more padding
+1557 // might be needed because of the packet type, or to fill the GSO segment size.
+1558 let padding =
+1559 self.calculate_padding(path_id, space_id, *coalesce, close, *pad_datagram, &builder);
+1560
+1561 Some(builder.finish_and_track(now, self, path_id, padding))
+1562 }
+1563
+1564 fn build_off_path_challenge(
+1565 &mut self,
+1566 now: Instant,
+1567 path_id: PathId,
+1568 space_id: SpaceId,
+1569 remote_cid: ConnectionId,
+1570 ack_eliciting: bool,
+1571 transmit: &mut TransmitBuf<'_>,
+1572 ) -> Option<Transmit> {
+1573 let mut builder = PacketBuilder::new(
+1574 now,
+1575 space_id,
+1576 path_id,
+1577 remote_cid,
+1578 transmit,
+1579 ack_eliciting,
+1580 self,
+1581 )?;
+1582 let path = self.path_data_mut(path_id);
+1583 if let Some((token, network_path)) = path.path_responses.pop_off_path(path.network_path) {
+1584 // TODO(flub): We need to use the right CID! We shouldn't use the same
+1585 // CID as the current active one for the path. Though see also
+1586 // https://github.com/quinn-rs/quinn/issues/2184
+1587 //
+1588 let stats = &mut self.stats.frame_tx;
+1589 let frame = frame::PathResponse(token);
+1590 builder.write_frame_with_log_msg(frame, stats, Some("(off-path)"));
+1591 builder.finish_and_track(now, self, path_id, PadDatagram::ToMinMtu);
+1592 self.stats.udp_tx.on_sent(1, transmit.len());
+1593 return Some(Transmit {
+1594 destination: network_path.remote,
+1595 size: transmit.len(),
+1596 ecn: None,
+1597 segment_size: None,
+1598 src_ip: network_path.local_ip,
+1599 });
+1600 }
+1601 None
+1602 }
+1603
+1604 fn calculate_padding(
+1605 &mut self,
+1606 path_id: PathId,
+1607 space_id: SpaceId,
+1608 coalesce: bool,
+1609 close: bool,
+1610 pad_datagram: PadDatagram,
+1611 builder: &PacketBuilder<'_, '_>,
+1612 ) -> PadDatagram {
+1613 // Are we allowed to coalesce AND is there enough space for another *packet* in
+1614 // this datagram AND is there another packet to send in this or the next space?
+1615 if coalesce
+1616 && builder
+1617 .buf
+1618 .datagram_remaining_mut()
+1619 .saturating_sub(builder.predict_packet_end())
+1620 > MIN_PACKET_SPACE
+1621 && self.has_next_send_space(space_id, path_id, builder.buf, close)
+1622 {
+1623 // We can append/coalesce the next packet into the current
+1624 // datagram. Finish the current packet without adding extra padding.
+1625 PadDatagram::No
+1626 } else {
+1627 // We need a new datagram for the next packet. Finish the current
+1628 // packet with padding.
+1629 if builder.buf.num_datagrams() > 1 && matches!(pad_datagram, PadDatagram::No) {
+1630 // If too many padding bytes would be required to continue the
+1631 // GSO batch after this packet, end the GSO batch here. Ensures
+1632 // that fixed-size frames with heterogeneous sizes
+1633 // (e.g. application datagrams) won't inadvertently waste large
+1634 // amounts of bandwidth. The exact threshold is a bit arbitrary
+1635 // and might benefit from further tuning, though there's no
+1636 // universally optimal value.
+1637 const MAX_PADDING: usize = 32;
+1638 if builder.buf.datagram_remaining_mut() > builder.predict_packet_end() + MAX_PADDING
+1639 {
+1640 trace!(
+1641 "GSO truncated by demand for {} padding bytes",
+1642 builder.buf.datagram_remaining_mut() - builder.predict_packet_end()
+1643 );
+1644 PadDatagram::No
+1645 } else {
+1646 // Pad the current datagram to GSO segment size so it can be
+1647 // included in the GSO batch.
+1648 PadDatagram::ToSegmentSize
+1649 }
+1650 } else {
+1651 pad_datagram
+1652 }
1653 }
-1654 let next_pn = self.spaces[SpaceId::Data]
-1655 .for_path(path_id)
-1656 .peek_tx_number();
-1657 let probe_size = self
-1658 .path_data_mut(path_id)
-1659 .mtud
-1660 .poll_transmit(now, next_pn)?;
-1661
-1662 Some((active_cid, probe_size))
-1663 }
-1664
-1665 /// Returns if there is anext packet space which has data to send
-1666 ///
-1667 /// This takes into account the space available to frames in the next datagram.
-1668 // TODO(flub): This duplication is not nice.
-1669 fn has_next_send_space(
-1670 &mut self,
-1671 current_space_id: SpaceId,
-1672 path_id: PathId,
-1673 buf: &TransmitBuf<'_>,
-1674 close: bool,
-1675 ) -> bool {
-1676 // Number of bytes available for frames if this is a 1-RTT packet. We're guaranteed
-1677 // to be able to send an individual frame at least this large in the next 1-RTT
-1678 // packet. This could be generalized to support every space, but it's only needed to
-1679 // handle large fixed-size frames, which only exist in 1-RTT (application
-1680 // datagrams). We don't account for coalesced packets potentially occupying space
-1681 // because frames can always spill into the next datagram.
-1682 let mut space_id = current_space_id;
-1683 loop {
-1684 let can_send = self.space_can_send(space_id, path_id, buf.segment_size(), close);
-1685 if !can_send.is_empty() || (close && self.spaces[space_id].crypto.is_some()) {
-1686 return true;
-1687 }
-1688 let Some(next_space) = space_id.next() else {
-1689 break;
-1690 };
-1691 space_id = next_space;
-1692 }
-1693 false
-1694 }
-1695
-1696 /// Checks if creating a new datagram would be blocked by congestion control
-1697 fn path_congestion_check(
-1698 &mut self,
-1699 space_id: SpaceId,
-1700 path_id: PathId,
-1701 transmit: &TransmitBuf<'_>,
-1702 can_send: &SendableFrames,
-1703 now: Instant,
-1704 ) -> PathBlocked {
-1705 // Anti-amplification is only based on `total_sent`, which gets updated after
-1706 // the transmit is sent. Therefore we pass the amount of bytes for datagrams
-1707 // that are already created, as well as 1 byte for starting another datagram. If
-1708 // there is any anti-amplification budget left, we always allow a full MTU to be
-1709 // sent (see https://github.com/quinn-rs/quinn/issues/1082).
-1710 if self.side().is_server()
-1711 && self
-1712 .path_data(path_id)
-1713 .anti_amplification_blocked(transmit.len() as u64 + 1)
-1714 {
-1715 trace!(?space_id, %path_id, "blocked by anti-amplification");
-1716 return PathBlocked::AntiAmplification;
-1717 }
-1718
-1719 // Congestion control check.
-1720 // Tail loss probes must not be blocked by congestion, or a deadlock could arise.
-1721 let bytes_to_send = transmit.segment_size() as u64;
-1722 let need_loss_probe = self.spaces[space_id].for_path(path_id).loss_probes > 0;
-1723
-1724 if can_send.other && !need_loss_probe && !can_send.close {
-1725 let path = self.path_data(path_id);
-1726 if path.in_flight.bytes + bytes_to_send >= path.congestion.window() {
-1727 trace!(?space_id, %path_id, "blocked by congestion control");
-1728 return PathBlocked::Congestion;
-1729 }
-1730 }
-1731
-1732 // Pacing check.
-1733 if let Some(delay) = self.path_data_mut(path_id).pacing_delay(bytes_to_send, now) {
-1734 self.timers.set(
-1735 Timer::PerPath(path_id, PathTimer::Pacing),
-1736 delay,
-1737 self.qlog.with_time(now),
-1738 );
-1739 // Loss probes and CONNECTION_CLOSE should be subject to pacing, even though
-1740 // they are not congestion controlled.
-1741 trace!(?space_id, %path_id, "blocked by pacing");
-1742 return PathBlocked::Pacing;
-1743 }
-1744
-1745 PathBlocked::No
-1746 }
-1747
-1748 /// Send PATH_CHALLENGE for a previous path if necessary
-1749 ///
-1750 /// QUIC-TRANSPORT section 9.3.3
-1751 /// <https://www.rfc-editor.org/rfc/rfc9000.html#name-off-path-packet-forwarding>
-1752 fn send_prev_path_challenge(
-1753 &mut self,
-1754 now: Instant,
-1755 buf: &mut TransmitBuf<'_>,
-1756 path_id: PathId,
-1757 ) -> Option<Transmit> {
-1758 let (prev_cid, prev_path) = self.paths.get_mut(&path_id)?.prev.as_mut()?;
-1759 // TODO (matheus23): We could use !prev_path.is_validating() here instead to
-1760 // (possibly) also re-send challenges when they get lost.
-1761 if !prev_path.send_new_challenge {
-1762 return None;
-1763 };
-1764 prev_path.send_new_challenge = false;
-1765 let network_path = prev_path.network_path;
-1766 let token = self.rng.random();
-1767 let info = paths::SentChallengeInfo {
-1768 sent_instant: now,
-1769 network_path,
-1770 };
-1771 prev_path.challenges_sent.insert(token, info);
-1772 debug_assert_eq!(
-1773 self.highest_space,
-1774 SpaceId::Data,
-1775 "PATH_CHALLENGE queued without 1-RTT keys"
-1776 );
-1777 buf.start_new_datagram_with_size(MIN_INITIAL_SIZE as usize);
-1778
-1779 // Use the previous CID to avoid linking the new path with the previous path. We
-1780 // don't bother accounting for possible retirement of that prev_cid because this is
-1781 // sent once, immediately after migration, when the CID is known to be valid. Even
-1782 // if a post-migration packet caused the CID to be retired, it's fair to pretend
-1783 // this is sent first.
-1784 debug_assert_eq!(buf.datagram_start_offset(), 0);
-1785 let mut builder =
-1786 PacketBuilder::new(now, SpaceId::Data, path_id, *prev_cid, buf, false, self)?;
-1787 let challenge = frame::PathChallenge(token);
-1788 let stats = &mut self.stats.frame_tx;
-1789 builder.write_frame_with_log_msg(challenge, stats, Some("validating previous path"));
-1790
-1791 // An endpoint MUST expand datagrams that contain a PATH_CHALLENGE frame
-1792 // to at least the smallest allowed maximum datagram size of 1200 bytes,
-1793 // unless the anti-amplification limit for the path does not permit
-1794 // sending a datagram of this size
-1795 builder.pad_to(MIN_INITIAL_SIZE);
-1796
-1797 builder.finish(self, now);
-1798 self.stats.udp_tx.on_sent(1, buf.len());
-1799
-1800 Some(Transmit {
-1801 destination: network_path.remote,
-1802 size: buf.len(),
-1803 ecn: None,
-1804 segment_size: None,
-1805 src_ip: network_path.local_ip,
-1806 })
-1807 }
-1808
-1809 /// Indicate what types of frames are ready to send for the given space
-1810 ///
-1811 /// *packet_size* is the number of bytes available to build the next packet. *close*
-1812 /// *indicates whether a CONNECTION_CLOSE frame needs to be sent.
-1813 fn space_can_send(
-1814 &mut self,
-1815 space_id: SpaceId,
-1816 path_id: PathId,
-1817 packet_size: usize,
-1818 close: bool,
-1819 ) -> SendableFrames {
-1820 let pn = self.spaces[SpaceId::Data]
-1821 .for_path(path_id)
-1822 .peek_tx_number();
-1823 let frame_space_1rtt = packet_size.saturating_sub(self.predict_1rtt_overhead(pn, path_id));
-1824 if self.spaces[space_id].crypto.is_none()
-1825 && (space_id != SpaceId::Data
-1826 || self.zero_rtt_crypto.is_none()
-1827 || self.side.is_server())
-1828 {
-1829 // No keys available for this space
-1830 return SendableFrames::empty();
-1831 }
-1832 let mut can_send = self.spaces[space_id].can_send(path_id, &self.streams);
-1833 if space_id == SpaceId::Data {
-1834 can_send |= self.can_send_1rtt(path_id, frame_space_1rtt);
-1835 }
-1836
-1837 can_send.close = close && self.spaces[space_id].crypto.is_some();
-1838
-1839 can_send
-1840 }
-1841
-1842 /// Process `ConnectionEvent`s generated by the associated `Endpoint`
-1843 ///
-1844 /// Will execute protocol logic upon receipt of a connection event, in turn preparing signals
-1845 /// (including application `Event`s, `EndpointEvent`s and outgoing datagrams) that should be
-1846 /// extracted through the relevant methods.
-1847 pub fn handle_event(&mut self, event: ConnectionEvent) {
-1848 use ConnectionEventInner::*;
-1849 match event.0 {
-1850 Datagram(DatagramConnectionEvent {
-1851 now,
-1852 network_path,
-1853 path_id,
-1854 ecn,
-1855 first_decode,
-1856 remaining,
-1857 }) => {
-1858 let span = trace_span!("pkt", %path_id);
-1859 let _guard = span.enter();
-1860
-1861 if self.update_network_path_or_discard(network_path, path_id) {
-1862 // A return value of true indicates we should discard this packet.
-1863 return;
-1864 }
-1865
-1866 let was_anti_amplification_blocked = self
-1867 .path(path_id)
-1868 .map(|path| path.anti_amplification_blocked(1))
-1869 .unwrap_or(true); // if we don't know about this path it's eagerly considered as unvalidated
-1870 // TODO(@divma): revisit this
-1871
-1872 self.stats.udp_rx.datagrams += 1;
-1873 self.stats.udp_rx.bytes += first_decode.len() as u64;
-1874 let data_len = first_decode.len();
-1875
-1876 self.handle_decode(now, network_path, path_id, ecn, first_decode);
-1877 // The current `path` might have changed inside `handle_decode` since the packet
-1878 // could have triggered a migration. The packet might also belong to an unknown
-1879 // path and have been rejected. Make sure the data received is accounted for the
-1880 // most recent path by accessing `path` after `handle_decode`.
-1881 if let Some(path) = self.path_mut(path_id) {
-1882 path.inc_total_recvd(data_len as u64);
-1883 }
-1884
-1885 if let Some(data) = remaining {
-1886 self.stats.udp_rx.bytes += data.len() as u64;
-1887 self.handle_coalesced(now, network_path, path_id, ecn, data);
-1888 }
-1889
-1890 if let Some(path) = self.paths.get_mut(&path_id) {
-1891 self.qlog
-1892 .emit_recovery_metrics(path_id, &mut path.data, now);
-1893 }
-1894
-1895 if was_anti_amplification_blocked {
-1896 // A prior attempt to set the loss detection timer may have failed due to
-1897 // anti-amplification, so ensure it's set now. Prevents a handshake deadlock if
-1898 // the server's first flight is lost.
-1899 self.set_loss_detection_timer(now, path_id);
-1900 }
-1901 }
-1902 NewIdentifiers(ids, now, cid_len, cid_lifetime) => {
-1903 let path_id = ids.first().map(|issued| issued.path_id).unwrap_or_default();
-1904 debug_assert!(ids.iter().all(|issued| issued.path_id == path_id));
-1905 let cid_state = self
-1906 .local_cid_state
-1907 .entry(path_id)
-1908 .or_insert_with(|| CidState::new(cid_len, cid_lifetime, now, 0));
-1909 cid_state.new_cids(&ids, now);
-1910
-1911 ids.into_iter().rev().for_each(|frame| {
-1912 self.spaces[SpaceId::Data].pending.new_cids.push(frame);
-1913 });
-1914 // Always update Timer::PushNewCid
-1915 self.reset_cid_retirement(now);
-1916 }
-1917 }
-1918 }
-1919
-1920 /// Updates the network path for `path_id`.
-1921 ///
-1922 /// Returns true if a packet coming in for this `path_id` over given `network_path` should be discarded.
-1923 /// Returns false if the path was updated and the packet doesn't need to be discarded.
-1924 fn update_network_path_or_discard(&mut self, network_path: FourTuple, path_id: PathId) -> bool {
-1925 let remote_may_migrate = self.side.remote_may_migrate(&self.state);
-1926 let local_ip_may_migrate = self.side.is_client();
-1927 // If this packet could initiate a migration and we're a client or a server that
-1928 // forbids migration, drop the datagram. This could be relaxed to heuristically
-1929 // permit NAT-rebinding-like migration.
-1930 if let Some(known_path) = self.path_mut(path_id) {
-1931 if network_path.remote != known_path.network_path.remote && !remote_may_migrate {
-1932 trace!(
-1933 %path_id,
-1934 %network_path,
-1935 %known_path.network_path,
-1936 "discarding packet from unrecognized peer"
-1937 );
-1938 return true;
-1939 }
-1940
-1941 if known_path.network_path.local_ip.is_some()
-1942 && network_path.local_ip.is_some()
-1943 && known_path.network_path.local_ip != network_path.local_ip
-1944 && !local_ip_may_migrate
-1945 {
-1946 trace!(
-1947 %path_id,
-1948 %network_path,
-1949 %known_path.network_path,
-1950 "discarding packet sent to incorrect interface"
-1951 );
-1952 return true;
-1953 }
-1954 // If the datagram indicates that we've changed our local IP, we update it.
-1955 // This is alluded to in Section 5.2 of the Multipath RFC draft 18:
-1956 // https://www.ietf.org/archive/id/draft-ietf-quic-multipath-18.html#name-using-multiple-paths-on-the
-1957 // > Client receives the packet, recognizes a path migration, updates the source address of path 2 to 192.0.2.1.
-1958 if let Some(local_ip) = network_path.local_ip {
-1959 if known_path
-1960 .network_path
-1961 .local_ip
-1962 .is_some_and(|ip| ip != local_ip)
-1963 {
-1964 debug!(
-1965 %path_id,
-1966 %network_path,
-1967 %known_path.network_path,
-1968 "path's local address seemingly migrated"
-1969 );
-1970 }
-1971 // We update the address without path validation on the client side.
-1972 // https://www.ietf.org/archive/id/draft-ietf-quic-multipath-18.html#section-5.1
-1973 // > Servers observing a 4-tuple change will perform path validation (see Section 9 of [QUIC-TRANSPORT]).
-1974 // This sounds like it's *only* the server endpoints that do this.
-1975 // TODO(matheus23): We should still consider doing a proper migration on the client side in the future.
-1976 // For now, this preserves the behavior of this code pre 4-tuple tracking.
-1977 known_path.network_path.local_ip = Some(local_ip);
-1978 }
-1979 }
-1980 false
-1981 }
-1982
-1983 /// Process timer expirations
-1984 ///
-1985 /// Executes protocol logic, potentially preparing signals (including application `Event`s,
-1986 /// `EndpointEvent`s and outgoing datagrams) that should be extracted through the relevant
-1987 /// methods.
-1988 ///
-1989 /// It is most efficient to call this immediately after the system clock reaches the latest
-1990 /// `Instant` that was output by `poll_timeout`; however spurious extra calls will simply
-1991 /// no-op and therefore are safe.
-1992 pub fn handle_timeout(&mut self, now: Instant) {
-1993 while let Some((timer, _time)) = self.timers.expire_before(now, &self.qlog) {
-1994 // TODO(@divma): remove `at` when the unicorn is born
-1995 trace!(?timer, at=?now, "timeout");
-1996 match timer {
-1997 Timer::Conn(timer) => match timer {
-1998 ConnTimer::Close => {
-1999 self.state.move_to_drained(None);
-2000 self.endpoint_events.push_back(EndpointEventInner::Drained);
-2001 }
-2002 ConnTimer::Idle => {
-2003 self.kill(ConnectionError::TimedOut);
-2004 }
-2005 ConnTimer::KeepAlive => {
-2006 trace!("sending keep-alive");
-2007 self.ping();
-2008 }
-2009 ConnTimer::KeyDiscard => {
-2010 self.zero_rtt_crypto = None;
-2011 self.prev_crypto = None;
-2012 }
-2013 ConnTimer::PushNewCid => {
-2014 while let Some((path_id, when)) = self.next_cid_retirement() {
-2015 if when > now {
-2016 break;
-2017 }
-2018 match self.local_cid_state.get_mut(&path_id) {
-2019 None => error!(%path_id, "No local CID state for path"),
-2020 Some(cid_state) => {
-2021 // Update `retire_prior_to` field in NEW_CONNECTION_ID frame
-2022 let num_new_cid = cid_state.on_cid_timeout().into();
-2023 if !self.state.is_closed() {
-2024 trace!(
-2025 "push a new CID to peer RETIRE_PRIOR_TO field {}",
-2026 cid_state.retire_prior_to()
-2027 );
-2028 self.endpoint_events.push_back(
-2029 EndpointEventInner::NeedIdentifiers(
-2030 path_id,
-2031 now,
-2032 num_new_cid,
-2033 ),
-2034 );
-2035 }
-2036 }
-2037 }
-2038 }
-2039 }
-2040 },
-2041 // TODO: add path_id as span somehow
-2042 Timer::PerPath(path_id, timer) => {
-2043 let span = trace_span!("per-path timer fired", %path_id, ?timer);
-2044 let _guard = span.enter();
-2045 match timer {
-2046 PathTimer::PathIdle => {
-2047 self.close_path(now, path_id, TransportErrorCode::NO_ERROR.into())
-2048 .ok();
-2049 }
-2050
-2051 PathTimer::PathKeepAlive => {
-2052 trace!("sending keep-alive on path");
-2053 self.ping_path(path_id).ok();
-2054 }
-2055 PathTimer::LossDetection => {
-2056 self.on_loss_detection_timeout(now, path_id);
-2057 self.qlog.emit_recovery_metrics(
-2058 path_id,
-2059 &mut self.paths.get_mut(&path_id).unwrap().data,
-2060 now,
-2061 );
-2062 }
-2063 PathTimer::PathValidation => {
-2064 let Some(path) = self.paths.get_mut(&path_id) else {
-2065 continue;
-2066 };
-2067 self.timers.stop(
-2068 Timer::PerPath(path_id, PathTimer::PathChallengeLost),
-2069 self.qlog.with_time(now),
-2070 );
-2071 debug!("path validation failed");
-2072 if let Some((_, prev)) = path.prev.take() {
-2073 path.data = prev;
-2074 }
-2075 path.data.challenges_sent.clear();
-2076 path.data.send_new_challenge = false;
-2077 }
-2078 PathTimer::PathChallengeLost => {
-2079 let Some(path) = self.paths.get_mut(&path_id) else {
-2080 continue;
-2081 };
-2082 trace!("path challenge deemed lost");
-2083 path.data.send_new_challenge = true;
-2084 }
-2085 PathTimer::PathOpen => {
-2086 let Some(path) = self.paths.get_mut(&path_id) else {
-2087 continue;
-2088 };
-2089 path.data.challenges_sent.clear();
-2090 path.data.send_new_challenge = false;
-2091 self.timers.stop(
-2092 Timer::PerPath(path_id, PathTimer::PathChallengeLost),
-2093 self.qlog.with_time(now),
-2094 );
-2095 debug!("new path validation failed");
-2096 if let Err(err) = self.close_path(
-2097 now,
-2098 path_id,
-2099 TransportErrorCode::PATH_UNSTABLE_OR_POOR.into(),
-2100 ) {
-2101 warn!(?err, "failed closing path");
-2102 }
-2103
-2104 self.events.push_back(Event::Path(PathEvent::LocallyClosed {
-2105 id: path_id,
-2106 error: PathError::ValidationFailed,
-2107 }));
-2108 }
-2109 PathTimer::Pacing => trace!("pacing timer expired"),
-2110 PathTimer::MaxAckDelay => {
-2111 trace!("max ack delay reached");
-2112 // This timer is only armed in the Data space
-2113 self.spaces[SpaceId::Data]
-2114 .for_path(path_id)
-2115 .pending_acks
-2116 .on_max_ack_delay_timeout()
-2117 }
-2118 PathTimer::DiscardPath => {
-2119 // The path was abandoned and 3*PTO has expired since. Clean up all
-2120 // remaining state and install stateless reset token.
-2121 self.timers.stop_per_path(path_id, self.qlog.with_time(now));
-2122 if let Some(loc_cid_state) = self.local_cid_state.remove(&path_id) {
-2123 let (min_seq, max_seq) = loc_cid_state.active_seq();
-2124 for seq in min_seq..=max_seq {
-2125 self.endpoint_events.push_back(
-2126 EndpointEventInner::RetireConnectionId(
-2127 now, path_id, seq, false,
-2128 ),
-2129 );
-2130 }
-2131 }
-2132 self.discard_path(path_id, now);
-2133 }
-2134 }
-2135 }
-2136 }
-2137 }
-2138 }
-2139
-2140 /// Close a connection immediately
-2141 ///
-2142 /// This does not ensure delivery of outstanding data. It is the application's responsibility to
-2143 /// call this only when all important communications have been completed, e.g. by calling
-2144 /// [`SendStream::finish`] on outstanding streams and waiting for the corresponding
-2145 /// [`StreamEvent::Finished`] event.
-2146 ///
-2147 /// If [`Streams::send_streams`] returns 0, all outstanding stream data has been
-2148 /// delivered. There may still be data from the peer that has not been received.
-2149 ///
-2150 /// [`StreamEvent::Finished`]: crate::StreamEvent::Finished
-2151 pub fn close(&mut self, now: Instant, error_code: VarInt, reason: Bytes) {
-2152 self.close_inner(
-2153 now,
-2154 Close::Application(frame::ApplicationClose { error_code, reason }),
-2155 )
-2156 }
-2157
-2158 fn close_inner(&mut self, now: Instant, reason: Close) {
-2159 let was_closed = self.state.is_closed();
-2160 if !was_closed {
-2161 self.close_common();
-2162 self.set_close_timer(now);
-2163 self.close = true;
-2164 self.state.move_to_closed_local(reason);
-2165 }
-2166 }
-2167
-2168 /// Control datagrams
-2169 pub fn datagrams(&mut self) -> Datagrams<'_> {
-2170 Datagrams { conn: self }
-2171 }
-2172
-2173 /// Returns connection statistics
-2174 pub fn stats(&mut self) -> ConnectionStats {
-2175 self.stats.clone()
-2176 }
-2177
-2178 /// Returns path statistics
-2179 pub fn path_stats(&mut self, path_id: PathId) -> Option<PathStats> {
-2180 let path = self.paths.get(&path_id)?;
-2181 let stats = self.path_stats.entry(path_id).or_default();
-2182 stats.rtt = path.data.rtt.get();
-2183 stats.cwnd = path.data.congestion.window();
-2184 stats.current_mtu = path.data.mtud.current_mtu();
-2185 Some(*stats)
-2186 }
-2187
-2188 /// Ping the remote endpoint
-2189 ///
-2190 /// Causes an ACK-eliciting packet to be transmitted on the connection.
-2191 pub fn ping(&mut self) {
-2192 // TODO(flub): This is very brute-force: it pings *all* the paths. Instead it would
-2193 // be nice if we could only send a single packet for this.
-2194 for path_data in self.spaces[self.highest_space].number_spaces.values_mut() {
-2195 path_data.ping_pending = true;
-2196 }
-2197 }
-2198
-2199 /// Ping the remote endpoint over a specific path
-2200 ///
-2201 /// Causes an ACK-eliciting packet to be transmitted on the path.
-2202 pub fn ping_path(&mut self, path: PathId) -> Result<(), ClosedPath> {
-2203 let path_data = self.spaces[self.highest_space]
-2204 .number_spaces
-2205 .get_mut(&path)
-2206 .ok_or(ClosedPath { _private: () })?;
-2207 path_data.ping_pending = true;
-2208 Ok(())
-2209 }
-2210
-2211 /// Update traffic keys spontaneously
-2212 ///
-2213 /// This can be useful for testing key updates, as they otherwise only happen infrequently.
-2214 pub fn force_key_update(&mut self) {
-2215 if !self.state.is_established() {
-2216 debug!("ignoring forced key update in illegal state");
-2217 return;
-2218 }
-2219 if self.prev_crypto.is_some() {
-2220 // We already just updated, or are currently updating, the keys. Concurrent key updates
-2221 // are illegal.
-2222 debug!("ignoring redundant forced key update");
-2223 return;
-2224 }
-2225 self.update_keys(None, false);
-2226 }
-2227
-2228 /// Get a session reference
-2229 pub fn crypto_session(&self) -> &dyn crypto::Session {
-2230 &*self.crypto
-2231 }
-2232
-2233 /// Whether the connection is in the process of being established
-2234 ///
-2235 /// If this returns `false`, the connection may be either established or closed, signaled by the
-2236 /// emission of a `Connected` or `ConnectionLost` message respectively.
-2237 pub fn is_handshaking(&self) -> bool {
-2238 self.state.is_handshake()
-2239 }
-2240
-2241 /// Whether the connection is closed
-2242 ///
-2243 /// Closed connections cannot transport any further data. A connection becomes closed when
-2244 /// either peer application intentionally closes it, or when either transport layer detects an
-2245 /// error such as a time-out or certificate validation failure.
-2246 ///
-2247 /// A `ConnectionLost` event is emitted with details when the connection becomes closed.
-2248 pub fn is_closed(&self) -> bool {
-2249 self.state.is_closed()
-2250 }
-2251
-2252 /// Whether there is no longer any need to keep the connection around
-2253 ///
-2254 /// Closed connections become drained after a brief timeout to absorb any remaining in-flight
-2255 /// packets from the peer. All drained connections have been closed.
-2256 pub fn is_drained(&self) -> bool {
-2257 self.state.is_drained()
-2258 }
-2259
-2260 /// For clients, if the peer accepted the 0-RTT data packets
-2261 ///
-2262 /// The value is meaningless until after the handshake completes.
-2263 pub fn accepted_0rtt(&self) -> bool {
-2264 self.accepted_0rtt
-2265 }
-2266
-2267 /// Whether 0-RTT is/was possible during the handshake
-2268 pub fn has_0rtt(&self) -> bool {
-2269 self.zero_rtt_enabled
-2270 }
-2271
-2272 /// Whether there are any pending retransmits
-2273 pub fn has_pending_retransmits(&self) -> bool {
-2274 !self.spaces[SpaceId::Data].pending.is_empty(&self.streams)
-2275 }
-2276
-2277 /// Look up whether we're the client or server of this Connection
-2278 pub fn side(&self) -> Side {
-2279 self.side.side()
-2280 }
-2281
-2282 /// Get the address observed by the remote over the given path
-2283 pub fn path_observed_address(&self, path_id: PathId) -> Result<Option<SocketAddr>, ClosedPath> {
-2284 self.path(path_id)
-2285 .map(|path_data| {
-2286 path_data
-2287 .last_observed_addr_report
-2288 .as_ref()
-2289 .map(|observed| observed.socket_addr())
-2290 })
-2291 .ok_or(ClosedPath { _private: () })
-2292 }
-2293
-2294 /// Current best estimate of this connection's latency (round-trip-time)
-2295 pub fn rtt(&self, path_id: PathId) -> Option<Duration> {
-2296 self.path(path_id).map(|d| d.rtt.get())
-2297 }
-2298
-2299 /// Current state of this connection's congestion controller, for debugging purposes
-2300 pub fn congestion_state(&self, path_id: PathId) -> Option<&dyn Controller> {
-2301 self.path(path_id).map(|d| d.congestion.as_ref())
-2302 }
-2303
-2304 /// Modify the number of remotely initiated streams that may be concurrently open
-2305 ///
-2306 /// No streams may be opened by the peer unless fewer than `count` are already open. Large
-2307 /// `count`s increase both minimum and worst-case memory consumption.
-2308 pub fn set_max_concurrent_streams(&mut self, dir: Dir, count: VarInt) {
-2309 self.streams.set_max_concurrent(dir, count);
-2310 // If the limit was reduced, then a flow control update previously deemed insignificant may
-2311 // now be significant.
-2312 let pending = &mut self.spaces[SpaceId::Data].pending;
-2313 self.streams.queue_max_stream_id(pending);
+1654 }
+1655
+1656 fn path_should_send(
+1657 &mut self,
+1658 path_id: PathId,
+1659 space_id: SpaceId,
+1660 have_available_path: bool,
+1661 sendable_frames: &SendableFrames,
+1662 ) -> bool {
+1663 let path_exclusive_only = space_id == SpaceId::Data
+1664 && have_available_path
+1665 && self.path_data(path_id).local_status() == PathStatus::Backup;
+1666 let path_should_send = if path_exclusive_only {
+1667 sendable_frames.path_exclusive
+1668 } else {
+1669 !sendable_frames.is_empty()
+1670 };
+1671 let needs_loss_probe = self.spaces[space_id].for_path(path_id).loss_probes > 0;
+1672 path_should_send || needs_loss_probe
+1673 }
+1674
+1675 fn on_remote_cids_exhausted(&mut self, now: Instant, path_id: PathId) {
+1676 if self.abandoned_paths.contains(&path_id) {
+1677 trace!(%path_id, "remote CIDs retired for abandoned path");
+1678 return;
+1679 }
+1680
+1681 let error = PathError::RemoteCidsExhausted;
+1682 debug!(?error, %path_id, "no active CID for path");
+1683 self.events
+1684 .push_back(Event::Path(PathEvent::LocallyClosed { id: path_id, error }));
+1685 // Locally we should have refused to open this path, the remote should
+1686 // have given us CIDs for this path before opening it. So we can always
+1687 // abandon this here.
+1688 self.close_path(
+1689 now,
+1690 path_id,
+1691 TransportErrorCode::NO_CID_AVAILABLE_FOR_PATH.into(),
+1692 )
+1693 .ok();
+1694 self.spaces[SpaceId::Data]
+1695 .pending
+1696 .path_cids_blocked
+1697 .insert(path_id);
+1698 }
+1699
+1700 fn poll_transmit_mtu_probe(
+1701 &mut self,
+1702 now: Instant,
+1703 transmit: &mut TransmitBuf<'_>,
+1704 path_id: PathId,
+1705 ) {
+1706 let Some((active_cid, probe_size)) = self.get_mtu_probe_data(now, path_id) else {
+1707 return;
+1708 };
+1709
+1710 // We are definitely sending a DPLPMTUD probe.
+1711 debug_assert_eq!(transmit.num_datagrams(), 0);
+1712 transmit.start_new_datagram_with_size(probe_size as usize);
+1713
+1714 let Some(mut builder) = PacketBuilder::new(
+1715 now,
+1716 SpaceId::Data,
+1717 path_id,
+1718 active_cid,
+1719 transmit,
+1720 true,
+1721 self,
+1722 ) else {
+1723 return;
+1724 };
+1725
+1726 // We implement MTU probes as ping packets padded up to the probe size
+1727 trace!(?probe_size, "writing MTUD probe");
+1728 builder.write_frame(frame::Ping, &mut self.stats.frame_tx);
+1729
+1730 // If supported by the peer, we want no delays to the probe's ACK
+1731 if self.peer_supports_ack_frequency() {
+1732 builder.write_frame(frame::ImmediateAck, &mut self.stats.frame_tx);
+1733 }
+1734
+1735 builder.finish_and_track(now, self, path_id, PadDatagram::ToSize(probe_size));
+1736
+1737 self.path_stats
+1738 .entry(path_id)
+1739 .or_default()
+1740 .sent_plpmtud_probes += 1;
+1741 }
+1742
+1743 fn get_mtu_probe_data(&mut self, now: Instant, path_id: PathId) -> Option<(ConnectionId, u16)> {
+1744 // We MTU probe all paths for which all of the following is true:
+1745 // - We have an active destination CID for the path.
+1746 // - The remote address *and* path are validated.
+1747 // - The path is not abandoned.
+1748 // - The MTU Discovery subsystem wants to probe the path.
+1749 let active_cid = self.rem_cids.get(&path_id).map(CidQueue::active)?;
+1750 let is_eligible = self.path_data(path_id).validated
+1751 && !self.path_data(path_id).is_validating_path()
+1752 && !self.abandoned_paths.contains(&path_id);
+1753
+1754 if !is_eligible {
+1755 return None;
+1756 }
+1757 let next_pn = self.spaces[SpaceId::Data]
+1758 .for_path(path_id)
+1759 .peek_tx_number();
+1760 let probe_size = self
+1761 .path_data_mut(path_id)
+1762 .mtud
+1763 .poll_transmit(now, next_pn)?;
+1764
+1765 Some((active_cid, probe_size))
+1766 }
+1767
+1768 /// Returns if there is anext packet space which has data to send
+1769 ///
+1770 /// This takes into account the space available to frames in the next datagram.
+1771 // TODO(flub): This duplication is not nice.
+1772 fn has_next_send_space(
+1773 &mut self,
+1774 current_space_id: SpaceId,
+1775 path_id: PathId,
+1776 buf: &TransmitBuf<'_>,
+1777 close: bool,
+1778 ) -> bool {
+1779 // Number of bytes available for frames if this is a 1-RTT packet. We're guaranteed
+1780 // to be able to send an individual frame at least this large in the next 1-RTT
+1781 // packet. This could be generalized to support every space, but it's only needed to
+1782 // handle large fixed-size frames, which only exist in 1-RTT (application
+1783 // datagrams). We don't account for coalesced packets potentially occupying space
+1784 // because frames can always spill into the next datagram.
+1785 let mut space_id = current_space_id;
+1786 loop {
+1787 let can_send = self.space_can_send(space_id, path_id, buf.segment_size(), close);
+1788 if !can_send.is_empty() || (close && self.spaces[space_id].crypto.is_some()) {
+1789 return true;
+1790 }
+1791 let Some(next_space) = space_id.next() else {
+1792 break;
+1793 };
+1794 space_id = next_space;
+1795 }
+1796 false
+1797 }
+1798
+1799 /// Checks if creating a new datagram would be blocked by congestion control
+1800 fn path_congestion_check(
+1801 &mut self,
+1802 space_id: SpaceId,
+1803 path_id: PathId,
+1804 transmit: &TransmitBuf<'_>,
+1805 can_send: &SendableFrames,
+1806 now: Instant,
+1807 ) -> PathBlocked {
+1808 // Anti-amplification is only based on `total_sent`, which gets updated after
+1809 // the transmit is sent. Therefore we pass the amount of bytes for datagrams
+1810 // that are already created, as well as 1 byte for starting another datagram. If
+1811 // there is any anti-amplification budget left, we always allow a full MTU to be
+1812 // sent (see https://github.com/quinn-rs/quinn/issues/1082).
+1813 if self.side().is_server()
+1814 && self
+1815 .path_data(path_id)
+1816 .anti_amplification_blocked(transmit.len() as u64 + 1)
+1817 {
+1818 trace!(?space_id, %path_id, "blocked by anti-amplification");
+1819 return PathBlocked::AntiAmplification;
+1820 }
+1821
+1822 // Congestion control check.
+1823 // Tail loss probes must not be blocked by congestion, or a deadlock could arise.
+1824 let bytes_to_send = transmit.segment_size() as u64;
+1825 let need_loss_probe = self.spaces[space_id].for_path(path_id).loss_probes > 0;
+1826
+1827 if can_send.other && !need_loss_probe && !can_send.close {
+1828 let path = self.path_data(path_id);
+1829 if path.in_flight.bytes + bytes_to_send >= path.congestion.window() {
+1830 trace!(?space_id, %path_id, "blocked by congestion control");
+1831 return PathBlocked::Congestion;
+1832 }
+1833 }
+1834
+1835 // Pacing check.
+1836 if let Some(delay) = self.path_data_mut(path_id).pacing_delay(bytes_to_send, now) {
+1837 self.timers.set(
+1838 Timer::PerPath(path_id, PathTimer::Pacing),
+1839 delay,
+1840 self.qlog.with_time(now),
+1841 );
+1842 // Loss probes and CONNECTION_CLOSE should be subject to pacing, even though
+1843 // they are not congestion controlled.
+1844 trace!(?space_id, %path_id, "blocked by pacing");
+1845 return PathBlocked::Pacing;
+1846 }
+1847
+1848 PathBlocked::No
+1849 }
+1850
+1851 /// Send PATH_CHALLENGE for a previous path if necessary
+1852 ///
+1853 /// QUIC-TRANSPORT section 9.3.3
+1854 /// <https://www.rfc-editor.org/rfc/rfc9000.html#name-off-path-packet-forwarding>
+1855 fn send_prev_path_challenge(
+1856 &mut self,
+1857 now: Instant,
+1858 buf: &mut TransmitBuf<'_>,
+1859 path_id: PathId,
+1860 ) -> Option<Transmit> {
+1861 let (prev_cid, prev_path) = self.paths.get_mut(&path_id)?.prev.as_mut()?;
+1862 // TODO (matheus23): We could use !prev_path.is_validating() here instead to
+1863 // (possibly) also re-send challenges when they get lost.
+1864 if !prev_path.send_new_challenge {
+1865 return None;
+1866 };
+1867 prev_path.send_new_challenge = false;
+1868 let network_path = prev_path.network_path;
+1869 let token = self.rng.random();
+1870 let info = paths::SentChallengeInfo {
+1871 sent_instant: now,
+1872 network_path,
+1873 };
+1874 prev_path.challenges_sent.insert(token, info);
+1875 debug_assert_eq!(
+1876 self.highest_space,
+1877 SpaceId::Data,
+1878 "PATH_CHALLENGE queued without 1-RTT keys"
+1879 );
+1880 buf.start_new_datagram_with_size(MIN_INITIAL_SIZE as usize);
+1881
+1882 // Use the previous CID to avoid linking the new path with the previous path. We
+1883 // don't bother accounting for possible retirement of that prev_cid because this is
+1884 // sent once, immediately after migration, when the CID is known to be valid. Even
+1885 // if a post-migration packet caused the CID to be retired, it's fair to pretend
+1886 // this is sent first.
+1887 debug_assert_eq!(buf.datagram_start_offset(), 0);
+1888 let mut builder =
+1889 PacketBuilder::new(now, SpaceId::Data, path_id, *prev_cid, buf, false, self)?;
+1890 let challenge = frame::PathChallenge(token);
+1891 let stats = &mut self.stats.frame_tx;
+1892 builder.write_frame_with_log_msg(challenge, stats, Some("validating previous path"));
+1893
+1894 // An endpoint MUST expand datagrams that contain a PATH_CHALLENGE frame
+1895 // to at least the smallest allowed maximum datagram size of 1200 bytes,
+1896 // unless the anti-amplification limit for the path does not permit
+1897 // sending a datagram of this size
+1898 builder.pad_to(MIN_INITIAL_SIZE);
+1899
+1900 builder.finish(self, now);
+1901 self.stats.udp_tx.on_sent(1, buf.len());
+1902
+1903 Some(Transmit {
+1904 destination: network_path.remote,
+1905 size: buf.len(),
+1906 ecn: None,
+1907 segment_size: None,
+1908 src_ip: network_path.local_ip,
+1909 })
+1910 }
+1911
+1912 /// Indicate what types of frames are ready to send for the given space
+1913 ///
+1914 /// *packet_size* is the number of bytes available to build the next packet.
+1915 /// *close* indicates whether a CONNECTION_CLOSE frame needs to be sent.
+1916 fn space_can_send(
+1917 &mut self,
+1918 space_id: SpaceId,
+1919 path_id: PathId,
+1920 packet_size: usize,
+1921 close: bool,
+1922 ) -> SendableFrames {
+1923 let space = &mut self.spaces[space_id];
+1924 let space_has_crypto = space.crypto.is_none();
+1925
+1926 if !space_has_crypto
+1927 && (space_id != SpaceId::Data
+1928 || self.zero_rtt_crypto.is_none()
+1929 || self.side.is_server())
+1930 {
+1931 // Nothing send in this space
+1932 return SendableFrames::empty();
+1933 }
+1934
+1935 let mut can_send = space.can_send(path_id, &self.streams);
+1936
+1937 if space_id == SpaceId::Data {
+1938 // Check for 1RTT space.
+1939 let pn = space.for_path(path_id).peek_tx_number();
+1940 let frame_space_1rtt =
+1941 packet_size.saturating_sub(self.predict_1rtt_overhead(pn, path_id));
+1942 can_send |= self.can_send_1rtt(path_id, frame_space_1rtt);
+1943 }
+1944
+1945 can_send.close = close && space_has_crypto;
+1946
+1947 can_send
+1948 }
+1949
+1950 fn space_can_send_non_data(
+1951 &mut self,
+1952 space_id: SpaceId,
+1953 path_id: PathId,
+1954 close: bool,
+1955 ) -> SendableFrames {
+1956 debug_assert!(space_id != SpaceId::Data);
+1957 let space = &mut self.spaces[space_id];
+1958 let space_has_crypto = space.crypto.is_none();
+1959
+1960 if !space_has_crypto && (self.zero_rtt_crypto.is_none() || self.side.is_server()) {
+1961 // Nothing send in this space
+1962 return SendableFrames::empty();
+1963 }
+1964
+1965 let mut can_send = space.can_send(path_id, &self.streams);
+1966 can_send.close = close && space_has_crypto;
+1967 can_send
+1968 }
+1969
+1970 /// Process `ConnectionEvent`s generated by the associated `Endpoint`
+1971 ///
+1972 /// Will execute protocol logic upon receipt of a connection event, in turn preparing signals
+1973 /// (including application `Event`s, `EndpointEvent`s and outgoing datagrams) that should be
+1974 /// extracted through the relevant methods.
+1975 pub fn handle_event(&mut self, event: ConnectionEvent) {
+1976 use ConnectionEventInner::*;
+1977 match event.0 {
+1978 Datagram(DatagramConnectionEvent {
+1979 now,
+1980 network_path,
+1981 path_id,
+1982 ecn,
+1983 first_decode,
+1984 remaining,
+1985 }) => {
+1986 let span = trace_span!("pkt", %path_id);
+1987 let _guard = span.enter();
+1988
+1989 if self.update_network_path_or_discard(network_path, path_id) {
+1990 // A return value of true indicates we should discard this packet.
+1991 return;
+1992 }
+1993
+1994 let was_anti_amplification_blocked = self
+1995 .path(path_id)
+1996 .map(|path| path.anti_amplification_blocked(1))
+1997 .unwrap_or(true); // if we don't know about this path it's eagerly considered as unvalidated
+1998 // TODO(@divma): revisit this
+1999
+2000 self.stats.udp_rx.datagrams += 1;
+2001 self.stats.udp_rx.bytes += first_decode.len() as u64;
+2002 let data_len = first_decode.len();
+2003
+2004 self.handle_decode(now, network_path, path_id, ecn, first_decode);
+2005 // The current `path` might have changed inside `handle_decode` since the packet
+2006 // could have triggered a migration. The packet might also belong to an unknown
+2007 // path and have been rejected. Make sure the data received is accounted for the
+2008 // most recent path by accessing `path` after `handle_decode`.
+2009 if let Some(path) = self.path_mut(path_id) {
+2010 path.inc_total_recvd(data_len as u64);
+2011 }
+2012
+2013 if let Some(data) = remaining {
+2014 self.stats.udp_rx.bytes += data.len() as u64;
+2015 self.handle_coalesced(now, network_path, path_id, ecn, data);
+2016 }
+2017
+2018 if let Some(path) = self.paths.get_mut(&path_id) {
+2019 self.qlog
+2020 .emit_recovery_metrics(path_id, &mut path.data, now);
+2021 }
+2022
+2023 if was_anti_amplification_blocked {
+2024 // A prior attempt to set the loss detection timer may have failed due to
+2025 // anti-amplification, so ensure it's set now. Prevents a handshake deadlock if
+2026 // the server's first flight is lost.
+2027 self.set_loss_detection_timer(now, path_id);
+2028 }
+2029 }
+2030 NewIdentifiers(ids, now, cid_len, cid_lifetime) => {
+2031 let path_id = ids.first().map(|issued| issued.path_id).unwrap_or_default();
+2032 debug_assert!(ids.iter().all(|issued| issued.path_id == path_id));
+2033 let cid_state = self
+2034 .local_cid_state
+2035 .entry(path_id)
+2036 .or_insert_with(|| CidState::new(cid_len, cid_lifetime, now, 0));
+2037 cid_state.new_cids(&ids, now);
+2038
+2039 ids.into_iter().rev().for_each(|frame| {
+2040 self.spaces[SpaceId::Data].pending.new_cids.push(frame);
+2041 });
+2042 // Always update Timer::PushNewCid
+2043 self.reset_cid_retirement(now);
+2044 }
+2045 }
+2046 }
+2047
+2048 /// Updates the network path for `path_id`.
+2049 ///
+2050 /// Returns true if a packet coming in for this `path_id` over given `network_path` should be discarded.
+2051 /// Returns false if the path was updated and the packet doesn't need to be discarded.
+2052 fn update_network_path_or_discard(&mut self, network_path: FourTuple, path_id: PathId) -> bool {
+2053 let remote_may_migrate = self.side.remote_may_migrate(&self.state);
+2054 let local_ip_may_migrate = self.side.is_client();
+2055 // If this packet could initiate a migration and we're a client or a server that
+2056 // forbids migration, drop the datagram. This could be relaxed to heuristically
+2057 // permit NAT-rebinding-like migration.
+2058 if let Some(known_path) = self.path_mut(path_id) {
+2059 if network_path.remote != known_path.network_path.remote && !remote_may_migrate {
+2060 trace!(
+2061 %path_id,
+2062 %network_path,
+2063 %known_path.network_path,
+2064 "discarding packet from unrecognized peer"
+2065 );
+2066 return true;
+2067 }
+2068
+2069 if known_path.network_path.local_ip.is_some()
+2070 && network_path.local_ip.is_some()
+2071 && known_path.network_path.local_ip != network_path.local_ip
+2072 && !local_ip_may_migrate
+2073 {
+2074 trace!(
+2075 %path_id,
+2076 %network_path,
+2077 %known_path.network_path,
+2078 "discarding packet sent to incorrect interface"
+2079 );
+2080 return true;
+2081 }
+2082 // If the datagram indicates that we've changed our local IP, we update it.
+2083 // This is alluded to in Section 5.2 of the Multipath RFC draft 18:
+2084 // https://www.ietf.org/archive/id/draft-ietf-quic-multipath-18.html#name-using-multiple-paths-on-the
+2085 // > Client receives the packet, recognizes a path migration, updates the source address of path 2 to 192.0.2.1.
+2086 if let Some(local_ip) = network_path.local_ip {
+2087 if known_path
+2088 .network_path
+2089 .local_ip
+2090 .is_some_and(|ip| ip != local_ip)
+2091 {
+2092 debug!(
+2093 %path_id,
+2094 %network_path,
+2095 %known_path.network_path,
+2096 "path's local address seemingly migrated"
+2097 );
+2098 }
+2099 // We update the address without path validation on the client side.
+2100 // https://www.ietf.org/archive/id/draft-ietf-quic-multipath-18.html#section-5.1
+2101 // > Servers observing a 4-tuple change will perform path validation (see Section 9 of [QUIC-TRANSPORT]).
+2102 // This sounds like it's *only* the server endpoints that do this.
+2103 // TODO(matheus23): We should still consider doing a proper migration on the client side in the future.
+2104 // For now, this preserves the behavior of this code pre 4-tuple tracking.
+2105 known_path.network_path.local_ip = Some(local_ip);
+2106 }
+2107 }
+2108 false
+2109 }
+2110
+2111 /// Process timer expirations
+2112 ///
+2113 /// Executes protocol logic, potentially preparing signals (including application `Event`s,
+2114 /// `EndpointEvent`s and outgoing datagrams) that should be extracted through the relevant
+2115 /// methods.
+2116 ///
+2117 /// It is most efficient to call this immediately after the system clock reaches the latest
+2118 /// `Instant` that was output by `poll_timeout`; however spurious extra calls will simply
+2119 /// no-op and therefore are safe.
+2120 pub fn handle_timeout(&mut self, now: Instant) {
+2121 while let Some((timer, _time)) = self.timers.expire_before(now, &self.qlog) {
+2122 // TODO(@divma): remove `at` when the unicorn is born
+2123 trace!(?timer, at=?now, "timeout");
+2124 match timer {
+2125 Timer::Conn(timer) => match timer {
+2126 ConnTimer::Close => {
+2127 self.state.move_to_drained(None);
+2128 self.endpoint_events.push_back(EndpointEventInner::Drained);
+2129 }
+2130 ConnTimer::Idle => {
+2131 self.kill(ConnectionError::TimedOut);
+2132 }
+2133 ConnTimer::KeepAlive => {
+2134 trace!("sending keep-alive");
+2135 self.ping();
+2136 }
+2137 ConnTimer::KeyDiscard => {
+2138 self.zero_rtt_crypto = None;
+2139 self.prev_crypto = None;
+2140 }
+2141 ConnTimer::PushNewCid => {
+2142 while let Some((path_id, when)) = self.next_cid_retirement() {
+2143 if when > now {
+2144 break;
+2145 }
+2146 match self.local_cid_state.get_mut(&path_id) {
+2147 None => error!(%path_id, "No local CID state for path"),
+2148 Some(cid_state) => {
+2149 // Update `retire_prior_to` field in NEW_CONNECTION_ID frame
+2150 let num_new_cid = cid_state.on_cid_timeout().into();
+2151 if !self.state.is_closed() {
+2152 trace!(
+2153 "push a new CID to peer RETIRE_PRIOR_TO field {}",
+2154 cid_state.retire_prior_to()
+2155 );
+2156 self.endpoint_events.push_back(
+2157 EndpointEventInner::NeedIdentifiers(
+2158 path_id,
+2159 now,
+2160 num_new_cid,
+2161 ),
+2162 );
+2163 }
+2164 }
+2165 }
+2166 }
+2167 }
+2168 },
+2169 // TODO: add path_id as span somehow
+2170 Timer::PerPath(path_id, timer) => {
+2171 let span = trace_span!("per-path timer fired", %path_id, ?timer);
+2172 let _guard = span.enter();
+2173 match timer {
+2174 PathTimer::PathIdle => {
+2175 self.close_path(now, path_id, TransportErrorCode::NO_ERROR.into())
+2176 .ok();
+2177 }
+2178
+2179 PathTimer::PathKeepAlive => {
+2180 trace!("sending keep-alive on path");
+2181 self.ping_path(path_id).ok();
+2182 }
+2183 PathTimer::LossDetection => {
+2184 self.on_loss_detection_timeout(now, path_id);
+2185 self.qlog.emit_recovery_metrics(
+2186 path_id,
+2187 &mut self.paths.get_mut(&path_id).unwrap().data,
+2188 now,
+2189 );
+2190 }
+2191 PathTimer::PathValidation => {
+2192 let Some(path) = self.paths.get_mut(&path_id) else {
+2193 continue;
+2194 };
+2195 self.timers.stop(
+2196 Timer::PerPath(path_id, PathTimer::PathChallengeLost),
+2197 self.qlog.with_time(now),
+2198 );
+2199 debug!("path validation failed");
+2200 if let Some((_, prev)) = path.prev.take() {
+2201 path.data = prev;
+2202 }
+2203 path.data.challenges_sent.clear();
+2204 path.data.send_new_challenge = false;
+2205 }
+2206 PathTimer::PathChallengeLost => {
+2207 let Some(path) = self.paths.get_mut(&path_id) else {
+2208 continue;
+2209 };
+2210 trace!("path challenge deemed lost");
+2211 path.data.send_new_challenge = true;
+2212 }
+2213 PathTimer::PathOpen => {
+2214 let Some(path) = self.paths.get_mut(&path_id) else {
+2215 continue;
+2216 };
+2217 path.data.challenges_sent.clear();
+2218 path.data.send_new_challenge = false;
+2219 self.timers.stop(
+2220 Timer::PerPath(path_id, PathTimer::PathChallengeLost),
+2221 self.qlog.with_time(now),
+2222 );
+2223 debug!("new path validation failed");
+2224 if let Err(err) = self.close_path(
+2225 now,
+2226 path_id,
+2227 TransportErrorCode::PATH_UNSTABLE_OR_POOR.into(),
+2228 ) {
+2229 warn!(?err, "failed closing path");
+2230 }
+2231
+2232 self.events.push_back(Event::Path(PathEvent::LocallyClosed {
+2233 id: path_id,
+2234 error: PathError::ValidationFailed,
+2235 }));
+2236 }
+2237 PathTimer::Pacing => trace!("pacing timer expired"),
+2238 PathTimer::MaxAckDelay => {
+2239 trace!("max ack delay reached");
+2240 // This timer is only armed in the Data space
+2241 self.spaces[SpaceId::Data]
+2242 .for_path(path_id)
+2243 .pending_acks
+2244 .on_max_ack_delay_timeout()
+2245 }
+2246 PathTimer::DiscardPath => {
+2247 // The path was abandoned and 3*PTO has expired since. Clean up all
+2248 // remaining state and install stateless reset token.
+2249 self.timers.stop_per_path(path_id, self.qlog.with_time(now));
+2250 if let Some(loc_cid_state) = self.local_cid_state.remove(&path_id) {
+2251 let (min_seq, max_seq) = loc_cid_state.active_seq();
+2252 for seq in min_seq..=max_seq {
+2253 self.endpoint_events.push_back(
+2254 EndpointEventInner::RetireConnectionId(
+2255 now, path_id, seq, false,
+2256 ),
+2257 );
+2258 }
+2259 }
+2260 self.discard_path(path_id, now);
+2261 }
+2262 }
+2263 }
+2264 }
+2265 }
+2266 }
+2267
+2268 /// Close a connection immediately
+2269 ///
+2270 /// This does not ensure delivery of outstanding data. It is the application's responsibility to
+2271 /// call this only when all important communications have been completed, e.g. by calling
+2272 /// [`SendStream::finish`] on outstanding streams and waiting for the corresponding
+2273 /// [`StreamEvent::Finished`] event.
+2274 ///
+2275 /// If [`Streams::send_streams`] returns 0, all outstanding stream data has been
+2276 /// delivered. There may still be data from the peer that has not been received.
+2277 ///
+2278 /// [`StreamEvent::Finished`]: crate::StreamEvent::Finished
+2279 pub fn close(&mut self, now: Instant, error_code: VarInt, reason: Bytes) {
+2280 self.close_inner(
+2281 now,
+2282 Close::Application(frame::ApplicationClose { error_code, reason }),
+2283 )
+2284 }
+2285
+2286 fn close_inner(&mut self, now: Instant, reason: Close) {
+2287 let was_closed = self.state.is_closed();
+2288 if !was_closed {
+2289 self.close_common();
+2290 self.set_close_timer(now);
+2291 self.close = true;
+2292 self.state.move_to_closed_local(reason);
+2293 }
+2294 }
+2295
+2296 /// Control datagrams
+2297 pub fn datagrams(&mut self) -> Datagrams<'_> {
+2298 Datagrams { conn: self }
+2299 }
+2300
+2301 /// Returns connection statistics
+2302 pub fn stats(&mut self) -> ConnectionStats {
+2303 self.stats.clone()
+2304 }
+2305
+2306 /// Returns path statistics
+2307 pub fn path_stats(&mut self, path_id: PathId) -> Option<PathStats> {
+2308 let path = self.paths.get(&path_id)?;
+2309 let stats = self.path_stats.entry(path_id).or_default();
+2310 stats.rtt = path.data.rtt.get();
+2311 stats.cwnd = path.data.congestion.window();
+2312 stats.current_mtu = path.data.mtud.current_mtu();
+2313 Some(*stats)
2314 }
2315
-2316 /// Modify the number of open paths allowed when multipath is enabled
+2316 /// Ping the remote endpoint
2317 ///
-2318 /// When reducing the number of concurrent paths this will only affect delaying sending
-2319 /// new MAX_PATH_ID frames until fewer than this number of paths are possible. To
-2320 /// actively reduce paths they must be closed using [`Connection::close_path`], which
-2321 /// can also be used to close not-yet-opened paths.
-2322 ///
-2323 /// If multipath is not negotiated (see the [`TransportConfig`]) this can not enable
-2324 /// multipath and will fail.
-2325 pub fn set_max_concurrent_paths(
-2326 &mut self,
-2327 now: Instant,
-2328 count: NonZeroU32,
-2329 ) -> Result<(), MultipathNotNegotiated> {
-2330 if !self.is_multipath_negotiated() {
-2331 return Err(MultipathNotNegotiated { _private: () });
-2332 }
-2333 self.max_concurrent_paths = count;
-2334
-2335 let in_use_count = self
-2336 .local_max_path_id
-2337 .next()
-2338 .saturating_sub(self.abandoned_paths.len() as u32)
-2339 .as_u32();
-2340 let extra_needed = count.get().saturating_sub(in_use_count);
-2341 let new_max_path_id = self.local_max_path_id.saturating_add(extra_needed);
-2342
-2343 self.set_max_path_id(now, new_max_path_id);
-2344
-2345 Ok(())
-2346 }
-2347
-2348 /// If needed, issues a new MAX_PATH_ID frame and new CIDs for any newly allowed paths
-2349 fn set_max_path_id(&mut self, now: Instant, max_path_id: PathId) {
-2350 if max_path_id <= self.local_max_path_id {
+2318 /// Causes an ACK-eliciting packet to be transmitted on the connection.
+2319 pub fn ping(&mut self) {
+2320 // TODO(flub): This is very brute-force: it pings *all* the paths. Instead it would
+2321 // be nice if we could only send a single packet for this.
+2322 for path_data in self.spaces[self.highest_space].number_spaces.values_mut() {
+2323 path_data.ping_pending = true;
+2324 }
+2325 }
+2326
+2327 /// Ping the remote endpoint over a specific path
+2328 ///
+2329 /// Causes an ACK-eliciting packet to be transmitted on the path.
+2330 pub fn ping_path(&mut self, path: PathId) -> Result<(), ClosedPath> {
+2331 let path_data = self.spaces[self.highest_space]
+2332 .number_spaces
+2333 .get_mut(&path)
+2334 .ok_or(ClosedPath { _private: () })?;
+2335 path_data.ping_pending = true;
+2336 Ok(())
+2337 }
+2338
+2339 /// Update traffic keys spontaneously
+2340 ///
+2341 /// This can be useful for testing key updates, as they otherwise only happen infrequently.
+2342 pub fn force_key_update(&mut self) {
+2343 if !self.state.is_established() {
+2344 debug!("ignoring forced key update in illegal state");
+2345 return;
+2346 }
+2347 if self.prev_crypto.is_some() {
+2348 // We already just updated, or are currently updating, the keys. Concurrent key updates
+2349 // are illegal.
+2350 debug!("ignoring redundant forced key update");
2351 return;
2352 }
-2353
-2354 self.local_max_path_id = max_path_id;
-2355 self.spaces[SpaceId::Data].pending.max_path_id = true;
-2356
-2357 self.issue_first_path_cids(now);
-2358 }
-2359
-2360 /// Current number of remotely initiated streams that may be concurrently open
-2361 ///
-2362 /// If the target for this limit is reduced using [`set_max_concurrent_streams`](Self::set_max_concurrent_streams),
-2363 /// it will not change immediately, even if fewer streams are open. Instead, it will
-2364 /// decrement by one for each time a remotely initiated stream of matching directionality is closed.
-2365 pub fn max_concurrent_streams(&self, dir: Dir) -> u64 {
-2366 self.streams.max_concurrent(dir)
+2353 self.update_keys(None, false);
+2354 }
+2355
+2356 /// Get a session reference
+2357 pub fn crypto_session(&self) -> &dyn crypto::Session {
+2358 &*self.crypto
+2359 }
+2360
+2361 /// Whether the connection is in the process of being established
+2362 ///
+2363 /// If this returns `false`, the connection may be either established or closed, signaled by the
+2364 /// emission of a `Connected` or `ConnectionLost` message respectively.
+2365 pub fn is_handshaking(&self) -> bool {
+2366 self.state.is_handshake()
2367 }
2368
-2369 /// See [`TransportConfig::send_window()`]
-2370 pub fn set_send_window(&mut self, send_window: u64) {
-2371 self.streams.set_send_window(send_window);
-2372 }
-2373
-2374 /// See [`TransportConfig::receive_window()`]
-2375 pub fn set_receive_window(&mut self, receive_window: VarInt) {
-2376 if self.streams.set_receive_window(receive_window) {
-2377 self.spaces[SpaceId::Data].pending.max_data = true;
-2378 }
-2379 }
-2380
-2381 /// Whether the Multipath for QUIC extension is enabled.
-2382 ///
-2383 /// Multipath is only enabled after the handshake is completed and if it was enabled by both
-2384 /// peers.
-2385 pub fn is_multipath_negotiated(&self) -> bool {
-2386 !self.is_handshaking()
-2387 && self.config.max_concurrent_multipath_paths.is_some()
-2388 && self.peer_params.initial_max_path_id.is_some()
-2389 }
-2390
-2391 fn on_ack_received(
-2392 &mut self,
-2393 now: Instant,
-2394 space: SpaceId,
-2395 ack: frame::Ack,
-2396 ) -> Result<(), TransportError> {
-2397 // All ACKs are referencing path 0
-2398 let path = PathId::ZERO;
-2399 self.inner_on_ack_received(now, space, path, ack)
-2400 }
-2401
-2402 fn on_path_ack_received(
-2403 &mut self,
-2404 now: Instant,
-2405 space: SpaceId,
-2406 path_ack: frame::PathAck,
-2407 ) -> Result<(), TransportError> {
-2408 let (ack, path) = path_ack.into_ack();
-2409 self.inner_on_ack_received(now, space, path, ack)
-2410 }
-2411
-2412 /// Handles an ACK frame acknowledging packets sent on *path*.
-2413 fn inner_on_ack_received(
-2414 &mut self,
-2415 now: Instant,
-2416 space: SpaceId,
-2417 path: PathId,
-2418 ack: frame::Ack,
-2419 ) -> Result<(), TransportError> {
-2420 if self.abandoned_paths.contains(&path) {
-2421 // See also https://www.ietf.org/archive/id/draft-ietf-quic-multipath-17.html#section-3.4.3-3
-2422 // > PATH_ACK frames received with an abandoned path ID are silently ignored, as specified in Section 4.
-2423 trace!("silently ignoring PATH_ACK on abandoned path");
-2424 return Ok(());
-2425 }
-2426 if ack.largest >= self.spaces[space].for_path(path).next_packet_number {
-2427 return Err(TransportError::PROTOCOL_VIOLATION("unsent packet acked"));
-2428 }
-2429 let new_largest = {
-2430 let space = &mut self.spaces[space].for_path(path);
-2431 if space.largest_acked_packet.is_none_or(|pn| ack.largest > pn) {
-2432 space.largest_acked_packet = Some(ack.largest);
-2433 if let Some(info) = space.sent_packets.get(ack.largest) {
-2434 // This should always succeed, but a misbehaving peer might ACK a packet we
-2435 // haven't sent. At worst, that will result in us spuriously reducing the
-2436 // congestion window.
-2437 space.largest_acked_packet_sent = info.time_sent;
-2438 }
-2439 true
-2440 } else {
-2441 false
-2442 }
-2443 };
-2444
-2445 if self.detect_spurious_loss(&ack, space, path) {
-2446 self.path_data_mut(path)
-2447 .congestion
-2448 .on_spurious_congestion_event();
-2449 }
-2450
-2451 // Avoid DoS from unreasonably huge ack ranges by filtering out just the new acks.
-2452 let mut newly_acked = ArrayRangeSet::new();
-2453 for range in ack.iter() {
-2454 self.spaces[space].for_path(path).check_ack(range.clone())?;
-2455 for (pn, _) in self.spaces[space]
-2456 .for_path(path)
-2457 .sent_packets
-2458 .iter_range(range)
-2459 {
-2460 newly_acked.insert_one(pn);
-2461 }
-2462 }
-2463
-2464 if newly_acked.is_empty() {
-2465 return Ok(());
-2466 }
-2467
-2468 let mut ack_eliciting_acked = false;
-2469 for packet in newly_acked.elts() {
-2470 if let Some(info) = self.spaces[space].for_path(path).take(packet) {
-2471 for (acked_path_id, acked_pn) in info.largest_acked.iter() {
-2472 // Assume ACKs for all packets below the largest acknowledged in
-2473 // `packet` have been received. This can cause the peer to spuriously
-2474 // retransmit if some of our earlier ACKs were lost, but allows for
-2475 // simpler state tracking. See discussion at
-2476 // https://www.rfc-editor.org/rfc/rfc9000.html#name-limiting-ranges-by-tracking
-2477 if let Some(pns) = self.spaces[space].path_space_mut(*acked_path_id) {
-2478 pns.pending_acks.subtract_below(*acked_pn);
-2479 }
-2480 }
-2481 ack_eliciting_acked |= info.ack_eliciting;
-2482
-2483 // Notify MTU discovery that a packet was acked, because it might be an MTU probe
-2484 let path_data = self.path_data_mut(path);
-2485 let mtu_updated = path_data.mtud.on_acked(space, packet, info.size);
-2486 if mtu_updated {
-2487 path_data
-2488 .congestion
-2489 .on_mtu_update(path_data.mtud.current_mtu());
-2490 }
-2491
-2492 // Notify ack frequency that a packet was acked, because it might contain an ACK_FREQUENCY frame
-2493 self.ack_frequency.on_acked(path, packet);
-2494
-2495 self.on_packet_acked(now, path, info);
-2496 }
-2497 }
-2498
-2499 let largest_ackd = self.spaces[space].for_path(path).largest_acked_packet;
-2500 let app_limited = self.app_limited;
-2501 let path_data = self.path_data_mut(path);
-2502 let in_flight = path_data.in_flight.bytes;
-2503
-2504 path_data
-2505 .congestion
-2506 .on_end_acks(now, in_flight, app_limited, largest_ackd);
-2507
-2508 if new_largest && ack_eliciting_acked {
-2509 let ack_delay = if space != SpaceId::Data {
-2510 Duration::from_micros(0)
-2511 } else {
-2512 cmp::min(
-2513 self.ack_frequency.peer_max_ack_delay,
-2514 Duration::from_micros(ack.delay << self.peer_params.ack_delay_exponent.0),
-2515 )
-2516 };
-2517 let rtt = now.saturating_duration_since(
-2518 self.spaces[space].for_path(path).largest_acked_packet_sent,
-2519 );
-2520
-2521 let next_pn = self.spaces[space].for_path(path).next_packet_number;
-2522 let path_data = self.path_data_mut(path);
-2523 // TODO(@divma): should be a method of path, should be contained in a single place
-2524 path_data.rtt.update(ack_delay, rtt);
-2525 if path_data.first_packet_after_rtt_sample.is_none() {
-2526 path_data.first_packet_after_rtt_sample = Some((space, next_pn));
-2527 }
-2528 }
+2369 /// Whether the connection is closed
+2370 ///
+2371 /// Closed connections cannot transport any further data. A connection becomes closed when
+2372 /// either peer application intentionally closes it, or when either transport layer detects an
+2373 /// error such as a time-out or certificate validation failure.
+2374 ///
+2375 /// A `ConnectionLost` event is emitted with details when the connection becomes closed.
+2376 pub fn is_closed(&self) -> bool {
+2377 self.state.is_closed()
+2378 }
+2379
+2380 /// Whether there is no longer any need to keep the connection around
+2381 ///
+2382 /// Closed connections become drained after a brief timeout to absorb any remaining in-flight
+2383 /// packets from the peer. All drained connections have been closed.
+2384 pub fn is_drained(&self) -> bool {
+2385 self.state.is_drained()
+2386 }
+2387
+2388 /// For clients, if the peer accepted the 0-RTT data packets
+2389 ///
+2390 /// The value is meaningless until after the handshake completes.
+2391 pub fn accepted_0rtt(&self) -> bool {
+2392 self.accepted_0rtt
+2393 }
+2394
+2395 /// Whether 0-RTT is/was possible during the handshake
+2396 pub fn has_0rtt(&self) -> bool {
+2397 self.zero_rtt_enabled
+2398 }
+2399
+2400 /// Whether there are any pending retransmits
+2401 pub fn has_pending_retransmits(&self) -> bool {
+2402 !self.spaces[SpaceId::Data].pending.is_empty(&self.streams)
+2403 }
+2404
+2405 /// Look up whether we're the client or server of this Connection
+2406 pub fn side(&self) -> Side {
+2407 self.side.side()
+2408 }
+2409
+2410 /// Get the address observed by the remote over the given path
+2411 pub fn path_observed_address(&self, path_id: PathId) -> Result<Option<SocketAddr>, ClosedPath> {
+2412 self.path(path_id)
+2413 .map(|path_data| {
+2414 path_data
+2415 .last_observed_addr_report
+2416 .as_ref()
+2417 .map(|observed| observed.socket_addr())
+2418 })
+2419 .ok_or(ClosedPath { _private: () })
+2420 }
+2421
+2422 /// Current best estimate of this connection's latency (round-trip-time)
+2423 pub fn rtt(&self, path_id: PathId) -> Option<Duration> {
+2424 self.path(path_id).map(|d| d.rtt.get())
+2425 }
+2426
+2427 /// Current state of this connection's congestion controller, for debugging purposes
+2428 pub fn congestion_state(&self, path_id: PathId) -> Option<&dyn Controller> {
+2429 self.path(path_id).map(|d| d.congestion.as_ref())
+2430 }
+2431
+2432 /// Modify the number of remotely initiated streams that may be concurrently open
+2433 ///
+2434 /// No streams may be opened by the peer unless fewer than `count` are already open. Large
+2435 /// `count`s increase both minimum and worst-case memory consumption.
+2436 pub fn set_max_concurrent_streams(&mut self, dir: Dir, count: VarInt) {
+2437 self.streams.set_max_concurrent(dir, count);
+2438 // If the limit was reduced, then a flow control update previously deemed insignificant may
+2439 // now be significant.
+2440 let pending = &mut self.spaces[SpaceId::Data].pending;
+2441 self.streams.queue_max_stream_id(pending);
+2442 }
+2443
+2444 /// Modify the number of open paths allowed when multipath is enabled
+2445 ///
+2446 /// When reducing the number of concurrent paths this will only affect delaying sending
+2447 /// new MAX_PATH_ID frames until fewer than this number of paths are possible. To
+2448 /// actively reduce paths they must be closed using [`Connection::close_path`], which
+2449 /// can also be used to close not-yet-opened paths.
+2450 ///
+2451 /// If multipath is not negotiated (see the [`TransportConfig`]) this can not enable
+2452 /// multipath and will fail.
+2453 pub fn set_max_concurrent_paths(
+2454 &mut self,
+2455 now: Instant,
+2456 count: NonZeroU32,
+2457 ) -> Result<(), MultipathNotNegotiated> {
+2458 if !self.is_multipath_negotiated() {
+2459 return Err(MultipathNotNegotiated { _private: () });
+2460 }
+2461 self.max_concurrent_paths = count;
+2462
+2463 let in_use_count = self
+2464 .local_max_path_id
+2465 .next()
+2466 .saturating_sub(self.abandoned_paths.len() as u32)
+2467 .as_u32();
+2468 let extra_needed = count.get().saturating_sub(in_use_count);
+2469 let new_max_path_id = self.local_max_path_id.saturating_add(extra_needed);
+2470
+2471 self.set_max_path_id(now, new_max_path_id);
+2472
+2473 Ok(())
+2474 }
+2475
+2476 /// If needed, issues a new MAX_PATH_ID frame and new CIDs for any newly allowed paths
+2477 fn set_max_path_id(&mut self, now: Instant, max_path_id: PathId) {
+2478 if max_path_id <= self.local_max_path_id {
+2479 return;
+2480 }
+2481
+2482 self.local_max_path_id = max_path_id;
+2483 self.spaces[SpaceId::Data].pending.max_path_id = true;
+2484
+2485 self.issue_first_path_cids(now);
+2486 }
+2487
+2488 /// Current number of remotely initiated streams that may be concurrently open
+2489 ///
+2490 /// If the target for this limit is reduced using [`set_max_concurrent_streams`](Self::set_max_concurrent_streams),
+2491 /// it will not change immediately, even if fewer streams are open. Instead, it will
+2492 /// decrement by one for each time a remotely initiated stream of matching directionality is closed.
+2493 pub fn max_concurrent_streams(&self, dir: Dir) -> u64 {
+2494 self.streams.max_concurrent(dir)
+2495 }
+2496
+2497 /// See [`TransportConfig::send_window()`]
+2498 pub fn set_send_window(&mut self, send_window: u64) {
+2499 self.streams.set_send_window(send_window);
+2500 }
+2501
+2502 /// See [`TransportConfig::receive_window()`]
+2503 pub fn set_receive_window(&mut self, receive_window: VarInt) {
+2504 if self.streams.set_receive_window(receive_window) {
+2505 self.spaces[SpaceId::Data].pending.max_data = true;
+2506 }
+2507 }
+2508
+2509 /// Whether the Multipath for QUIC extension is enabled.
+2510 ///
+2511 /// Multipath is only enabled after the handshake is completed and if it was enabled by both
+2512 /// peers.
+2513 pub fn is_multipath_negotiated(&self) -> bool {
+2514 !self.is_handshaking()
+2515 && self.config.max_concurrent_multipath_paths.is_some()
+2516 && self.peer_params.initial_max_path_id.is_some()
+2517 }
+2518
+2519 fn on_ack_received(
+2520 &mut self,
+2521 now: Instant,
+2522 space: SpaceId,
+2523 ack: frame::Ack,
+2524 ) -> Result<(), TransportError> {
+2525 // All ACKs are referencing path 0
+2526 let path = PathId::ZERO;
+2527 self.inner_on_ack_received(now, space, path, ack)
+2528 }
2529
-2530 // Must be called before crypto/pto_count are clobbered
-2531 self.detect_lost_packets(now, space, path, true);
-2532
-2533 if self.peer_completed_address_validation(path) {
-2534 self.path_data_mut(path).pto_count = 0;
-2535 }
-2536
-2537 // Explicit congestion notification
-2538 // TODO(@divma): this code is a good example of logic that should be contained in a single
-2539 // place but it's split between the path data and the packet number space data, we should
-2540 // find a way to make this work without two lookups
-2541 if self.path_data(path).sending_ecn {
-2542 if let Some(ecn) = ack.ecn {
-2543 // We only examine ECN counters from ACKs that we are certain we received in transmit
-2544 // order, allowing us to compute an increase in ECN counts to compare against the number
-2545 // of newly acked packets that remains well-defined in the presence of arbitrary packet
-2546 // reordering.
-2547 if new_largest {
-2548 let sent = self.spaces[space].for_path(path).largest_acked_packet_sent;
-2549 self.process_ecn(now, space, path, newly_acked.len() as u64, ecn, sent);
-2550 }
-2551 } else {
-2552 // We always start out sending ECN, so any ack that doesn't acknowledge it disables it.
-2553 debug!("ECN not acknowledged by peer");
-2554 self.path_data_mut(path).sending_ecn = false;
-2555 }
+2530 fn on_path_ack_received(
+2531 &mut self,
+2532 now: Instant,
+2533 space: SpaceId,
+2534 path_ack: frame::PathAck,
+2535 ) -> Result<(), TransportError> {
+2536 let (ack, path) = path_ack.into_ack();
+2537 self.inner_on_ack_received(now, space, path, ack)
+2538 }
+2539
+2540 /// Handles an ACK frame acknowledging packets sent on *path*.
+2541 fn inner_on_ack_received(
+2542 &mut self,
+2543 now: Instant,
+2544 space: SpaceId,
+2545 path: PathId,
+2546 ack: frame::Ack,
+2547 ) -> Result<(), TransportError> {
+2548 if self.abandoned_paths.contains(&path) {
+2549 // See also https://www.ietf.org/archive/id/draft-ietf-quic-multipath-17.html#section-3.4.3-3
+2550 // > PATH_ACK frames received with an abandoned path ID are silently ignored, as specified in Section 4.
+2551 trace!("silently ignoring PATH_ACK on abandoned path");
+2552 return Ok(());
+2553 }
+2554 if ack.largest >= self.spaces[space].for_path(path).next_packet_number {
+2555 return Err(TransportError::PROTOCOL_VIOLATION("unsent packet acked"));
2556 }
-2557
-2558 self.set_loss_detection_timer(now, path);
-2559 Ok(())
-2560 }
-2561
-2562 fn detect_spurious_loss(&mut self, ack: &frame::Ack, space: SpaceId, path: PathId) -> bool {
-2563 let lost_packets = &mut self.spaces[space].for_path(path).lost_packets;
-2564
-2565 if lost_packets.is_empty() {
-2566 return false;
-2567 }
-2568
-2569 for range in ack.iter() {
-2570 let spurious_losses: Vec<u64> = lost_packets
-2571 .iter_range(range.clone())
-2572 .map(|(pn, _info)| pn)
-2573 .collect();
-2574
-2575 for pn in spurious_losses {
-2576 lost_packets.remove(pn);
-2577 }
-2578 }
-2579
-2580 // If this ACK frame acknowledged all deemed lost packets,
-2581 // then we have raised a spurious congestion event in the past.
-2582 // We cannot conclude when there are remaining packets,
-2583 // but future ACK frames might indicate a spurious loss detection.
-2584 lost_packets.is_empty()
-2585 }
-2586
-2587 /// Drain lost packets that we reasonably think will never arrive
-2588 ///
-2589 /// The current criterion is copied from `msquic`:
-2590 /// discard packets that were sent earlier than 2 probe timeouts ago.
-2591 fn drain_lost_packets(&mut self, now: Instant, space: SpaceId, path: PathId) {
-2592 let two_pto = 2 * self.path_data(path).rtt.pto_base();
-2593
-2594 let lost_packets = &mut self.spaces[space].for_path(path).lost_packets;
-2595 lost_packets.retain(|_pn, info| now.saturating_duration_since(info.time_sent) <= two_pto);
-2596 }
-2597
-2598 /// Process a new ECN block from an in-order ACK
-2599 fn process_ecn(
-2600 &mut self,
-2601 now: Instant,
-2602 space: SpaceId,
-2603 path: PathId,
-2604 newly_acked: u64,
-2605 ecn: frame::EcnCounts,
-2606 largest_sent_time: Instant,
-2607 ) {
-2608 match self.spaces[space]
-2609 .for_path(path)
-2610 .detect_ecn(newly_acked, ecn)
-2611 {
-2612 Err(e) => {
-2613 debug!("halting ECN due to verification failure: {}", e);
-2614
-2615 self.path_data_mut(path).sending_ecn = false;
-2616 // Wipe out the existing value because it might be garbage and could interfere with
-2617 // future attempts to use ECN on new paths.
-2618 self.spaces[space].for_path(path).ecn_feedback = frame::EcnCounts::ZERO;
-2619 }
-2620 Ok(false) => {}
-2621 Ok(true) => {
-2622 self.path_stats.entry(path).or_default().congestion_events += 1;
-2623 self.path_data_mut(path).congestion.on_congestion_event(
-2624 now,
-2625 largest_sent_time,
-2626 false,
-2627 true,
-2628 0,
-2629 );
-2630 }
-2631 }
-2632 }
-2633
-2634 // Not timing-aware, so it's safe to call this for inferred acks, such as arise from
-2635 // high-latency handshakes
-2636 fn on_packet_acked(&mut self, now: Instant, path_id: PathId, info: SentPacket) {
-2637 self.paths
-2638 .get_mut(&path_id)
-2639 .expect("known path")
-2640 .remove_in_flight(&info);
-2641 let app_limited = self.app_limited;
-2642 let path = self.path_data_mut(path_id);
-2643 if info.ack_eliciting && !path.is_validating_path() {
-2644 // Only pass ACKs to the congestion controller if we are not validating the current
-2645 // path, so as to ignore any ACKs from older paths still coming in.
-2646 let rtt = path.rtt;
-2647 path.congestion
-2648 .on_ack(now, info.time_sent, info.size.into(), app_limited, &rtt);
-2649 }
-2650
-2651 // Update state for confirmed delivery of frames
-2652 if let Some(retransmits) = info.retransmits.get() {
-2653 for (id, _) in retransmits.reset_stream.iter() {
-2654 self.streams.reset_acked(*id);
+2557 let new_largest = {
+2558 let space = &mut self.spaces[space].for_path(path);
+2559 if space.largest_acked_packet.is_none_or(|pn| ack.largest > pn) {
+2560 space.largest_acked_packet = Some(ack.largest);
+2561 if let Some(info) = space.sent_packets.get(ack.largest) {
+2562 // This should always succeed, but a misbehaving peer might ACK a packet we
+2563 // haven't sent. At worst, that will result in us spuriously reducing the
+2564 // congestion window.
+2565 space.largest_acked_packet_sent = info.time_sent;
+2566 }
+2567 true
+2568 } else {
+2569 false
+2570 }
+2571 };
+2572
+2573 if self.detect_spurious_loss(&ack, space, path) {
+2574 self.path_data_mut(path)
+2575 .congestion
+2576 .on_spurious_congestion_event();
+2577 }
+2578
+2579 // Avoid DoS from unreasonably huge ack ranges by filtering out just the new acks.
+2580 let mut newly_acked = ArrayRangeSet::new();
+2581 for range in ack.iter() {
+2582 self.spaces[space].for_path(path).check_ack(range.clone())?;
+2583 for (pn, _) in self.spaces[space]
+2584 .for_path(path)
+2585 .sent_packets
+2586 .iter_range(range)
+2587 {
+2588 newly_acked.insert_one(pn);
+2589 }
+2590 }
+2591
+2592 if newly_acked.is_empty() {
+2593 return Ok(());
+2594 }
+2595
+2596 let mut ack_eliciting_acked = false;
+2597 for packet in newly_acked.elts() {
+2598 if let Some(info) = self.spaces[space].for_path(path).take(packet) {
+2599 for (acked_path_id, acked_pn) in info.largest_acked.iter() {
+2600 // Assume ACKs for all packets below the largest acknowledged in
+2601 // `packet` have been received. This can cause the peer to spuriously
+2602 // retransmit if some of our earlier ACKs were lost, but allows for
+2603 // simpler state tracking. See discussion at
+2604 // https://www.rfc-editor.org/rfc/rfc9000.html#name-limiting-ranges-by-tracking
+2605 if let Some(pns) = self.spaces[space].path_space_mut(*acked_path_id) {
+2606 pns.pending_acks.subtract_below(*acked_pn);
+2607 }
+2608 }
+2609 ack_eliciting_acked |= info.ack_eliciting;
+2610
+2611 // Notify MTU discovery that a packet was acked, because it might be an MTU probe
+2612 let path_data = self.path_data_mut(path);
+2613 let mtu_updated = path_data.mtud.on_acked(space, packet, info.size);
+2614 if mtu_updated {
+2615 path_data
+2616 .congestion
+2617 .on_mtu_update(path_data.mtud.current_mtu());
+2618 }
+2619
+2620 // Notify ack frequency that a packet was acked, because it might contain an ACK_FREQUENCY frame
+2621 self.ack_frequency.on_acked(path, packet);
+2622
+2623 self.on_packet_acked(now, path, info);
+2624 }
+2625 }
+2626
+2627 let largest_ackd = self.spaces[space].for_path(path).largest_acked_packet;
+2628 let app_limited = self.app_limited;
+2629 let path_data = self.path_data_mut(path);
+2630 let in_flight = path_data.in_flight.bytes;
+2631
+2632 path_data
+2633 .congestion
+2634 .on_end_acks(now, in_flight, app_limited, largest_ackd);
+2635
+2636 if new_largest && ack_eliciting_acked {
+2637 let ack_delay = if space != SpaceId::Data {
+2638 Duration::from_micros(0)
+2639 } else {
+2640 cmp::min(
+2641 self.ack_frequency.peer_max_ack_delay,
+2642 Duration::from_micros(ack.delay << self.peer_params.ack_delay_exponent.0),
+2643 )
+2644 };
+2645 let rtt = now.saturating_duration_since(
+2646 self.spaces[space].for_path(path).largest_acked_packet_sent,
+2647 );
+2648
+2649 let next_pn = self.spaces[space].for_path(path).next_packet_number;
+2650 let path_data = self.path_data_mut(path);
+2651 // TODO(@divma): should be a method of path, should be contained in a single place
+2652 path_data.rtt.update(ack_delay, rtt);
+2653 if path_data.first_packet_after_rtt_sample.is_none() {
+2654 path_data.first_packet_after_rtt_sample = Some((space, next_pn));
2655 }
2656 }
2657
-2658 for frame in info.stream_frames {
-2659 self.streams.received_ack_of(frame);
-2660 }
-2661 }
-2662
-2663 fn set_key_discard_timer(&mut self, now: Instant, space: SpaceId) {
-2664 let start = if self.zero_rtt_crypto.is_some() {
-2665 now
-2666 } else {
-2667 self.prev_crypto
-2668 .as_ref()
-2669 .expect("no previous keys")
-2670 .end_packet
-2671 .as_ref()
-2672 .expect("update not acknowledged yet")
-2673 .1
-2674 };
-2675
-2676 // QUIC-MULTIPATH § 2.5 Key Phase Update Process: use largest PTO off all paths.
-2677 self.timers.set(
-2678 Timer::Conn(ConnTimer::KeyDiscard),
-2679 start + self.pto_max_path(space, false) * 3,
-2680 self.qlog.with_time(now),
-2681 );
-2682 }
-2683
-2684 /// Handle a [`PathTimer::LossDetection`] timeout.
-2685 ///
-2686 /// This timer expires for two reasons:
-2687 /// - An ACK-eliciting packet we sent should be considered lost.
-2688 /// - The PTO may have expired and a tail-loss probe needs to be scheduled.
-2689 ///
-2690 /// The former needs us to schedule re-transmission of the lost data.
-2691 ///
-2692 /// The latter means we have not received an ACK for an ack-eliciting packet we sent
-2693 /// within the PTO time-window. We need to schedule a tail-loss probe, an ack-eliciting
-2694 /// packet, to try and elicit new acknowledgements. These new acknowledgements will
-2695 /// indicate whether the previously sent packets were lost or not.
-2696 fn on_loss_detection_timeout(&mut self, now: Instant, path_id: PathId) {
-2697 if let Some((_, pn_space)) = self.loss_time_and_space(path_id) {
-2698 // Time threshold loss Detection
-2699 self.detect_lost_packets(now, pn_space, path_id, false);
-2700 self.set_loss_detection_timer(now, path_id);
-2701 return;
-2702 }
-2703
-2704 let (_, space) = match self.pto_time_and_space(now, path_id) {
-2705 Some(x) => x,
-2706 None => {
-2707 error!(%path_id, "PTO expired while unset");
-2708 return;
-2709 }
-2710 };
-2711 trace!(
-2712 in_flight = self.path_data(path_id).in_flight.bytes,
-2713 count = self.path_data(path_id).pto_count,
-2714 ?space,
-2715 %path_id,
-2716 "PTO fired"
-2717 );
-2718
-2719 let count = match self.path_data(path_id).in_flight.ack_eliciting {
-2720 // A PTO when we're not expecting any ACKs must be due to handshake anti-amplification
-2721 // deadlock preventions
-2722 0 => {
-2723 debug_assert!(!self.peer_completed_address_validation(path_id));
-2724 1
-2725 }
-2726 // Conventional loss probe
-2727 _ => 2,
-2728 };
-2729 let pns = self.spaces[space].for_path(path_id);
-2730 pns.loss_probes = pns.loss_probes.saturating_add(count);
-2731 let path_data = self.path_data_mut(path_id);
-2732 path_data.pto_count = path_data.pto_count.saturating_add(1);
-2733 self.set_loss_detection_timer(now, path_id);
-2734 }
-2735
-2736 /// Detect any lost packets
-2737 ///
-2738 /// There are two cases in which we detects lost packets:
-2739 ///
-2740 /// - We received an ACK packet.
-2741 /// - The [`PathTimer::LossDetection`] timer expired. So there is an un-acknowledged packet
-2742 /// that was followed by an acknowledged packet. The loss timer for this
-2743 /// un-acknowledged packet expired and we need to detect that packet as lost.
-2744 ///
-2745 /// Packets are lost if they are both (See RFC9002 §6.1):
-2746 ///
-2747 /// - Unacknowledged, in flight and sent prior to an acknowledged packet.
-2748 /// - Old enough by either:
-2749 /// - Having a packet number [`TransportConfig::packet_threshold`] lower then the last
-2750 /// acknowledged packet.
-2751 /// - Being sent [`TransportConfig::time_threshold`] * RTT in the past.
-2752 fn detect_lost_packets(
-2753 &mut self,
-2754 now: Instant,
-2755 pn_space: SpaceId,
-2756 path_id: PathId,
-2757 due_to_ack: bool,
-2758 ) {
-2759 let mut lost_packets = Vec::<u64>::new();
-2760 let mut lost_mtu_probe = None;
-2761 let mut in_persistent_congestion = false;
-2762 let mut size_of_lost_packets = 0u64;
-2763 self.spaces[pn_space].for_path(path_id).loss_time = None;
-2764
-2765 // Find all the lost packets, populating all variables initialised above.
-2766
-2767 let path = self.path_data(path_id);
-2768 let in_flight_mtu_probe = path.mtud.in_flight_mtu_probe();
-2769 let loss_delay = path
-2770 .rtt
-2771 .conservative()
-2772 .mul_f32(self.config.time_threshold)
-2773 .max(TIMER_GRANULARITY);
-2774 let first_packet_after_rtt_sample = path.first_packet_after_rtt_sample;
-2775
-2776 let largest_acked_packet = self.spaces[pn_space]
-2777 .for_path(path_id)
-2778 .largest_acked_packet
-2779 .expect("detect_lost_packets only to be called if path received at least one ACK");
-2780 let packet_threshold = self.config.packet_threshold as u64;
-2781
-2782 // InPersistentCongestion: Determine if all packets in the time period before the newest
-2783 // lost packet, including the edges, are marked lost. PTO computation must always
-2784 // include max ACK delay, i.e. operate as if in Data space (see RFC9001 §7.6.1).
-2785 let congestion_period = self
-2786 .pto(SpaceId::Data, path_id)
-2787 .saturating_mul(self.config.persistent_congestion_threshold);
-2788 let mut persistent_congestion_start: Option<Instant> = None;
-2789 let mut prev_packet = None;
-2790 let space = self.spaces[pn_space].for_path(path_id);
-2791
-2792 for (packet, info) in space.sent_packets.iter_range(0..largest_acked_packet) {
-2793 if prev_packet != Some(packet.wrapping_sub(1)) {
-2794 // An intervening packet was acknowledged
-2795 persistent_congestion_start = None;
-2796 }
-2797
-2798 // Packets sent before now - loss_delay are deemed lost.
-2799 // However, we avoid subtraction as it can panic and there's no
-2800 // saturating equivalent of this subtraction operation with a Duration.
-2801 let packet_too_old = now.saturating_duration_since(info.time_sent) >= loss_delay;
-2802 if packet_too_old || largest_acked_packet >= packet + packet_threshold {
-2803 // The packet should be declared lost.
-2804 if Some(packet) == in_flight_mtu_probe {
-2805 // Lost MTU probes are not included in `lost_packets`, because they
-2806 // should not trigger a congestion control response
-2807 lost_mtu_probe = in_flight_mtu_probe;
-2808 } else {
-2809 lost_packets.push(packet);
-2810 size_of_lost_packets += info.size as u64;
-2811 if info.ack_eliciting && due_to_ack {
-2812 match persistent_congestion_start {
-2813 // Two ACK-eliciting packets lost more than
-2814 // congestion_period apart, with no ACKed packets in between
-2815 Some(start) if info.time_sent - start > congestion_period => {
-2816 in_persistent_congestion = true;
-2817 }
-2818 // Persistent congestion must start after the first RTT sample
-2819 None if first_packet_after_rtt_sample
-2820 .is_some_and(|x| x < (pn_space, packet)) =>
-2821 {
-2822 persistent_congestion_start = Some(info.time_sent);
-2823 }
-2824 _ => {}
-2825 }
-2826 }
-2827 }
-2828 } else {
-2829 // The packet should not yet be declared lost.
-2830 if space.loss_time.is_none() {
-2831 // Since we iterate in order the lowest packet number's loss time will
-2832 // always be the earliest.
-2833 space.loss_time = Some(info.time_sent + loss_delay);
-2834 }
-2835 persistent_congestion_start = None;
-2836 }
-2837
-2838 prev_packet = Some(packet);
-2839 }
-2840
-2841 self.handle_lost_packets(
-2842 pn_space,
-2843 path_id,
-2844 now,
-2845 lost_packets,
-2846 lost_mtu_probe,
-2847 loss_delay,
-2848 in_persistent_congestion,
-2849 size_of_lost_packets,
-2850 );
-2851 }
-2852
-2853 /// Drops the path state, declaring any remaining in-flight packets as lost
-2854 fn discard_path(&mut self, path_id: PathId, now: Instant) {
-2855 trace!(%path_id, "dropping path state");
-2856 let path = self.path_data(path_id);
-2857 let in_flight_mtu_probe = path.mtud.in_flight_mtu_probe();
-2858
-2859 let mut size_of_lost_packets = 0u64; // add to path_stats.lost_bytes;
-2860 let lost_pns: Vec<_> = self.spaces[SpaceId::Data]
-2861 .for_path(path_id)
-2862 .sent_packets
-2863 .iter()
-2864 .filter(|(pn, _info)| Some(*pn) != in_flight_mtu_probe)
-2865 .map(|(pn, info)| {
-2866 size_of_lost_packets += info.size as u64;
-2867 pn
-2868 })
-2869 .collect();
-2870
-2871 if !lost_pns.is_empty() {
-2872 trace!(
-2873 %path_id,
-2874 count = lost_pns.len(),
-2875 lost_bytes = size_of_lost_packets,
-2876 "packets lost on path abandon"
-2877 );
-2878 self.handle_lost_packets(
-2879 SpaceId::Data,
-2880 path_id,
-2881 now,
-2882 lost_pns,
-2883 in_flight_mtu_probe,
-2884 Duration::ZERO,
-2885 false,
-2886 size_of_lost_packets,
-2887 );
-2888 }
-2889 self.paths.remove(&path_id);
-2890 self.spaces[SpaceId::Data].number_spaces.remove(&path_id);
-2891
-2892 let path_stats = self.path_stats.remove(&path_id).unwrap_or_default();
-2893 self.events.push_back(
-2894 PathEvent::Abandoned {
-2895 id: path_id,
-2896 path_stats,
-2897 }
-2898 .into(),
-2899 );
-2900 }
-2901
-2902 fn handle_lost_packets(
-2903 &mut self,
-2904 pn_space: SpaceId,
-2905 path_id: PathId,
-2906 now: Instant,
-2907 lost_packets: Vec<u64>,
-2908 lost_mtu_probe: Option<u64>,
-2909 loss_delay: Duration,
-2910 in_persistent_congestion: bool,
-2911 size_of_lost_packets: u64,
-2912 ) {
-2913 debug_assert!(
-2914 {
-2915 let mut sorted = lost_packets.clone();
-2916 sorted.sort();
-2917 sorted == lost_packets
-2918 },
-2919 "lost_packets must be sorted"
-2920 );
-2921
-2922 self.drain_lost_packets(now, pn_space, path_id);
-2923
-2924 // OnPacketsLost
-2925 if let Some(largest_lost) = lost_packets.last().cloned() {
-2926 let old_bytes_in_flight = self.path_data_mut(path_id).in_flight.bytes;
-2927 let largest_lost_sent = self.spaces[pn_space]
-2928 .for_path(path_id)
-2929 .sent_packets
-2930 .get(largest_lost)
-2931 .unwrap()
-2932 .time_sent;
-2933 let path_stats = self.path_stats.entry(path_id).or_default();
-2934 path_stats.lost_packets += lost_packets.len() as u64;
-2935 path_stats.lost_bytes += size_of_lost_packets;
-2936 trace!(
-2937 %path_id,
-2938 count = lost_packets.len(),
-2939 lost_bytes = size_of_lost_packets,
-2940 "packets lost",
-2941 );
-2942
-2943 for &packet in &lost_packets {
-2944 let Some(info) = self.spaces[pn_space].for_path(path_id).take(packet) else {
-2945 continue;
-2946 };
-2947 self.qlog
-2948 .emit_packet_lost(packet, &info, loss_delay, pn_space, now);
-2949 self.paths
-2950 .get_mut(&path_id)
-2951 .unwrap()
-2952 .remove_in_flight(&info);
-2953
-2954 for frame in info.stream_frames {
-2955 self.streams.retransmit(frame);
-2956 }
-2957 self.spaces[pn_space].pending |= info.retransmits;
-2958 self.path_data_mut(path_id)
-2959 .mtud
-2960 .on_non_probe_lost(packet, info.size);
-2961
-2962 self.spaces[pn_space].for_path(path_id).lost_packets.insert(
-2963 packet,
-2964 LostPacket {
-2965 time_sent: info.time_sent,
-2966 },
-2967 );
-2968 }
-2969
-2970 let path = self.path_data_mut(path_id);
-2971 if path.mtud.black_hole_detected(now) {
-2972 path.congestion.on_mtu_update(path.mtud.current_mtu());
-2973 if let Some(max_datagram_size) = self.datagrams().max_size() {
-2974 if self.datagrams.drop_oversized(max_datagram_size)
-2975 && self.datagrams.send_blocked
-2976 {
-2977 self.datagrams.send_blocked = false;
-2978 self.events.push_back(Event::DatagramsUnblocked);
-2979 }
-2980 }
-2981 self.path_stats
-2982 .entry(path_id)
-2983 .or_default()
-2984 .black_holes_detected += 1;
-2985 }
+2658 // Must be called before crypto/pto_count are clobbered
+2659 self.detect_lost_packets(now, space, path, true);
+2660
+2661 if self.peer_completed_address_validation(path) {
+2662 self.path_data_mut(path).pto_count = 0;
+2663 }
+2664
+2665 // Explicit congestion notification
+2666 // TODO(@divma): this code is a good example of logic that should be contained in a single
+2667 // place but it's split between the path data and the packet number space data, we should
+2668 // find a way to make this work without two lookups
+2669 if self.path_data(path).sending_ecn {
+2670 if let Some(ecn) = ack.ecn {
+2671 // We only examine ECN counters from ACKs that we are certain we received in transmit
+2672 // order, allowing us to compute an increase in ECN counts to compare against the number
+2673 // of newly acked packets that remains well-defined in the presence of arbitrary packet
+2674 // reordering.
+2675 if new_largest {
+2676 let sent = self.spaces[space].for_path(path).largest_acked_packet_sent;
+2677 self.process_ecn(now, space, path, newly_acked.len() as u64, ecn, sent);
+2678 }
+2679 } else {
+2680 // We always start out sending ECN, so any ack that doesn't acknowledge it disables it.
+2681 debug!("ECN not acknowledged by peer");
+2682 self.path_data_mut(path).sending_ecn = false;
+2683 }
+2684 }
+2685
+2686 self.set_loss_detection_timer(now, path);
+2687 Ok(())
+2688 }
+2689
+2690 fn detect_spurious_loss(&mut self, ack: &frame::Ack, space: SpaceId, path: PathId) -> bool {
+2691 let lost_packets = &mut self.spaces[space].for_path(path).lost_packets;
+2692
+2693 if lost_packets.is_empty() {
+2694 return false;
+2695 }
+2696
+2697 for range in ack.iter() {
+2698 let spurious_losses: Vec<u64> = lost_packets
+2699 .iter_range(range.clone())
+2700 .map(|(pn, _info)| pn)
+2701 .collect();
+2702
+2703 for pn in spurious_losses {
+2704 lost_packets.remove(pn);
+2705 }
+2706 }
+2707
+2708 // If this ACK frame acknowledged all deemed lost packets,
+2709 // then we have raised a spurious congestion event in the past.
+2710 // We cannot conclude when there are remaining packets,
+2711 // but future ACK frames might indicate a spurious loss detection.
+2712 lost_packets.is_empty()
+2713 }
+2714
+2715 /// Drain lost packets that we reasonably think will never arrive
+2716 ///
+2717 /// The current criterion is copied from `msquic`:
+2718 /// discard packets that were sent earlier than 2 probe timeouts ago.
+2719 fn drain_lost_packets(&mut self, now: Instant, space: SpaceId, path: PathId) {
+2720 let two_pto = 2 * self.path_data(path).rtt.pto_base();
+2721
+2722 let lost_packets = &mut self.spaces[space].for_path(path).lost_packets;
+2723 lost_packets.retain(|_pn, info| now.saturating_duration_since(info.time_sent) <= two_pto);
+2724 }
+2725
+2726 /// Process a new ECN block from an in-order ACK
+2727 fn process_ecn(
+2728 &mut self,
+2729 now: Instant,
+2730 space: SpaceId,
+2731 path: PathId,
+2732 newly_acked: u64,
+2733 ecn: frame::EcnCounts,
+2734 largest_sent_time: Instant,
+2735 ) {
+2736 match self.spaces[space]
+2737 .for_path(path)
+2738 .detect_ecn(newly_acked, ecn)
+2739 {
+2740 Err(e) => {
+2741 debug!("halting ECN due to verification failure: {}", e);
+2742
+2743 self.path_data_mut(path).sending_ecn = false;
+2744 // Wipe out the existing value because it might be garbage and could interfere with
+2745 // future attempts to use ECN on new paths.
+2746 self.spaces[space].for_path(path).ecn_feedback = frame::EcnCounts::ZERO;
+2747 }
+2748 Ok(false) => {}
+2749 Ok(true) => {
+2750 self.path_stats.entry(path).or_default().congestion_events += 1;
+2751 self.path_data_mut(path).congestion.on_congestion_event(
+2752 now,
+2753 largest_sent_time,
+2754 false,
+2755 true,
+2756 0,
+2757 );
+2758 }
+2759 }
+2760 }
+2761
+2762 // Not timing-aware, so it's safe to call this for inferred acks, such as arise from
+2763 // high-latency handshakes
+2764 fn on_packet_acked(&mut self, now: Instant, path_id: PathId, info: SentPacket) {
+2765 self.paths
+2766 .get_mut(&path_id)
+2767 .expect("known path")
+2768 .remove_in_flight(&info);
+2769 let app_limited = self.app_limited;
+2770 let path = self.path_data_mut(path_id);
+2771 if info.ack_eliciting && !path.is_validating_path() {
+2772 // Only pass ACKs to the congestion controller if we are not validating the current
+2773 // path, so as to ignore any ACKs from older paths still coming in.
+2774 let rtt = path.rtt;
+2775 path.congestion
+2776 .on_ack(now, info.time_sent, info.size.into(), app_limited, &rtt);
+2777 }
+2778
+2779 // Update state for confirmed delivery of frames
+2780 if let Some(retransmits) = info.retransmits.get() {
+2781 for (id, _) in retransmits.reset_stream.iter() {
+2782 self.streams.reset_acked(*id);
+2783 }
+2784 }
+2785
+2786 for frame in info.stream_frames {
+2787 self.streams.received_ack_of(frame);
+2788 }
+2789 }
+2790
+2791 fn set_key_discard_timer(&mut self, now: Instant, space: SpaceId) {
+2792 let start = if self.zero_rtt_crypto.is_some() {
+2793 now
+2794 } else {
+2795 self.prev_crypto
+2796 .as_ref()
+2797 .expect("no previous keys")
+2798 .end_packet
+2799 .as_ref()
+2800 .expect("update not acknowledged yet")
+2801 .1
+2802 };
+2803
+2804 // QUIC-MULTIPATH § 2.5 Key Phase Update Process: use largest PTO off all paths.
+2805 self.timers.set(
+2806 Timer::Conn(ConnTimer::KeyDiscard),
+2807 start + self.pto_max_path(space, false) * 3,
+2808 self.qlog.with_time(now),
+2809 );
+2810 }
+2811
+2812 /// Handle a [`PathTimer::LossDetection`] timeout.
+2813 ///
+2814 /// This timer expires for two reasons:
+2815 /// - An ACK-eliciting packet we sent should be considered lost.
+2816 /// - The PTO may have expired and a tail-loss probe needs to be scheduled.
+2817 ///
+2818 /// The former needs us to schedule re-transmission of the lost data.
+2819 ///
+2820 /// The latter means we have not received an ACK for an ack-eliciting packet we sent
+2821 /// within the PTO time-window. We need to schedule a tail-loss probe, an ack-eliciting
+2822 /// packet, to try and elicit new acknowledgements. These new acknowledgements will
+2823 /// indicate whether the previously sent packets were lost or not.
+2824 fn on_loss_detection_timeout(&mut self, now: Instant, path_id: PathId) {
+2825 if let Some((_, pn_space)) = self.loss_time_and_space(path_id) {
+2826 // Time threshold loss Detection
+2827 self.detect_lost_packets(now, pn_space, path_id, false);
+2828 self.set_loss_detection_timer(now, path_id);
+2829 return;
+2830 }
+2831
+2832 let (_, space) = match self.pto_time_and_space(now, path_id) {
+2833 Some(x) => x,
+2834 None => {
+2835 error!(%path_id, "PTO expired while unset");
+2836 return;
+2837 }
+2838 };
+2839 trace!(
+2840 in_flight = self.path_data(path_id).in_flight.bytes,
+2841 count = self.path_data(path_id).pto_count,
+2842 ?space,
+2843 %path_id,
+2844 "PTO fired"
+2845 );
+2846
+2847 let count = match self.path_data(path_id).in_flight.ack_eliciting {
+2848 // A PTO when we're not expecting any ACKs must be due to handshake anti-amplification
+2849 // deadlock preventions
+2850 0 => {
+2851 debug_assert!(!self.peer_completed_address_validation(path_id));
+2852 1
+2853 }
+2854 // Conventional loss probe
+2855 _ => 2,
+2856 };
+2857 let pns = self.spaces[space].for_path(path_id);
+2858 pns.loss_probes = pns.loss_probes.saturating_add(count);
+2859 let path_data = self.path_data_mut(path_id);
+2860 path_data.pto_count = path_data.pto_count.saturating_add(1);
+2861 self.set_loss_detection_timer(now, path_id);
+2862 }
+2863
+2864 /// Detect any lost packets
+2865 ///
+2866 /// There are two cases in which we detects lost packets:
+2867 ///
+2868 /// - We received an ACK packet.
+2869 /// - The [`PathTimer::LossDetection`] timer expired. So there is an un-acknowledged packet
+2870 /// that was followed by an acknowledged packet. The loss timer for this
+2871 /// un-acknowledged packet expired and we need to detect that packet as lost.
+2872 ///
+2873 /// Packets are lost if they are both (See RFC9002 §6.1):
+2874 ///
+2875 /// - Unacknowledged, in flight and sent prior to an acknowledged packet.
+2876 /// - Old enough by either:
+2877 /// - Having a packet number [`TransportConfig::packet_threshold`] lower then the last
+2878 /// acknowledged packet.
+2879 /// - Being sent [`TransportConfig::time_threshold`] * RTT in the past.
+2880 fn detect_lost_packets(
+2881 &mut self,
+2882 now: Instant,
+2883 pn_space: SpaceId,
+2884 path_id: PathId,
+2885 due_to_ack: bool,
+2886 ) {
+2887 let mut lost_packets = Vec::<u64>::new();
+2888 let mut lost_mtu_probe = None;
+2889 let mut in_persistent_congestion = false;
+2890 let mut size_of_lost_packets = 0u64;
+2891 self.spaces[pn_space].for_path(path_id).loss_time = None;
+2892
+2893 // Find all the lost packets, populating all variables initialised above.
+2894
+2895 let path = self.path_data(path_id);
+2896 let in_flight_mtu_probe = path.mtud.in_flight_mtu_probe();
+2897 let loss_delay = path
+2898 .rtt
+2899 .conservative()
+2900 .mul_f32(self.config.time_threshold)
+2901 .max(TIMER_GRANULARITY);
+2902 let first_packet_after_rtt_sample = path.first_packet_after_rtt_sample;
+2903
+2904 let largest_acked_packet = self.spaces[pn_space]
+2905 .for_path(path_id)
+2906 .largest_acked_packet
+2907 .expect("detect_lost_packets only to be called if path received at least one ACK");
+2908 let packet_threshold = self.config.packet_threshold as u64;
+2909
+2910 // InPersistentCongestion: Determine if all packets in the time period before the newest
+2911 // lost packet, including the edges, are marked lost. PTO computation must always
+2912 // include max ACK delay, i.e. operate as if in Data space (see RFC9001 §7.6.1).
+2913 let congestion_period = self
+2914 .pto(SpaceId::Data, path_id)
+2915 .saturating_mul(self.config.persistent_congestion_threshold);
+2916 let mut persistent_congestion_start: Option<Instant> = None;
+2917 let mut prev_packet = None;
+2918 let space = self.spaces[pn_space].for_path(path_id);
+2919
+2920 for (packet, info) in space.sent_packets.iter_range(0..largest_acked_packet) {
+2921 if prev_packet != Some(packet.wrapping_sub(1)) {
+2922 // An intervening packet was acknowledged
+2923 persistent_congestion_start = None;
+2924 }
+2925
+2926 // Packets sent before now - loss_delay are deemed lost.
+2927 // However, we avoid subtraction as it can panic and there's no
+2928 // saturating equivalent of this subtraction operation with a Duration.
+2929 let packet_too_old = now.saturating_duration_since(info.time_sent) >= loss_delay;
+2930 if packet_too_old || largest_acked_packet >= packet + packet_threshold {
+2931 // The packet should be declared lost.
+2932 if Some(packet) == in_flight_mtu_probe {
+2933 // Lost MTU probes are not included in `lost_packets`, because they
+2934 // should not trigger a congestion control response
+2935 lost_mtu_probe = in_flight_mtu_probe;
+2936 } else {
+2937 lost_packets.push(packet);
+2938 size_of_lost_packets += info.size as u64;
+2939 if info.ack_eliciting && due_to_ack {
+2940 match persistent_congestion_start {
+2941 // Two ACK-eliciting packets lost more than
+2942 // congestion_period apart, with no ACKed packets in between
+2943 Some(start) if info.time_sent - start > congestion_period => {
+2944 in_persistent_congestion = true;
+2945 }
+2946 // Persistent congestion must start after the first RTT sample
+2947 None if first_packet_after_rtt_sample
+2948 .is_some_and(|x| x < (pn_space, packet)) =>
+2949 {
+2950 persistent_congestion_start = Some(info.time_sent);
+2951 }
+2952 _ => {}
+2953 }
+2954 }
+2955 }
+2956 } else {
+2957 // The packet should not yet be declared lost.
+2958 if space.loss_time.is_none() {
+2959 // Since we iterate in order the lowest packet number's loss time will
+2960 // always be the earliest.
+2961 space.loss_time = Some(info.time_sent + loss_delay);
+2962 }
+2963 persistent_congestion_start = None;
+2964 }
+2965
+2966 prev_packet = Some(packet);
+2967 }
+2968
+2969 self.handle_lost_packets(
+2970 pn_space,
+2971 path_id,
+2972 now,
+2973 lost_packets,
+2974 lost_mtu_probe,
+2975 loss_delay,
+2976 in_persistent_congestion,
+2977 size_of_lost_packets,
+2978 );
+2979 }
+2980
+2981 /// Drops the path state, declaring any remaining in-flight packets as lost
+2982 fn discard_path(&mut self, path_id: PathId, now: Instant) {
+2983 trace!(%path_id, "dropping path state");
+2984 let path = self.path_data(path_id);
+2985 let in_flight_mtu_probe = path.mtud.in_flight_mtu_probe();
2986
-2987 // Don't apply congestion penalty for lost ack-only packets
-2988 let lost_ack_eliciting =
-2989 old_bytes_in_flight != self.path_data_mut(path_id).in_flight.bytes;
-2990
-2991 if lost_ack_eliciting {
-2992 self.path_stats
-2993 .entry(path_id)
-2994 .or_default()
-2995 .congestion_events += 1;
-2996 self.path_data_mut(path_id).congestion.on_congestion_event(
-2997 now,
-2998 largest_lost_sent,
-2999 in_persistent_congestion,
-3000 false,
-3001 size_of_lost_packets,
-3002 );
-3003 }
-3004 }
-3005
-3006 // Handle a lost MTU probe
-3007 if let Some(packet) = lost_mtu_probe {
-3008 let info = self.spaces[SpaceId::Data]
-3009 .for_path(path_id)
-3010 .take(packet)
-3011 .unwrap(); // safe: lost_mtu_probe is omitted from lost_packets, and
-3012 // therefore must not have been removed yet
-3013 self.paths
-3014 .get_mut(&path_id)
-3015 .unwrap()
-3016 .remove_in_flight(&info);
-3017 self.path_data_mut(path_id).mtud.on_probe_lost();
-3018 self.path_stats
-3019 .entry(path_id)
-3020 .or_default()
-3021 .lost_plpmtud_probes += 1;
-3022 }
-3023 }
-3024
-3025 /// Returns the earliest time packets should be declared lost for all spaces on a path.
-3026 ///
-3027 /// If a path has an acknowledged packet with any prior un-acknowledged packets, the
-3028 /// earliest un-acknowledged packet can be declared lost after a timeout has elapsed.
-3029 /// The time returned is when this packet should be declared lost.
-3030 fn loss_time_and_space(&self, path_id: PathId) -> Option<(Instant, SpaceId)> {
-3031 SpaceId::iter()
-3032 .filter_map(|id| {
-3033 self.spaces[id]
-3034 .number_spaces
-3035 .get(&path_id)
-3036 .and_then(|pns| pns.loss_time)
-3037 .map(|time| (time, id))
-3038 })
-3039 .min_by_key(|&(time, _)| time)
-3040 }
-3041
-3042 /// Returns the earliest next PTO should fire for all spaces on a path.
-3043 fn pto_time_and_space(&mut self, now: Instant, path_id: PathId) -> Option<(Instant, SpaceId)> {
-3044 let path = self.path(path_id)?;
-3045 let pto_count = path.pto_count;
-3046 let backoff = 2u32.pow(pto_count.min(MAX_BACKOFF_EXPONENT));
-3047 let mut duration = path.rtt.pto_base() * backoff;
-3048
-3049 if path_id == PathId::ZERO
-3050 && path.in_flight.ack_eliciting == 0
-3051 && !self.peer_completed_address_validation(PathId::ZERO)
-3052 {
-3053 // Address Validation during Connection Establishment:
-3054 // https://www.rfc-editor.org/rfc/rfc9000.html#section-8.1. To prevent a
-3055 // deadlock if an Initial or Handshake packet from the server is lost and the
-3056 // server can not send more due to its anti-amplification limit the client must
-3057 // send another packet on PTO.
-3058 let space = match self.highest_space {
-3059 SpaceId::Handshake => SpaceId::Handshake,
-3060 _ => SpaceId::Initial,
-3061 };
-3062
-3063 return Some((now + duration, space));
-3064 }
-3065
-3066 let mut result = None;
-3067 for space in SpaceId::iter() {
-3068 let Some(pns) = self.spaces[space].number_spaces.get(&path_id) else {
-3069 continue;
-3070 };
-3071
-3072 if !pns.has_in_flight() {
-3073 continue;
-3074 }
-3075 if space == SpaceId::Data {
-3076 // Skip ApplicationData until handshake completes.
-3077 if self.is_handshaking() {
-3078 return result;
-3079 }
-3080 // Include max_ack_delay and backoff for ApplicationData.
-3081 duration += self.ack_frequency.max_ack_delay_for_pto() * backoff;
-3082 }
-3083 let Some(last_ack_eliciting) = pns.time_of_last_ack_eliciting_packet else {
-3084 continue;
-3085 };
-3086 let pto = last_ack_eliciting + duration;
-3087 if result.is_none_or(|(earliest_pto, _)| pto < earliest_pto) {
-3088 if path.anti_amplification_blocked(1) {
-3089 // Nothing would be able to be sent.
-3090 continue;
-3091 }
-3092 if path.in_flight.ack_eliciting == 0 {
-3093 // Nothing ack-eliciting, no PTO to arm/fire.
-3094 continue;
-3095 }
-3096 result = Some((pto, space));
-3097 }
-3098 }
-3099 result
-3100 }
-3101
-3102 fn peer_completed_address_validation(&self, path: PathId) -> bool {
-3103 // TODO(flub): This logic needs updating for multipath
-3104 if self.side.is_server() || self.state.is_closed() {
-3105 return true;
-3106 }
-3107 // The server is guaranteed to have validated our address if any of our handshake or 1-RTT
-3108 // packets are acknowledged or we've seen HANDSHAKE_DONE and discarded handshake keys.
-3109 self.spaces[SpaceId::Handshake]
-3110 .path_space(PathId::ZERO)
-3111 .and_then(|pns| pns.largest_acked_packet)
-3112 .is_some()
-3113 || self.spaces[SpaceId::Data]
-3114 .path_space(path)
-3115 .and_then(|pns| pns.largest_acked_packet)
-3116 .is_some()
-3117 || (self.spaces[SpaceId::Data].crypto.is_some()
-3118 && self.spaces[SpaceId::Handshake].crypto.is_none())
-3119 }
-3120
-3121 /// Resets the the [`PathTimer::LossDetection`] timer to the next instant it may be needed
-3122 ///
-3123 /// The timer must fire if either:
-3124 /// - An ack-eliciting packet we sent needs to be declared lost.
-3125 /// - A tail-loss probe needs to be sent.
-3126 ///
-3127 /// See [`Connection::on_loss_detection_timeout`] for details.
-3128 fn set_loss_detection_timer(&mut self, now: Instant, path_id: PathId) {
-3129 if self.state.is_closed() {
-3130 // No loss detection takes place on closed connections, and `close_common` already
-3131 // stopped time timer. Ensure we don't restart it inadvertently, e.g. in response to a
-3132 // reordered packet being handled by state-insensitive code.
-3133 return;
-3134 }
-3135
-3136 if let Some((loss_time, _)) = self.loss_time_and_space(path_id) {
-3137 // Time threshold loss detection.
-3138 self.timers.set(
-3139 Timer::PerPath(path_id, PathTimer::LossDetection),
-3140 loss_time,
-3141 self.qlog.with_time(now),
-3142 );
-3143 return;
-3144 }
-3145
-3146 // Determine which PN space to arm PTO for.
-3147 // Calculate PTO duration
-3148 if let Some((timeout, _)) = self.pto_time_and_space(now, path_id) {
-3149 self.timers.set(
-3150 Timer::PerPath(path_id, PathTimer::LossDetection),
-3151 timeout,
-3152 self.qlog.with_time(now),
-3153 );
-3154 } else {
-3155 self.timers.stop(
-3156 Timer::PerPath(path_id, PathTimer::LossDetection),
-3157 self.qlog.with_time(now),
-3158 );
-3159 }
-3160 }
-3161
-3162 /// The maximum probe timeout across all paths
-3163 ///
-3164 /// If `is_closing` is set to `true` it will filter out paths that have not yet been used.
-3165 ///
-3166 /// See [`Connection::pto`]
-3167 fn pto_max_path(&self, space: SpaceId, is_closing: bool) -> Duration {
-3168 match space {
-3169 SpaceId::Initial | SpaceId::Handshake => self.pto(space, PathId::ZERO),
-3170 SpaceId::Data => self
-3171 .paths
-3172 .iter()
-3173 .filter_map(|(path_id, state)| {
-3174 if is_closing && state.data.total_sent == 0 && state.data.total_recvd == 0 {
-3175 // If we are closing and haven't sent anything yet, do not include
-3176 None
-3177 } else {
-3178 let pto = self.pto(space, *path_id);
-3179 Some(pto)
-3180 }
-3181 })
-3182 .max()
-3183 .expect("there should be at least one path"),
-3184 }
-3185 }
-3186
-3187 /// Probe Timeout
-3188 ///
-3189 /// The PTO is logically the time in which you'd expect to receive an acknowledgement
-3190 /// for a packet. So approximately RTT + max_ack_delay.
-3191 fn pto(&self, space: SpaceId, path_id: PathId) -> Duration {
-3192 let max_ack_delay = match space {
-3193 SpaceId::Initial | SpaceId::Handshake => Duration::ZERO,
-3194 SpaceId::Data => self.ack_frequency.max_ack_delay_for_pto(),
-3195 };
-3196 self.path_data(path_id).rtt.pto_base() + max_ack_delay
-3197 }
-3198
-3199 fn on_packet_authenticated(
-3200 &mut self,
-3201 now: Instant,
-3202 space_id: SpaceId,
-3203 path_id: PathId,
-3204 ecn: Option<EcnCodepoint>,
-3205 packet: Option<u64>,
-3206 spin: bool,
-3207 is_1rtt: bool,
-3208 ) {
-3209 self.total_authed_packets += 1;
-3210 if let Some(AbandonState::ExpectingPathAbandon { deadline }) = self
-3211 .paths
-3212 .get(&path_id)
-3213 .map(|path| &path.data.abandon_state)
-3214 {
-3215 if now > *deadline {
-3216 warn!("received data on path which we abandoned more than 3 * PTO ago");
-3217 // The peer failed to respond with a PATH_ABANDON in time.
-3218 if !self.state.is_closed() {
-3219 self.state
-3220 .move_to_closed(TransportError::PROTOCOL_VIOLATION(
-3221 "peer failed to respond with PATH_ABANDON in time",
-3222 ));
-3223 self.close_common();
-3224 self.set_close_timer(now);
-3225 self.close = true;
-3226 }
-3227 return;
-3228 }
-3229 }
-3230
-3231 self.reset_keep_alive(path_id, now);
-3232 self.reset_idle_timeout(now, space_id, path_id);
-3233 self.permit_idle_reset = true;
-3234 self.receiving_ecn |= ecn.is_some();
-3235 if let Some(x) = ecn {
-3236 let space = &mut self.spaces[space_id];
-3237 space.for_path(path_id).ecn_counters += x;
-3238
-3239 if x.is_ce() {
-3240 space
-3241 .for_path(path_id)
-3242 .pending_acks
-3243 .set_immediate_ack_required();
-3244 }
-3245 }
-3246
-3247 let packet = match packet {
-3248 Some(x) => x,
-3249 None => return,
-3250 };
-3251 match &self.side {
-3252 ConnectionSide::Client { .. } => {
-3253 // If we received a handshake packet that authenticated, then we're talking to
-3254 // the real server. From now on we should no longer allow the server to migrate
-3255 // its address.
-3256 if space_id == SpaceId::Handshake {
-3257 if let Some(hs) = self.state.as_handshake_mut() {
-3258 hs.allow_server_migration = false;
-3259 }
-3260 }
-3261 }
-3262 ConnectionSide::Server { .. } => {
-3263 if self.spaces[SpaceId::Initial].crypto.is_some() && space_id == SpaceId::Handshake
-3264 {
-3265 // A server stops sending and processing Initial packets when it receives its first Handshake packet.
-3266 self.discard_space(now, SpaceId::Initial);
-3267 }
-3268 if self.zero_rtt_crypto.is_some() && is_1rtt {
-3269 // Discard 0-RTT keys soon after receiving a 1-RTT packet
-3270 self.set_key_discard_timer(now, space_id)
-3271 }
-3272 }
-3273 }
-3274 let space = self.spaces[space_id].for_path(path_id);
-3275 space.pending_acks.insert_one(packet, now);
-3276 if packet >= space.rx_packet.unwrap_or_default() {
-3277 space.rx_packet = Some(packet);
-3278 // Update outgoing spin bit, inverting iff we're the client
-3279 self.spin = self.side.is_client() ^ spin;
-3280 }
-3281 }
-3282
-3283 /// Resets the idle timeout timers
-3284 ///
-3285 /// Without multipath there is only the connection-wide idle timeout. When multipath is
-3286 /// enabled there is an additional per-path idle timeout.
-3287 fn reset_idle_timeout(&mut self, now: Instant, space: SpaceId, path_id: PathId) {
-3288 // First reset the global idle timeout.
-3289 if let Some(timeout) = self.idle_timeout {
-3290 if self.state.is_closed() {
-3291 self.timers
-3292 .stop(Timer::Conn(ConnTimer::Idle), self.qlog.with_time(now));
-3293 } else {
-3294 let dt = cmp::max(timeout, 3 * self.pto_max_path(space, false));
-3295 self.timers.set(
-3296 Timer::Conn(ConnTimer::Idle),
-3297 now + dt,
-3298 self.qlog.with_time(now),
-3299 );
-3300 }
-3301 }
-3302
-3303 // Now handle the per-path state
-3304 if let Some(timeout) = self.path_data(path_id).idle_timeout {
-3305 if self.state.is_closed() {
-3306 self.timers.stop(
-3307 Timer::PerPath(path_id, PathTimer::PathIdle),
-3308 self.qlog.with_time(now),
-3309 );
-3310 } else {
-3311 let dt = cmp::max(timeout, 3 * self.pto(space, path_id));
-3312 self.timers.set(
-3313 Timer::PerPath(path_id, PathTimer::PathIdle),
-3314 now + dt,
-3315 self.qlog.with_time(now),
-3316 );
-3317 }
-3318 }
-3319 }
-3320
-3321 /// Resets both the [`ConnTimer::KeepAlive`] and [`PathTimer::PathKeepAlive`] timers
-3322 fn reset_keep_alive(&mut self, path_id: PathId, now: Instant) {
-3323 if !self.state.is_established() {
-3324 return;
-3325 }
+2987 let mut size_of_lost_packets = 0u64; // add to path_stats.lost_bytes;
+2988 let lost_pns: Vec<_> = self.spaces[SpaceId::Data]
+2989 .for_path(path_id)
+2990 .sent_packets
+2991 .iter()
+2992 .filter(|(pn, _info)| Some(*pn) != in_flight_mtu_probe)
+2993 .map(|(pn, info)| {
+2994 size_of_lost_packets += info.size as u64;
+2995 pn
+2996 })
+2997 .collect();
+2998
+2999 if !lost_pns.is_empty() {
+3000 trace!(
+3001 %path_id,
+3002 count = lost_pns.len(),
+3003 lost_bytes = size_of_lost_packets,
+3004 "packets lost on path abandon"
+3005 );
+3006 self.handle_lost_packets(
+3007 SpaceId::Data,
+3008 path_id,
+3009 now,
+3010 lost_pns,
+3011 in_flight_mtu_probe,
+3012 Duration::ZERO,
+3013 false,
+3014 size_of_lost_packets,
+3015 );
+3016 }
+3017 self.paths.remove(&path_id);
+3018 self.spaces[SpaceId::Data].number_spaces.remove(&path_id);
+3019
+3020 let path_stats = self.path_stats.remove(&path_id).unwrap_or_default();
+3021 self.events.push_back(
+3022 PathEvent::Abandoned {
+3023 id: path_id,
+3024 path_stats,
+3025 }
+3026 .into(),
+3027 );
+3028 }
+3029
+3030 fn handle_lost_packets(
+3031 &mut self,
+3032 pn_space: SpaceId,
+3033 path_id: PathId,
+3034 now: Instant,
+3035 lost_packets: Vec<u64>,
+3036 lost_mtu_probe: Option<u64>,
+3037 loss_delay: Duration,
+3038 in_persistent_congestion: bool,
+3039 size_of_lost_packets: u64,
+3040 ) {
+3041 debug_assert!(
+3042 {
+3043 let mut sorted = lost_packets.clone();
+3044 sorted.sort();
+3045 sorted == lost_packets
+3046 },
+3047 "lost_packets must be sorted"
+3048 );
+3049
+3050 self.drain_lost_packets(now, pn_space, path_id);
+3051
+3052 // OnPacketsLost
+3053 if let Some(largest_lost) = lost_packets.last().cloned() {
+3054 let old_bytes_in_flight = self.path_data_mut(path_id).in_flight.bytes;
+3055 let largest_lost_sent = self.spaces[pn_space]
+3056 .for_path(path_id)
+3057 .sent_packets
+3058 .get(largest_lost)
+3059 .unwrap()
+3060 .time_sent;
+3061 let path_stats = self.path_stats.entry(path_id).or_default();
+3062 path_stats.lost_packets += lost_packets.len() as u64;
+3063 path_stats.lost_bytes += size_of_lost_packets;
+3064 trace!(
+3065 %path_id,
+3066 count = lost_packets.len(),
+3067 lost_bytes = size_of_lost_packets,
+3068 "packets lost",
+3069 );
+3070
+3071 for &packet in &lost_packets {
+3072 let Some(info) = self.spaces[pn_space].for_path(path_id).take(packet) else {
+3073 continue;
+3074 };
+3075 self.qlog
+3076 .emit_packet_lost(packet, &info, loss_delay, pn_space, now);
+3077 self.paths
+3078 .get_mut(&path_id)
+3079 .unwrap()
+3080 .remove_in_flight(&info);
+3081
+3082 for frame in info.stream_frames {
+3083 self.streams.retransmit(frame);
+3084 }
+3085 self.spaces[pn_space].pending |= info.retransmits;
+3086 self.path_data_mut(path_id)
+3087 .mtud
+3088 .on_non_probe_lost(packet, info.size);
+3089
+3090 self.spaces[pn_space].for_path(path_id).lost_packets.insert(
+3091 packet,
+3092 LostPacket {
+3093 time_sent: info.time_sent,
+3094 },
+3095 );
+3096 }
+3097
+3098 let path = self.path_data_mut(path_id);
+3099 if path.mtud.black_hole_detected(now) {
+3100 path.congestion.on_mtu_update(path.mtud.current_mtu());
+3101 if let Some(max_datagram_size) = self.datagrams().max_size() {
+3102 if self.datagrams.drop_oversized(max_datagram_size)
+3103 && self.datagrams.send_blocked
+3104 {
+3105 self.datagrams.send_blocked = false;
+3106 self.events.push_back(Event::DatagramsUnblocked);
+3107 }
+3108 }
+3109 self.path_stats
+3110 .entry(path_id)
+3111 .or_default()
+3112 .black_holes_detected += 1;
+3113 }
+3114
+3115 // Don't apply congestion penalty for lost ack-only packets
+3116 let lost_ack_eliciting =
+3117 old_bytes_in_flight != self.path_data_mut(path_id).in_flight.bytes;
+3118
+3119 if lost_ack_eliciting {
+3120 self.path_stats
+3121 .entry(path_id)
+3122 .or_default()
+3123 .congestion_events += 1;
+3124 self.path_data_mut(path_id).congestion.on_congestion_event(
+3125 now,
+3126 largest_lost_sent,
+3127 in_persistent_congestion,
+3128 false,
+3129 size_of_lost_packets,
+3130 );
+3131 }
+3132 }
+3133
+3134 // Handle a lost MTU probe
+3135 if let Some(packet) = lost_mtu_probe {
+3136 let info = self.spaces[SpaceId::Data]
+3137 .for_path(path_id)
+3138 .take(packet)
+3139 .unwrap(); // safe: lost_mtu_probe is omitted from lost_packets, and
+3140 // therefore must not have been removed yet
+3141 self.paths
+3142 .get_mut(&path_id)
+3143 .unwrap()
+3144 .remove_in_flight(&info);
+3145 self.path_data_mut(path_id).mtud.on_probe_lost();
+3146 self.path_stats
+3147 .entry(path_id)
+3148 .or_default()
+3149 .lost_plpmtud_probes += 1;
+3150 }
+3151 }
+3152
+3153 /// Returns the earliest time packets should be declared lost for all spaces on a path.
+3154 ///
+3155 /// If a path has an acknowledged packet with any prior un-acknowledged packets, the
+3156 /// earliest un-acknowledged packet can be declared lost after a timeout has elapsed.
+3157 /// The time returned is when this packet should be declared lost.
+3158 fn loss_time_and_space(&self, path_id: PathId) -> Option<(Instant, SpaceId)> {
+3159 SpaceId::iter()
+3160 .filter_map(|id| {
+3161 self.spaces[id]
+3162 .number_spaces
+3163 .get(&path_id)
+3164 .and_then(|pns| pns.loss_time)
+3165 .map(|time| (time, id))
+3166 })
+3167 .min_by_key(|&(time, _)| time)
+3168 }
+3169
+3170 /// Returns the earliest next PTO should fire for all spaces on a path.
+3171 fn pto_time_and_space(&mut self, now: Instant, path_id: PathId) -> Option<(Instant, SpaceId)> {
+3172 let path = self.path(path_id)?;
+3173 let pto_count = path.pto_count;
+3174 let backoff = 2u32.pow(pto_count.min(MAX_BACKOFF_EXPONENT));
+3175 let mut duration = path.rtt.pto_base() * backoff;
+3176
+3177 if path_id == PathId::ZERO
+3178 && path.in_flight.ack_eliciting == 0
+3179 && !self.peer_completed_address_validation(PathId::ZERO)
+3180 {
+3181 // Address Validation during Connection Establishment:
+3182 // https://www.rfc-editor.org/rfc/rfc9000.html#section-8.1. To prevent a
+3183 // deadlock if an Initial or Handshake packet from the server is lost and the
+3184 // server can not send more due to its anti-amplification limit the client must
+3185 // send another packet on PTO.
+3186 let space = match self.highest_space {
+3187 SpaceId::Handshake => SpaceId::Handshake,
+3188 _ => SpaceId::Initial,
+3189 };
+3190
+3191 return Some((now + duration, space));
+3192 }
+3193
+3194 let mut result = None;
+3195 for space in SpaceId::iter() {
+3196 let Some(pns) = self.spaces[space].number_spaces.get(&path_id) else {
+3197 continue;
+3198 };
+3199
+3200 if !pns.has_in_flight() {
+3201 continue;
+3202 }
+3203 if space == SpaceId::Data {
+3204 // Skip ApplicationData until handshake completes.
+3205 if self.is_handshaking() {
+3206 return result;
+3207 }
+3208 // Include max_ack_delay and backoff for ApplicationData.
+3209 duration += self.ack_frequency.max_ack_delay_for_pto() * backoff;
+3210 }
+3211 let Some(last_ack_eliciting) = pns.time_of_last_ack_eliciting_packet else {
+3212 continue;
+3213 };
+3214 let pto = last_ack_eliciting + duration;
+3215 if result.is_none_or(|(earliest_pto, _)| pto < earliest_pto) {
+3216 if path.anti_amplification_blocked(1) {
+3217 // Nothing would be able to be sent.
+3218 continue;
+3219 }
+3220 if path.in_flight.ack_eliciting == 0 {
+3221 // Nothing ack-eliciting, no PTO to arm/fire.
+3222 continue;
+3223 }
+3224 result = Some((pto, space));
+3225 }
+3226 }
+3227 result
+3228 }
+3229
+3230 fn peer_completed_address_validation(&self, path: PathId) -> bool {
+3231 // TODO(flub): This logic needs updating for multipath
+3232 if self.side.is_server() || self.state.is_closed() {
+3233 return true;
+3234 }
+3235 // The server is guaranteed to have validated our address if any of our handshake or 1-RTT
+3236 // packets are acknowledged or we've seen HANDSHAKE_DONE and discarded handshake keys.
+3237 self.spaces[SpaceId::Handshake]
+3238 .path_space(PathId::ZERO)
+3239 .and_then(|pns| pns.largest_acked_packet)
+3240 .is_some()
+3241 || self.spaces[SpaceId::Data]
+3242 .path_space(path)
+3243 .and_then(|pns| pns.largest_acked_packet)
+3244 .is_some()
+3245 || (self.spaces[SpaceId::Data].crypto.is_some()
+3246 && self.spaces[SpaceId::Handshake].crypto.is_none())
+3247 }
+3248
+3249 /// Resets the the [`PathTimer::LossDetection`] timer to the next instant it may be needed
+3250 ///
+3251 /// The timer must fire if either:
+3252 /// - An ack-eliciting packet we sent needs to be declared lost.
+3253 /// - A tail-loss probe needs to be sent.
+3254 ///
+3255 /// See [`Connection::on_loss_detection_timeout`] for details.
+3256 fn set_loss_detection_timer(&mut self, now: Instant, path_id: PathId) {
+3257 if self.state.is_closed() {
+3258 // No loss detection takes place on closed connections, and `close_common` already
+3259 // stopped time timer. Ensure we don't restart it inadvertently, e.g. in response to a
+3260 // reordered packet being handled by state-insensitive code.
+3261 return;
+3262 }
+3263
+3264 if let Some((loss_time, _)) = self.loss_time_and_space(path_id) {
+3265 // Time threshold loss detection.
+3266 self.timers.set(
+3267 Timer::PerPath(path_id, PathTimer::LossDetection),
+3268 loss_time,
+3269 self.qlog.with_time(now),
+3270 );
+3271 return;
+3272 }
+3273
+3274 // Determine which PN space to arm PTO for.
+3275 // Calculate PTO duration
+3276 if let Some((timeout, _)) = self.pto_time_and_space(now, path_id) {
+3277 self.timers.set(
+3278 Timer::PerPath(path_id, PathTimer::LossDetection),
+3279 timeout,
+3280 self.qlog.with_time(now),
+3281 );
+3282 } else {
+3283 self.timers.stop(
+3284 Timer::PerPath(path_id, PathTimer::LossDetection),
+3285 self.qlog.with_time(now),
+3286 );
+3287 }
+3288 }
+3289
+3290 /// The maximum probe timeout across all paths
+3291 ///
+3292 /// If `is_closing` is set to `true` it will filter out paths that have not yet been used.
+3293 ///
+3294 /// See [`Connection::pto`]
+3295 fn pto_max_path(&self, space: SpaceId, is_closing: bool) -> Duration {
+3296 match space {
+3297 SpaceId::Initial | SpaceId::Handshake => self.pto(space, PathId::ZERO),
+3298 SpaceId::Data => self
+3299 .paths
+3300 .iter()
+3301 .filter_map(|(path_id, state)| {
+3302 if is_closing && state.data.total_sent == 0 && state.data.total_recvd == 0 {
+3303 // If we are closing and haven't sent anything yet, do not include
+3304 None
+3305 } else {
+3306 let pto = self.pto(space, *path_id);
+3307 Some(pto)
+3308 }
+3309 })
+3310 .max()
+3311 .expect("there should be at least one path"),
+3312 }
+3313 }
+3314
+3315 /// Probe Timeout
+3316 ///
+3317 /// The PTO is logically the time in which you'd expect to receive an acknowledgement
+3318 /// for a packet. So approximately RTT + max_ack_delay.
+3319 fn pto(&self, space: SpaceId, path_id: PathId) -> Duration {
+3320 let max_ack_delay = match space {
+3321 SpaceId::Initial | SpaceId::Handshake => Duration::ZERO,
+3322 SpaceId::Data => self.ack_frequency.max_ack_delay_for_pto(),
+3323 };
+3324 self.path_data(path_id).rtt.pto_base() + max_ack_delay
+3325 }
3326
-3327 if let Some(interval) = self.config.keep_alive_interval {
-3328 self.timers.set(
-3329 Timer::Conn(ConnTimer::KeepAlive),
-3330 now + interval,
-3331 self.qlog.with_time(now),
-3332 );
-3333 }
-3334
-3335 if let Some(interval) = self.path_data(path_id).keep_alive {
-3336 self.timers.set(
-3337 Timer::PerPath(path_id, PathTimer::PathKeepAlive),
-3338 now + interval,
-3339 self.qlog.with_time(now),
-3340 );
-3341 }
-3342 }
-3343
-3344 /// Sets the timer for when a previously issued CID should be retired next
-3345 fn reset_cid_retirement(&mut self, now: Instant) {
-3346 if let Some((_path, t)) = self.next_cid_retirement() {
-3347 self.timers.set(
-3348 Timer::Conn(ConnTimer::PushNewCid),
-3349 t,
-3350 self.qlog.with_time(now),
-3351 );
-3352 }
-3353 }
-3354
-3355 /// The next time when a previously issued CID should be retired
-3356 fn next_cid_retirement(&self) -> Option<(PathId, Instant)> {
-3357 self.local_cid_state
-3358 .iter()
-3359 .filter_map(|(path_id, cid_state)| cid_state.next_timeout().map(|t| (*path_id, t)))
-3360 .min_by_key(|(_path_id, timeout)| *timeout)
-3361 }
-3362
-3363 /// Handle the already-decrypted first packet from the client
-3364 ///
-3365 /// Decrypting the first packet in the `Endpoint` allows stateless packet handling to be more
-3366 /// efficient.
-3367 pub(crate) fn handle_first_packet(
-3368 &mut self,
-3369 now: Instant,
-3370 network_path: FourTuple,
-3371 ecn: Option<EcnCodepoint>,
-3372 packet_number: u64,
-3373 packet: InitialPacket,
-3374 remaining: Option<BytesMut>,
-3375 ) -> Result<(), ConnectionError> {
-3376 let span = trace_span!("first recv");
-3377 let _guard = span.enter();
-3378 debug_assert!(self.side.is_server());
-3379 let len = packet.header_data.len() + packet.payload.len();
-3380 let path_id = PathId::ZERO;
-3381 self.path_data_mut(path_id).total_recvd = len as u64;
-3382
-3383 if let Some(hs) = self.state.as_handshake_mut() {
-3384 hs.expected_token = packet.header.token.clone();
-3385 } else {
-3386 unreachable!("first packet must be delivered in Handshake state");
-3387 }
-3388
-3389 // The first packet is always on PathId::ZERO
-3390 self.on_packet_authenticated(
-3391 now,
-3392 SpaceId::Initial,
-3393 path_id,
-3394 ecn,
-3395 Some(packet_number),
-3396 false,
-3397 false,
-3398 );
-3399
-3400 let packet: Packet = packet.into();
-3401
-3402 let mut qlog = QlogRecvPacket::new(len);
-3403 qlog.header(&packet.header, Some(packet_number), path_id);
-3404
-3405 self.process_decrypted_packet(
-3406 now,
-3407 network_path,
-3408 path_id,
-3409 Some(packet_number),
-3410 packet,
-3411 &mut qlog,
-3412 )?;
-3413 self.qlog.emit_packet_received(qlog, now);
-3414 if let Some(data) = remaining {
-3415 self.handle_coalesced(now, network_path, path_id, ecn, data);
-3416 }
-3417
-3418 self.qlog.emit_recovery_metrics(
-3419 path_id,
-3420 &mut self.paths.get_mut(&path_id).unwrap().data,
-3421 now,
-3422 );
-3423
-3424 Ok(())
-3425 }
-3426
-3427 fn init_0rtt(&mut self, now: Instant) {
-3428 let (header, packet) = match self.crypto.early_crypto() {
-3429 Some(x) => x,
-3430 None => return,
-3431 };
-3432 if self.side.is_client() {
-3433 match self.crypto.transport_parameters() {
-3434 Ok(params) => {
-3435 let params = params
-3436 .expect("crypto layer didn't supply transport parameters with ticket");
-3437 // Certain values must not be cached
-3438 let params = TransportParameters {
-3439 initial_src_cid: None,
-3440 original_dst_cid: None,
-3441 preferred_address: None,
-3442 retry_src_cid: None,
-3443 stateless_reset_token: None,
-3444 min_ack_delay: None,
-3445 ack_delay_exponent: TransportParameters::default().ack_delay_exponent,
-3446 max_ack_delay: TransportParameters::default().max_ack_delay,
-3447 initial_max_path_id: None,
-3448 ..params
-3449 };
-3450 self.set_peer_params(params);
-3451 self.qlog.emit_peer_transport_params_restored(self, now);
-3452 }
-3453 Err(e) => {
-3454 error!("session ticket has malformed transport parameters: {}", e);
-3455 return;
-3456 }
-3457 }
-3458 }
-3459 trace!("0-RTT enabled");
-3460 self.zero_rtt_enabled = true;
-3461 self.zero_rtt_crypto = Some(ZeroRttCrypto { header, packet });
-3462 }
-3463
-3464 fn read_crypto(
-3465 &mut self,
-3466 space: SpaceId,
-3467 crypto: &frame::Crypto,
-3468 payload_len: usize,
-3469 ) -> Result<(), TransportError> {
-3470 let expected = if !self.state.is_handshake() {
-3471 SpaceId::Data
-3472 } else if self.highest_space == SpaceId::Initial {
-3473 SpaceId::Initial
-3474 } else {
-3475 // On the server, self.highest_space can be Data after receiving the client's first
-3476 // flight, but we expect Handshake CRYPTO until the handshake is complete.
-3477 SpaceId::Handshake
-3478 };
-3479 // We can't decrypt Handshake packets when highest_space is Initial, CRYPTO frames in 0-RTT
-3480 // packets are illegal, and we don't process 1-RTT packets until the handshake is
-3481 // complete. Therefore, we will never see CRYPTO data from a later-than-expected space.
-3482 debug_assert!(space <= expected, "received out-of-order CRYPTO data");
-3483
-3484 let end = crypto.offset + crypto.data.len() as u64;
-3485 if space < expected && end > self.spaces[space].crypto_stream.bytes_read() {
-3486 warn!(
-3487 "received new {:?} CRYPTO data when expecting {:?}",
-3488 space, expected
-3489 );
-3490 return Err(TransportError::PROTOCOL_VIOLATION(
-3491 "new data at unexpected encryption level",
-3492 ));
-3493 }
-3494
-3495 let space = &mut self.spaces[space];
-3496 let max = end.saturating_sub(space.crypto_stream.bytes_read());
-3497 if max > self.config.crypto_buffer_size as u64 {
-3498 return Err(TransportError::CRYPTO_BUFFER_EXCEEDED(""));
-3499 }
-3500
-3501 space
-3502 .crypto_stream
-3503 .insert(crypto.offset, crypto.data.clone(), payload_len);
-3504 while let Some(chunk) = space.crypto_stream.read(usize::MAX, true) {
-3505 trace!("consumed {} CRYPTO bytes", chunk.bytes.len());
-3506 if self.crypto.read_handshake(&chunk.bytes)? {
-3507 self.events.push_back(Event::HandshakeDataReady);
-3508 }
-3509 }
+3327 fn on_packet_authenticated(
+3328 &mut self,
+3329 now: Instant,
+3330 space_id: SpaceId,
+3331 path_id: PathId,
+3332 ecn: Option<EcnCodepoint>,
+3333 packet: Option<u64>,
+3334 spin: bool,
+3335 is_1rtt: bool,
+3336 ) {
+3337 self.total_authed_packets += 1;
+3338 if let Some(AbandonState::ExpectingPathAbandon { deadline }) = self
+3339 .paths
+3340 .get(&path_id)
+3341 .map(|path| &path.data.abandon_state)
+3342 {
+3343 if now > *deadline {
+3344 warn!("received data on path which we abandoned more than 3 * PTO ago");
+3345 // The peer failed to respond with a PATH_ABANDON in time.
+3346 if !self.state.is_closed() {
+3347 self.state
+3348 .move_to_closed(TransportError::PROTOCOL_VIOLATION(
+3349 "peer failed to respond with PATH_ABANDON in time",
+3350 ));
+3351 self.close_common();
+3352 self.set_close_timer(now);
+3353 self.close = true;
+3354 }
+3355 return;
+3356 }
+3357 }
+3358
+3359 self.reset_keep_alive(path_id, now);
+3360 self.reset_idle_timeout(now, space_id, path_id);
+3361 self.permit_idle_reset = true;
+3362 self.receiving_ecn |= ecn.is_some();
+3363 if let Some(x) = ecn {
+3364 let space = &mut self.spaces[space_id];
+3365 space.for_path(path_id).ecn_counters += x;
+3366
+3367 if x.is_ce() {
+3368 space
+3369 .for_path(path_id)
+3370 .pending_acks
+3371 .set_immediate_ack_required();
+3372 }
+3373 }
+3374
+3375 let packet = match packet {
+3376 Some(x) => x,
+3377 None => return,
+3378 };
+3379 match &self.side {
+3380 ConnectionSide::Client { .. } => {
+3381 // If we received a handshake packet that authenticated, then we're talking to
+3382 // the real server. From now on we should no longer allow the server to migrate
+3383 // its address.
+3384 if space_id == SpaceId::Handshake {
+3385 if let Some(hs) = self.state.as_handshake_mut() {
+3386 hs.allow_server_migration = false;
+3387 }
+3388 }
+3389 }
+3390 ConnectionSide::Server { .. } => {
+3391 if self.spaces[SpaceId::Initial].crypto.is_some() && space_id == SpaceId::Handshake
+3392 {
+3393 // A server stops sending and processing Initial packets when it receives its first Handshake packet.
+3394 self.discard_space(now, SpaceId::Initial);
+3395 }
+3396 if self.zero_rtt_crypto.is_some() && is_1rtt {
+3397 // Discard 0-RTT keys soon after receiving a 1-RTT packet
+3398 self.set_key_discard_timer(now, space_id)
+3399 }
+3400 }
+3401 }
+3402 let space = self.spaces[space_id].for_path(path_id);
+3403 space.pending_acks.insert_one(packet, now);
+3404 if packet >= space.rx_packet.unwrap_or_default() {
+3405 space.rx_packet = Some(packet);
+3406 // Update outgoing spin bit, inverting iff we're the client
+3407 self.spin = self.side.is_client() ^ spin;
+3408 }
+3409 }
+3410
+3411 /// Resets the idle timeout timers
+3412 ///
+3413 /// Without multipath there is only the connection-wide idle timeout. When multipath is
+3414 /// enabled there is an additional per-path idle timeout.
+3415 fn reset_idle_timeout(&mut self, now: Instant, space: SpaceId, path_id: PathId) {
+3416 // First reset the global idle timeout.
+3417 if let Some(timeout) = self.idle_timeout {
+3418 if self.state.is_closed() {
+3419 self.timers
+3420 .stop(Timer::Conn(ConnTimer::Idle), self.qlog.with_time(now));
+3421 } else {
+3422 let dt = cmp::max(timeout, 3 * self.pto_max_path(space, false));
+3423 self.timers.set(
+3424 Timer::Conn(ConnTimer::Idle),
+3425 now + dt,
+3426 self.qlog.with_time(now),
+3427 );
+3428 }
+3429 }
+3430
+3431 // Now handle the per-path state
+3432 if let Some(timeout) = self.path_data(path_id).idle_timeout {
+3433 if self.state.is_closed() {
+3434 self.timers.stop(
+3435 Timer::PerPath(path_id, PathTimer::PathIdle),
+3436 self.qlog.with_time(now),
+3437 );
+3438 } else {
+3439 let dt = cmp::max(timeout, 3 * self.pto(space, path_id));
+3440 self.timers.set(
+3441 Timer::PerPath(path_id, PathTimer::PathIdle),
+3442 now + dt,
+3443 self.qlog.with_time(now),
+3444 );
+3445 }
+3446 }
+3447 }
+3448
+3449 /// Resets both the [`ConnTimer::KeepAlive`] and [`PathTimer::PathKeepAlive`] timers
+3450 fn reset_keep_alive(&mut self, path_id: PathId, now: Instant) {
+3451 if !self.state.is_established() {
+3452 return;
+3453 }
+3454
+3455 if let Some(interval) = self.config.keep_alive_interval {
+3456 self.timers.set(
+3457 Timer::Conn(ConnTimer::KeepAlive),
+3458 now + interval,
+3459 self.qlog.with_time(now),
+3460 );
+3461 }
+3462
+3463 if let Some(interval) = self.path_data(path_id).keep_alive {
+3464 self.timers.set(
+3465 Timer::PerPath(path_id, PathTimer::PathKeepAlive),
+3466 now + interval,
+3467 self.qlog.with_time(now),
+3468 );
+3469 }
+3470 }
+3471
+3472 /// Sets the timer for when a previously issued CID should be retired next
+3473 fn reset_cid_retirement(&mut self, now: Instant) {
+3474 if let Some((_path, t)) = self.next_cid_retirement() {
+3475 self.timers.set(
+3476 Timer::Conn(ConnTimer::PushNewCid),
+3477 t,
+3478 self.qlog.with_time(now),
+3479 );
+3480 }
+3481 }
+3482
+3483 /// The next time when a previously issued CID should be retired
+3484 fn next_cid_retirement(&self) -> Option<(PathId, Instant)> {
+3485 self.local_cid_state
+3486 .iter()
+3487 .filter_map(|(path_id, cid_state)| cid_state.next_timeout().map(|t| (*path_id, t)))
+3488 .min_by_key(|(_path_id, timeout)| *timeout)
+3489 }
+3490
+3491 /// Handle the already-decrypted first packet from the client
+3492 ///
+3493 /// Decrypting the first packet in the `Endpoint` allows stateless packet handling to be more
+3494 /// efficient.
+3495 pub(crate) fn handle_first_packet(
+3496 &mut self,
+3497 now: Instant,
+3498 network_path: FourTuple,
+3499 ecn: Option<EcnCodepoint>,
+3500 packet_number: u64,
+3501 packet: InitialPacket,
+3502 remaining: Option<BytesMut>,
+3503 ) -> Result<(), ConnectionError> {
+3504 let span = trace_span!("first recv");
+3505 let _guard = span.enter();
+3506 debug_assert!(self.side.is_server());
+3507 let len = packet.header_data.len() + packet.payload.len();
+3508 let path_id = PathId::ZERO;
+3509 self.path_data_mut(path_id).total_recvd = len as u64;
3510
-3511 Ok(())
-3512 }
-3513
-3514 fn write_crypto(&mut self) {
-3515 loop {
-3516 let space = self.highest_space;
-3517 let mut outgoing = Vec::new();
-3518 if let Some(crypto) = self.crypto.write_handshake(&mut outgoing) {
-3519 match space {
-3520 SpaceId::Initial => {
-3521 self.upgrade_crypto(SpaceId::Handshake, crypto);
-3522 }
-3523 SpaceId::Handshake => {
-3524 self.upgrade_crypto(SpaceId::Data, crypto);
-3525 }
-3526 _ => unreachable!("got updated secrets during 1-RTT"),
-3527 }
-3528 }
-3529 if outgoing.is_empty() {
-3530 if space == self.highest_space {
-3531 break;
-3532 } else {
-3533 // Keys updated, check for more data to send
-3534 continue;
-3535 }
-3536 }
-3537 let offset = self.spaces[space].crypto_offset;
-3538 let outgoing = Bytes::from(outgoing);
-3539 if let Some(hs) = self.state.as_handshake_mut() {
-3540 if space == SpaceId::Initial && offset == 0 && self.side.is_client() {
-3541 hs.client_hello = Some(outgoing.clone());
-3542 }
-3543 }
-3544 self.spaces[space].crypto_offset += outgoing.len() as u64;
-3545 trace!("wrote {} {:?} CRYPTO bytes", outgoing.len(), space);
-3546 self.spaces[space].pending.crypto.push_back(frame::Crypto {
-3547 offset,
-3548 data: outgoing,
-3549 });
-3550 }
-3551 }
-3552
-3553 /// Switch to stronger cryptography during handshake
-3554 fn upgrade_crypto(&mut self, space: SpaceId, crypto: Keys) {
-3555 debug_assert!(
-3556 self.spaces[space].crypto.is_none(),
-3557 "already reached packet space {space:?}"
-3558 );
-3559 trace!("{:?} keys ready", space);
-3560 if space == SpaceId::Data {
-3561 // Precompute the first key update
-3562 self.next_crypto = Some(
-3563 self.crypto
-3564 .next_1rtt_keys()
-3565 .expect("handshake should be complete"),
-3566 );
-3567 }
-3568
-3569 self.spaces[space].crypto = Some(crypto);
-3570 debug_assert!(space as usize > self.highest_space as usize);
-3571 self.highest_space = space;
-3572 if space == SpaceId::Data && self.side.is_client() {
-3573 // Discard 0-RTT keys because 1-RTT keys are available.
-3574 self.zero_rtt_crypto = None;
-3575 }
-3576 }
-3577
-3578 fn discard_space(&mut self, now: Instant, space_id: SpaceId) {
-3579 debug_assert!(space_id != SpaceId::Data);
-3580 trace!("discarding {:?} keys", space_id);
-3581 if space_id == SpaceId::Initial {
-3582 // No longer needed
-3583 if let ConnectionSide::Client { token, .. } = &mut self.side {
-3584 *token = Bytes::new();
+3511 if let Some(hs) = self.state.as_handshake_mut() {
+3512 hs.expected_token = packet.header.token.clone();
+3513 } else {
+3514 unreachable!("first packet must be delivered in Handshake state");
+3515 }
+3516
+3517 // The first packet is always on PathId::ZERO
+3518 self.on_packet_authenticated(
+3519 now,
+3520 SpaceId::Initial,
+3521 path_id,
+3522 ecn,
+3523 Some(packet_number),
+3524 false,
+3525 false,
+3526 );
+3527
+3528 let packet: Packet = packet.into();
+3529
+3530 let mut qlog = QlogRecvPacket::new(len);
+3531 qlog.header(&packet.header, Some(packet_number), path_id);
+3532
+3533 self.process_decrypted_packet(
+3534 now,
+3535 network_path,
+3536 path_id,
+3537 Some(packet_number),
+3538 packet,
+3539 &mut qlog,
+3540 )?;
+3541 self.qlog.emit_packet_received(qlog, now);
+3542 if let Some(data) = remaining {
+3543 self.handle_coalesced(now, network_path, path_id, ecn, data);
+3544 }
+3545
+3546 self.qlog.emit_recovery_metrics(
+3547 path_id,
+3548 &mut self.paths.get_mut(&path_id).unwrap().data,
+3549 now,
+3550 );
+3551
+3552 Ok(())
+3553 }
+3554
+3555 fn init_0rtt(&mut self, now: Instant) {
+3556 let (header, packet) = match self.crypto.early_crypto() {
+3557 Some(x) => x,
+3558 None => return,
+3559 };
+3560 if self.side.is_client() {
+3561 match self.crypto.transport_parameters() {
+3562 Ok(params) => {
+3563 let params = params
+3564 .expect("crypto layer didn't supply transport parameters with ticket");
+3565 // Certain values must not be cached
+3566 let params = TransportParameters {
+3567 initial_src_cid: None,
+3568 original_dst_cid: None,
+3569 preferred_address: None,
+3570 retry_src_cid: None,
+3571 stateless_reset_token: None,
+3572 min_ack_delay: None,
+3573 ack_delay_exponent: TransportParameters::default().ack_delay_exponent,
+3574 max_ack_delay: TransportParameters::default().max_ack_delay,
+3575 initial_max_path_id: None,
+3576 ..params
+3577 };
+3578 self.set_peer_params(params);
+3579 self.qlog.emit_peer_transport_params_restored(self, now);
+3580 }
+3581 Err(e) => {
+3582 error!("session ticket has malformed transport parameters: {}", e);
+3583 return;
+3584 }
3585 }
3586 }
-3587 let space = &mut self.spaces[space_id];
-3588 space.crypto = None;
-3589 let pns = space.for_path(PathId::ZERO);
-3590 pns.time_of_last_ack_eliciting_packet = None;
-3591 pns.loss_time = None;
-3592 pns.loss_probes = 0;
-3593 let sent_packets = mem::take(&mut pns.sent_packets);
-3594 let path = self.paths.get_mut(&PathId::ZERO).unwrap();
-3595 for (_, packet) in sent_packets.into_iter() {
-3596 path.data.remove_in_flight(&packet);
-3597 }
-3598
-3599 self.set_loss_detection_timer(now, PathId::ZERO)
-3600 }
-3601
-3602 fn handle_coalesced(
-3603 &mut self,
-3604 now: Instant,
-3605 network_path: FourTuple,
-3606 path_id: PathId,
-3607 ecn: Option<EcnCodepoint>,
-3608 data: BytesMut,
-3609 ) {
-3610 self.path_data_mut(path_id)
-3611 .inc_total_recvd(data.len() as u64);
-3612 let mut remaining = Some(data);
-3613 let cid_len = self
-3614 .local_cid_state
-3615 .values()
-3616 .map(|cid_state| cid_state.cid_len())
-3617 .next()
-3618 .expect("one cid_state must exist");
-3619 while let Some(data) = remaining {
-3620 match PartialDecode::new(
-3621 data,
-3622 &FixedLengthConnectionIdParser::new(cid_len),
-3623 &[self.version],
-3624 self.endpoint_config.grease_quic_bit,
-3625 ) {
-3626 Ok((partial_decode, rest)) => {
-3627 remaining = rest;
-3628 self.handle_decode(now, network_path, path_id, ecn, partial_decode);
-3629 }
-3630 Err(e) => {
-3631 trace!("malformed header: {}", e);
-3632 return;
-3633 }
-3634 }
-3635 }
-3636 }
-3637
-3638 fn handle_decode(
-3639 &mut self,
-3640 now: Instant,
-3641 network_path: FourTuple,
-3642 path_id: PathId,
-3643 ecn: Option<EcnCodepoint>,
-3644 partial_decode: PartialDecode,
-3645 ) {
-3646 let qlog = QlogRecvPacket::new(partial_decode.len());
-3647 if let Some(decoded) = packet_crypto::unprotect_header(
-3648 partial_decode,
-3649 &self.spaces,
-3650 self.zero_rtt_crypto.as_ref(),
-3651 self.peer_params.stateless_reset_token,
-3652 ) {
-3653 self.handle_packet(
-3654 now,
-3655 network_path,
-3656 path_id,
-3657 ecn,
-3658 decoded.packet,
-3659 decoded.stateless_reset,
-3660 qlog,
-3661 );
-3662 }
-3663 }
-3664
-3665 fn handle_packet(
-3666 &mut self,
-3667 now: Instant,
-3668 network_path: FourTuple,
-3669 path_id: PathId,
-3670 ecn: Option<EcnCodepoint>,
-3671 packet: Option<Packet>,
-3672 stateless_reset: bool,
-3673 mut qlog: QlogRecvPacket,
-3674 ) {
-3675 self.stats.udp_rx.ios += 1;
-3676 if let Some(ref packet) = packet {
-3677 trace!(
-3678 "got {:?} packet ({} bytes) from {} using id {}",
-3679 packet.header.space(),
-3680 packet.payload.len() + packet.header_data.len(),
-3681 network_path,
-3682 packet.header.dst_cid(),
-3683 );
-3684 }
-3685
-3686 if self.is_handshaking() {
-3687 if path_id != PathId::ZERO {
-3688 debug!(%network_path, %path_id, "discarding multipath packet during handshake");
-3689 return;
-3690 }
-3691 if network_path != self.path_data_mut(path_id).network_path {
-3692 if let Some(hs) = self.state.as_handshake() {
-3693 if hs.allow_server_migration {
-3694 trace!(%network_path, prev = %self.path_data(path_id).network_path, "server migrated to new remote");
-3695 self.path_data_mut(path_id).network_path = network_path;
-3696 self.qlog.emit_tuple_assigned(path_id, network_path, now);
-3697 } else {
-3698 debug!("discarding packet with unexpected remote during handshake");
-3699 return;
-3700 }
-3701 } else {
-3702 debug!("discarding packet with unexpected remote during handshake");
-3703 return;
-3704 }
-3705 }
-3706 }
-3707
-3708 let was_closed = self.state.is_closed();
-3709 let was_drained = self.state.is_drained();
-3710
-3711 let decrypted = match packet {
-3712 None => Err(None),
-3713 Some(mut packet) => self
-3714 .decrypt_packet(now, path_id, &mut packet)
-3715 .map(move |number| (packet, number)),
-3716 };
-3717 let result = match decrypted {
-3718 _ if stateless_reset => {
-3719 debug!("got stateless reset");
-3720 Err(ConnectionError::Reset)
-3721 }
-3722 Err(Some(e)) => {
-3723 warn!("illegal packet: {}", e);
-3724 Err(e.into())
-3725 }
-3726 Err(None) => {
-3727 debug!("failed to authenticate packet");
-3728 self.authentication_failures += 1;
-3729 let integrity_limit = self.spaces[self.highest_space]
-3730 .crypto
-3731 .as_ref()
-3732 .unwrap()
-3733 .packet
-3734 .local
-3735 .integrity_limit();
-3736 if self.authentication_failures > integrity_limit {
-3737 Err(TransportError::AEAD_LIMIT_REACHED("integrity limit violated").into())
-3738 } else {
-3739 return;
-3740 }
-3741 }
-3742 Ok((packet, number)) => {
-3743 qlog.header(&packet.header, number, path_id);
-3744 let span = match number {
-3745 Some(pn) => trace_span!("recv", space = ?packet.header.space(), pn),
-3746 None => trace_span!("recv", space = ?packet.header.space()),
-3747 };
-3748 let _guard = span.enter();
-3749
-3750 let dedup = self.spaces[packet.header.space()]
-3751 .path_space_mut(path_id)
-3752 .map(|pns| &mut pns.dedup);
-3753 if number.zip(dedup).is_some_and(|(n, d)| d.insert(n)) {
-3754 debug!("discarding possible duplicate packet");
-3755 self.qlog.emit_packet_received(qlog, now);
-3756 return;
-3757 } else if self.state.is_handshake() && packet.header.is_short() {
-3758 // TODO: SHOULD buffer these to improve reordering tolerance.
-3759 trace!("dropping short packet during handshake");
-3760 self.qlog.emit_packet_received(qlog, now);
-3761 return;
-3762 } else {
-3763 if let Header::Initial(InitialHeader { ref token, .. }) = packet.header {
-3764 if let Some(hs) = self.state.as_handshake() {
-3765 if self.side.is_server() && token != &hs.expected_token {
-3766 // Clients must send the same retry token in every Initial. Initial
-3767 // packets can be spoofed, so we discard rather than killing the
-3768 // connection.
-3769 warn!("discarding Initial with invalid retry token");
-3770 self.qlog.emit_packet_received(qlog, now);
-3771 return;
-3772 }
-3773 }
-3774 }
-3775
-3776 if !self.state.is_closed() {
-3777 let spin = match packet.header {
-3778 Header::Short { spin, .. } => spin,
-3779 _ => false,
-3780 };
-3781
-3782 if self.side().is_server() && !self.abandoned_paths.contains(&path_id) {
-3783 // Only the client is allowed to open paths
-3784 self.ensure_path(path_id, network_path, now, number);
-3785 }
-3786 if self.paths.contains_key(&path_id) {
-3787 self.on_packet_authenticated(
-3788 now,
-3789 packet.header.space(),
-3790 path_id,
-3791 ecn,
-3792 number,
-3793 spin,
-3794 packet.header.is_1rtt(),
-3795 );
-3796 }
-3797 }
-3798
-3799 let res = self.process_decrypted_packet(
-3800 now,
-3801 network_path,
-3802 path_id,
-3803 number,
-3804 packet,
-3805 &mut qlog,
-3806 );
-3807
-3808 self.qlog.emit_packet_received(qlog, now);
-3809 res
-3810 }
-3811 }
-3812 };
+3587 trace!("0-RTT enabled");
+3588 self.zero_rtt_enabled = true;
+3589 self.zero_rtt_crypto = Some(ZeroRttCrypto { header, packet });
+3590 }
+3591
+3592 fn read_crypto(
+3593 &mut self,
+3594 space: SpaceId,
+3595 crypto: &frame::Crypto,
+3596 payload_len: usize,
+3597 ) -> Result<(), TransportError> {
+3598 let expected = if !self.state.is_handshake() {
+3599 SpaceId::Data
+3600 } else if self.highest_space == SpaceId::Initial {
+3601 SpaceId::Initial
+3602 } else {
+3603 // On the server, self.highest_space can be Data after receiving the client's first
+3604 // flight, but we expect Handshake CRYPTO until the handshake is complete.
+3605 SpaceId::Handshake
+3606 };
+3607 // We can't decrypt Handshake packets when highest_space is Initial, CRYPTO frames in 0-RTT
+3608 // packets are illegal, and we don't process 1-RTT packets until the handshake is
+3609 // complete. Therefore, we will never see CRYPTO data from a later-than-expected space.
+3610 debug_assert!(space <= expected, "received out-of-order CRYPTO data");
+3611
+3612 let end = crypto.offset + crypto.data.len() as u64;
+3613 if space < expected && end > self.spaces[space].crypto_stream.bytes_read() {
+3614 warn!(
+3615 "received new {:?} CRYPTO data when expecting {:?}",
+3616 space, expected
+3617 );
+3618 return Err(TransportError::PROTOCOL_VIOLATION(
+3619 "new data at unexpected encryption level",
+3620 ));
+3621 }
+3622
+3623 let space = &mut self.spaces[space];
+3624 let max = end.saturating_sub(space.crypto_stream.bytes_read());
+3625 if max > self.config.crypto_buffer_size as u64 {
+3626 return Err(TransportError::CRYPTO_BUFFER_EXCEEDED(""));
+3627 }
+3628
+3629 space
+3630 .crypto_stream
+3631 .insert(crypto.offset, crypto.data.clone(), payload_len);
+3632 while let Some(chunk) = space.crypto_stream.read(usize::MAX, true) {
+3633 trace!("consumed {} CRYPTO bytes", chunk.bytes.len());
+3634 if self.crypto.read_handshake(&chunk.bytes)? {
+3635 self.events.push_back(Event::HandshakeDataReady);
+3636 }
+3637 }
+3638
+3639 Ok(())
+3640 }
+3641
+3642 fn write_crypto(&mut self) {
+3643 loop {
+3644 let space = self.highest_space;
+3645 let mut outgoing = Vec::new();
+3646 if let Some(crypto) = self.crypto.write_handshake(&mut outgoing) {
+3647 match space {
+3648 SpaceId::Initial => {
+3649 self.upgrade_crypto(SpaceId::Handshake, crypto);
+3650 }
+3651 SpaceId::Handshake => {
+3652 self.upgrade_crypto(SpaceId::Data, crypto);
+3653 }
+3654 _ => unreachable!("got updated secrets during 1-RTT"),
+3655 }
+3656 }
+3657 if outgoing.is_empty() {
+3658 if space == self.highest_space {
+3659 break;
+3660 } else {
+3661 // Keys updated, check for more data to send
+3662 continue;
+3663 }
+3664 }
+3665 let offset = self.spaces[space].crypto_offset;
+3666 let outgoing = Bytes::from(outgoing);
+3667 if let Some(hs) = self.state.as_handshake_mut() {
+3668 if space == SpaceId::Initial && offset == 0 && self.side.is_client() {
+3669 hs.client_hello = Some(outgoing.clone());
+3670 }
+3671 }
+3672 self.spaces[space].crypto_offset += outgoing.len() as u64;
+3673 trace!("wrote {} {:?} CRYPTO bytes", outgoing.len(), space);
+3674 self.spaces[space].pending.crypto.push_back(frame::Crypto {
+3675 offset,
+3676 data: outgoing,
+3677 });
+3678 }
+3679 }
+3680
+3681 /// Switch to stronger cryptography during handshake
+3682 fn upgrade_crypto(&mut self, space: SpaceId, crypto: Keys) {
+3683 debug_assert!(
+3684 self.spaces[space].crypto.is_none(),
+3685 "already reached packet space {space:?}"
+3686 );
+3687 trace!("{:?} keys ready", space);
+3688 if space == SpaceId::Data {
+3689 // Precompute the first key update
+3690 self.next_crypto = Some(
+3691 self.crypto
+3692 .next_1rtt_keys()
+3693 .expect("handshake should be complete"),
+3694 );
+3695 }
+3696
+3697 self.spaces[space].crypto = Some(crypto);
+3698 debug_assert!(space as usize > self.highest_space as usize);
+3699 self.highest_space = space;
+3700 if space == SpaceId::Data && self.side.is_client() {
+3701 // Discard 0-RTT keys because 1-RTT keys are available.
+3702 self.zero_rtt_crypto = None;
+3703 }
+3704 }
+3705
+3706 fn discard_space(&mut self, now: Instant, space_id: SpaceId) {
+3707 debug_assert!(space_id != SpaceId::Data);
+3708 trace!("discarding {:?} keys", space_id);
+3709 if space_id == SpaceId::Initial {
+3710 // No longer needed
+3711 if let ConnectionSide::Client { token, .. } = &mut self.side {
+3712 *token = Bytes::new();
+3713 }
+3714 }
+3715 let space = &mut self.spaces[space_id];
+3716 space.crypto = None;
+3717 let pns = space.for_path(PathId::ZERO);
+3718 pns.time_of_last_ack_eliciting_packet = None;
+3719 pns.loss_time = None;
+3720 pns.loss_probes = 0;
+3721 let sent_packets = mem::take(&mut pns.sent_packets);
+3722 let path = self.paths.get_mut(&PathId::ZERO).unwrap();
+3723 for (_, packet) in sent_packets.into_iter() {
+3724 path.data.remove_in_flight(&packet);
+3725 }
+3726
+3727 self.set_loss_detection_timer(now, PathId::ZERO)
+3728 }
+3729
+3730 fn handle_coalesced(
+3731 &mut self,
+3732 now: Instant,
+3733 network_path: FourTuple,
+3734 path_id: PathId,
+3735 ecn: Option<EcnCodepoint>,
+3736 data: BytesMut,
+3737 ) {
+3738 self.path_data_mut(path_id)
+3739 .inc_total_recvd(data.len() as u64);
+3740 let mut remaining = Some(data);
+3741 let cid_len = self
+3742 .local_cid_state
+3743 .values()
+3744 .map(|cid_state| cid_state.cid_len())
+3745 .next()
+3746 .expect("one cid_state must exist");
+3747 while let Some(data) = remaining {
+3748 match PartialDecode::new(
+3749 data,
+3750 &FixedLengthConnectionIdParser::new(cid_len),
+3751 &[self.version],
+3752 self.endpoint_config.grease_quic_bit,
+3753 ) {
+3754 Ok((partial_decode, rest)) => {
+3755 remaining = rest;
+3756 self.handle_decode(now, network_path, path_id, ecn, partial_decode);
+3757 }
+3758 Err(e) => {
+3759 trace!("malformed header: {}", e);
+3760 return;
+3761 }
+3762 }
+3763 }
+3764 }
+3765
+3766 fn handle_decode(
+3767 &mut self,
+3768 now: Instant,
+3769 network_path: FourTuple,
+3770 path_id: PathId,
+3771 ecn: Option<EcnCodepoint>,
+3772 partial_decode: PartialDecode,
+3773 ) {
+3774 let qlog = QlogRecvPacket::new(partial_decode.len());
+3775 if let Some(decoded) = packet_crypto::unprotect_header(
+3776 partial_decode,
+3777 &self.spaces,
+3778 self.zero_rtt_crypto.as_ref(),
+3779 self.peer_params.stateless_reset_token,
+3780 ) {
+3781 self.handle_packet(
+3782 now,
+3783 network_path,
+3784 path_id,
+3785 ecn,
+3786 decoded.packet,
+3787 decoded.stateless_reset,
+3788 qlog,
+3789 );
+3790 }
+3791 }
+3792
+3793 fn handle_packet(
+3794 &mut self,
+3795 now: Instant,
+3796 network_path: FourTuple,
+3797 path_id: PathId,
+3798 ecn: Option<EcnCodepoint>,
+3799 packet: Option<Packet>,
+3800 stateless_reset: bool,
+3801 mut qlog: QlogRecvPacket,
+3802 ) {
+3803 self.stats.udp_rx.ios += 1;
+3804 if let Some(ref packet) = packet {
+3805 trace!(
+3806 "got {:?} packet ({} bytes) from {} using id {}",
+3807 packet.header.space(),
+3808 packet.payload.len() + packet.header_data.len(),
+3809 network_path,
+3810 packet.header.dst_cid(),
+3811 );
+3812 }
3813
-3814 // State transitions for error cases
-3815 if let Err(conn_err) = result {
-3816 match conn_err {
-3817 ConnectionError::ApplicationClosed(reason) => self.state.move_to_closed(reason),
-3818 ConnectionError::ConnectionClosed(reason) => self.state.move_to_closed(reason),
-3819 ConnectionError::Reset
-3820 | ConnectionError::TransportError(TransportError {
-3821 code: TransportErrorCode::AEAD_LIMIT_REACHED,
-3822 ..
-3823 }) => {
-3824 self.state.move_to_drained(Some(conn_err));
-3825 }
-3826 ConnectionError::TimedOut => {
-3827 unreachable!("timeouts aren't generated by packet processing");
-3828 }
-3829 ConnectionError::TransportError(err) => {
-3830 debug!("closing connection due to transport error: {}", err);
-3831 self.state.move_to_closed(err);
+3814 if self.is_handshaking() {
+3815 if path_id != PathId::ZERO {
+3816 debug!(%network_path, %path_id, "discarding multipath packet during handshake");
+3817 return;
+3818 }
+3819 if network_path != self.path_data_mut(path_id).network_path {
+3820 if let Some(hs) = self.state.as_handshake() {
+3821 if hs.allow_server_migration {
+3822 trace!(%network_path, prev = %self.path_data(path_id).network_path, "server migrated to new remote");
+3823 self.path_data_mut(path_id).network_path = network_path;
+3824 self.qlog.emit_tuple_assigned(path_id, network_path, now);
+3825 } else {
+3826 debug!("discarding packet with unexpected remote during handshake");
+3827 return;
+3828 }
+3829 } else {
+3830 debug!("discarding packet with unexpected remote during handshake");
+3831 return;
3832 }
-3833 ConnectionError::VersionMismatch => {
-3834 self.state.move_to_draining(Some(conn_err));
-3835 }
-3836 ConnectionError::LocallyClosed => {
-3837 unreachable!("LocallyClosed isn't generated by packet processing");
-3838 }
-3839 ConnectionError::CidsExhausted => {
-3840 unreachable!("CidsExhausted isn't generated by packet processing");
-3841 }
-3842 };
-3843 }
-3844
-3845 if !was_closed && self.state.is_closed() {
-3846 self.close_common();
-3847 if !self.state.is_drained() {
-3848 self.set_close_timer(now);
+3833 }
+3834 }
+3835
+3836 let was_closed = self.state.is_closed();
+3837 let was_drained = self.state.is_drained();
+3838
+3839 let decrypted = match packet {
+3840 None => Err(None),
+3841 Some(mut packet) => self
+3842 .decrypt_packet(now, path_id, &mut packet)
+3843 .map(move |number| (packet, number)),
+3844 };
+3845 let result = match decrypted {
+3846 _ if stateless_reset => {
+3847 debug!("got stateless reset");
+3848 Err(ConnectionError::Reset)
3849 }
-3850 }
-3851 if !was_drained && self.state.is_drained() {
-3852 self.endpoint_events.push_back(EndpointEventInner::Drained);
-3853 // Close timer may have been started previously, e.g. if we sent a close and got a
-3854 // stateless reset in response
-3855 self.timers
-3856 .stop(Timer::Conn(ConnTimer::Close), self.qlog.with_time(now));
-3857 }
-3858
-3859 // Transmit CONNECTION_CLOSE if necessary
-3860 if matches!(self.state.as_type(), StateType::Closed) {
-3861 // If there is no PathData for this PathId the packet was for a brand new
-3862 // path. It was a valid packet however, so the remote is valid and we want to
-3863 // send CONNECTION_CLOSE.
-3864 let path_remote = self
-3865 .paths
-3866 .get(&path_id)
-3867 .map(|p| p.data.network_path)
-3868 .unwrap_or(network_path);
-3869 self.close = network_path == path_remote;
-3870 }
-3871 }
-3872
-3873 fn process_decrypted_packet(
-3874 &mut self,
-3875 now: Instant,
-3876 network_path: FourTuple,
-3877 path_id: PathId,
-3878 number: Option<u64>,
-3879 packet: Packet,
-3880 qlog: &mut QlogRecvPacket,
-3881 ) -> Result<(), ConnectionError> {
-3882 if !self.paths.contains_key(&path_id) {
-3883 // There is a chance this is a server side, first (for this path) packet, which would
-3884 // be a protocol violation. It's more likely, however, that this is a packet of a
-3885 // pruned path
-3886 trace!(%path_id, ?number, "discarding packet for unknown path");
-3887 return Ok(());
-3888 }
-3889 let state = match self.state.as_type() {
-3890 StateType::Established => {
-3891 match packet.header.space() {
-3892 SpaceId::Data => self.process_payload(
-3893 now,
-3894 network_path,
-3895 path_id,
-3896 number.unwrap(),
-3897 packet,
-3898 qlog,
-3899 )?,
-3900 _ if packet.header.has_frames() => {
-3901 self.process_early_payload(now, path_id, packet, qlog)?
-3902 }
-3903 _ => {
-3904 trace!("discarding unexpected pre-handshake packet");
-3905 }
-3906 }
-3907 return Ok(());
-3908 }
-3909 StateType::Closed => {
-3910 for result in frame::Iter::new(packet.payload.freeze())? {
-3911 let frame = match result {
-3912 Ok(frame) => frame,
-3913 Err(err) => {
-3914 debug!("frame decoding error: {err:?}");
-3915 continue;
-3916 }
-3917 };
-3918 qlog.frame(&frame);
-3919
-3920 if let Frame::Padding = frame {
-3921 continue;
-3922 };
-3923
-3924 self.stats.frame_rx.record(frame.ty());
-3925
-3926 if let Frame::Close(_error) = frame {
-3927 self.state.move_to_draining(None);
-3928 break;
-3929 }
-3930 }
-3931 return Ok(());
-3932 }
-3933 StateType::Draining | StateType::Drained => return Ok(()),
-3934 StateType::Handshake => self.state.as_handshake_mut().expect("checked"),
-3935 };
-3936
-3937 match packet.header {
-3938 Header::Retry {
-3939 src_cid: rem_cid, ..
-3940 } => {
-3941 debug_assert_eq!(path_id, PathId::ZERO);
-3942 if self.side.is_server() {
-3943 return Err(TransportError::PROTOCOL_VIOLATION("client sent Retry").into());
-3944 }
-3945
-3946 let is_valid_retry = self
-3947 .rem_cids
-3948 .get(&path_id)
-3949 .map(|cids| cids.active())
-3950 .map(|orig_dst_cid| {
-3951 self.crypto.is_valid_retry(
-3952 orig_dst_cid,
-3953 &packet.header_data,
-3954 &packet.payload,
-3955 )
-3956 })
-3957 .unwrap_or_default();
-3958 if self.total_authed_packets > 1
-3959 || packet.payload.len() <= 16 // token + 16 byte tag
-3960 || !is_valid_retry
-3961 {
-3962 trace!("discarding invalid Retry");
-3963 // - After the client has received and processed an Initial or Retry
-3964 // packet from the server, it MUST discard any subsequent Retry
-3965 // packets that it receives.
-3966 // - A client MUST discard a Retry packet with a zero-length Retry Token
-3967 // field.
-3968 // - Clients MUST discard Retry packets that have a Retry Integrity Tag
-3969 // that cannot be validated
-3970 return Ok(());
-3971 }
+3850 Err(Some(e)) => {
+3851 warn!("illegal packet: {}", e);
+3852 Err(e.into())
+3853 }
+3854 Err(None) => {
+3855 debug!("failed to authenticate packet");
+3856 self.authentication_failures += 1;
+3857 let integrity_limit = self.spaces[self.highest_space]
+3858 .crypto
+3859 .as_ref()
+3860 .unwrap()
+3861 .packet
+3862 .local
+3863 .integrity_limit();
+3864 if self.authentication_failures > integrity_limit {
+3865 Err(TransportError::AEAD_LIMIT_REACHED("integrity limit violated").into())
+3866 } else {
+3867 return;
+3868 }
+3869 }
+3870 Ok((packet, number)) => {
+3871 qlog.header(&packet.header, number, path_id);
+3872 let span = match number {
+3873 Some(pn) => trace_span!("recv", space = ?packet.header.space(), pn),
+3874 None => trace_span!("recv", space = ?packet.header.space()),
+3875 };
+3876 let _guard = span.enter();
+3877
+3878 let dedup = self.spaces[packet.header.space()]
+3879 .path_space_mut(path_id)
+3880 .map(|pns| &mut pns.dedup);
+3881 if number.zip(dedup).is_some_and(|(n, d)| d.insert(n)) {
+3882 debug!("discarding possible duplicate packet");
+3883 self.qlog.emit_packet_received(qlog, now);
+3884 return;
+3885 } else if self.state.is_handshake() && packet.header.is_short() {
+3886 // TODO: SHOULD buffer these to improve reordering tolerance.
+3887 trace!("dropping short packet during handshake");
+3888 self.qlog.emit_packet_received(qlog, now);
+3889 return;
+3890 } else {
+3891 if let Header::Initial(InitialHeader { ref token, .. }) = packet.header {
+3892 if let Some(hs) = self.state.as_handshake() {
+3893 if self.side.is_server() && token != &hs.expected_token {
+3894 // Clients must send the same retry token in every Initial. Initial
+3895 // packets can be spoofed, so we discard rather than killing the
+3896 // connection.
+3897 warn!("discarding Initial with invalid retry token");
+3898 self.qlog.emit_packet_received(qlog, now);
+3899 return;
+3900 }
+3901 }
+3902 }
+3903
+3904 if !self.state.is_closed() {
+3905 let spin = match packet.header {
+3906 Header::Short { spin, .. } => spin,
+3907 _ => false,
+3908 };
+3909
+3910 if self.side().is_server() && !self.abandoned_paths.contains(&path_id) {
+3911 // Only the client is allowed to open paths
+3912 self.ensure_path(path_id, network_path, now, number);
+3913 }
+3914 if self.paths.contains_key(&path_id) {
+3915 self.on_packet_authenticated(
+3916 now,
+3917 packet.header.space(),
+3918 path_id,
+3919 ecn,
+3920 number,
+3921 spin,
+3922 packet.header.is_1rtt(),
+3923 );
+3924 }
+3925 }
+3926
+3927 let res = self.process_decrypted_packet(
+3928 now,
+3929 network_path,
+3930 path_id,
+3931 number,
+3932 packet,
+3933 &mut qlog,
+3934 );
+3935
+3936 self.qlog.emit_packet_received(qlog, now);
+3937 res
+3938 }
+3939 }
+3940 };
+3941
+3942 // State transitions for error cases
+3943 if let Err(conn_err) = result {
+3944 match conn_err {
+3945 ConnectionError::ApplicationClosed(reason) => self.state.move_to_closed(reason),
+3946 ConnectionError::ConnectionClosed(reason) => self.state.move_to_closed(reason),
+3947 ConnectionError::Reset
+3948 | ConnectionError::TransportError(TransportError {
+3949 code: TransportErrorCode::AEAD_LIMIT_REACHED,
+3950 ..
+3951 }) => {
+3952 self.state.move_to_drained(Some(conn_err));
+3953 }
+3954 ConnectionError::TimedOut => {
+3955 unreachable!("timeouts aren't generated by packet processing");
+3956 }
+3957 ConnectionError::TransportError(err) => {
+3958 debug!("closing connection due to transport error: {}", err);
+3959 self.state.move_to_closed(err);
+3960 }
+3961 ConnectionError::VersionMismatch => {
+3962 self.state.move_to_draining(Some(conn_err));
+3963 }
+3964 ConnectionError::LocallyClosed => {
+3965 unreachable!("LocallyClosed isn't generated by packet processing");
+3966 }
+3967 ConnectionError::CidsExhausted => {
+3968 unreachable!("CidsExhausted isn't generated by packet processing");
+3969 }
+3970 };
+3971 }
3972
-3973 trace!("retrying with CID {}", rem_cid);
-3974 let client_hello = state.client_hello.take().unwrap();
-3975 self.retry_src_cid = Some(rem_cid);
-3976 self.rem_cids
-3977 .get_mut(&path_id)
-3978 .expect("PathId::ZERO not yet abandoned, is_valid_retry would have been false")
-3979 .update_initial_cid(rem_cid);
-3980 self.rem_handshake_cid = rem_cid;
-3981
-3982 let space = &mut self.spaces[SpaceId::Initial];
-3983 if let Some(info) = space.for_path(PathId::ZERO).take(0) {
-3984 self.on_packet_acked(now, PathId::ZERO, info);
-3985 };
+3973 if !was_closed && self.state.is_closed() {
+3974 self.close_common();
+3975 if !self.state.is_drained() {
+3976 self.set_close_timer(now);
+3977 }
+3978 }
+3979 if !was_drained && self.state.is_drained() {
+3980 self.endpoint_events.push_back(EndpointEventInner::Drained);
+3981 // Close timer may have been started previously, e.g. if we sent a close and got a
+3982 // stateless reset in response
+3983 self.timers
+3984 .stop(Timer::Conn(ConnTimer::Close), self.qlog.with_time(now));
+3985 }
3986
-3987 self.discard_space(now, SpaceId::Initial); // Make sure we clean up after
-3988 // any retransmitted Initials
-3989 self.spaces[SpaceId::Initial] = {
-3990 let mut space = PacketSpace::new(now, SpaceId::Initial, &mut self.rng);
-3991 space.crypto = Some(self.crypto.initial_keys(rem_cid, self.side.side()));
-3992 space.crypto_offset = client_hello.len() as u64;
-3993 space.for_path(path_id).next_packet_number = self.spaces[SpaceId::Initial]
-3994 .for_path(path_id)
-3995 .next_packet_number;
-3996 space.pending.crypto.push_back(frame::Crypto {
-3997 offset: 0,
-3998 data: client_hello,
-3999 });
-4000 space
-4001 };
-4002
-4003 // Retransmit all 0-RTT data
-4004 let zero_rtt = mem::take(
-4005 &mut self.spaces[SpaceId::Data]
-4006 .for_path(PathId::ZERO)
-4007 .sent_packets,
-4008 );
-4009 for (_, info) in zero_rtt.into_iter() {
-4010 self.paths
-4011 .get_mut(&PathId::ZERO)
-4012 .unwrap()
-4013 .remove_in_flight(&info);
-4014 self.spaces[SpaceId::Data].pending |= info.retransmits;
-4015 }
-4016 self.streams.retransmit_all_for_0rtt();
-4017
-4018 let token_len = packet.payload.len() - 16;
-4019 let ConnectionSide::Client { ref mut token, .. } = self.side else {
-4020 unreachable!("we already short-circuited if we're server");
-4021 };
-4022 *token = packet.payload.freeze().split_to(token_len);
-4023
-4024 self.state = State::handshake(state::Handshake {
-4025 expected_token: Bytes::new(),
-4026 rem_cid_set: false,
-4027 client_hello: None,
-4028 allow_server_migration: true,
-4029 });
-4030 Ok(())
-4031 }
-4032 Header::Long {
-4033 ty: LongType::Handshake,
-4034 src_cid: rem_cid,
-4035 dst_cid: loc_cid,
-4036 ..
-4037 } => {
-4038 debug_assert_eq!(path_id, PathId::ZERO);
-4039 if rem_cid != self.rem_handshake_cid {
-4040 debug!(
-4041 "discarding packet with mismatched remote CID: {} != {}",
-4042 self.rem_handshake_cid, rem_cid
-4043 );
-4044 return Ok(());
-4045 }
-4046 self.on_path_validated(path_id);
+3987 // Transmit CONNECTION_CLOSE if necessary
+3988 if matches!(self.state.as_type(), StateType::Closed) {
+3989 // If there is no PathData for this PathId the packet was for a brand new
+3990 // path. It was a valid packet however, so the remote is valid and we want to
+3991 // send CONNECTION_CLOSE.
+3992 let path_remote = self
+3993 .paths
+3994 .get(&path_id)
+3995 .map(|p| p.data.network_path)
+3996 .unwrap_or(network_path);
+3997 self.close = network_path == path_remote;
+3998 }
+3999 }
+4000
+4001 fn process_decrypted_packet(
+4002 &mut self,
+4003 now: Instant,
+4004 network_path: FourTuple,
+4005 path_id: PathId,
+4006 number: Option<u64>,
+4007 packet: Packet,
+4008 qlog: &mut QlogRecvPacket,
+4009 ) -> Result<(), ConnectionError> {
+4010 if !self.paths.contains_key(&path_id) {
+4011 // There is a chance this is a server side, first (for this path) packet, which would
+4012 // be a protocol violation. It's more likely, however, that this is a packet of a
+4013 // pruned path
+4014 trace!(%path_id, ?number, "discarding packet for unknown path");
+4015 return Ok(());
+4016 }
+4017 let state = match self.state.as_type() {
+4018 StateType::Established => {
+4019 match packet.header.space() {
+4020 SpaceId::Data => self.process_payload(
+4021 now,
+4022 network_path,
+4023 path_id,
+4024 number.unwrap(),
+4025 packet,
+4026 qlog,
+4027 )?,
+4028 _ if packet.header.has_frames() => {
+4029 self.process_early_payload(now, path_id, packet, qlog)?
+4030 }
+4031 _ => {
+4032 trace!("discarding unexpected pre-handshake packet");
+4033 }
+4034 }
+4035 return Ok(());
+4036 }
+4037 StateType::Closed => {
+4038 for result in frame::Iter::new(packet.payload.freeze())? {
+4039 let frame = match result {
+4040 Ok(frame) => frame,
+4041 Err(err) => {
+4042 debug!("frame decoding error: {err:?}");
+4043 continue;
+4044 }
+4045 };
+4046 qlog.frame(&frame);
4047
-4048 self.process_early_payload(now, path_id, packet, qlog)?;
-4049 if self.state.is_closed() {
-4050 return Ok(());
-4051 }
-4052
-4053 if self.crypto.is_handshaking() {
-4054 trace!("handshake ongoing");
-4055 return Ok(());
-4056 }
-4057
-4058 if self.side.is_client() {
-4059 // Client-only because server params were set from the client's Initial
-4060 let params = self.crypto.transport_parameters()?.ok_or_else(|| {
-4061 TransportError::new(
-4062 TransportErrorCode::crypto(0x6d),
-4063 "transport parameters missing".to_owned(),
-4064 )
-4065 })?;
-4066
-4067 if self.has_0rtt() {
-4068 if !self.crypto.early_data_accepted().unwrap() {
-4069 debug_assert!(self.side.is_client());
-4070 debug!("0-RTT rejected");
-4071 self.accepted_0rtt = false;
-4072 self.streams.zero_rtt_rejected();
+4048 if let Frame::Padding = frame {
+4049 continue;
+4050 };
+4051
+4052 self.stats.frame_rx.record(frame.ty());
+4053
+4054 if let Frame::Close(_error) = frame {
+4055 self.state.move_to_draining(None);
+4056 break;
+4057 }
+4058 }
+4059 return Ok(());
+4060 }
+4061 StateType::Draining | StateType::Drained => return Ok(()),
+4062 StateType::Handshake => self.state.as_handshake_mut().expect("checked"),
+4063 };
+4064
+4065 match packet.header {
+4066 Header::Retry {
+4067 src_cid: rem_cid, ..
+4068 } => {
+4069 debug_assert_eq!(path_id, PathId::ZERO);
+4070 if self.side.is_server() {
+4071 return Err(TransportError::PROTOCOL_VIOLATION("client sent Retry").into());
+4072 }
4073
-4074 // Discard already-queued frames
-4075 self.spaces[SpaceId::Data].pending = Retransmits::default();
-4076
-4077 // Discard 0-RTT packets
-4078 let sent_packets = mem::take(
-4079 &mut self.spaces[SpaceId::Data].for_path(path_id).sent_packets,
-4080 );
-4081 for (_, packet) in sent_packets.into_iter() {
-4082 self.paths
-4083 .get_mut(&path_id)
-4084 .unwrap()
-4085 .remove_in_flight(&packet);
-4086 }
-4087 } else {
-4088 self.accepted_0rtt = true;
-4089 params.validate_resumption_from(&self.peer_params)?;
-4090 }
-4091 }
-4092 if let Some(token) = params.stateless_reset_token {
-4093 // TODO(matheus23): Reset token for a remote, or for a 4-tuple?
-4094 let remote = self.path_data(path_id).network_path.remote;
-4095 self.endpoint_events
-4096 .push_back(EndpointEventInner::ResetToken(path_id, remote, token));
-4097 }
-4098 self.handle_peer_params(params, loc_cid, rem_cid, now)?;
-4099 self.issue_first_cids(now);
-4100 } else {
-4101 // Server-only
-4102 self.spaces[SpaceId::Data].pending.handshake_done = true;
-4103 self.discard_space(now, SpaceId::Handshake);
-4104 self.events.push_back(Event::HandshakeConfirmed);
-4105 trace!("handshake confirmed");
-4106 }
-4107
-4108 self.events.push_back(Event::Connected);
-4109 self.state.move_to_established();
-4110 trace!("established");
-4111
-4112 // Multipath can only be enabled after the state has reached Established.
-4113 // So this can not happen any earlier.
-4114 self.issue_first_path_cids(now);
-4115 Ok(())
-4116 }
-4117 Header::Initial(InitialHeader {
-4118 src_cid: rem_cid,
-4119 dst_cid: loc_cid,
-4120 ..
-4121 }) => {
-4122 debug_assert_eq!(path_id, PathId::ZERO);
-4123 if !state.rem_cid_set {
-4124 trace!("switching remote CID to {}", rem_cid);
-4125 let mut state = state.clone();
-4126 self.rem_cids
-4127 .get_mut(&path_id)
-4128 .expect("PathId::ZERO not yet abandoned")
-4129 .update_initial_cid(rem_cid);
-4130 self.rem_handshake_cid = rem_cid;
-4131 self.orig_rem_cid = rem_cid;
-4132 state.rem_cid_set = true;
-4133 self.state.move_to_handshake(state);
-4134 } else if rem_cid != self.rem_handshake_cid {
-4135 debug!(
-4136 "discarding packet with mismatched remote CID: {} != {}",
-4137 self.rem_handshake_cid, rem_cid
-4138 );
-4139 return Ok(());
-4140 }
-4141
-4142 let starting_space = self.highest_space;
-4143 self.process_early_payload(now, path_id, packet, qlog)?;
-4144
-4145 if self.side.is_server()
-4146 && starting_space == SpaceId::Initial
-4147 && self.highest_space != SpaceId::Initial
-4148 {
-4149 let params = self.crypto.transport_parameters()?.ok_or_else(|| {
-4150 TransportError::new(
-4151 TransportErrorCode::crypto(0x6d),
-4152 "transport parameters missing".to_owned(),
-4153 )
-4154 })?;
-4155 self.handle_peer_params(params, loc_cid, rem_cid, now)?;
-4156 self.issue_first_cids(now);
-4157 self.init_0rtt(now);
-4158 }
-4159 Ok(())
-4160 }
-4161 Header::Long {
-4162 ty: LongType::ZeroRtt,
-4163 ..
-4164 } => {
-4165 self.process_payload(now, network_path, path_id, number.unwrap(), packet, qlog)?;
-4166 Ok(())
-4167 }
-4168 Header::VersionNegotiate { .. } => {
-4169 if self.total_authed_packets > 1 {
-4170 return Ok(());
-4171 }
-4172 let supported = packet
-4173 .payload
-4174 .chunks(4)
-4175 .any(|x| match <[u8; 4]>::try_from(x) {
-4176 Ok(version) => self.version == u32::from_be_bytes(version),
-4177 Err(_) => false,
-4178 });
-4179 if supported {
-4180 return Ok(());
-4181 }
-4182 debug!("remote doesn't support our version");
-4183 Err(ConnectionError::VersionMismatch)
-4184 }
-4185 Header::Short { .. } => unreachable!(
-4186 "short packets received during handshake are discarded in handle_packet"
-4187 ),
-4188 }
-4189 }
-4190
-4191 /// Process an Initial or Handshake packet payload
-4192 fn process_early_payload(
-4193 &mut self,
-4194 now: Instant,
-4195 path_id: PathId,
-4196 packet: Packet,
-4197 #[allow(unused)] qlog: &mut QlogRecvPacket,
-4198 ) -> Result<(), TransportError> {
-4199 debug_assert_ne!(packet.header.space(), SpaceId::Data);
-4200 debug_assert_eq!(path_id, PathId::ZERO);
-4201 let payload_len = packet.payload.len();
-4202 let mut ack_eliciting = false;
-4203 for result in frame::Iter::new(packet.payload.freeze())? {
-4204 let frame = result?;
-4205 qlog.frame(&frame);
-4206 let span = match frame {
-4207 Frame::Padding => continue,
-4208 _ => Some(trace_span!("frame", ty = %frame.ty(), path = tracing::field::Empty)),
-4209 };
-4210
-4211 self.stats.frame_rx.record(frame.ty());
-4212
-4213 let _guard = span.as_ref().map(|x| x.enter());
-4214 ack_eliciting |= frame.is_ack_eliciting();
-4215
-4216 // Process frames
-4217 if frame.is_1rtt() && packet.header.space() != SpaceId::Data {
-4218 return Err(TransportError::PROTOCOL_VIOLATION(
-4219 "illegal frame type in handshake",
-4220 ));
-4221 }
-4222
-4223 match frame {
-4224 Frame::Padding | Frame::Ping => {}
-4225 Frame::Crypto(frame) => {
-4226 self.read_crypto(packet.header.space(), &frame, payload_len)?;
-4227 }
-4228 Frame::Ack(ack) => {
-4229 self.on_ack_received(now, packet.header.space(), ack)?;
-4230 }
-4231 Frame::PathAck(ack) => {
-4232 span.as_ref()
-4233 .map(|span| span.record("path", tracing::field::debug(&ack.path_id)));
-4234 self.on_path_ack_received(now, packet.header.space(), ack)?;
-4235 }
-4236 Frame::Close(reason) => {
-4237 self.state.move_to_draining(Some(reason.into()));
-4238 return Ok(());
-4239 }
-4240 _ => {
-4241 let mut err =
-4242 TransportError::PROTOCOL_VIOLATION("illegal frame type in handshake");
-4243 err.frame = frame::MaybeFrame::Known(frame.ty());
-4244 return Err(err);
-4245 }
-4246 }
-4247 }
-4248
-4249 if ack_eliciting {
-4250 // In the initial and handshake spaces, ACKs must be sent immediately
-4251 self.spaces[packet.header.space()]
-4252 .for_path(path_id)
-4253 .pending_acks
-4254 .set_immediate_ack_required();
-4255 }
-4256
-4257 self.write_crypto();
-4258 Ok(())
-4259 }
-4260
-4261 /// Processes the packet payload, always in the data space.
-4262 fn process_payload(
-4263 &mut self,
-4264 now: Instant,
-4265 network_path: FourTuple,
-4266 path_id: PathId,
-4267 number: u64,
-4268 packet: Packet,
-4269 #[allow(unused)] qlog: &mut QlogRecvPacket,
-4270 ) -> Result<(), TransportError> {
-4271 let payload = packet.payload.freeze();
-4272 let mut is_probing_packet = true;
-4273 let mut close = None;
-4274 let payload_len = payload.len();
-4275 let mut ack_eliciting = false;
-4276 // if this packet triggers a path migration and includes a observed address frame, it's
-4277 // stored here
-4278 let mut migration_observed_addr = None;
-4279 for result in frame::Iter::new(payload)? {
-4280 let frame = result?;
-4281 qlog.frame(&frame);
-4282 let span = match frame {
-4283 Frame::Padding => continue,
-4284 _ => trace_span!("frame", ty = %frame.ty(), path = tracing::field::Empty),
-4285 };
-4286
-4287 self.stats.frame_rx.record(frame.ty());
-4288 // Crypto, Stream and Datagram frames are special cased in order no pollute
-4289 // the log with payload data
-4290 match &frame {
-4291 Frame::Crypto(f) => {
-4292 trace!(offset = f.offset, len = f.data.len(), "got crypto frame");
-4293 }
-4294 Frame::Stream(f) => {
-4295 trace!(id = %f.id, offset = f.offset, len = f.data.len(), fin = f.fin, "got stream frame");
-4296 }
-4297 Frame::Datagram(f) => {
-4298 trace!(len = f.data.len(), "got datagram frame");
+4074 let is_valid_retry = self
+4075 .rem_cids
+4076 .get(&path_id)
+4077 .map(|cids| cids.active())
+4078 .map(|orig_dst_cid| {
+4079 self.crypto.is_valid_retry(
+4080 orig_dst_cid,
+4081 &packet.header_data,
+4082 &packet.payload,
+4083 )
+4084 })
+4085 .unwrap_or_default();
+4086 if self.total_authed_packets > 1
+4087 || packet.payload.len() <= 16 // token + 16 byte tag
+4088 || !is_valid_retry
+4089 {
+4090 trace!("discarding invalid Retry");
+4091 // - After the client has received and processed an Initial or Retry
+4092 // packet from the server, it MUST discard any subsequent Retry
+4093 // packets that it receives.
+4094 // - A client MUST discard a Retry packet with a zero-length Retry Token
+4095 // field.
+4096 // - Clients MUST discard Retry packets that have a Retry Integrity Tag
+4097 // that cannot be validated
+4098 return Ok(());
+4099 }
+4100
+4101 trace!("retrying with CID {}", rem_cid);
+4102 let client_hello = state.client_hello.take().unwrap();
+4103 self.retry_src_cid = Some(rem_cid);
+4104 self.rem_cids
+4105 .get_mut(&path_id)
+4106 .expect("PathId::ZERO not yet abandoned, is_valid_retry would have been false")
+4107 .update_initial_cid(rem_cid);
+4108 self.rem_handshake_cid = rem_cid;
+4109
+4110 let space = &mut self.spaces[SpaceId::Initial];
+4111 if let Some(info) = space.for_path(PathId::ZERO).take(0) {
+4112 self.on_packet_acked(now, PathId::ZERO, info);
+4113 };
+4114
+4115 self.discard_space(now, SpaceId::Initial); // Make sure we clean up after
+4116 // any retransmitted Initials
+4117 self.spaces[SpaceId::Initial] = {
+4118 let mut space = PacketSpace::new(now, SpaceId::Initial, &mut self.rng);
+4119 space.crypto = Some(self.crypto.initial_keys(rem_cid, self.side.side()));
+4120 space.crypto_offset = client_hello.len() as u64;
+4121 space.for_path(path_id).next_packet_number = self.spaces[SpaceId::Initial]
+4122 .for_path(path_id)
+4123 .next_packet_number;
+4124 space.pending.crypto.push_back(frame::Crypto {
+4125 offset: 0,
+4126 data: client_hello,
+4127 });
+4128 space
+4129 };
+4130
+4131 // Retransmit all 0-RTT data
+4132 let zero_rtt = mem::take(
+4133 &mut self.spaces[SpaceId::Data]
+4134 .for_path(PathId::ZERO)
+4135 .sent_packets,
+4136 );
+4137 for (_, info) in zero_rtt.into_iter() {
+4138 self.paths
+4139 .get_mut(&PathId::ZERO)
+4140 .unwrap()
+4141 .remove_in_flight(&info);
+4142 self.spaces[SpaceId::Data].pending |= info.retransmits;
+4143 }
+4144 self.streams.retransmit_all_for_0rtt();
+4145
+4146 let token_len = packet.payload.len() - 16;
+4147 let ConnectionSide::Client { ref mut token, .. } = self.side else {
+4148 unreachable!("we already short-circuited if we're server");
+4149 };
+4150 *token = packet.payload.freeze().split_to(token_len);
+4151
+4152 self.state = State::handshake(state::Handshake {
+4153 expected_token: Bytes::new(),
+4154 rem_cid_set: false,
+4155 client_hello: None,
+4156 allow_server_migration: true,
+4157 });
+4158 Ok(())
+4159 }
+4160 Header::Long {
+4161 ty: LongType::Handshake,
+4162 src_cid: rem_cid,
+4163 dst_cid: loc_cid,
+4164 ..
+4165 } => {
+4166 debug_assert_eq!(path_id, PathId::ZERO);
+4167 if rem_cid != self.rem_handshake_cid {
+4168 debug!(
+4169 "discarding packet with mismatched remote CID: {} != {}",
+4170 self.rem_handshake_cid, rem_cid
+4171 );
+4172 return Ok(());
+4173 }
+4174 self.on_path_validated(path_id);
+4175
+4176 self.process_early_payload(now, path_id, packet, qlog)?;
+4177 if self.state.is_closed() {
+4178 return Ok(());
+4179 }
+4180
+4181 if self.crypto.is_handshaking() {
+4182 trace!("handshake ongoing");
+4183 return Ok(());
+4184 }
+4185
+4186 if self.side.is_client() {
+4187 // Client-only because server params were set from the client's Initial
+4188 let params = self.crypto.transport_parameters()?.ok_or_else(|| {
+4189 TransportError::new(
+4190 TransportErrorCode::crypto(0x6d),
+4191 "transport parameters missing".to_owned(),
+4192 )
+4193 })?;
+4194
+4195 if self.has_0rtt() {
+4196 if !self.crypto.early_data_accepted().unwrap() {
+4197 debug_assert!(self.side.is_client());
+4198 debug!("0-RTT rejected");
+4199 self.accepted_0rtt = false;
+4200 self.streams.zero_rtt_rejected();
+4201
+4202 // Discard already-queued frames
+4203 self.spaces[SpaceId::Data].pending = Retransmits::default();
+4204
+4205 // Discard 0-RTT packets
+4206 let sent_packets = mem::take(
+4207 &mut self.spaces[SpaceId::Data].for_path(path_id).sent_packets,
+4208 );
+4209 for (_, packet) in sent_packets.into_iter() {
+4210 self.paths
+4211 .get_mut(&path_id)
+4212 .unwrap()
+4213 .remove_in_flight(&packet);
+4214 }
+4215 } else {
+4216 self.accepted_0rtt = true;
+4217 params.validate_resumption_from(&self.peer_params)?;
+4218 }
+4219 }
+4220 if let Some(token) = params.stateless_reset_token {
+4221 // TODO(matheus23): Reset token for a remote, or for a 4-tuple?
+4222 let remote = self.path_data(path_id).network_path.remote;
+4223 self.endpoint_events
+4224 .push_back(EndpointEventInner::ResetToken(path_id, remote, token));
+4225 }
+4226 self.handle_peer_params(params, loc_cid, rem_cid, now)?;
+4227 self.issue_first_cids(now);
+4228 } else {
+4229 // Server-only
+4230 self.spaces[SpaceId::Data].pending.handshake_done = true;
+4231 self.discard_space(now, SpaceId::Handshake);
+4232 self.events.push_back(Event::HandshakeConfirmed);
+4233 trace!("handshake confirmed");
+4234 }
+4235
+4236 self.events.push_back(Event::Connected);
+4237 self.state.move_to_established();
+4238 trace!("established");
+4239
+4240 // Multipath can only be enabled after the state has reached Established.
+4241 // So this can not happen any earlier.
+4242 self.issue_first_path_cids(now);
+4243 Ok(())
+4244 }
+4245 Header::Initial(InitialHeader {
+4246 src_cid: rem_cid,
+4247 dst_cid: loc_cid,
+4248 ..
+4249 }) => {
+4250 debug_assert_eq!(path_id, PathId::ZERO);
+4251 if !state.rem_cid_set {
+4252 trace!("switching remote CID to {}", rem_cid);
+4253 let mut state = state.clone();
+4254 self.rem_cids
+4255 .get_mut(&path_id)
+4256 .expect("PathId::ZERO not yet abandoned")
+4257 .update_initial_cid(rem_cid);
+4258 self.rem_handshake_cid = rem_cid;
+4259 self.orig_rem_cid = rem_cid;
+4260 state.rem_cid_set = true;
+4261 self.state.move_to_handshake(state);
+4262 } else if rem_cid != self.rem_handshake_cid {
+4263 debug!(
+4264 "discarding packet with mismatched remote CID: {} != {}",
+4265 self.rem_handshake_cid, rem_cid
+4266 );
+4267 return Ok(());
+4268 }
+4269
+4270 let starting_space = self.highest_space;
+4271 self.process_early_payload(now, path_id, packet, qlog)?;
+4272
+4273 if self.side.is_server()
+4274 && starting_space == SpaceId::Initial
+4275 && self.highest_space != SpaceId::Initial
+4276 {
+4277 let params = self.crypto.transport_parameters()?.ok_or_else(|| {
+4278 TransportError::new(
+4279 TransportErrorCode::crypto(0x6d),
+4280 "transport parameters missing".to_owned(),
+4281 )
+4282 })?;
+4283 self.handle_peer_params(params, loc_cid, rem_cid, now)?;
+4284 self.issue_first_cids(now);
+4285 self.init_0rtt(now);
+4286 }
+4287 Ok(())
+4288 }
+4289 Header::Long {
+4290 ty: LongType::ZeroRtt,
+4291 ..
+4292 } => {
+4293 self.process_payload(now, network_path, path_id, number.unwrap(), packet, qlog)?;
+4294 Ok(())
+4295 }
+4296 Header::VersionNegotiate { .. } => {
+4297 if self.total_authed_packets > 1 {
+4298 return Ok(());
4299 }
-4300 f => {
-4301 trace!("got frame {f}");
-4302 }
-4303 }
-4304
-4305 let _guard = span.enter();
-4306 if packet.header.is_0rtt() {
-4307 match frame {
-4308 Frame::Crypto(_) | Frame::Close(Close::Application(_)) => {
-4309 return Err(TransportError::PROTOCOL_VIOLATION(
-4310 "illegal frame type in 0-RTT",
-4311 ));
-4312 }
-4313 _ => {
-4314 if frame.is_1rtt() {
-4315 return Err(TransportError::PROTOCOL_VIOLATION(
-4316 "illegal frame type in 0-RTT",
-4317 ));
-4318 }
-4319 }
-4320 }
-4321 }
-4322 ack_eliciting |= frame.is_ack_eliciting();
-4323
-4324 // Check whether this could be a probing packet
-4325 match frame {
-4326 Frame::Padding
-4327 | Frame::PathChallenge(_)
-4328 | Frame::PathResponse(_)
-4329 | Frame::NewConnectionId(_)
-4330 | Frame::ObservedAddr(_) => {}
-4331 _ => {
-4332 is_probing_packet = false;
-4333 }
-4334 }
-4335
-4336 match frame {
-4337 Frame::Crypto(frame) => {
-4338 self.read_crypto(SpaceId::Data, &frame, payload_len)?;
-4339 }
-4340 Frame::Stream(frame) => {
-4341 if self.streams.received(frame, payload_len)?.should_transmit() {
-4342 self.spaces[SpaceId::Data].pending.max_data = true;
-4343 }
-4344 }
-4345 Frame::Ack(ack) => {
-4346 self.on_ack_received(now, SpaceId::Data, ack)?;
-4347 }
-4348 Frame::PathAck(ack) => {
-4349 span.record("path", tracing::field::debug(&ack.path_id));
-4350 self.on_path_ack_received(now, SpaceId::Data, ack)?;
-4351 }
+4300 let supported = packet
+4301 .payload
+4302 .chunks(4)
+4303 .any(|x| match <[u8; 4]>::try_from(x) {
+4304 Ok(version) => self.version == u32::from_be_bytes(version),
+4305 Err(_) => false,
+4306 });
+4307 if supported {
+4308 return Ok(());
+4309 }
+4310 debug!("remote doesn't support our version");
+4311 Err(ConnectionError::VersionMismatch)
+4312 }
+4313 Header::Short { .. } => unreachable!(
+4314 "short packets received during handshake are discarded in handle_packet"
+4315 ),
+4316 }
+4317 }
+4318
+4319 /// Process an Initial or Handshake packet payload
+4320 fn process_early_payload(
+4321 &mut self,
+4322 now: Instant,
+4323 path_id: PathId,
+4324 packet: Packet,
+4325 #[allow(unused)] qlog: &mut QlogRecvPacket,
+4326 ) -> Result<(), TransportError> {
+4327 debug_assert_ne!(packet.header.space(), SpaceId::Data);
+4328 debug_assert_eq!(path_id, PathId::ZERO);
+4329 let payload_len = packet.payload.len();
+4330 let mut ack_eliciting = false;
+4331 for result in frame::Iter::new(packet.payload.freeze())? {
+4332 let frame = result?;
+4333 qlog.frame(&frame);
+4334 let span = match frame {
+4335 Frame::Padding => continue,
+4336 _ => Some(trace_span!("frame", ty = %frame.ty(), path = tracing::field::Empty)),
+4337 };
+4338
+4339 self.stats.frame_rx.record(frame.ty());
+4340
+4341 let _guard = span.as_ref().map(|x| x.enter());
+4342 ack_eliciting |= frame.is_ack_eliciting();
+4343
+4344 // Process frames
+4345 if frame.is_1rtt() && packet.header.space() != SpaceId::Data {
+4346 return Err(TransportError::PROTOCOL_VIOLATION(
+4347 "illegal frame type in handshake",
+4348 ));
+4349 }
+4350
+4351 match frame {
4352 Frame::Padding | Frame::Ping => {}
-4353 Frame::Close(reason) => {
-4354 close = Some(reason);
+4353 Frame::Crypto(frame) => {
+4354 self.read_crypto(packet.header.space(), &frame, payload_len)?;
4355 }
-4356 Frame::PathChallenge(challenge) => {
-4357 let path = &mut self
-4358 .path_mut(path_id)
-4359 .expect("payload is processed only after the path becomes known");
-4360 path.path_responses.push(number, challenge.0, network_path);
-4361 // At this point, update_network_path_or_discard was already called, so
-4362 // we don't need to be lenient about `local_ip` possibly mis-matching.
-4363 if network_path == path.network_path {
-4364 // PATH_CHALLENGE on active path, possible off-path packet forwarding
-4365 // attack. Send a non-probing packet to recover the active path.
-4366 // TODO(flub): No longer true! We now path_challege also to validate
-4367 // the path if the path is new, without an RFC9000-style
-4368 // migration involved. This means we add in an extra
-4369 // IMMEDIATE_ACK on some challenges. It isn't really wrong to do
-4370 // so, but it still is something untidy. We should instead
-4371 // suppress this when we know the remote is still validating the
-4372 // path.
-4373 match self.peer_supports_ack_frequency() {
-4374 true => self.immediate_ack(path_id),
-4375 false => {
-4376 self.ping_path(path_id).ok();
-4377 }
-4378 }
-4379 }
-4380 }
-4381 Frame::PathResponse(response) => {
-4382 let path = self
-4383 .paths
-4384 .get_mut(&path_id)
-4385 .expect("payload is processed only after the path becomes known");
-4386
-4387 use PathTimer::*;
-4388 use paths::OnPathResponseReceived::*;
-4389 match path
-4390 .data
-4391 .on_path_response_received(now, response.0, network_path)
-4392 {
-4393 OnPath { was_open } => {
-4394 let qlog = self.qlog.with_time(now);
-4395
-4396 self.timers
-4397 .stop(Timer::PerPath(path_id, PathValidation), qlog.clone());
-4398 self.timers
-4399 .stop(Timer::PerPath(path_id, PathOpen), qlog.clone());
-4400
-4401 let next_challenge = path
-4402 .data
-4403 .earliest_expiring_challenge()
-4404 .map(|time| time + self.ack_frequency.max_ack_delay_for_pto());
-4405 self.timers.set_or_stop(
-4406 Timer::PerPath(path_id, PathChallengeLost),
-4407 next_challenge,
-4408 qlog,
-4409 );
-4410
-4411 if !was_open {
-4412 self.events
-4413 .push_back(Event::Path(PathEvent::Opened { id: path_id }));
-4414 if let Some(observed) = path.data.last_observed_addr_report.as_ref()
-4415 {
-4416 self.events.push_back(Event::Path(PathEvent::ObservedAddr {
-4417 id: path_id,
-4418 addr: observed.socket_addr(),
-4419 }));
-4420 }
-4421 }
-4422 if let Some((_, ref mut prev)) = path.prev {
-4423 prev.challenges_sent.clear();
-4424 prev.send_new_challenge = false;
-4425 }
-4426 }
-4427 OffPath => {
-4428 debug!("Response to off-path PathChallenge!");
-4429 let next_challenge = path
-4430 .data
-4431 .earliest_expiring_challenge()
-4432 .map(|time| time + self.ack_frequency.max_ack_delay_for_pto());
-4433 self.timers.set_or_stop(
-4434 Timer::PerPath(path_id, PathChallengeLost),
-4435 next_challenge,
-4436 self.qlog.with_time(now),
-4437 );
-4438 }
-4439 Invalid { expected } => {
-4440 debug!(%response, %network_path, %expected, "ignoring invalid PATH_RESPONSE")
-4441 }
-4442 Unknown => debug!(%response, "ignoring invalid PATH_RESPONSE"),
-4443 }
-4444 }
-4445 Frame::MaxData(frame::MaxData(bytes)) => {
-4446 self.streams.received_max_data(bytes);
-4447 }
-4448 Frame::MaxStreamData(frame::MaxStreamData { id, offset }) => {
-4449 self.streams.received_max_stream_data(id, offset)?;
-4450 }
-4451 Frame::MaxStreams(frame::MaxStreams { dir, count }) => {
-4452 self.streams.received_max_streams(dir, count)?;
-4453 }
-4454 Frame::ResetStream(frame) => {
-4455 if self.streams.received_reset(frame)?.should_transmit() {
-4456 self.spaces[SpaceId::Data].pending.max_data = true;
-4457 }
-4458 }
-4459 Frame::DataBlocked { offset } => {
-4460 debug!(offset, "peer claims to be blocked at connection level");
+4356 Frame::Ack(ack) => {
+4357 self.on_ack_received(now, packet.header.space(), ack)?;
+4358 }
+4359 Frame::PathAck(ack) => {
+4360 span.as_ref()
+4361 .map(|span| span.record("path", tracing::field::debug(&ack.path_id)));
+4362 self.on_path_ack_received(now, packet.header.space(), ack)?;
+4363 }
+4364 Frame::Close(reason) => {
+4365 self.state.move_to_draining(Some(reason.into()));
+4366 return Ok(());
+4367 }
+4368 _ => {
+4369 let mut err =
+4370 TransportError::PROTOCOL_VIOLATION("illegal frame type in handshake");
+4371 err.frame = frame::MaybeFrame::Known(frame.ty());
+4372 return Err(err);
+4373 }
+4374 }
+4375 }
+4376
+4377 if ack_eliciting {
+4378 // In the initial and handshake spaces, ACKs must be sent immediately
+4379 self.spaces[packet.header.space()]
+4380 .for_path(path_id)
+4381 .pending_acks
+4382 .set_immediate_ack_required();
+4383 }
+4384
+4385 self.write_crypto();
+4386 Ok(())
+4387 }
+4388
+4389 /// Processes the packet payload, always in the data space.
+4390 fn process_payload(
+4391 &mut self,
+4392 now: Instant,
+4393 network_path: FourTuple,
+4394 path_id: PathId,
+4395 number: u64,
+4396 packet: Packet,
+4397 #[allow(unused)] qlog: &mut QlogRecvPacket,
+4398 ) -> Result<(), TransportError> {
+4399 let payload = packet.payload.freeze();
+4400 let mut is_probing_packet = true;
+4401 let mut close = None;
+4402 let payload_len = payload.len();
+4403 let mut ack_eliciting = false;
+4404 // if this packet triggers a path migration and includes a observed address frame, it's
+4405 // stored here
+4406 let mut migration_observed_addr = None;
+4407 for result in frame::Iter::new(payload)? {
+4408 let frame = result?;
+4409 qlog.frame(&frame);
+4410 let span = match frame {
+4411 Frame::Padding => continue,
+4412 _ => trace_span!("frame", ty = %frame.ty(), path = tracing::field::Empty),
+4413 };
+4414
+4415 self.stats.frame_rx.record(frame.ty());
+4416 // Crypto, Stream and Datagram frames are special cased in order no pollute
+4417 // the log with payload data
+4418 match &frame {
+4419 Frame::Crypto(f) => {
+4420 trace!(offset = f.offset, len = f.data.len(), "got crypto frame");
+4421 }
+4422 Frame::Stream(f) => {
+4423 trace!(id = %f.id, offset = f.offset, len = f.data.len(), fin = f.fin, "got stream frame");
+4424 }
+4425 Frame::Datagram(f) => {
+4426 trace!(len = f.data.len(), "got datagram frame");
+4427 }
+4428 f => {
+4429 trace!("got frame {f}");
+4430 }
+4431 }
+4432
+4433 let _guard = span.enter();
+4434 if packet.header.is_0rtt() {
+4435 match frame {
+4436 Frame::Crypto(_) | Frame::Close(Close::Application(_)) => {
+4437 return Err(TransportError::PROTOCOL_VIOLATION(
+4438 "illegal frame type in 0-RTT",
+4439 ));
+4440 }
+4441 _ => {
+4442 if frame.is_1rtt() {
+4443 return Err(TransportError::PROTOCOL_VIOLATION(
+4444 "illegal frame type in 0-RTT",
+4445 ));
+4446 }
+4447 }
+4448 }
+4449 }
+4450 ack_eliciting |= frame.is_ack_eliciting();
+4451
+4452 // Check whether this could be a probing packet
+4453 match frame {
+4454 Frame::Padding
+4455 | Frame::PathChallenge(_)
+4456 | Frame::PathResponse(_)
+4457 | Frame::NewConnectionId(_)
+4458 | Frame::ObservedAddr(_) => {}
+4459 _ => {
+4460 is_probing_packet = false;
4461 }
-4462 Frame::StreamDataBlocked { id, offset } => {
-4463 if id.initiator() == self.side.side() && id.dir() == Dir::Uni {
-4464 debug!("got STREAM_DATA_BLOCKED on send-only {}", id);
-4465 return Err(TransportError::STREAM_STATE_ERROR(
-4466 "STREAM_DATA_BLOCKED on send-only stream",
-4467 ));
-4468 }
-4469 debug!(
-4470 stream = %id,
-4471 offset, "peer claims to be blocked at stream level"
-4472 );
-4473 }
-4474 Frame::StreamsBlocked { dir, limit } => {
-4475 if limit > MAX_STREAM_COUNT {
-4476 return Err(TransportError::FRAME_ENCODING_ERROR(
-4477 "unrepresentable stream limit",
-4478 ));
-4479 }
-4480 debug!(
-4481 "peer claims to be blocked opening more than {} {} streams",
-4482 limit, dir
-4483 );
-4484 }
-4485 Frame::StopSending(frame::StopSending { id, error_code }) => {
-4486 if id.initiator() != self.side.side() {
-4487 if id.dir() == Dir::Uni {
-4488 debug!("got STOP_SENDING on recv-only {}", id);
-4489 return Err(TransportError::STREAM_STATE_ERROR(
-4490 "STOP_SENDING on recv-only stream",
-4491 ));
-4492 }
-4493 } else if self.streams.is_local_unopened(id) {
-4494 return Err(TransportError::STREAM_STATE_ERROR(
-4495 "STOP_SENDING on unopened stream",
-4496 ));
-4497 }
-4498 self.streams.received_stop_sending(id, error_code);
-4499 }
-4500 Frame::RetireConnectionId(frame::RetireConnectionId { path_id, sequence }) => {
-4501 if let Some(ref path_id) = path_id {
-4502 span.record("path", tracing::field::debug(&path_id));
-4503 }
-4504 let path_id = path_id.unwrap_or_default();
-4505 match self.local_cid_state.get_mut(&path_id) {
-4506 None => error!(?path_id, "RETIRE_CONNECTION_ID for unknown path"),
-4507 Some(cid_state) => {
-4508 let allow_more_cids = cid_state
-4509 .on_cid_retirement(sequence, self.peer_params.issue_cids_limit())?;
-4510
-4511 // If the path has closed, we do not issue more CIDs for this path
-4512 // For details see https://www.ietf.org/archive/id/draft-ietf-quic-multipath-17.html#section-3.2.2
-4513 // > an endpoint SHOULD provide new connection IDs for that path, if still open, using PATH_NEW_CONNECTION_ID frames.
-4514 let has_path = !self.abandoned_paths.contains(&path_id);
-4515 let allow_more_cids = allow_more_cids && has_path;
-4516
-4517 self.endpoint_events
-4518 .push_back(EndpointEventInner::RetireConnectionId(
-4519 now,
-4520 path_id,
-4521 sequence,
-4522 allow_more_cids,
-4523 ));
-4524 }
-4525 }
-4526 }
-4527 Frame::NewConnectionId(frame) => {
-4528 let path_id = if let Some(path_id) = frame.path_id {
-4529 if !self.is_multipath_negotiated() {
-4530 return Err(TransportError::PROTOCOL_VIOLATION(
-4531 "received PATH_NEW_CONNECTION_ID frame when multipath was not negotiated",
-4532 ));
-4533 }
-4534 if path_id > self.local_max_path_id {
-4535 return Err(TransportError::PROTOCOL_VIOLATION(
-4536 "PATH_NEW_CONNECTION_ID contains path_id exceeding current max",
-4537 ));
-4538 }
-4539 path_id
-4540 } else {
-4541 PathId::ZERO
-4542 };
-4543
-4544 if self.abandoned_paths.contains(&path_id) {
-4545 trace!("ignoring issued CID for abandoned path");
-4546 continue;
-4547 }
-4548 if let Some(ref path_id) = frame.path_id {
-4549 span.record("path", tracing::field::debug(&path_id));
-4550 }
-4551 let rem_cids = self
-4552 .rem_cids
-4553 .entry(path_id)
-4554 .or_insert_with(|| CidQueue::new(frame.id));
-4555 if rem_cids.active().is_empty() {
-4556 return Err(TransportError::PROTOCOL_VIOLATION(
-4557 "NEW_CONNECTION_ID when CIDs aren't in use",
-4558 ));
-4559 }
-4560 if frame.retire_prior_to > frame.sequence {
-4561 return Err(TransportError::PROTOCOL_VIOLATION(
-4562 "NEW_CONNECTION_ID retiring unissued CIDs",
-4563 ));
-4564 }
-4565
-4566 use crate::cid_queue::InsertError;
-4567 match rem_cids.insert(frame) {
-4568 Ok(None) if self.path(path_id).is_none() => {
-4569 // if this gives us CIDs to open a new path and a nat traversal attempt
-4570 // is underway we could try to probe a pending remote
-4571 self.continue_nat_traversal_round(now);
-4572 }
-4573 Ok(None) => {}
-4574 Ok(Some((retired, reset_token))) => {
-4575 let pending_retired =
-4576 &mut self.spaces[SpaceId::Data].pending.retire_cids;
-4577 /// Ensure `pending_retired` cannot grow without bound. Limit is
-4578 /// somewhat arbitrary but very permissive.
-4579 const MAX_PENDING_RETIRED_CIDS: u64 = CidQueue::LEN as u64 * 10;
-4580 // We don't bother counting in-flight frames because those are bounded
-4581 // by congestion control.
-4582 if (pending_retired.len() as u64)
-4583 .saturating_add(retired.end.saturating_sub(retired.start))
-4584 > MAX_PENDING_RETIRED_CIDS
-4585 {
-4586 return Err(TransportError::CONNECTION_ID_LIMIT_ERROR(
-4587 "queued too many retired CIDs",
-4588 ));
-4589 }
-4590 pending_retired.extend(retired.map(|seq| (path_id, seq)));
-4591 // TODO(matheus23): Reset token for a remote or a full 4-tuple?
-4592 self.set_reset_token(path_id, network_path.remote, reset_token);
-4593 }
-4594 Err(InsertError::ExceedsLimit) => {
-4595 return Err(TransportError::CONNECTION_ID_LIMIT_ERROR(""));
-4596 }
-4597 Err(InsertError::Retired) => {
-4598 trace!("discarding already-retired");
-4599 // RETIRE_CONNECTION_ID might not have been previously sent if e.g. a
-4600 // range of connection IDs larger than the active connection ID limit
-4601 // was retired all at once via retire_prior_to.
-4602 self.spaces[SpaceId::Data]
-4603 .pending
-4604 .retire_cids
-4605 .push((path_id, frame.sequence));
-4606 continue;
-4607 }
-4608 };
-4609
-4610 if self.side.is_server()
-4611 && path_id == PathId::ZERO
-4612 && self
-4613 .rem_cids
-4614 .get(&PathId::ZERO)
-4615 .map(|cids| cids.active_seq() == 0)
-4616 .unwrap_or_default()
-4617 {
-4618 // We're a server still using the initial remote CID for the client, so
-4619 // let's switch immediately to enable clientside stateless resets.
-4620 self.update_rem_cid(PathId::ZERO);
-4621 }
-4622 }
-4623 Frame::NewToken(NewToken { token }) => {
-4624 let ConnectionSide::Client {
-4625 token_store,
-4626 server_name,
-4627 ..
-4628 } = &self.side
-4629 else {
-4630 return Err(TransportError::PROTOCOL_VIOLATION("client sent NEW_TOKEN"));
-4631 };
-4632 if token.is_empty() {
-4633 return Err(TransportError::FRAME_ENCODING_ERROR("empty token"));
-4634 }
-4635 trace!("got new token");
-4636 token_store.insert(server_name, token);
-4637 }
-4638 Frame::Datagram(datagram) => {
-4639 if self
-4640 .datagrams
-4641 .received(datagram, &self.config.datagram_receive_buffer_size)?
-4642 {
-4643 self.events.push_back(Event::DatagramReceived);
-4644 }
-4645 }
-4646 Frame::AckFrequency(ack_frequency) => {
-4647 // This frame can only be sent in the Data space
-4648
-4649 if !self.ack_frequency.ack_frequency_received(&ack_frequency)? {
-4650 // The AckFrequency frame is stale (we have already received a more
-4651 // recent one)
-4652 continue;
+4462 }
+4463
+4464 match frame {
+4465 Frame::Crypto(frame) => {
+4466 self.read_crypto(SpaceId::Data, &frame, payload_len)?;
+4467 }
+4468 Frame::Stream(frame) => {
+4469 if self.streams.received(frame, payload_len)?.should_transmit() {
+4470 self.spaces[SpaceId::Data].pending.max_data = true;
+4471 }
+4472 }
+4473 Frame::Ack(ack) => {
+4474 self.on_ack_received(now, SpaceId::Data, ack)?;
+4475 }
+4476 Frame::PathAck(ack) => {
+4477 span.record("path", tracing::field::debug(&ack.path_id));
+4478 self.on_path_ack_received(now, SpaceId::Data, ack)?;
+4479 }
+4480 Frame::Padding | Frame::Ping => {}
+4481 Frame::Close(reason) => {
+4482 close = Some(reason);
+4483 }
+4484 Frame::PathChallenge(challenge) => {
+4485 let path = &mut self
+4486 .path_mut(path_id)
+4487 .expect("payload is processed only after the path becomes known");
+4488 path.path_responses.push(number, challenge.0, network_path);
+4489 // At this point, update_network_path_or_discard was already called, so
+4490 // we don't need to be lenient about `local_ip` possibly mis-matching.
+4491 if network_path == path.network_path {
+4492 // PATH_CHALLENGE on active path, possible off-path packet forwarding
+4493 // attack. Send a non-probing packet to recover the active path.
+4494 // TODO(flub): No longer true! We now path_challege also to validate
+4495 // the path if the path is new, without an RFC9000-style
+4496 // migration involved. This means we add in an extra
+4497 // IMMEDIATE_ACK on some challenges. It isn't really wrong to do
+4498 // so, but it still is something untidy. We should instead
+4499 // suppress this when we know the remote is still validating the
+4500 // path.
+4501 match self.peer_supports_ack_frequency() {
+4502 true => self.immediate_ack(path_id),
+4503 false => {
+4504 self.ping_path(path_id).ok();
+4505 }
+4506 }
+4507 }
+4508 }
+4509 Frame::PathResponse(response) => {
+4510 let path = self
+4511 .paths
+4512 .get_mut(&path_id)
+4513 .expect("payload is processed only after the path becomes known");
+4514
+4515 use PathTimer::*;
+4516 use paths::OnPathResponseReceived::*;
+4517 match path
+4518 .data
+4519 .on_path_response_received(now, response.0, network_path)
+4520 {
+4521 OnPath { was_open } => {
+4522 let qlog = self.qlog.with_time(now);
+4523
+4524 self.timers
+4525 .stop(Timer::PerPath(path_id, PathValidation), qlog.clone());
+4526 self.timers
+4527 .stop(Timer::PerPath(path_id, PathOpen), qlog.clone());
+4528
+4529 let next_challenge = path
+4530 .data
+4531 .earliest_expiring_challenge()
+4532 .map(|time| time + self.ack_frequency.max_ack_delay_for_pto());
+4533 self.timers.set_or_stop(
+4534 Timer::PerPath(path_id, PathChallengeLost),
+4535 next_challenge,
+4536 qlog,
+4537 );
+4538
+4539 if !was_open {
+4540 self.events
+4541 .push_back(Event::Path(PathEvent::Opened { id: path_id }));
+4542 if let Some(observed) = path.data.last_observed_addr_report.as_ref()
+4543 {
+4544 self.events.push_back(Event::Path(PathEvent::ObservedAddr {
+4545 id: path_id,
+4546 addr: observed.socket_addr(),
+4547 }));
+4548 }
+4549 }
+4550 if let Some((_, ref mut prev)) = path.prev {
+4551 prev.challenges_sent.clear();
+4552 prev.send_new_challenge = false;
+4553 }
+4554 }
+4555 OffPath => {
+4556 debug!("Response to off-path PathChallenge!");
+4557 let next_challenge = path
+4558 .data
+4559 .earliest_expiring_challenge()
+4560 .map(|time| time + self.ack_frequency.max_ack_delay_for_pto());
+4561 self.timers.set_or_stop(
+4562 Timer::PerPath(path_id, PathChallengeLost),
+4563 next_challenge,
+4564 self.qlog.with_time(now),
+4565 );
+4566 }
+4567 Invalid { expected } => {
+4568 debug!(%response, %network_path, %expected, "ignoring invalid PATH_RESPONSE")
+4569 }
+4570 Unknown => debug!(%response, "ignoring invalid PATH_RESPONSE"),
+4571 }
+4572 }
+4573 Frame::MaxData(frame::MaxData(bytes)) => {
+4574 self.streams.received_max_data(bytes);
+4575 }
+4576 Frame::MaxStreamData(frame::MaxStreamData { id, offset }) => {
+4577 self.streams.received_max_stream_data(id, offset)?;
+4578 }
+4579 Frame::MaxStreams(frame::MaxStreams { dir, count }) => {
+4580 self.streams.received_max_streams(dir, count)?;
+4581 }
+4582 Frame::ResetStream(frame) => {
+4583 if self.streams.received_reset(frame)?.should_transmit() {
+4584 self.spaces[SpaceId::Data].pending.max_data = true;
+4585 }
+4586 }
+4587 Frame::DataBlocked { offset } => {
+4588 debug!(offset, "peer claims to be blocked at connection level");
+4589 }
+4590 Frame::StreamDataBlocked { id, offset } => {
+4591 if id.initiator() == self.side.side() && id.dir() == Dir::Uni {
+4592 debug!("got STREAM_DATA_BLOCKED on send-only {}", id);
+4593 return Err(TransportError::STREAM_STATE_ERROR(
+4594 "STREAM_DATA_BLOCKED on send-only stream",
+4595 ));
+4596 }
+4597 debug!(
+4598 stream = %id,
+4599 offset, "peer claims to be blocked at stream level"
+4600 );
+4601 }
+4602 Frame::StreamsBlocked { dir, limit } => {
+4603 if limit > MAX_STREAM_COUNT {
+4604 return Err(TransportError::FRAME_ENCODING_ERROR(
+4605 "unrepresentable stream limit",
+4606 ));
+4607 }
+4608 debug!(
+4609 "peer claims to be blocked opening more than {} {} streams",
+4610 limit, dir
+4611 );
+4612 }
+4613 Frame::StopSending(frame::StopSending { id, error_code }) => {
+4614 if id.initiator() != self.side.side() {
+4615 if id.dir() == Dir::Uni {
+4616 debug!("got STOP_SENDING on recv-only {}", id);
+4617 return Err(TransportError::STREAM_STATE_ERROR(
+4618 "STOP_SENDING on recv-only stream",
+4619 ));
+4620 }
+4621 } else if self.streams.is_local_unopened(id) {
+4622 return Err(TransportError::STREAM_STATE_ERROR(
+4623 "STOP_SENDING on unopened stream",
+4624 ));
+4625 }
+4626 self.streams.received_stop_sending(id, error_code);
+4627 }
+4628 Frame::RetireConnectionId(frame::RetireConnectionId { path_id, sequence }) => {
+4629 if let Some(ref path_id) = path_id {
+4630 span.record("path", tracing::field::debug(&path_id));
+4631 }
+4632 let path_id = path_id.unwrap_or_default();
+4633 match self.local_cid_state.get_mut(&path_id) {
+4634 None => error!(?path_id, "RETIRE_CONNECTION_ID for unknown path"),
+4635 Some(cid_state) => {
+4636 let allow_more_cids = cid_state
+4637 .on_cid_retirement(sequence, self.peer_params.issue_cids_limit())?;
+4638
+4639 // If the path has closed, we do not issue more CIDs for this path
+4640 // For details see https://www.ietf.org/archive/id/draft-ietf-quic-multipath-17.html#section-3.2.2
+4641 // > an endpoint SHOULD provide new connection IDs for that path, if still open, using PATH_NEW_CONNECTION_ID frames.
+4642 let has_path = !self.abandoned_paths.contains(&path_id);
+4643 let allow_more_cids = allow_more_cids && has_path;
+4644
+4645 self.endpoint_events
+4646 .push_back(EndpointEventInner::RetireConnectionId(
+4647 now,
+4648 path_id,
+4649 sequence,
+4650 allow_more_cids,
+4651 ));
+4652 }
4653 }
-4654
-4655 // Update the params for all of our paths
-4656 for (path_id, space) in self.spaces[SpaceId::Data].number_spaces.iter_mut() {
-4657 space.pending_acks.set_ack_frequency_params(&ack_frequency);
-4658
-4659 // Our `max_ack_delay` has been updated, so we may need to adjust
-4660 // its associated timeout
-4661 if let Some(timeout) = space
-4662 .pending_acks
-4663 .max_ack_delay_timeout(self.ack_frequency.max_ack_delay)
-4664 {
-4665 self.timers.set(
-4666 Timer::PerPath(*path_id, PathTimer::MaxAckDelay),
-4667 timeout,
-4668 self.qlog.with_time(now),
-4669 );
-4670 }
-4671 }
-4672 }
-4673 Frame::ImmediateAck => {
-4674 // This frame can only be sent in the Data space
-4675 for pns in self.spaces[SpaceId::Data].iter_paths_mut() {
-4676 pns.pending_acks.set_immediate_ack_required();
-4677 }
-4678 }
-4679 Frame::HandshakeDone => {
-4680 if self.side.is_server() {
-4681 return Err(TransportError::PROTOCOL_VIOLATION(
-4682 "client sent HANDSHAKE_DONE",
-4683 ));
-4684 }
-4685 if self.spaces[SpaceId::Handshake].crypto.is_some() {
-4686 self.discard_space(now, SpaceId::Handshake);
+4654 }
+4655 Frame::NewConnectionId(frame) => {
+4656 let path_id = if let Some(path_id) = frame.path_id {
+4657 if !self.is_multipath_negotiated() {
+4658 return Err(TransportError::PROTOCOL_VIOLATION(
+4659 "received PATH_NEW_CONNECTION_ID frame when multipath was not negotiated",
+4660 ));
+4661 }
+4662 if path_id > self.local_max_path_id {
+4663 return Err(TransportError::PROTOCOL_VIOLATION(
+4664 "PATH_NEW_CONNECTION_ID contains path_id exceeding current max",
+4665 ));
+4666 }
+4667 path_id
+4668 } else {
+4669 PathId::ZERO
+4670 };
+4671
+4672 if self.abandoned_paths.contains(&path_id) {
+4673 trace!("ignoring issued CID for abandoned path");
+4674 continue;
+4675 }
+4676 if let Some(ref path_id) = frame.path_id {
+4677 span.record("path", tracing::field::debug(&path_id));
+4678 }
+4679 let rem_cids = self
+4680 .rem_cids
+4681 .entry(path_id)
+4682 .or_insert_with(|| CidQueue::new(frame.id));
+4683 if rem_cids.active().is_empty() {
+4684 return Err(TransportError::PROTOCOL_VIOLATION(
+4685 "NEW_CONNECTION_ID when CIDs aren't in use",
+4686 ));
4687 }
-4688 self.events.push_back(Event::HandshakeConfirmed);
-4689 trace!("handshake confirmed");
-4690 }
-4691 Frame::ObservedAddr(observed) => {
-4692 // check if params allows the peer to send report and this node to receive it
-4693 trace!(seq_no = %observed.seq_no, ip = %observed.ip, port = observed.port);
-4694 if !self
-4695 .peer_params
-4696 .address_discovery_role
-4697 .should_report(&self.config.address_discovery_role)
-4698 {
-4699 return Err(TransportError::PROTOCOL_VIOLATION(
-4700 "received OBSERVED_ADDRESS frame when not negotiated",
-4701 ));
-4702 }
-4703 // must only be sent in data space
-4704 if packet.header.space() != SpaceId::Data {
-4705 return Err(TransportError::PROTOCOL_VIOLATION(
-4706 "OBSERVED_ADDRESS frame outside data space",
-4707 ));
-4708 }
-4709
-4710 let path = self.path_data_mut(path_id);
-4711 if network_path == path.network_path {
-4712 if let Some(updated) = path.update_observed_addr_report(observed) {
-4713 if path.open {
-4714 self.events.push_back(Event::Path(PathEvent::ObservedAddr {
-4715 id: path_id,
-4716 addr: updated,
-4717 }));
-4718 }
-4719 // otherwise the event is reported when the path is deemed open
-4720 }
-4721 } else {
-4722 // include in migration
-4723 migration_observed_addr = Some(observed)
-4724 }
-4725 }
-4726 Frame::PathAbandon(frame::PathAbandon {
-4727 path_id,
-4728 error_code,
-4729 }) => {
-4730 span.record("path", tracing::field::debug(&path_id));
-4731 // TODO(flub): don't really know which error code to use here.
-4732 match self.close_path(now, path_id, error_code.into()) {
-4733 Ok(()) => {
-4734 trace!("peer abandoned path");
+4688 if frame.retire_prior_to > frame.sequence {
+4689 return Err(TransportError::PROTOCOL_VIOLATION(
+4690 "NEW_CONNECTION_ID retiring unissued CIDs",
+4691 ));
+4692 }
+4693
+4694 use crate::cid_queue::InsertError;
+4695 match rem_cids.insert(frame) {
+4696 Ok(None) if self.path(path_id).is_none() => {
+4697 // if this gives us CIDs to open a new path and a nat traversal attempt
+4698 // is underway we could try to probe a pending remote
+4699 self.continue_nat_traversal_round(now);
+4700 }
+4701 Ok(None) => {}
+4702 Ok(Some((retired, reset_token))) => {
+4703 let pending_retired =
+4704 &mut self.spaces[SpaceId::Data].pending.retire_cids;
+4705 /// Ensure `pending_retired` cannot grow without bound. Limit is
+4706 /// somewhat arbitrary but very permissive.
+4707 const MAX_PENDING_RETIRED_CIDS: u64 = CidQueue::LEN as u64 * 10;
+4708 // We don't bother counting in-flight frames because those are bounded
+4709 // by congestion control.
+4710 if (pending_retired.len() as u64)
+4711 .saturating_add(retired.end.saturating_sub(retired.start))
+4712 > MAX_PENDING_RETIRED_CIDS
+4713 {
+4714 return Err(TransportError::CONNECTION_ID_LIMIT_ERROR(
+4715 "queued too many retired CIDs",
+4716 ));
+4717 }
+4718 pending_retired.extend(retired.map(|seq| (path_id, seq)));
+4719 // TODO(matheus23): Reset token for a remote or a full 4-tuple?
+4720 self.set_reset_token(path_id, network_path.remote, reset_token);
+4721 }
+4722 Err(InsertError::ExceedsLimit) => {
+4723 return Err(TransportError::CONNECTION_ID_LIMIT_ERROR(""));
+4724 }
+4725 Err(InsertError::Retired) => {
+4726 trace!("discarding already-retired");
+4727 // RETIRE_CONNECTION_ID might not have been previously sent if e.g. a
+4728 // range of connection IDs larger than the active connection ID limit
+4729 // was retired all at once via retire_prior_to.
+4730 self.spaces[SpaceId::Data]
+4731 .pending
+4732 .retire_cids
+4733 .push((path_id, frame.sequence));
+4734 continue;
4735 }
-4736 Err(ClosePathError::LastOpenPath) => {
-4737 trace!("peer abandoned last path, closing connection");
-4738 return Err(TransportError::NO_VIABLE_PATH(
-4739 "last path abandoned by peer",
-4740 ));
-4741 }
-4742 Err(ClosePathError::ClosedPath) => {
-4743 trace!("peer abandoned already closed path");
-4744 }
-4745 };
-4746 // If we receive a retransmit of PATH_ABANDON then we may already have
-4747 // abandoned this path locally. In that case the DiscardPath timer
-4748 // may already have fired and we no longer have any state for this path.
-4749 // Only set this timer if we still have path state.
-4750 if let Some(path) = self.paths.get_mut(&path_id) {
-4751 if !matches!(path.data.abandon_state, AbandonState::ReceivedPathAbandon) {
-4752 let ack_delay = self.ack_frequency.max_ack_delay_for_pto();
-4753 let pto = path.data.rtt.pto_base() + ack_delay;
-4754 self.timers.set(
-4755 Timer::PerPath(path_id, PathTimer::DiscardPath),
-4756 now + 3 * pto,
-4757 self.qlog.with_time(now),
-4758 );
-4759 // We received a PATH_ABANDON, we don't expect another one by a certain time.
-4760 path.data.abandon_state = AbandonState::ReceivedPathAbandon;
-4761 }
+4736 };
+4737
+4738 if self.side.is_server()
+4739 && path_id == PathId::ZERO
+4740 && self
+4741 .rem_cids
+4742 .get(&PathId::ZERO)
+4743 .map(|cids| cids.active_seq() == 0)
+4744 .unwrap_or_default()
+4745 {
+4746 // We're a server still using the initial remote CID for the client, so
+4747 // let's switch immediately to enable clientside stateless resets.
+4748 self.update_rem_cid(PathId::ZERO);
+4749 }
+4750 }
+4751 Frame::NewToken(NewToken { token }) => {
+4752 let ConnectionSide::Client {
+4753 token_store,
+4754 server_name,
+4755 ..
+4756 } = &self.side
+4757 else {
+4758 return Err(TransportError::PROTOCOL_VIOLATION("client sent NEW_TOKEN"));
+4759 };
+4760 if token.is_empty() {
+4761 return Err(TransportError::FRAME_ENCODING_ERROR("empty token"));
4762 }
-4763 }
-4764 Frame::PathStatusAvailable(info) => {
-4765 span.record("path", tracing::field::debug(&info.path_id));
-4766 if self.is_multipath_negotiated() {
-4767 self.on_path_status(
-4768 info.path_id,
-4769 PathStatus::Available,
-4770 info.status_seq_no,
-4771 );
-4772 } else {
-4773 return Err(TransportError::PROTOCOL_VIOLATION(
-4774 "received PATH_STATUS_AVAILABLE frame when multipath was not negotiated",
-4775 ));
-4776 }
-4777 }
-4778 Frame::PathStatusBackup(info) => {
-4779 span.record("path", tracing::field::debug(&info.path_id));
-4780 if self.is_multipath_negotiated() {
-4781 self.on_path_status(info.path_id, PathStatus::Backup, info.status_seq_no);
-4782 } else {
-4783 return Err(TransportError::PROTOCOL_VIOLATION(
-4784 "received PATH_STATUS_BACKUP frame when multipath was not negotiated",
-4785 ));
-4786 }
-4787 }
-4788 Frame::MaxPathId(frame::MaxPathId(path_id)) => {
-4789 span.record("path", tracing::field::debug(&path_id));
-4790 if !self.is_multipath_negotiated() {
-4791 return Err(TransportError::PROTOCOL_VIOLATION(
-4792 "received MAX_PATH_ID frame when multipath was not negotiated",
-4793 ));
-4794 }
-4795 // frames that do not increase the path id are ignored
-4796 if path_id > self.remote_max_path_id {
-4797 self.remote_max_path_id = path_id;
-4798 self.issue_first_path_cids(now);
-4799 while let Some(true) = self.continue_nat_traversal_round(now) {}
-4800 }
-4801 }
-4802 Frame::PathsBlocked(frame::PathsBlocked(max_path_id)) => {
-4803 // Receipt of a value of Maximum Path Identifier or Path Identifier that is higher than the local maximum value MUST
-4804 // be treated as a connection error of type PROTOCOL_VIOLATION.
-4805 // Ref <https://www.ietf.org/archive/id/draft-ietf-quic-multipath-14.html#name-paths_blocked-and-path_cids>
-4806 if self.is_multipath_negotiated() {
-4807 if max_path_id > self.local_max_path_id {
-4808 return Err(TransportError::PROTOCOL_VIOLATION(
-4809 "PATHS_BLOCKED maximum path identifier was larger than local maximum",
-4810 ));
-4811 }
-4812 debug!("received PATHS_BLOCKED({:?})", max_path_id);
-4813 // TODO(@divma): ensure max concurrent paths
-4814 } else {
-4815 return Err(TransportError::PROTOCOL_VIOLATION(
-4816 "received PATHS_BLOCKED frame when not multipath was not negotiated",
-4817 ));
-4818 }
-4819 }
-4820 Frame::PathCidsBlocked(frame::PathCidsBlocked { path_id, next_seq }) => {
-4821 // Nothing to do. This is recorded in the frame stats, but otherwise we
-4822 // always issue all CIDs we're allowed to issue, so either this is an
-4823 // impatient peer or a bug on our side.
-4824
-4825 // Receipt of a value of Maximum Path Identifier or Path Identifier that is higher than the local maximum value MUST
-4826 // be treated as a connection error of type PROTOCOL_VIOLATION.
-4827 // Ref <https://www.ietf.org/archive/id/draft-ietf-quic-multipath-14.html#name-paths_blocked-and-path_cids>
-4828 if self.is_multipath_negotiated() {
-4829 if path_id > self.local_max_path_id {
-4830 return Err(TransportError::PROTOCOL_VIOLATION(
-4831 "PATH_CIDS_BLOCKED path identifier was larger than local maximum",
-4832 ));
-4833 }
-4834 if next_seq.0
-4835 > self
-4836 .local_cid_state
-4837 .get(&path_id)
-4838 .map(|cid_state| cid_state.active_seq().1 + 1)
-4839 .unwrap_or_default()
-4840 {
-4841 return Err(TransportError::PROTOCOL_VIOLATION(
-4842 "PATH_CIDS_BLOCKED next sequence number larger than in local state",
-4843 ));
-4844 }
-4845 debug!(%path_id, %next_seq, "received PATH_CIDS_BLOCKED");
-4846 } else {
-4847 return Err(TransportError::PROTOCOL_VIOLATION(
-4848 "received PATH_CIDS_BLOCKED frame when not multipath was not negotiated",
-4849 ));
-4850 }
-4851 }
-4852 Frame::AddAddress(addr) => {
-4853 let client_state = match self.iroh_hp.client_side_mut() {
-4854 Ok(state) => state,
-4855 Err(err) => {
-4856 return Err(TransportError::PROTOCOL_VIOLATION(format!(
-4857 "Nat traversal(ADD_ADDRESS): {err}"
-4858 )));
-4859 }
-4860 };
-4861
-4862 if !client_state.check_remote_address(&addr) {
-4863 // if the address is not valid we flag it, but update anyway
-4864 warn!(?addr, "server sent illegal ADD_ADDRESS frame");
-4865 }
-4866
-4867 match client_state.add_remote_address(addr) {
-4868 Ok(maybe_added) => {
-4869 if let Some(added) = maybe_added {
-4870 self.events.push_back(Event::NatTraversal(
-4871 iroh_hp::Event::AddressAdded(added),
-4872 ));
-4873 }
-4874 }
-4875 Err(e) => {
-4876 warn!(%e, "failed to add remote address")
-4877 }
-4878 }
-4879 }
-4880 Frame::RemoveAddress(addr) => {
-4881 let client_state = match self.iroh_hp.client_side_mut() {
-4882 Ok(state) => state,
-4883 Err(err) => {
-4884 return Err(TransportError::PROTOCOL_VIOLATION(format!(
-4885 "Nat traversal(REMOVE_ADDRESS): {err}"
-4886 )));
-4887 }
-4888 };
-4889 if let Some(removed_addr) = client_state.remove_remote_address(addr) {
-4890 self.events
-4891 .push_back(Event::NatTraversal(iroh_hp::Event::AddressRemoved(
-4892 removed_addr,
-4893 )));
-4894 }
-4895 }
-4896 Frame::ReachOut(reach_out) => {
-4897 let server_state = match self.iroh_hp.server_side_mut() {
-4898 Ok(state) => state,
-4899 Err(err) => {
-4900 return Err(TransportError::PROTOCOL_VIOLATION(format!(
-4901 "Nat traversal(REACH_OUT): {err}"
-4902 )));
-4903 }
-4904 };
-4905
-4906 if let Err(err) = server_state.handle_reach_out(reach_out) {
-4907 return Err(TransportError::PROTOCOL_VIOLATION(format!(
-4908 "Nat traversal(REACH_OUT): {err}"
-4909 )));
-4910 }
-4911 }
-4912 }
-4913 }
-4914
-4915 let space = self.spaces[SpaceId::Data].for_path(path_id);
-4916 if space
-4917 .pending_acks
-4918 .packet_received(now, number, ack_eliciting, &space.dedup)
-4919 {
-4920 if self.abandoned_paths.contains(&path_id) {
-4921 // § 3.4.3 QUIC-MULTIPATH: promptly send ACKs for packets received from
-4922 // abandoned paths.
-4923 space.pending_acks.set_immediate_ack_required();
-4924 } else {
-4925 self.timers.set(
-4926 Timer::PerPath(path_id, PathTimer::MaxAckDelay),
-4927 now + self.ack_frequency.max_ack_delay,
-4928 self.qlog.with_time(now),
-4929 );
-4930 }
-4931 }
-4932
-4933 // Issue stream ID credit due to ACKs of outgoing finish/resets and incoming finish/resets
-4934 // on stopped streams. Incoming finishes/resets on open streams are not handled here as they
-4935 // are only freed, and hence only issue credit, once the application has been notified
-4936 // during a read on the stream.
-4937 let pending = &mut self.spaces[SpaceId::Data].pending;
-4938 self.streams.queue_max_stream_id(pending);
-4939
-4940 if let Some(reason) = close {
-4941 self.state.move_to_draining(Some(reason.into()));
-4942 self.close = true;
-4943 }
-4944
-4945 if Some(number) == self.spaces[SpaceId::Data].for_path(path_id).rx_packet
-4946 && !is_probing_packet
-4947 && network_path != self.path_data(path_id).network_path
-4948 {
-4949 let ConnectionSide::Server { ref server_config } = self.side else {
-4950 panic!("packets from unknown remote should be dropped by clients");
-4951 };
-4952 debug_assert!(
-4953 server_config.migration,
-4954 "migration-initiating packets should have been dropped immediately"
-4955 );
-4956 self.migrate(path_id, now, network_path, migration_observed_addr);
-4957 // Break linkability, if possible
-4958 self.update_rem_cid(path_id);
-4959 self.spin = false;
-4960 }
-4961
-4962 Ok(())
-4963 }
-4964
-4965 fn migrate(
-4966 &mut self,
-4967 path_id: PathId,
-4968 now: Instant,
-4969 network_path: FourTuple,
-4970 observed_addr: Option<ObservedAddr>,
-4971 ) {
-4972 trace!(%network_path, %path_id, "migration initiated");
-4973 self.path_generation_counter = self.path_generation_counter.wrapping_add(1);
-4974 // TODO(@divma): conditions for path migration in multipath are very specific, check them
-4975 // again to prevent path migrations that should actually create a new path
-4976
-4977 // Reset rtt/congestion state for new path unless it looks like a NAT rebinding.
-4978 // Note that the congestion window will not grow until validation terminates. Helps mitigate
-4979 // amplification attacks performed by spoofing source addresses.
-4980 let prev_pto = self.pto(SpaceId::Data, path_id);
-4981 let known_path = self.paths.get_mut(&path_id).expect("known path");
-4982 let path = &mut known_path.data;
-4983 let mut new_path = if network_path.remote.is_ipv4()
-4984 && network_path.remote.ip() == path.network_path.remote.ip()
-4985 {
-4986 PathData::from_previous(network_path, path, self.path_generation_counter, now)
-4987 } else {
-4988 let peer_max_udp_payload_size =
-4989 u16::try_from(self.peer_params.max_udp_payload_size.into_inner())
-4990 .unwrap_or(u16::MAX);
-4991 PathData::new(
-4992 network_path,
-4993 self.allow_mtud,
-4994 Some(peer_max_udp_payload_size),
-4995 self.path_generation_counter,
-4996 now,
-4997 &self.config,
-4998 )
-4999 };
-5000 new_path.last_observed_addr_report = path.last_observed_addr_report.clone();
-5001 if let Some(report) = observed_addr {
-5002 if let Some(updated) = new_path.update_observed_addr_report(report) {
-5003 tracing::info!("adding observed addr event from migration");
-5004 self.events.push_back(Event::Path(PathEvent::ObservedAddr {
-5005 id: path_id,
-5006 addr: updated,
-5007 }));
-5008 }
-5009 }
-5010 new_path.send_new_challenge = true;
-5011
-5012 let mut prev = mem::replace(path, new_path);
-5013 // Don't clobber the original path if the previous one hasn't been validated yet
-5014 if !prev.is_validating_path() {
-5015 prev.send_new_challenge = true;
-5016 // We haven't updated the remote CID yet, this captures the remote CID we were using on
-5017 // the previous path.
-5018
-5019 known_path.prev = Some((self.rem_cids.get(&path_id).unwrap().active(), prev));
-5020 }
-5021
-5022 // We need to re-assign the correct remote to this path in qlog
-5023 self.qlog.emit_tuple_assigned(path_id, network_path, now);
-5024
-5025 self.timers.set(
-5026 Timer::PerPath(path_id, PathTimer::PathValidation),
-5027 now + 3 * cmp::max(self.pto(SpaceId::Data, path_id), prev_pto),
-5028 self.qlog.with_time(now),
-5029 );
-5030 }
-5031
-5032 /// Handle a change in the local address, i.e. an active migration
-5033 pub fn local_address_changed(&mut self) {
-5034 // TODO(flub): if multipath is enabled this needs to create a new path entirely.
-5035 self.update_rem_cid(PathId::ZERO);
-5036 self.ping();
-5037 }
-5038
-5039 /// Switch to a previously unused remote connection ID, if possible
-5040 fn update_rem_cid(&mut self, path_id: PathId) {
-5041 let Some((reset_token, retired)) =
-5042 self.rem_cids.get_mut(&path_id).and_then(|cids| cids.next())
-5043 else {
-5044 return;
-5045 };
-5046
-5047 // Retire the current remote CID and any CIDs we had to skip.
-5048 self.spaces[SpaceId::Data]
-5049 .pending
-5050 .retire_cids
-5051 .extend(retired.map(|seq| (path_id, seq)));
-5052 let remote = self.path_data(path_id).network_path.remote;
-5053 self.set_reset_token(path_id, remote, reset_token);
-5054 }
-5055
-5056 /// Sends this reset token to the endpoint
-5057 ///
-5058 /// The endpoint needs to know the reset tokens issued by the peer, so that if the peer
-5059 /// sends a reset token it knows to route it to this connection. See RFC 9000 section
-5060 /// 10.3. Stateless Reset.
-5061 ///
-5062 /// Reset tokens are different for each path, the endpoint identifies paths by peer
-5063 /// socket address however, not by path ID.
-5064 fn set_reset_token(&mut self, path_id: PathId, remote: SocketAddr, reset_token: ResetToken) {
-5065 self.endpoint_events
-5066 .push_back(EndpointEventInner::ResetToken(path_id, remote, reset_token));
+4763 trace!("got new token");
+4764 token_store.insert(server_name, token);
+4765 }
+4766 Frame::Datagram(datagram) => {
+4767 if self
+4768 .datagrams
+4769 .received(datagram, &self.config.datagram_receive_buffer_size)?
+4770 {
+4771 self.events.push_back(Event::DatagramReceived);
+4772 }
+4773 }
+4774 Frame::AckFrequency(ack_frequency) => {
+4775 // This frame can only be sent in the Data space
+4776
+4777 if !self.ack_frequency.ack_frequency_received(&ack_frequency)? {
+4778 // The AckFrequency frame is stale (we have already received a more
+4779 // recent one)
+4780 continue;
+4781 }
+4782
+4783 // Update the params for all of our paths
+4784 for (path_id, space) in self.spaces[SpaceId::Data].number_spaces.iter_mut() {
+4785 space.pending_acks.set_ack_frequency_params(&ack_frequency);
+4786
+4787 // Our `max_ack_delay` has been updated, so we may need to adjust
+4788 // its associated timeout
+4789 if let Some(timeout) = space
+4790 .pending_acks
+4791 .max_ack_delay_timeout(self.ack_frequency.max_ack_delay)
+4792 {
+4793 self.timers.set(
+4794 Timer::PerPath(*path_id, PathTimer::MaxAckDelay),
+4795 timeout,
+4796 self.qlog.with_time(now),
+4797 );
+4798 }
+4799 }
+4800 }
+4801 Frame::ImmediateAck => {
+4802 // This frame can only be sent in the Data space
+4803 for pns in self.spaces[SpaceId::Data].iter_paths_mut() {
+4804 pns.pending_acks.set_immediate_ack_required();
+4805 }
+4806 }
+4807 Frame::HandshakeDone => {
+4808 if self.side.is_server() {
+4809 return Err(TransportError::PROTOCOL_VIOLATION(
+4810 "client sent HANDSHAKE_DONE",
+4811 ));
+4812 }
+4813 if self.spaces[SpaceId::Handshake].crypto.is_some() {
+4814 self.discard_space(now, SpaceId::Handshake);
+4815 }
+4816 self.events.push_back(Event::HandshakeConfirmed);
+4817 trace!("handshake confirmed");
+4818 }
+4819 Frame::ObservedAddr(observed) => {
+4820 // check if params allows the peer to send report and this node to receive it
+4821 trace!(seq_no = %observed.seq_no, ip = %observed.ip, port = observed.port);
+4822 if !self
+4823 .peer_params
+4824 .address_discovery_role
+4825 .should_report(&self.config.address_discovery_role)
+4826 {
+4827 return Err(TransportError::PROTOCOL_VIOLATION(
+4828 "received OBSERVED_ADDRESS frame when not negotiated",
+4829 ));
+4830 }
+4831 // must only be sent in data space
+4832 if packet.header.space() != SpaceId::Data {
+4833 return Err(TransportError::PROTOCOL_VIOLATION(
+4834 "OBSERVED_ADDRESS frame outside data space",
+4835 ));
+4836 }
+4837
+4838 let path = self.path_data_mut(path_id);
+4839 if network_path == path.network_path {
+4840 if let Some(updated) = path.update_observed_addr_report(observed) {
+4841 if path.open {
+4842 self.events.push_back(Event::Path(PathEvent::ObservedAddr {
+4843 id: path_id,
+4844 addr: updated,
+4845 }));
+4846 }
+4847 // otherwise the event is reported when the path is deemed open
+4848 }
+4849 } else {
+4850 // include in migration
+4851 migration_observed_addr = Some(observed)
+4852 }
+4853 }
+4854 Frame::PathAbandon(frame::PathAbandon {
+4855 path_id,
+4856 error_code,
+4857 }) => {
+4858 span.record("path", tracing::field::debug(&path_id));
+4859 // TODO(flub): don't really know which error code to use here.
+4860 match self.close_path(now, path_id, error_code.into()) {
+4861 Ok(()) => {
+4862 trace!("peer abandoned path");
+4863 }
+4864 Err(ClosePathError::LastOpenPath) => {
+4865 trace!("peer abandoned last path, closing connection");
+4866 return Err(TransportError::NO_VIABLE_PATH(
+4867 "last path abandoned by peer",
+4868 ));
+4869 }
+4870 Err(ClosePathError::ClosedPath) => {
+4871 trace!("peer abandoned already closed path");
+4872 }
+4873 };
+4874 // If we receive a retransmit of PATH_ABANDON then we may already have
+4875 // abandoned this path locally. In that case the DiscardPath timer
+4876 // may already have fired and we no longer have any state for this path.
+4877 // Only set this timer if we still have path state.
+4878 if let Some(path) = self.paths.get_mut(&path_id) {
+4879 if !matches!(path.data.abandon_state, AbandonState::ReceivedPathAbandon) {
+4880 let ack_delay = self.ack_frequency.max_ack_delay_for_pto();
+4881 let pto = path.data.rtt.pto_base() + ack_delay;
+4882 self.timers.set(
+4883 Timer::PerPath(path_id, PathTimer::DiscardPath),
+4884 now + 3 * pto,
+4885 self.qlog.with_time(now),
+4886 );
+4887 // We received a PATH_ABANDON, we don't expect another one by a certain time.
+4888 path.data.abandon_state = AbandonState::ReceivedPathAbandon;
+4889 }
+4890 }
+4891 }
+4892 Frame::PathStatusAvailable(info) => {
+4893 span.record("path", tracing::field::debug(&info.path_id));
+4894 if self.is_multipath_negotiated() {
+4895 self.on_path_status(
+4896 info.path_id,
+4897 PathStatus::Available,
+4898 info.status_seq_no,
+4899 );
+4900 } else {
+4901 return Err(TransportError::PROTOCOL_VIOLATION(
+4902 "received PATH_STATUS_AVAILABLE frame when multipath was not negotiated",
+4903 ));
+4904 }
+4905 }
+4906 Frame::PathStatusBackup(info) => {
+4907 span.record("path", tracing::field::debug(&info.path_id));
+4908 if self.is_multipath_negotiated() {
+4909 self.on_path_status(info.path_id, PathStatus::Backup, info.status_seq_no);
+4910 } else {
+4911 return Err(TransportError::PROTOCOL_VIOLATION(
+4912 "received PATH_STATUS_BACKUP frame when multipath was not negotiated",
+4913 ));
+4914 }
+4915 }
+4916 Frame::MaxPathId(frame::MaxPathId(path_id)) => {
+4917 span.record("path", tracing::field::debug(&path_id));
+4918 if !self.is_multipath_negotiated() {
+4919 return Err(TransportError::PROTOCOL_VIOLATION(
+4920 "received MAX_PATH_ID frame when multipath was not negotiated",
+4921 ));
+4922 }
+4923 // frames that do not increase the path id are ignored
+4924 if path_id > self.remote_max_path_id {
+4925 self.remote_max_path_id = path_id;
+4926 self.issue_first_path_cids(now);
+4927 while let Some(true) = self.continue_nat_traversal_round(now) {}
+4928 }
+4929 }
+4930 Frame::PathsBlocked(frame::PathsBlocked(max_path_id)) => {
+4931 // Receipt of a value of Maximum Path Identifier or Path Identifier that is higher than the local maximum value MUST
+4932 // be treated as a connection error of type PROTOCOL_VIOLATION.
+4933 // Ref <https://www.ietf.org/archive/id/draft-ietf-quic-multipath-14.html#name-paths_blocked-and-path_cids>
+4934 if self.is_multipath_negotiated() {
+4935 if max_path_id > self.local_max_path_id {
+4936 return Err(TransportError::PROTOCOL_VIOLATION(
+4937 "PATHS_BLOCKED maximum path identifier was larger than local maximum",
+4938 ));
+4939 }
+4940 debug!("received PATHS_BLOCKED({:?})", max_path_id);
+4941 // TODO(@divma): ensure max concurrent paths
+4942 } else {
+4943 return Err(TransportError::PROTOCOL_VIOLATION(
+4944 "received PATHS_BLOCKED frame when not multipath was not negotiated",
+4945 ));
+4946 }
+4947 }
+4948 Frame::PathCidsBlocked(frame::PathCidsBlocked { path_id, next_seq }) => {
+4949 // Nothing to do. This is recorded in the frame stats, but otherwise we
+4950 // always issue all CIDs we're allowed to issue, so either this is an
+4951 // impatient peer or a bug on our side.
+4952
+4953 // Receipt of a value of Maximum Path Identifier or Path Identifier that is higher than the local maximum value MUST
+4954 // be treated as a connection error of type PROTOCOL_VIOLATION.
+4955 // Ref <https://www.ietf.org/archive/id/draft-ietf-quic-multipath-14.html#name-paths_blocked-and-path_cids>
+4956 if self.is_multipath_negotiated() {
+4957 if path_id > self.local_max_path_id {
+4958 return Err(TransportError::PROTOCOL_VIOLATION(
+4959 "PATH_CIDS_BLOCKED path identifier was larger than local maximum",
+4960 ));
+4961 }
+4962 if next_seq.0
+4963 > self
+4964 .local_cid_state
+4965 .get(&path_id)
+4966 .map(|cid_state| cid_state.active_seq().1 + 1)
+4967 .unwrap_or_default()
+4968 {
+4969 return Err(TransportError::PROTOCOL_VIOLATION(
+4970 "PATH_CIDS_BLOCKED next sequence number larger than in local state",
+4971 ));
+4972 }
+4973 debug!(%path_id, %next_seq, "received PATH_CIDS_BLOCKED");
+4974 } else {
+4975 return Err(TransportError::PROTOCOL_VIOLATION(
+4976 "received PATH_CIDS_BLOCKED frame when not multipath was not negotiated",
+4977 ));
+4978 }
+4979 }
+4980 Frame::AddAddress(addr) => {
+4981 let client_state = match self.iroh_hp.client_side_mut() {
+4982 Ok(state) => state,
+4983 Err(err) => {
+4984 return Err(TransportError::PROTOCOL_VIOLATION(format!(
+4985 "Nat traversal(ADD_ADDRESS): {err}"
+4986 )));
+4987 }
+4988 };
+4989
+4990 if !client_state.check_remote_address(&addr) {
+4991 // if the address is not valid we flag it, but update anyway
+4992 warn!(?addr, "server sent illegal ADD_ADDRESS frame");
+4993 }
+4994
+4995 match client_state.add_remote_address(addr) {
+4996 Ok(maybe_added) => {
+4997 if let Some(added) = maybe_added {
+4998 self.events.push_back(Event::NatTraversal(
+4999 iroh_hp::Event::AddressAdded(added),
+5000 ));
+5001 }
+5002 }
+5003 Err(e) => {
+5004 warn!(%e, "failed to add remote address")
+5005 }
+5006 }
+5007 }
+5008 Frame::RemoveAddress(addr) => {
+5009 let client_state = match self.iroh_hp.client_side_mut() {
+5010 Ok(state) => state,
+5011 Err(err) => {
+5012 return Err(TransportError::PROTOCOL_VIOLATION(format!(
+5013 "Nat traversal(REMOVE_ADDRESS): {err}"
+5014 )));
+5015 }
+5016 };
+5017 if let Some(removed_addr) = client_state.remove_remote_address(addr) {
+5018 self.events
+5019 .push_back(Event::NatTraversal(iroh_hp::Event::AddressRemoved(
+5020 removed_addr,
+5021 )));
+5022 }
+5023 }
+5024 Frame::ReachOut(reach_out) => {
+5025 let server_state = match self.iroh_hp.server_side_mut() {
+5026 Ok(state) => state,
+5027 Err(err) => {
+5028 return Err(TransportError::PROTOCOL_VIOLATION(format!(
+5029 "Nat traversal(REACH_OUT): {err}"
+5030 )));
+5031 }
+5032 };
+5033
+5034 if let Err(err) = server_state.handle_reach_out(reach_out) {
+5035 return Err(TransportError::PROTOCOL_VIOLATION(format!(
+5036 "Nat traversal(REACH_OUT): {err}"
+5037 )));
+5038 }
+5039 }
+5040 }
+5041 }
+5042
+5043 let space = self.spaces[SpaceId::Data].for_path(path_id);
+5044 if space
+5045 .pending_acks
+5046 .packet_received(now, number, ack_eliciting, &space.dedup)
+5047 {
+5048 if self.abandoned_paths.contains(&path_id) {
+5049 // § 3.4.3 QUIC-MULTIPATH: promptly send ACKs for packets received from
+5050 // abandoned paths.
+5051 space.pending_acks.set_immediate_ack_required();
+5052 } else {
+5053 self.timers.set(
+5054 Timer::PerPath(path_id, PathTimer::MaxAckDelay),
+5055 now + self.ack_frequency.max_ack_delay,
+5056 self.qlog.with_time(now),
+5057 );
+5058 }
+5059 }
+5060
+5061 // Issue stream ID credit due to ACKs of outgoing finish/resets and incoming finish/resets
+5062 // on stopped streams. Incoming finishes/resets on open streams are not handled here as they
+5063 // are only freed, and hence only issue credit, once the application has been notified
+5064 // during a read on the stream.
+5065 let pending = &mut self.spaces[SpaceId::Data].pending;
+5066 self.streams.queue_max_stream_id(pending);
5067
-5068 // During the handshake the server sends a reset token in the transport
-5069 // parameters. When we are the client and we receive the reset token during the
-5070 // handshake we want this to affect our peer transport parameters.
-5071 // TODO(flub): Pretty sure this is pointless, the entire params is overwritten
-5072 // shortly after this was called. And then the params don't have this anymore.
-5073 if path_id == PathId::ZERO {
-5074 self.peer_params.stateless_reset_token = Some(reset_token);
-5075 }
-5076 }
-5077
-5078 /// Issue an initial set of connection IDs to the peer upon connection
-5079 fn issue_first_cids(&mut self, now: Instant) {
-5080 if self
-5081 .local_cid_state
-5082 .get(&PathId::ZERO)
-5083 .expect("PathId::ZERO exists when the connection is created")
-5084 .cid_len()
-5085 == 0
-5086 {
-5087 return;
+5068 if let Some(reason) = close {
+5069 self.state.move_to_draining(Some(reason.into()));
+5070 self.close = true;
+5071 }
+5072
+5073 if Some(number) == self.spaces[SpaceId::Data].for_path(path_id).rx_packet
+5074 && !is_probing_packet
+5075 && network_path != self.path_data(path_id).network_path
+5076 {
+5077 let ConnectionSide::Server { ref server_config } = self.side else {
+5078 panic!("packets from unknown remote should be dropped by clients");
+5079 };
+5080 debug_assert!(
+5081 server_config.migration,
+5082 "migration-initiating packets should have been dropped immediately"
+5083 );
+5084 self.migrate(path_id, now, network_path, migration_observed_addr);
+5085 // Break linkability, if possible
+5086 self.update_rem_cid(path_id);
+5087 self.spin = false;
5088 }
5089
-5090 // Subtract 1 to account for the CID we supplied while handshaking
-5091 let mut n = self.peer_params.issue_cids_limit() - 1;
-5092 if let ConnectionSide::Server { server_config } = &self.side {
-5093 if server_config.has_preferred_address() {
-5094 // We also sent a CID in the transport parameters
-5095 n -= 1;
-5096 }
-5097 }
-5098 self.endpoint_events
-5099 .push_back(EndpointEventInner::NeedIdentifiers(PathId::ZERO, now, n));
-5100 }
-5101
-5102 /// Issues an initial set of CIDs for paths that have not yet had any CIDs issued
-5103 ///
-5104 /// Later CIDs are issued when CIDs expire or are retired by the peer.
-5105 fn issue_first_path_cids(&mut self, now: Instant) {
-5106 if let Some(max_path_id) = self.max_path_id() {
-5107 let mut path_id = self.max_path_id_with_cids.next();
-5108 while path_id <= max_path_id {
-5109 self.endpoint_events
-5110 .push_back(EndpointEventInner::NeedIdentifiers(
-5111 path_id,
-5112 now,
-5113 self.peer_params.issue_cids_limit(),
-5114 ));
-5115 path_id = path_id.next();
-5116 }
-5117 self.max_path_id_with_cids = max_path_id;
-5118 }
-5119 }
-5120
-5121 /// Populates a packet with frames
-5122 ///
-5123 /// This tries to fit as many frames as possible into the packet.
-5124 ///
-5125 /// *path_exclusive_only* means to only build frames which can only be sent on this
-5126 /// *path. This is used in multipath for backup paths while there is still an active
-5127 /// *path.
-5128 fn populate_packet<'a, 'b>(
-5129 &mut self,
-5130 now: Instant,
-5131 space_id: SpaceId,
-5132 path_id: PathId,
-5133 path_exclusive_only: bool,
-5134 builder: &mut PacketBuilder<'a, 'b>,
-5135 ) {
-5136 let pn = builder.exact_number;
-5137 let is_multipath_negotiated = self.is_multipath_negotiated();
-5138 let stats = &mut self.stats.frame_tx;
-5139 let space = &mut self.spaces[space_id];
-5140 let path = &mut self.paths.get_mut(&path_id).expect("known path").data;
-5141 let is_0rtt = space_id == SpaceId::Data && space.crypto.is_none();
-5142 space
-5143 .for_path(path_id)
-5144 .pending_acks
-5145 .maybe_ack_non_eliciting();
+5090 Ok(())
+5091 }
+5092
+5093 fn migrate(
+5094 &mut self,
+5095 path_id: PathId,
+5096 now: Instant,
+5097 network_path: FourTuple,
+5098 observed_addr: Option<ObservedAddr>,
+5099 ) {
+5100 trace!(%network_path, %path_id, "migration initiated");
+5101 self.path_generation_counter = self.path_generation_counter.wrapping_add(1);
+5102 // TODO(@divma): conditions for path migration in multipath are very specific, check them
+5103 // again to prevent path migrations that should actually create a new path
+5104
+5105 // Reset rtt/congestion state for new path unless it looks like a NAT rebinding.
+5106 // Note that the congestion window will not grow until validation terminates. Helps mitigate
+5107 // amplification attacks performed by spoofing source addresses.
+5108 let prev_pto = self.pto(SpaceId::Data, path_id);
+5109 let known_path = self.paths.get_mut(&path_id).expect("known path");
+5110 let path = &mut known_path.data;
+5111 let mut new_path = if network_path.remote.is_ipv4()
+5112 && network_path.remote.ip() == path.network_path.remote.ip()
+5113 {
+5114 PathData::from_previous(network_path, path, self.path_generation_counter, now)
+5115 } else {
+5116 let peer_max_udp_payload_size =
+5117 u16::try_from(self.peer_params.max_udp_payload_size.into_inner())
+5118 .unwrap_or(u16::MAX);
+5119 PathData::new(
+5120 network_path,
+5121 self.allow_mtud,
+5122 Some(peer_max_udp_payload_size),
+5123 self.path_generation_counter,
+5124 now,
+5125 &self.config,
+5126 )
+5127 };
+5128 new_path.last_observed_addr_report = path.last_observed_addr_report.clone();
+5129 if let Some(report) = observed_addr {
+5130 if let Some(updated) = new_path.update_observed_addr_report(report) {
+5131 tracing::info!("adding observed addr event from migration");
+5132 self.events.push_back(Event::Path(PathEvent::ObservedAddr {
+5133 id: path_id,
+5134 addr: updated,
+5135 }));
+5136 }
+5137 }
+5138 new_path.send_new_challenge = true;
+5139
+5140 let mut prev = mem::replace(path, new_path);
+5141 // Don't clobber the original path if the previous one hasn't been validated yet
+5142 if !prev.is_validating_path() {
+5143 prev.send_new_challenge = true;
+5144 // We haven't updated the remote CID yet, this captures the remote CID we were using on
+5145 // the previous path.
5146
-5147 // HANDSHAKE_DONE
-5148 if !is_0rtt && mem::replace(&mut space.pending.handshake_done, false) {
-5149 builder.write_frame(frame::HandshakeDone, stats);
-5150 }
-5151
-5152 // REACH_OUT
-5153 // TODO(@divma): path explusive considerations
-5154 if let Some((round, addresses)) = space.pending.reach_out.as_mut() {
-5155 while let Some(local_addr) = addresses.pop() {
-5156 let reach_out = frame::ReachOut::new(*round, local_addr);
-5157 if builder.frame_space_remaining() > reach_out.size() {
-5158 builder.write_frame(reach_out, stats);
-5159 } else {
-5160 addresses.push(local_addr);
-5161 break;
-5162 }
-5163 }
-5164 if addresses.is_empty() {
-5165 space.pending.reach_out = None;
-5166 }
-5167 }
-5168
-5169 // OBSERVED_ADDR
-5170 if !path_exclusive_only
-5171 && space_id == SpaceId::Data
-5172 && self
-5173 .config
-5174 .address_discovery_role
-5175 .should_report(&self.peer_params.address_discovery_role)
-5176 && (!path.observed_addr_sent || space.pending.observed_addr)
-5177 {
-5178 let frame =
-5179 frame::ObservedAddr::new(path.network_path.remote, self.next_observed_addr_seq_no);
-5180 if builder.frame_space_remaining() > frame.size() {
-5181 builder.write_frame(frame, stats);
-5182
-5183 self.next_observed_addr_seq_no = self.next_observed_addr_seq_no.saturating_add(1u8);
-5184 path.observed_addr_sent = true;
-5185
-5186 space.pending.observed_addr = false;
-5187 }
-5188 }
-5189
-5190 // PING
-5191 if mem::replace(&mut space.for_path(path_id).ping_pending, false) {
-5192 builder.write_frame(frame::Ping, stats);
-5193 }
-5194
-5195 // IMMEDIATE_ACK
-5196 if mem::replace(&mut space.for_path(path_id).immediate_ack_pending, false) {
-5197 debug_assert_eq!(
-5198 space_id,
-5199 SpaceId::Data,
-5200 "immediate acks must be sent in the data space"
-5201 );
-5202 builder.write_frame(frame::ImmediateAck, stats);
+5147 known_path.prev = Some((self.rem_cids.get(&path_id).unwrap().active(), prev));
+5148 }
+5149
+5150 // We need to re-assign the correct remote to this path in qlog
+5151 self.qlog.emit_tuple_assigned(path_id, network_path, now);
+5152
+5153 self.timers.set(
+5154 Timer::PerPath(path_id, PathTimer::PathValidation),
+5155 now + 3 * cmp::max(self.pto(SpaceId::Data, path_id), prev_pto),
+5156 self.qlog.with_time(now),
+5157 );
+5158 }
+5159
+5160 /// Handle a change in the local address, i.e. an active migration
+5161 pub fn local_address_changed(&mut self) {
+5162 // TODO(flub): if multipath is enabled this needs to create a new path entirely.
+5163 self.update_rem_cid(PathId::ZERO);
+5164 self.ping();
+5165 }
+5166
+5167 /// Switch to a previously unused remote connection ID, if possible
+5168 fn update_rem_cid(&mut self, path_id: PathId) {
+5169 let Some((reset_token, retired)) =
+5170 self.rem_cids.get_mut(&path_id).and_then(|cids| cids.next())
+5171 else {
+5172 return;
+5173 };
+5174
+5175 // Retire the current remote CID and any CIDs we had to skip.
+5176 self.spaces[SpaceId::Data]
+5177 .pending
+5178 .retire_cids
+5179 .extend(retired.map(|seq| (path_id, seq)));
+5180 let remote = self.path_data(path_id).network_path.remote;
+5181 self.set_reset_token(path_id, remote, reset_token);
+5182 }
+5183
+5184 /// Sends this reset token to the endpoint
+5185 ///
+5186 /// The endpoint needs to know the reset tokens issued by the peer, so that if the peer
+5187 /// sends a reset token it knows to route it to this connection. See RFC 9000 section
+5188 /// 10.3. Stateless Reset.
+5189 ///
+5190 /// Reset tokens are different for each path, the endpoint identifies paths by peer
+5191 /// socket address however, not by path ID.
+5192 fn set_reset_token(&mut self, path_id: PathId, remote: SocketAddr, reset_token: ResetToken) {
+5193 self.endpoint_events
+5194 .push_back(EndpointEventInner::ResetToken(path_id, remote, reset_token));
+5195
+5196 // During the handshake the server sends a reset token in the transport
+5197 // parameters. When we are the client and we receive the reset token during the
+5198 // handshake we want this to affect our peer transport parameters.
+5199 // TODO(flub): Pretty sure this is pointless, the entire params is overwritten
+5200 // shortly after this was called. And then the params don't have this anymore.
+5201 if path_id == PathId::ZERO {
+5202 self.peer_params.stateless_reset_token = Some(reset_token);
5203 }
-5204
-5205 // ACK
-5206 // TODO(flub): Should this send acks for this path anyway?
-5207
-5208 if !path_exclusive_only {
-5209 for path_id in space
-5210 .number_spaces
-5211 .iter_mut()
-5212 .filter(|(_, pns)| pns.pending_acks.can_send())
-5213 .map(|(&path_id, _)| path_id)
-5214 .collect::<Vec<_>>()
-5215 {
-5216 Self::populate_acks(
-5217 now,
-5218 self.receiving_ecn,
-5219 path_id,
-5220 space_id,
-5221 space,
-5222 is_multipath_negotiated,
-5223 builder,
-5224 stats,
-5225 );
-5226 }
-5227 }
-5228
-5229 // ACK_FREQUENCY
-5230 if !path_exclusive_only && mem::replace(&mut space.pending.ack_frequency, false) {
-5231 let sequence_number = self.ack_frequency.next_sequence_number();
-5232
-5233 // Safe to unwrap because this is always provided when ACK frequency is enabled
-5234 let config = self.config.ack_frequency_config.as_ref().unwrap();
-5235
-5236 // Ensure the delay is within bounds to avoid a PROTOCOL_VIOLATION error
-5237 let max_ack_delay = self.ack_frequency.candidate_max_ack_delay(
-5238 path.rtt.get(),
-5239 config,
-5240 &self.peer_params,
-5241 );
-5242
-5243 let frame = frame::AckFrequency {
-5244 sequence: sequence_number,
-5245 ack_eliciting_threshold: config.ack_eliciting_threshold,
-5246 request_max_ack_delay: max_ack_delay.as_micros().try_into().unwrap_or(VarInt::MAX),
-5247 reordering_threshold: config.reordering_threshold,
-5248 };
-5249 builder.write_frame(frame, stats);
-5250
-5251 self.ack_frequency
-5252 .ack_frequency_sent(path_id, pn, max_ack_delay);
-5253 }
-5254
-5255 // PATH_CHALLENGE
-5256 if builder.frame_space_remaining() > frame::PathChallenge::SIZE_BOUND
-5257 && space_id == SpaceId::Data
-5258 && path.send_new_challenge
-5259 && !self.state.is_closed()
-5260 // we don't want to send new challenges if we are already closing
-5261 {
-5262 path.send_new_challenge = false;
-5263
-5264 // Generate a new challenge every time we send a new PATH_CHALLENGE
-5265 let token = self.rng.random();
-5266 let info = paths::SentChallengeInfo {
-5267 sent_instant: now,
-5268 network_path: path.network_path,
-5269 };
-5270 path.challenges_sent.insert(token, info);
-5271 let challenge = frame::PathChallenge(token);
-5272 trace!(frame = %challenge);
-5273 builder.write_frame(challenge, stats);
-5274 builder.require_padding();
-5275 let pto = self.ack_frequency.max_ack_delay_for_pto() + path.rtt.pto_base();
-5276 self.timers.set(
-5277 Timer::PerPath(path_id, PathTimer::PathChallengeLost),
-5278 now + pto,
-5279 self.qlog.with_time(now),
-5280 );
-5281
-5282 if is_multipath_negotiated && !path.validated && path.send_new_challenge {
-5283 // queue informing the path status along with the challenge
-5284 space.pending.path_status.insert(path_id);
-5285 }
-5286
-5287 // Always include an OBSERVED_ADDR frame with a PATH_CHALLENGE, regardless
-5288 // of whether one has already been sent on this path.
-5289 if space_id == SpaceId::Data
-5290 && self
-5291 .config
-5292 .address_discovery_role
-5293 .should_report(&self.peer_params.address_discovery_role)
-5294 {
-5295 let frame = frame::ObservedAddr::new(
-5296 path.network_path.remote,
-5297 self.next_observed_addr_seq_no,
-5298 );
-5299 if builder.frame_space_remaining() > frame.size() {
-5300 builder.write_frame(frame, stats);
-5301
-5302 self.next_observed_addr_seq_no =
-5303 self.next_observed_addr_seq_no.saturating_add(1u8);
-5304 path.observed_addr_sent = true;
-5305
-5306 space.pending.observed_addr = false;
-5307 }
-5308 }
-5309 }
+5204 }
+5205
+5206 /// Issue an initial set of connection IDs to the peer upon connection
+5207 fn issue_first_cids(&mut self, now: Instant) {
+5208 if self
+5209 .local_cid_state
+5210 .get(&PathId::ZERO)
+5211 .expect("PathId::ZERO exists when the connection is created")
+5212 .cid_len()
+5213 == 0
+5214 {
+5215 return;
+5216 }
+5217
+5218 // Subtract 1 to account for the CID we supplied while handshaking
+5219 let mut n = self.peer_params.issue_cids_limit() - 1;
+5220 if let ConnectionSide::Server { server_config } = &self.side {
+5221 if server_config.has_preferred_address() {
+5222 // We also sent a CID in the transport parameters
+5223 n -= 1;
+5224 }
+5225 }
+5226 self.endpoint_events
+5227 .push_back(EndpointEventInner::NeedIdentifiers(PathId::ZERO, now, n));
+5228 }
+5229
+5230 /// Issues an initial set of CIDs for paths that have not yet had any CIDs issued
+5231 ///
+5232 /// Later CIDs are issued when CIDs expire or are retired by the peer.
+5233 fn issue_first_path_cids(&mut self, now: Instant) {
+5234 if let Some(max_path_id) = self.max_path_id() {
+5235 let mut path_id = self.max_path_id_with_cids.next();
+5236 while path_id <= max_path_id {
+5237 self.endpoint_events
+5238 .push_back(EndpointEventInner::NeedIdentifiers(
+5239 path_id,
+5240 now,
+5241 self.peer_params.issue_cids_limit(),
+5242 ));
+5243 path_id = path_id.next();
+5244 }
+5245 self.max_path_id_with_cids = max_path_id;
+5246 }
+5247 }
+5248
+5249 /// Populates a packet with frames
+5250 ///
+5251 /// This tries to fit as many frames as possible into the packet.
+5252 ///
+5253 /// *path_exclusive_only* means to only build frames which can only be sent on this
+5254 /// *path. This is used in multipath for backup paths while there is still an active
+5255 /// *path.
+5256 fn populate_packet<'a, 'b>(
+5257 &mut self,
+5258 now: Instant,
+5259 space_id: SpaceId,
+5260 path_id: PathId,
+5261 path_exclusive_only: bool,
+5262 builder: &mut PacketBuilder<'a, 'b>,
+5263 ) {
+5264 let pn = builder.exact_number;
+5265 let is_multipath_negotiated = self.is_multipath_negotiated();
+5266 let stats = &mut self.stats.frame_tx;
+5267 let space = &mut self.spaces[space_id];
+5268 let path = &mut self.paths.get_mut(&path_id).expect("known path").data;
+5269 let is_0rtt = space_id == SpaceId::Data && space.crypto.is_none();
+5270 space
+5271 .for_path(path_id)
+5272 .pending_acks
+5273 .maybe_ack_non_eliciting();
+5274
+5275 // HANDSHAKE_DONE
+5276 if !is_0rtt && mem::replace(&mut space.pending.handshake_done, false) {
+5277 builder.write_frame(frame::HandshakeDone, stats);
+5278 }
+5279
+5280 // REACH_OUT
+5281 // TODO(@divma): path explusive considerations
+5282 if let Some((round, addresses)) = space.pending.reach_out.as_mut() {
+5283 while let Some(local_addr) = addresses.pop() {
+5284 let reach_out = frame::ReachOut::new(*round, local_addr);
+5285 if builder.frame_space_remaining() > reach_out.size() {
+5286 builder.write_frame(reach_out, stats);
+5287 } else {
+5288 addresses.push(local_addr);
+5289 break;
+5290 }
+5291 }
+5292 if addresses.is_empty() {
+5293 space.pending.reach_out = None;
+5294 }
+5295 }
+5296
+5297 // OBSERVED_ADDR
+5298 if !path_exclusive_only
+5299 && space_id == SpaceId::Data
+5300 && self
+5301 .config
+5302 .address_discovery_role
+5303 .should_report(&self.peer_params.address_discovery_role)
+5304 && (!path.observed_addr_sent || space.pending.observed_addr)
+5305 {
+5306 let frame =
+5307 frame::ObservedAddr::new(path.network_path.remote, self.next_observed_addr_seq_no);
+5308 if builder.frame_space_remaining() > frame.size() {
+5309 builder.write_frame(frame, stats);
5310
-5311 // PATH_RESPONSE
-5312 if builder.frame_space_remaining() > frame::PathResponse::SIZE_BOUND
-5313 && space_id == SpaceId::Data
-5314 {
-5315 if let Some(token) = path.path_responses.pop_on_path(path.network_path) {
-5316 let response = frame::PathResponse(token);
-5317 trace!(frame = %response);
-5318 builder.write_frame(response, stats);
-5319 builder.require_padding();
-5320
-5321 // NOTE: this is technically not required but might be useful to ride the
-5322 // request/response nature of path challenges to refresh an observation
-5323 // Since PATH_RESPONSE is a probing frame, this is allowed by the spec.
-5324 if space_id == SpaceId::Data
-5325 && self
-5326 .config
-5327 .address_discovery_role
-5328 .should_report(&self.peer_params.address_discovery_role)
-5329 {
-5330 let frame = frame::ObservedAddr::new(
-5331 path.network_path.remote,
-5332 self.next_observed_addr_seq_no,
-5333 );
-5334 if builder.frame_space_remaining() > frame.size() {
-5335 builder.write_frame(frame, stats);
-5336
-5337 self.next_observed_addr_seq_no =
-5338 self.next_observed_addr_seq_no.saturating_add(1u8);
-5339 path.observed_addr_sent = true;
-5340
-5341 space.pending.observed_addr = false;
-5342 }
-5343 }
-5344 }
-5345 }
-5346
-5347 // CRYPTO
-5348 while !path_exclusive_only
-5349 && builder.frame_space_remaining() > frame::Crypto::SIZE_BOUND
-5350 && !is_0rtt
-5351 {
-5352 let mut frame = match space.pending.crypto.pop_front() {
-5353 Some(x) => x,
-5354 None => break,
-5355 };
+5311 self.next_observed_addr_seq_no = self.next_observed_addr_seq_no.saturating_add(1u8);
+5312 path.observed_addr_sent = true;
+5313
+5314 space.pending.observed_addr = false;
+5315 }
+5316 }
+5317
+5318 // PING
+5319 if mem::replace(&mut space.for_path(path_id).ping_pending, false) {
+5320 builder.write_frame(frame::Ping, stats);
+5321 }
+5322
+5323 // IMMEDIATE_ACK
+5324 if mem::replace(&mut space.for_path(path_id).immediate_ack_pending, false) {
+5325 debug_assert_eq!(
+5326 space_id,
+5327 SpaceId::Data,
+5328 "immediate acks must be sent in the data space"
+5329 );
+5330 builder.write_frame(frame::ImmediateAck, stats);
+5331 }
+5332
+5333 // ACK
+5334 // TODO(flub): Should this send acks for this path anyway?
+5335
+5336 if !path_exclusive_only {
+5337 for path_id in space
+5338 .number_spaces
+5339 .iter_mut()
+5340 .filter(|(_, pns)| pns.pending_acks.can_send())
+5341 .map(|(&path_id, _)| path_id)
+5342 .collect::<Vec<_>>()
+5343 {
+5344 Self::populate_acks(
+5345 now,
+5346 self.receiving_ecn,
+5347 path_id,
+5348 space_id,
+5349 space,
+5350 is_multipath_negotiated,
+5351 builder,
+5352 stats,
+5353 );
+5354 }
+5355 }
5356
-5357 // Calculate the maximum amount of crypto data we can store in the buffer.
-5358 // Since the offset is known, we can reserve the exact size required to encode it.
-5359 // For length we reserve 2bytes which allows to encode up to 2^14,
-5360 // which is more than what fits into normally sized QUIC frames.
-5361 let max_crypto_data_size = builder.frame_space_remaining()
-5362 - 1 // Frame Type
-5363 - VarInt::size(unsafe { VarInt::from_u64_unchecked(frame.offset) })
-5364 - 2; // Maximum encoded length for frame size, given we send less than 2^14 bytes
-5365
-5366 let len = frame
-5367 .data
-5368 .len()
-5369 .min(2usize.pow(14) - 1)
-5370 .min(max_crypto_data_size);
-5371
-5372 let data = frame.data.split_to(len);
-5373 let offset = frame.offset;
-5374 let truncated = frame::Crypto { offset, data };
-5375 builder.write_frame(truncated, stats);
-5376
-5377 if !frame.data.is_empty() {
-5378 frame.offset += len as u64;
-5379 space.pending.crypto.push_front(frame);
-5380 }
+5357 // ACK_FREQUENCY
+5358 if !path_exclusive_only && mem::replace(&mut space.pending.ack_frequency, false) {
+5359 let sequence_number = self.ack_frequency.next_sequence_number();
+5360
+5361 // Safe to unwrap because this is always provided when ACK frequency is enabled
+5362 let config = self.config.ack_frequency_config.as_ref().unwrap();
+5363
+5364 // Ensure the delay is within bounds to avoid a PROTOCOL_VIOLATION error
+5365 let max_ack_delay = self.ack_frequency.candidate_max_ack_delay(
+5366 path.rtt.get(),
+5367 config,
+5368 &self.peer_params,
+5369 );
+5370
+5371 let frame = frame::AckFrequency {
+5372 sequence: sequence_number,
+5373 ack_eliciting_threshold: config.ack_eliciting_threshold,
+5374 request_max_ack_delay: max_ack_delay.as_micros().try_into().unwrap_or(VarInt::MAX),
+5375 reordering_threshold: config.reordering_threshold,
+5376 };
+5377 builder.write_frame(frame, stats);
+5378
+5379 self.ack_frequency
+5380 .ack_frequency_sent(path_id, pn, max_ack_delay);
5381 }
5382
-5383 // TODO(flub): maybe this is much higher priority?
-5384 // PATH_ABANDON
-5385 while !path_exclusive_only
-5386 && space_id == SpaceId::Data
-5387 && frame::PathAbandon::SIZE_BOUND <= builder.frame_space_remaining()
-5388 {
-5389 let Some((abandoned_path_id, error_code)) = space.pending.path_abandon.pop_first()
-5390 else {
-5391 break;
-5392 };
-5393 let frame = frame::PathAbandon {
-5394 path_id: abandoned_path_id,
-5395 error_code,
-5396 };
-5397 builder.write_frame(frame, stats);
-5398
-5399 let ack_delay = self.ack_frequency.max_ack_delay_for_pto();
-5400 // We can't access path here anymore due to borrowing issues.
-5401 let send_pto = self.paths.get(&path_id).unwrap().data.rtt.pto_base() + ack_delay;
-5402 if let Some(abandoned_path) = self.paths.get_mut(&abandoned_path_id) {
-5403 // We only want to set the deadline on the *first* PATH_ABANDON we send.
-5404 // Retransmits shouldn't run this code again
-5405 if matches!(
-5406 abandoned_path.data.abandon_state,
-5407 AbandonState::NotAbandoned
-5408 ) {
-5409 // The peer MUST respond with a corresponding PATH_ABANDON frame.
-5410 // The other peer has 3 * PTO to do that.
-5411 // This uses the PTO of the path we send on!
-5412 // If the PATH_ABANDON comes in within the deadline we're giving here, then this
-5413 // state will be set to `AbandonState::ReceivedPathAbandon`, essentially clearing
-5414 // the deadline. If we receive a frame after the deadline, we error out with a
-5415 // protocol violation.
-5416 // Receiving other frames before the deadline is fine, as those might be packets
-5417 // that were still in-flight.
-5418 abandoned_path.data.abandon_state = AbandonState::ExpectingPathAbandon {
-5419 deadline: now + 3 * send_pto,
-5420 };
-5421
-5422 // At some point, we need to forget about the path.
-5423 // If we do so too early, then we'll have discarded the CIDs of that path and won't
-5424 // handle incoming packets on that path correctly.
-5425 // To give this path enough time, we assume that the peer will have received our
-5426 // PATH_ABANDON within 3 * PTO of the path we sent the abandon on,
-5427 // and then we give the path 3 * PTO time to make it very unlikely that there will
-5428 // still be packets incoming on the path at that point.
-5429 // This timer will actually get reset to a value that's likely to be even earlier
-5430 // once we actually receive the PATH_ABANDON frame itself.
-5431 let abandoned_pto =
-5432 self.paths.get(&path_id).unwrap().data.rtt.pto_base() + ack_delay;
-5433 self.timers.set(
-5434 Timer::PerPath(abandoned_path_id, PathTimer::DiscardPath),
-5435 now + 3 * send_pto + 3 * abandoned_pto,
-5436 self.qlog.with_time(now),
-5437 );
-5438 }
-5439 } else {
-5440 warn!("sent PATH_ABANDON after path was already discarded");
-5441 }
-5442 }
-5443
-5444 // PATH_STATUS_AVAILABLE & PATH_STATUS_BACKUP
-5445 while !path_exclusive_only
-5446 && space_id == SpaceId::Data
-5447 && frame::PathStatusAvailable::SIZE_BOUND <= builder.frame_space_remaining()
-5448 {
-5449 let Some(path_id) = space.pending.path_status.pop_first() else {
-5450 break;
-5451 };
-5452 let Some(path) = self.paths.get(&path_id).map(|path_state| &path_state.data) else {
-5453 trace!(%path_id, "discarding queued path status for unknown path");
-5454 continue;
-5455 };
-5456
-5457 let seq = path.status.seq();
-5458 match path.local_status() {
-5459 PathStatus::Available => {
-5460 let frame = frame::PathStatusAvailable {
-5461 path_id,
-5462 status_seq_no: seq,
-5463 };
-5464 builder.write_frame(frame, stats);
-5465 }
-5466 PathStatus::Backup => {
-5467 let frame = frame::PathStatusBackup {
-5468 path_id,
-5469 status_seq_no: seq,
-5470 };
-5471 builder.write_frame(frame, stats);
-5472 }
-5473 }
-5474 }
-5475
-5476 // MAX_PATH_ID
-5477 if space_id == SpaceId::Data
-5478 && space.pending.max_path_id
-5479 && frame::MaxPathId::SIZE_BOUND <= builder.frame_space_remaining()
-5480 {
-5481 let frame = frame::MaxPathId(self.local_max_path_id);
-5482 builder.write_frame(frame, stats);
-5483 space.pending.max_path_id = false;
-5484 }
-5485
-5486 // PATHS_BLOCKED
-5487 if space_id == SpaceId::Data
-5488 && space.pending.paths_blocked
-5489 && frame::PathsBlocked::SIZE_BOUND <= builder.frame_space_remaining()
-5490 {
-5491 let frame = frame::PathsBlocked(self.remote_max_path_id);
-5492 builder.write_frame(frame, stats);
-5493 space.pending.paths_blocked = false;
-5494 }
-5495
-5496 // PATH_CIDS_BLOCKED
-5497 while space_id == SpaceId::Data
-5498 && frame::PathCidsBlocked::SIZE_BOUND <= builder.frame_space_remaining()
-5499 {
-5500 let Some(path_id) = space.pending.path_cids_blocked.pop_first() else {
-5501 break;
-5502 };
-5503 let next_seq = match self.rem_cids.get(&path_id) {
-5504 Some(cid_queue) => VarInt(cid_queue.active_seq() + 1),
-5505 None => VarInt(0),
-5506 };
-5507 let frame = frame::PathCidsBlocked { path_id, next_seq };
-5508 builder.write_frame(frame, stats);
+5383 // PATH_CHALLENGE
+5384 if builder.frame_space_remaining() > frame::PathChallenge::SIZE_BOUND
+5385 && space_id == SpaceId::Data
+5386 && path.send_new_challenge
+5387 && !self.state.is_closed()
+5388 // we don't want to send new challenges if we are already closing
+5389 {
+5390 path.send_new_challenge = false;
+5391
+5392 // Generate a new challenge every time we send a new PATH_CHALLENGE
+5393 let token = self.rng.random();
+5394 let info = paths::SentChallengeInfo {
+5395 sent_instant: now,
+5396 network_path: path.network_path,
+5397 };
+5398 path.challenges_sent.insert(token, info);
+5399 let challenge = frame::PathChallenge(token);
+5400 trace!(frame = %challenge);
+5401 builder.write_frame(challenge, stats);
+5402 builder.require_padding();
+5403 let pto = self.ack_frequency.max_ack_delay_for_pto() + path.rtt.pto_base();
+5404 self.timers.set(
+5405 Timer::PerPath(path_id, PathTimer::PathChallengeLost),
+5406 now + pto,
+5407 self.qlog.with_time(now),
+5408 );
+5409
+5410 if is_multipath_negotiated && !path.validated && path.send_new_challenge {
+5411 // queue informing the path status along with the challenge
+5412 space.pending.path_status.insert(path_id);
+5413 }
+5414
+5415 // Always include an OBSERVED_ADDR frame with a PATH_CHALLENGE, regardless
+5416 // of whether one has already been sent on this path.
+5417 if space_id == SpaceId::Data
+5418 && self
+5419 .config
+5420 .address_discovery_role
+5421 .should_report(&self.peer_params.address_discovery_role)
+5422 {
+5423 let frame = frame::ObservedAddr::new(
+5424 path.network_path.remote,
+5425 self.next_observed_addr_seq_no,
+5426 );
+5427 if builder.frame_space_remaining() > frame.size() {
+5428 builder.write_frame(frame, stats);
+5429
+5430 self.next_observed_addr_seq_no =
+5431 self.next_observed_addr_seq_no.saturating_add(1u8);
+5432 path.observed_addr_sent = true;
+5433
+5434 space.pending.observed_addr = false;
+5435 }
+5436 }
+5437 }
+5438
+5439 // PATH_RESPONSE
+5440 if builder.frame_space_remaining() > frame::PathResponse::SIZE_BOUND
+5441 && space_id == SpaceId::Data
+5442 {
+5443 if let Some(token) = path.path_responses.pop_on_path(path.network_path) {
+5444 let response = frame::PathResponse(token);
+5445 trace!(frame = %response);
+5446 builder.write_frame(response, stats);
+5447 builder.require_padding();
+5448
+5449 // NOTE: this is technically not required but might be useful to ride the
+5450 // request/response nature of path challenges to refresh an observation
+5451 // Since PATH_RESPONSE is a probing frame, this is allowed by the spec.
+5452 if space_id == SpaceId::Data
+5453 && self
+5454 .config
+5455 .address_discovery_role
+5456 .should_report(&self.peer_params.address_discovery_role)
+5457 {
+5458 let frame = frame::ObservedAddr::new(
+5459 path.network_path.remote,
+5460 self.next_observed_addr_seq_no,
+5461 );
+5462 if builder.frame_space_remaining() > frame.size() {
+5463 builder.write_frame(frame, stats);
+5464
+5465 self.next_observed_addr_seq_no =
+5466 self.next_observed_addr_seq_no.saturating_add(1u8);
+5467 path.observed_addr_sent = true;
+5468
+5469 space.pending.observed_addr = false;
+5470 }
+5471 }
+5472 }
+5473 }
+5474
+5475 // CRYPTO
+5476 while !path_exclusive_only
+5477 && builder.frame_space_remaining() > frame::Crypto::SIZE_BOUND
+5478 && !is_0rtt
+5479 {
+5480 let mut frame = match space.pending.crypto.pop_front() {
+5481 Some(x) => x,
+5482 None => break,
+5483 };
+5484
+5485 // Calculate the maximum amount of crypto data we can store in the buffer.
+5486 // Since the offset is known, we can reserve the exact size required to encode it.
+5487 // For length we reserve 2bytes which allows to encode up to 2^14,
+5488 // which is more than what fits into normally sized QUIC frames.
+5489 let max_crypto_data_size = builder.frame_space_remaining()
+5490 - 1 // Frame Type
+5491 - VarInt::size(unsafe { VarInt::from_u64_unchecked(frame.offset) })
+5492 - 2; // Maximum encoded length for frame size, given we send less than 2^14 bytes
+5493
+5494 let len = frame
+5495 .data
+5496 .len()
+5497 .min(2usize.pow(14) - 1)
+5498 .min(max_crypto_data_size);
+5499
+5500 let data = frame.data.split_to(len);
+5501 let offset = frame.offset;
+5502 let truncated = frame::Crypto { offset, data };
+5503 builder.write_frame(truncated, stats);
+5504
+5505 if !frame.data.is_empty() {
+5506 frame.offset += len as u64;
+5507 space.pending.crypto.push_front(frame);
+5508 }
5509 }
5510
-5511 // RESET_STREAM, STOP_SENDING, MAX_DATA, MAX_STREAM_DATA, MAX_STREAMS
-5512 if space_id == SpaceId::Data {
-5513 self.streams
-5514 .write_control_frames(builder, &mut space.pending, stats);
-5515 }
-5516
-5517 // NEW_CONNECTION_ID
-5518 let cid_len = self
-5519 .local_cid_state
-5520 .values()
-5521 .map(|cid_state| cid_state.cid_len())
-5522 .max()
-5523 .expect("some local CID state must exist");
-5524 let new_cid_size_bound =
-5525 frame::NewConnectionId::size_bound(is_multipath_negotiated, cid_len);
-5526 while !path_exclusive_only && builder.frame_space_remaining() > new_cid_size_bound {
-5527 let issued = match space.pending.new_cids.pop() {
-5528 Some(x) => x,
-5529 None => break,
-5530 };
-5531 let retire_prior_to = self
-5532 .local_cid_state
-5533 .get(&issued.path_id)
-5534 .map(|cid_state| cid_state.retire_prior_to())
-5535 .unwrap_or_else(|| panic!("missing local CID state for path={}", issued.path_id));
-5536
-5537 let cid_path_id = match is_multipath_negotiated {
-5538 true => Some(issued.path_id),
-5539 false => {
-5540 debug_assert_eq!(issued.path_id, PathId::ZERO);
-5541 None
-5542 }
-5543 };
-5544 let frame = frame::NewConnectionId {
-5545 path_id: cid_path_id,
-5546 sequence: issued.sequence,
-5547 retire_prior_to,
-5548 id: issued.id,
-5549 reset_token: issued.reset_token,
-5550 };
-5551 builder.write_frame(frame, stats);
-5552 }
-5553
-5554 // RETIRE_CONNECTION_ID
-5555 let retire_cid_bound = frame::RetireConnectionId::size_bound(is_multipath_negotiated);
-5556 while !path_exclusive_only && builder.frame_space_remaining() > retire_cid_bound {
-5557 let (path_id, sequence) = match space.pending.retire_cids.pop() {
-5558 Some((PathId::ZERO, seq)) if !is_multipath_negotiated => (None, seq),
-5559 Some((path_id, seq)) => (Some(path_id), seq),
-5560 None => break,
-5561 };
-5562 let frame = frame::RetireConnectionId { path_id, sequence };
-5563 builder.write_frame(frame, stats);
-5564 }
-5565
-5566 // DATAGRAM
-5567 let mut sent_datagrams = false;
-5568 while !path_exclusive_only
-5569 && builder.frame_space_remaining() > Datagram::SIZE_BOUND
-5570 && space_id == SpaceId::Data
-5571 {
-5572 match self.datagrams.write(builder, stats) {
-5573 true => {
-5574 sent_datagrams = true;
-5575 }
-5576 false => break,
-5577 }
-5578 }
-5579 if self.datagrams.send_blocked && sent_datagrams {
-5580 self.events.push_back(Event::DatagramsUnblocked);
-5581 self.datagrams.send_blocked = false;
-5582 }
-5583
-5584 let path = &mut self.paths.get_mut(&path_id).expect("known path").data;
-5585
-5586 // NEW_TOKEN
-5587 if !path_exclusive_only {
-5588 while let Some(network_path) = space.pending.new_tokens.pop() {
-5589 debug_assert_eq!(space_id, SpaceId::Data);
-5590 let ConnectionSide::Server { server_config } = &self.side else {
-5591 panic!("NEW_TOKEN frames should not be enqueued by clients");
-5592 };
-5593
-5594 if !network_path.is_probably_same_path(&path.network_path) {
-5595 // NEW_TOKEN frames contain tokens bound to a client's IP address, and are only
-5596 // useful if used from the same IP address. Thus, we abandon enqueued NEW_TOKEN
-5597 // frames upon an path change. Instead, when the new path becomes validated,
-5598 // NEW_TOKEN frames may be enqueued for the new path instead.
-5599 continue;
+5511 // TODO(flub): maybe this is much higher priority?
+5512 // PATH_ABANDON
+5513 while !path_exclusive_only
+5514 && space_id == SpaceId::Data
+5515 && frame::PathAbandon::SIZE_BOUND <= builder.frame_space_remaining()
+5516 {
+5517 let Some((abandoned_path_id, error_code)) = space.pending.path_abandon.pop_first()
+5518 else {
+5519 break;
+5520 };
+5521 let frame = frame::PathAbandon {
+5522 path_id: abandoned_path_id,
+5523 error_code,
+5524 };
+5525 builder.write_frame(frame, stats);
+5526
+5527 let ack_delay = self.ack_frequency.max_ack_delay_for_pto();
+5528 // We can't access path here anymore due to borrowing issues.
+5529 let send_pto = self.paths.get(&path_id).unwrap().data.rtt.pto_base() + ack_delay;
+5530 if let Some(abandoned_path) = self.paths.get_mut(&abandoned_path_id) {
+5531 // We only want to set the deadline on the *first* PATH_ABANDON we send.
+5532 // Retransmits shouldn't run this code again
+5533 if matches!(
+5534 abandoned_path.data.abandon_state,
+5535 AbandonState::NotAbandoned
+5536 ) {
+5537 // The peer MUST respond with a corresponding PATH_ABANDON frame.
+5538 // The other peer has 3 * PTO to do that.
+5539 // This uses the PTO of the path we send on!
+5540 // If the PATH_ABANDON comes in within the deadline we're giving here, then this
+5541 // state will be set to `AbandonState::ReceivedPathAbandon`, essentially clearing
+5542 // the deadline. If we receive a frame after the deadline, we error out with a
+5543 // protocol violation.
+5544 // Receiving other frames before the deadline is fine, as those might be packets
+5545 // that were still in-flight.
+5546 abandoned_path.data.abandon_state = AbandonState::ExpectingPathAbandon {
+5547 deadline: now + 3 * send_pto,
+5548 };
+5549
+5550 // At some point, we need to forget about the path.
+5551 // If we do so too early, then we'll have discarded the CIDs of that path and won't
+5552 // handle incoming packets on that path correctly.
+5553 // To give this path enough time, we assume that the peer will have received our
+5554 // PATH_ABANDON within 3 * PTO of the path we sent the abandon on,
+5555 // and then we give the path 3 * PTO time to make it very unlikely that there will
+5556 // still be packets incoming on the path at that point.
+5557 // This timer will actually get reset to a value that's likely to be even earlier
+5558 // once we actually receive the PATH_ABANDON frame itself.
+5559 let abandoned_pto =
+5560 self.paths.get(&path_id).unwrap().data.rtt.pto_base() + ack_delay;
+5561 self.timers.set(
+5562 Timer::PerPath(abandoned_path_id, PathTimer::DiscardPath),
+5563 now + 3 * send_pto + 3 * abandoned_pto,
+5564 self.qlog.with_time(now),
+5565 );
+5566 }
+5567 } else {
+5568 warn!("sent PATH_ABANDON after path was already discarded");
+5569 }
+5570 }
+5571
+5572 // PATH_STATUS_AVAILABLE & PATH_STATUS_BACKUP
+5573 while !path_exclusive_only
+5574 && space_id == SpaceId::Data
+5575 && frame::PathStatusAvailable::SIZE_BOUND <= builder.frame_space_remaining()
+5576 {
+5577 let Some(path_id) = space.pending.path_status.pop_first() else {
+5578 break;
+5579 };
+5580 let Some(path) = self.paths.get(&path_id).map(|path_state| &path_state.data) else {
+5581 trace!(%path_id, "discarding queued path status for unknown path");
+5582 continue;
+5583 };
+5584
+5585 let seq = path.status.seq();
+5586 match path.local_status() {
+5587 PathStatus::Available => {
+5588 let frame = frame::PathStatusAvailable {
+5589 path_id,
+5590 status_seq_no: seq,
+5591 };
+5592 builder.write_frame(frame, stats);
+5593 }
+5594 PathStatus::Backup => {
+5595 let frame = frame::PathStatusBackup {
+5596 path_id,
+5597 status_seq_no: seq,
+5598 };
+5599 builder.write_frame(frame, stats);
5600 }
-5601
-5602 let token = Token::new(
-5603 TokenPayload::Validation {
-5604 ip: network_path.remote.ip(),
-5605 issued: server_config.time_source.now(),
-5606 },
-5607 &mut self.rng,
-5608 );
-5609 let new_token = NewToken {
-5610 token: token.encode(&*server_config.token_key).into(),
-5611 };
-5612
-5613 if builder.frame_space_remaining() < new_token.size() {
-5614 space.pending.new_tokens.push(network_path);
-5615 break;
-5616 }
-5617
-5618 builder.write_frame(new_token, stats);
-5619 builder.retransmits_mut().new_tokens.push(network_path);
-5620 }
-5621 }
-5622
-5623 // STREAM
-5624 if !path_exclusive_only && space_id == SpaceId::Data {
-5625 self.streams
-5626 .write_stream_frames(builder, self.config.send_fairness, stats);
-5627 }
-5628
-5629 // ADD_ADDRESS
-5630 // TODO(@divma): check if we need to do path exclusive filters
-5631 while space_id == SpaceId::Data
-5632 && frame::AddAddress::SIZE_BOUND <= builder.frame_space_remaining()
-5633 {
-5634 if let Some(added_address) = space.pending.add_address.pop_last() {
-5635 builder.write_frame(added_address, stats);
-5636 } else {
-5637 break;
-5638 }
-5639 }
-5640
-5641 // REMOVE_ADDRESS
-5642 while space_id == SpaceId::Data
-5643 && frame::RemoveAddress::SIZE_BOUND <= builder.frame_space_remaining()
-5644 {
-5645 if let Some(removed_address) = space.pending.remove_address.pop_last() {
-5646 builder.write_frame(removed_address, stats);
-5647 } else {
-5648 break;
-5649 }
-5650 }
-5651 }
-5652
-5653 /// Write pending ACKs into a buffer
-5654 fn populate_acks<'a, 'b>(
-5655 now: Instant,
-5656 receiving_ecn: bool,
-5657 path_id: PathId,
-5658 space_id: SpaceId,
-5659 space: &mut PacketSpace,
-5660 is_multipath_negotiated: bool,
-5661 builder: &mut PacketBuilder<'a, 'b>,
-5662 stats: &mut FrameStats,
-5663 ) {
-5664 // 0-RTT packets must never carry acks (which would have to be of handshake packets)
-5665 debug_assert!(space.crypto.is_some(), "tried to send ACK in 0-RTT");
-5666
-5667 debug_assert!(
-5668 is_multipath_negotiated || path_id == PathId::ZERO,
-5669 "Only PathId::ZERO allowed without multipath (have {path_id:?})"
-5670 );
-5671 if is_multipath_negotiated {
-5672 debug_assert!(
-5673 space_id == SpaceId::Data || path_id == PathId::ZERO,
-5674 "path acks must be sent in 1RTT space (have {space_id:?})"
-5675 );
-5676 }
-5677
-5678 let pns = space.for_path(path_id);
-5679 let ranges = pns.pending_acks.ranges();
-5680 debug_assert!(!ranges.is_empty(), "can not send empty ACK range");
-5681 let ecn = if receiving_ecn {
-5682 Some(&pns.ecn_counters)
-5683 } else {
-5684 None
-5685 };
-5686
-5687 let delay_micros = pns.pending_acks.ack_delay(now).as_micros() as u64;
-5688 // TODO: This should come from `TransportConfig` if that gets configurable.
-5689 let ack_delay_exp = TransportParameters::default().ack_delay_exponent;
-5690 let delay = delay_micros >> ack_delay_exp.into_inner();
-5691
-5692 if is_multipath_negotiated && space_id == SpaceId::Data {
-5693 if !ranges.is_empty() {
-5694 let frame = frame::PathAck::encoder(path_id, delay, ranges, ecn);
-5695 builder.write_frame(frame, stats);
-5696 }
-5697 } else {
-5698 builder.write_frame(frame::Ack::encoder(delay, ranges, ecn), stats);
-5699 }
-5700 }
-5701
-5702 fn close_common(&mut self) {
-5703 trace!("connection closed");
-5704 self.timers.reset();
-5705 }
-5706
-5707 fn set_close_timer(&mut self, now: Instant) {
-5708 // QUIC-MULTIPATH § 2.6 Connection Closure: draining for 3*PTO with PTO the max of
-5709 // the PTO for all paths.
-5710 let pto_max = self.pto_max_path(self.highest_space, true);
-5711 self.timers.set(
-5712 Timer::Conn(ConnTimer::Close),
-5713 now + 3 * pto_max,
-5714 self.qlog.with_time(now),
-5715 );
-5716 }
-5717
-5718 /// Handle transport parameters received from the peer
-5719 ///
-5720 /// *rem_cid* and *loc_cid* are the source and destination CIDs respectively of the
-5721 /// *packet into which the transport parameters arrived.
-5722 fn handle_peer_params(
-5723 &mut self,
-5724 params: TransportParameters,
-5725 loc_cid: ConnectionId,
-5726 rem_cid: ConnectionId,
-5727 now: Instant,
-5728 ) -> Result<(), TransportError> {
-5729 if Some(self.orig_rem_cid) != params.initial_src_cid
-5730 || (self.side.is_client()
-5731 && (Some(self.initial_dst_cid) != params.original_dst_cid
-5732 || self.retry_src_cid != params.retry_src_cid))
-5733 {
-5734 return Err(TransportError::TRANSPORT_PARAMETER_ERROR(
-5735 "CID authentication failure",
-5736 ));
-5737 }
-5738 if params.initial_max_path_id.is_some() && (loc_cid.is_empty() || rem_cid.is_empty()) {
-5739 return Err(TransportError::PROTOCOL_VIOLATION(
-5740 "multipath must not use zero-length CIDs",
-5741 ));
-5742 }
-5743
-5744 self.set_peer_params(params);
-5745 self.qlog.emit_peer_transport_params_received(self, now);
-5746
-5747 Ok(())
-5748 }
-5749
-5750 fn set_peer_params(&mut self, params: TransportParameters) {
-5751 self.streams.set_params(¶ms);
-5752 self.idle_timeout =
-5753 negotiate_max_idle_timeout(self.config.max_idle_timeout, Some(params.max_idle_timeout));
-5754 trace!("negotiated max idle timeout {:?}", self.idle_timeout);
-5755
-5756 if let Some(ref info) = params.preferred_address {
-5757 // During the handshake PathId::ZERO exists.
-5758 self.rem_cids.get_mut(&PathId::ZERO).expect("not yet abandoned").insert(frame::NewConnectionId {
-5759 path_id: None,
-5760 sequence: 1,
-5761 id: info.connection_id,
-5762 reset_token: info.stateless_reset_token,
-5763 retire_prior_to: 0,
-5764 })
-5765 .expect(
-5766 "preferred address CID is the first received, and hence is guaranteed to be legal",
-5767 );
-5768 let remote = self.path_data(PathId::ZERO).network_path.remote;
-5769 self.set_reset_token(PathId::ZERO, remote, info.stateless_reset_token);
-5770 }
-5771 self.ack_frequency.peer_max_ack_delay = get_max_ack_delay(¶ms);
-5772
-5773 let mut multipath_enabled = None;
-5774 if let (Some(local_max_path_id), Some(remote_max_path_id)) = (
-5775 self.config.get_initial_max_path_id(),
-5776 params.initial_max_path_id,
-5777 ) {
-5778 // multipath is enabled, register the local and remote maximums
-5779 self.local_max_path_id = local_max_path_id;
-5780 self.remote_max_path_id = remote_max_path_id;
-5781 let initial_max_path_id = local_max_path_id.min(remote_max_path_id);
-5782 debug!(%initial_max_path_id, "multipath negotiated");
-5783 multipath_enabled = Some(initial_max_path_id);
-5784 }
-5785
-5786 if let Some((max_locally_allowed_remote_addresses, max_remotely_allowed_remote_addresses)) =
-5787 self.config
-5788 .max_remote_nat_traversal_addresses
-5789 .zip(params.max_remote_nat_traversal_addresses)
-5790 {
-5791 if let Some(max_initial_paths) =
-5792 multipath_enabled.map(|path_id| path_id.saturating_add(1u8))
-5793 {
-5794 let max_local_addresses = max_remotely_allowed_remote_addresses.get();
-5795 let max_remote_addresses = max_locally_allowed_remote_addresses.get();
-5796 self.iroh_hp =
-5797 iroh_hp::State::new(max_remote_addresses, max_local_addresses, self.side());
-5798 debug!(
-5799 %max_remote_addresses, %max_local_addresses,
-5800 "iroh hole punching negotiated"
-5801 );
-5802
-5803 match self.side() {
-5804 Side::Client => {
-5805 if max_initial_paths.as_u32() < max_remote_addresses as u32 + 1 {
-5806 // in this case the client might try to open `max_remote_addresses` new
-5807 // paths, but the current multipath configuration will not allow it
-5808 warn!(%max_initial_paths, %max_remote_addresses, "local client configuration might cause nat traversal issues")
-5809 } else if max_local_addresses as u64
-5810 > params.active_connection_id_limit.into_inner()
-5811 {
-5812 // the server allows us to send at most `params.active_connection_id_limit`
-5813 // but they might need at least `max_local_addresses` to effectively send
-5814 // `PATH_CHALLENGE` frames to each advertised local address
-5815 warn!(%max_local_addresses, remote_cid_limit=%params.active_connection_id_limit.into_inner(), "remote server configuration might cause nat traversal issues")
-5816 }
-5817 }
-5818 Side::Server => {
-5819 if (max_initial_paths.as_u32() as u64) < crate::LOC_CID_COUNT {
-5820 warn!(%max_initial_paths, local_cid_limit=%crate::LOC_CID_COUNT, "local server configuration might cause nat traversal issues")
-5821 }
-5822 }
-5823 }
-5824 } else {
-5825 debug!("iroh nat traversal enabled for both endpoints, but multipath is missing")
-5826 }
+5601 }
+5602 }
+5603
+5604 // MAX_PATH_ID
+5605 if space_id == SpaceId::Data
+5606 && space.pending.max_path_id
+5607 && frame::MaxPathId::SIZE_BOUND <= builder.frame_space_remaining()
+5608 {
+5609 let frame = frame::MaxPathId(self.local_max_path_id);
+5610 builder.write_frame(frame, stats);
+5611 space.pending.max_path_id = false;
+5612 }
+5613
+5614 // PATHS_BLOCKED
+5615 if space_id == SpaceId::Data
+5616 && space.pending.paths_blocked
+5617 && frame::PathsBlocked::SIZE_BOUND <= builder.frame_space_remaining()
+5618 {
+5619 let frame = frame::PathsBlocked(self.remote_max_path_id);
+5620 builder.write_frame(frame, stats);
+5621 space.pending.paths_blocked = false;
+5622 }
+5623
+5624 // PATH_CIDS_BLOCKED
+5625 while space_id == SpaceId::Data
+5626 && frame::PathCidsBlocked::SIZE_BOUND <= builder.frame_space_remaining()
+5627 {
+5628 let Some(path_id) = space.pending.path_cids_blocked.pop_first() else {
+5629 break;
+5630 };
+5631 let next_seq = match self.rem_cids.get(&path_id) {
+5632 Some(cid_queue) => VarInt(cid_queue.active_seq() + 1),
+5633 None => VarInt(0),
+5634 };
+5635 let frame = frame::PathCidsBlocked { path_id, next_seq };
+5636 builder.write_frame(frame, stats);
+5637 }
+5638
+5639 // RESET_STREAM, STOP_SENDING, MAX_DATA, MAX_STREAM_DATA, MAX_STREAMS
+5640 if space_id == SpaceId::Data {
+5641 self.streams
+5642 .write_control_frames(builder, &mut space.pending, stats);
+5643 }
+5644
+5645 // NEW_CONNECTION_ID
+5646 let cid_len = self
+5647 .local_cid_state
+5648 .values()
+5649 .map(|cid_state| cid_state.cid_len())
+5650 .max()
+5651 .expect("some local CID state must exist");
+5652 let new_cid_size_bound =
+5653 frame::NewConnectionId::size_bound(is_multipath_negotiated, cid_len);
+5654 while !path_exclusive_only && builder.frame_space_remaining() > new_cid_size_bound {
+5655 let issued = match space.pending.new_cids.pop() {
+5656 Some(x) => x,
+5657 None => break,
+5658 };
+5659 let retire_prior_to = self
+5660 .local_cid_state
+5661 .get(&issued.path_id)
+5662 .map(|cid_state| cid_state.retire_prior_to())
+5663 .unwrap_or_else(|| panic!("missing local CID state for path={}", issued.path_id));
+5664
+5665 let cid_path_id = match is_multipath_negotiated {
+5666 true => Some(issued.path_id),
+5667 false => {
+5668 debug_assert_eq!(issued.path_id, PathId::ZERO);
+5669 None
+5670 }
+5671 };
+5672 let frame = frame::NewConnectionId {
+5673 path_id: cid_path_id,
+5674 sequence: issued.sequence,
+5675 retire_prior_to,
+5676 id: issued.id,
+5677 reset_token: issued.reset_token,
+5678 };
+5679 builder.write_frame(frame, stats);
+5680 }
+5681
+5682 // RETIRE_CONNECTION_ID
+5683 let retire_cid_bound = frame::RetireConnectionId::size_bound(is_multipath_negotiated);
+5684 while !path_exclusive_only && builder.frame_space_remaining() > retire_cid_bound {
+5685 let (path_id, sequence) = match space.pending.retire_cids.pop() {
+5686 Some((PathId::ZERO, seq)) if !is_multipath_negotiated => (None, seq),
+5687 Some((path_id, seq)) => (Some(path_id), seq),
+5688 None => break,
+5689 };
+5690 let frame = frame::RetireConnectionId { path_id, sequence };
+5691 builder.write_frame(frame, stats);
+5692 }
+5693
+5694 // DATAGRAM
+5695 let mut sent_datagrams = false;
+5696 while !path_exclusive_only
+5697 && builder.frame_space_remaining() > Datagram::SIZE_BOUND
+5698 && space_id == SpaceId::Data
+5699 {
+5700 match self.datagrams.write(builder, stats) {
+5701 true => {
+5702 sent_datagrams = true;
+5703 }
+5704 false => break,
+5705 }
+5706 }
+5707 if self.datagrams.send_blocked && sent_datagrams {
+5708 self.events.push_back(Event::DatagramsUnblocked);
+5709 self.datagrams.send_blocked = false;
+5710 }
+5711
+5712 let path = &mut self.paths.get_mut(&path_id).expect("known path").data;
+5713
+5714 // NEW_TOKEN
+5715 if !path_exclusive_only {
+5716 while let Some(network_path) = space.pending.new_tokens.pop() {
+5717 debug_assert_eq!(space_id, SpaceId::Data);
+5718 let ConnectionSide::Server { server_config } = &self.side else {
+5719 panic!("NEW_TOKEN frames should not be enqueued by clients");
+5720 };
+5721
+5722 if !network_path.is_probably_same_path(&path.network_path) {
+5723 // NEW_TOKEN frames contain tokens bound to a client's IP address, and are only
+5724 // useful if used from the same IP address. Thus, we abandon enqueued NEW_TOKEN
+5725 // frames upon an path change. Instead, when the new path becomes validated,
+5726 // NEW_TOKEN frames may be enqueued for the new path instead.
+5727 continue;
+5728 }
+5729
+5730 let token = Token::new(
+5731 TokenPayload::Validation {
+5732 ip: network_path.remote.ip(),
+5733 issued: server_config.time_source.now(),
+5734 },
+5735 &mut self.rng,
+5736 );
+5737 let new_token = NewToken {
+5738 token: token.encode(&*server_config.token_key).into(),
+5739 };
+5740
+5741 if builder.frame_space_remaining() < new_token.size() {
+5742 space.pending.new_tokens.push(network_path);
+5743 break;
+5744 }
+5745
+5746 builder.write_frame(new_token, stats);
+5747 builder.retransmits_mut().new_tokens.push(network_path);
+5748 }
+5749 }
+5750
+5751 // STREAM
+5752 if !path_exclusive_only && space_id == SpaceId::Data {
+5753 self.streams
+5754 .write_stream_frames(builder, self.config.send_fairness, stats);
+5755 }
+5756
+5757 // ADD_ADDRESS
+5758 // TODO(@divma): check if we need to do path exclusive filters
+5759 while space_id == SpaceId::Data
+5760 && frame::AddAddress::SIZE_BOUND <= builder.frame_space_remaining()
+5761 {
+5762 if let Some(added_address) = space.pending.add_address.pop_last() {
+5763 builder.write_frame(added_address, stats);
+5764 } else {
+5765 break;
+5766 }
+5767 }
+5768
+5769 // REMOVE_ADDRESS
+5770 while space_id == SpaceId::Data
+5771 && frame::RemoveAddress::SIZE_BOUND <= builder.frame_space_remaining()
+5772 {
+5773 if let Some(removed_address) = space.pending.remove_address.pop_last() {
+5774 builder.write_frame(removed_address, stats);
+5775 } else {
+5776 break;
+5777 }
+5778 }
+5779 }
+5780
+5781 /// Write pending ACKs into a buffer
+5782 fn populate_acks<'a, 'b>(
+5783 now: Instant,
+5784 receiving_ecn: bool,
+5785 path_id: PathId,
+5786 space_id: SpaceId,
+5787 space: &mut PacketSpace,
+5788 is_multipath_negotiated: bool,
+5789 builder: &mut PacketBuilder<'a, 'b>,
+5790 stats: &mut FrameStats,
+5791 ) {
+5792 // 0-RTT packets must never carry acks (which would have to be of handshake packets)
+5793 debug_assert!(space.crypto.is_some(), "tried to send ACK in 0-RTT");
+5794
+5795 debug_assert!(
+5796 is_multipath_negotiated || path_id == PathId::ZERO,
+5797 "Only PathId::ZERO allowed without multipath (have {path_id:?})"
+5798 );
+5799 if is_multipath_negotiated {
+5800 debug_assert!(
+5801 space_id == SpaceId::Data || path_id == PathId::ZERO,
+5802 "path acks must be sent in 1RTT space (have {space_id:?})"
+5803 );
+5804 }
+5805
+5806 let pns = space.for_path(path_id);
+5807 let ranges = pns.pending_acks.ranges();
+5808 debug_assert!(!ranges.is_empty(), "can not send empty ACK range");
+5809 let ecn = if receiving_ecn {
+5810 Some(&pns.ecn_counters)
+5811 } else {
+5812 None
+5813 };
+5814
+5815 let delay_micros = pns.pending_acks.ack_delay(now).as_micros() as u64;
+5816 // TODO: This should come from `TransportConfig` if that gets configurable.
+5817 let ack_delay_exp = TransportParameters::default().ack_delay_exponent;
+5818 let delay = delay_micros >> ack_delay_exp.into_inner();
+5819
+5820 if is_multipath_negotiated && space_id == SpaceId::Data {
+5821 if !ranges.is_empty() {
+5822 let frame = frame::PathAck::encoder(path_id, delay, ranges, ecn);
+5823 builder.write_frame(frame, stats);
+5824 }
+5825 } else {
+5826 builder.write_frame(frame::Ack::encoder(delay, ranges, ecn), stats);
5827 }
-5828
-5829 self.peer_params = params;
-5830 let peer_max_udp_payload_size =
-5831 u16::try_from(self.peer_params.max_udp_payload_size.into_inner()).unwrap_or(u16::MAX);
-5832 self.path_data_mut(PathId::ZERO)
-5833 .mtud
-5834 .on_peer_max_udp_payload_size_received(peer_max_udp_payload_size);
-5835 }
-5836
-5837 /// Decrypts a packet, returning the packet number on success
-5838 fn decrypt_packet(
-5839 &mut self,
-5840 now: Instant,
-5841 path_id: PathId,
-5842 packet: &mut Packet,
-5843 ) -> Result<Option<u64>, Option<TransportError>> {
-5844 let result = packet_crypto::decrypt_packet_body(
-5845 packet,
-5846 path_id,
-5847 &self.spaces,
-5848 self.zero_rtt_crypto.as_ref(),
-5849 self.key_phase,
-5850 self.prev_crypto.as_ref(),
-5851 self.next_crypto.as_ref(),
-5852 )?;
-5853
-5854 let result = match result {
-5855 Some(r) => r,
-5856 None => return Ok(None),
-5857 };
-5858
-5859 if result.outgoing_key_update_acked {
-5860 if let Some(prev) = self.prev_crypto.as_mut() {
-5861 prev.end_packet = Some((result.number, now));
-5862 self.set_key_discard_timer(now, packet.header.space());
-5863 }
-5864 }
-5865
-5866 if result.incoming_key_update {
-5867 trace!("key update authenticated");
-5868 self.update_keys(Some((result.number, now)), true);
-5869 self.set_key_discard_timer(now, packet.header.space());
+5828 }
+5829
+5830 fn close_common(&mut self) {
+5831 trace!("connection closed");
+5832 self.timers.reset();
+5833 }
+5834
+5835 fn set_close_timer(&mut self, now: Instant) {
+5836 // QUIC-MULTIPATH § 2.6 Connection Closure: draining for 3*PTO with PTO the max of
+5837 // the PTO for all paths.
+5838 let pto_max = self.pto_max_path(self.highest_space, true);
+5839 self.timers.set(
+5840 Timer::Conn(ConnTimer::Close),
+5841 now + 3 * pto_max,
+5842 self.qlog.with_time(now),
+5843 );
+5844 }
+5845
+5846 /// Handle transport parameters received from the peer
+5847 ///
+5848 /// *rem_cid* and *loc_cid* are the source and destination CIDs respectively of the
+5849 /// *packet into which the transport parameters arrived.
+5850 fn handle_peer_params(
+5851 &mut self,
+5852 params: TransportParameters,
+5853 loc_cid: ConnectionId,
+5854 rem_cid: ConnectionId,
+5855 now: Instant,
+5856 ) -> Result<(), TransportError> {
+5857 if Some(self.orig_rem_cid) != params.initial_src_cid
+5858 || (self.side.is_client()
+5859 && (Some(self.initial_dst_cid) != params.original_dst_cid
+5860 || self.retry_src_cid != params.retry_src_cid))
+5861 {
+5862 return Err(TransportError::TRANSPORT_PARAMETER_ERROR(
+5863 "CID authentication failure",
+5864 ));
+5865 }
+5866 if params.initial_max_path_id.is_some() && (loc_cid.is_empty() || rem_cid.is_empty()) {
+5867 return Err(TransportError::PROTOCOL_VIOLATION(
+5868 "multipath must not use zero-length CIDs",
+5869 ));
5870 }
5871
-5872 Ok(Some(result.number))
-5873 }
+5872 self.set_peer_params(params);
+5873 self.qlog.emit_peer_transport_params_received(self, now);
5874
-5875 fn update_keys(&mut self, end_packet: Option<(u64, Instant)>, remote: bool) {
-5876 trace!("executing key update");
-5877 // Generate keys for the key phase after the one we're switching to, store them in
-5878 // `next_crypto`, make the contents of `next_crypto` current, and move the current keys into
-5879 // `prev_crypto`.
-5880 let new = self
-5881 .crypto
-5882 .next_1rtt_keys()
-5883 .expect("only called for `Data` packets");
-5884 self.key_phase_size = new
-5885 .local
-5886 .confidentiality_limit()
-5887 .saturating_sub(KEY_UPDATE_MARGIN);
-5888 let old = mem::replace(
-5889 &mut self.spaces[SpaceId::Data]
-5890 .crypto
-5891 .as_mut()
-5892 .unwrap() // safe because update_keys() can only be triggered by short packets
-5893 .packet,
-5894 mem::replace(self.next_crypto.as_mut().unwrap(), new),
-5895 );
-5896 self.spaces[SpaceId::Data]
-5897 .iter_paths_mut()
-5898 .for_each(|s| s.sent_with_keys = 0);
-5899 self.prev_crypto = Some(PrevCrypto {
-5900 crypto: old,
-5901 end_packet,
-5902 update_unacked: remote,
-5903 });
-5904 self.key_phase = !self.key_phase;
-5905 }
-5906
-5907 fn peer_supports_ack_frequency(&self) -> bool {
-5908 self.peer_params.min_ack_delay.is_some()
-5909 }
-5910
-5911 /// Send an IMMEDIATE_ACK frame to the remote endpoint
-5912 ///
-5913 /// According to the spec, this will result in an error if the remote endpoint does not support
-5914 /// the Acknowledgement Frequency extension
-5915 pub(crate) fn immediate_ack(&mut self, path_id: PathId) {
-5916 debug_assert_eq!(
-5917 self.highest_space,
-5918 SpaceId::Data,
-5919 "immediate ack must be written in the data space"
-5920 );
-5921 self.spaces[self.highest_space]
-5922 .for_path(path_id)
-5923 .immediate_ack_pending = true;
-5924 }
-5925
-5926 /// Decodes a packet, returning its decrypted payload, so it can be inspected in tests
-5927 #[cfg(test)]
-5928 pub(crate) fn decode_packet(&self, event: &ConnectionEvent) -> Option<Vec<u8>> {
-5929 let (path_id, first_decode, remaining) = match &event.0 {
-5930 ConnectionEventInner::Datagram(DatagramConnectionEvent {
-5931 path_id,
-5932 first_decode,
-5933 remaining,
-5934 ..
-5935 }) => (path_id, first_decode, remaining),
-5936 _ => return None,
-5937 };
-5938
-5939 if remaining.is_some() {
-5940 panic!("Packets should never be coalesced in tests");
-5941 }
-5942
-5943 let decrypted_header = packet_crypto::unprotect_header(
-5944 first_decode.clone(),
-5945 &self.spaces,
-5946 self.zero_rtt_crypto.as_ref(),
-5947 self.peer_params.stateless_reset_token,
-5948 )?;
-5949
-5950 let mut packet = decrypted_header.packet?;
-5951 packet_crypto::decrypt_packet_body(
-5952 &mut packet,
-5953 *path_id,
-5954 &self.spaces,
-5955 self.zero_rtt_crypto.as_ref(),
-5956 self.key_phase,
-5957 self.prev_crypto.as_ref(),
-5958 self.next_crypto.as_ref(),
-5959 )
-5960 .ok()?;
-5961
-5962 Some(packet.payload.to_vec())
+5875 Ok(())
+5876 }
+5877
+5878 fn set_peer_params(&mut self, params: TransportParameters) {
+5879 self.streams.set_params(¶ms);
+5880 self.idle_timeout =
+5881 negotiate_max_idle_timeout(self.config.max_idle_timeout, Some(params.max_idle_timeout));
+5882 trace!("negotiated max idle timeout {:?}", self.idle_timeout);
+5883
+5884 if let Some(ref info) = params.preferred_address {
+5885 // During the handshake PathId::ZERO exists.
+5886 self.rem_cids.get_mut(&PathId::ZERO).expect("not yet abandoned").insert(frame::NewConnectionId {
+5887 path_id: None,
+5888 sequence: 1,
+5889 id: info.connection_id,
+5890 reset_token: info.stateless_reset_token,
+5891 retire_prior_to: 0,
+5892 })
+5893 .expect(
+5894 "preferred address CID is the first received, and hence is guaranteed to be legal",
+5895 );
+5896 let remote = self.path_data(PathId::ZERO).network_path.remote;
+5897 self.set_reset_token(PathId::ZERO, remote, info.stateless_reset_token);
+5898 }
+5899 self.ack_frequency.peer_max_ack_delay = get_max_ack_delay(¶ms);
+5900
+5901 let mut multipath_enabled = None;
+5902 if let (Some(local_max_path_id), Some(remote_max_path_id)) = (
+5903 self.config.get_initial_max_path_id(),
+5904 params.initial_max_path_id,
+5905 ) {
+5906 // multipath is enabled, register the local and remote maximums
+5907 self.local_max_path_id = local_max_path_id;
+5908 self.remote_max_path_id = remote_max_path_id;
+5909 let initial_max_path_id = local_max_path_id.min(remote_max_path_id);
+5910 debug!(%initial_max_path_id, "multipath negotiated");
+5911 multipath_enabled = Some(initial_max_path_id);
+5912 }
+5913
+5914 if let Some((max_locally_allowed_remote_addresses, max_remotely_allowed_remote_addresses)) =
+5915 self.config
+5916 .max_remote_nat_traversal_addresses
+5917 .zip(params.max_remote_nat_traversal_addresses)
+5918 {
+5919 if let Some(max_initial_paths) =
+5920 multipath_enabled.map(|path_id| path_id.saturating_add(1u8))
+5921 {
+5922 let max_local_addresses = max_remotely_allowed_remote_addresses.get();
+5923 let max_remote_addresses = max_locally_allowed_remote_addresses.get();
+5924 self.iroh_hp =
+5925 iroh_hp::State::new(max_remote_addresses, max_local_addresses, self.side());
+5926 debug!(
+5927 %max_remote_addresses, %max_local_addresses,
+5928 "iroh hole punching negotiated"
+5929 );
+5930
+5931 match self.side() {
+5932 Side::Client => {
+5933 if max_initial_paths.as_u32() < max_remote_addresses as u32 + 1 {
+5934 // in this case the client might try to open `max_remote_addresses` new
+5935 // paths, but the current multipath configuration will not allow it
+5936 warn!(%max_initial_paths, %max_remote_addresses, "local client configuration might cause nat traversal issues")
+5937 } else if max_local_addresses as u64
+5938 > params.active_connection_id_limit.into_inner()
+5939 {
+5940 // the server allows us to send at most `params.active_connection_id_limit`
+5941 // but they might need at least `max_local_addresses` to effectively send
+5942 // `PATH_CHALLENGE` frames to each advertised local address
+5943 warn!(%max_local_addresses, remote_cid_limit=%params.active_connection_id_limit.into_inner(), "remote server configuration might cause nat traversal issues")
+5944 }
+5945 }
+5946 Side::Server => {
+5947 if (max_initial_paths.as_u32() as u64) < crate::LOC_CID_COUNT {
+5948 warn!(%max_initial_paths, local_cid_limit=%crate::LOC_CID_COUNT, "local server configuration might cause nat traversal issues")
+5949 }
+5950 }
+5951 }
+5952 } else {
+5953 debug!("iroh nat traversal enabled for both endpoints, but multipath is missing")
+5954 }
+5955 }
+5956
+5957 self.peer_params = params;
+5958 let peer_max_udp_payload_size =
+5959 u16::try_from(self.peer_params.max_udp_payload_size.into_inner()).unwrap_or(u16::MAX);
+5960 self.path_data_mut(PathId::ZERO)
+5961 .mtud
+5962 .on_peer_max_udp_payload_size_received(peer_max_udp_payload_size);
5963 }
5964
-5965 /// The number of bytes of packets containing retransmittable frames that have not been
-5966 /// acknowledged or declared lost.
-5967 #[cfg(test)]
-5968 pub(crate) fn bytes_in_flight(&self) -> u64 {
-5969 // TODO(@divma): consider including for multipath?
-5970 self.path_data(PathId::ZERO).in_flight.bytes
-5971 }
-5972
-5973 /// Number of bytes worth of non-ack-only packets that may be sent
-5974 #[cfg(test)]
-5975 pub(crate) fn congestion_window(&self) -> u64 {
-5976 let path = self.path_data(PathId::ZERO);
-5977 path.congestion
-5978 .window()
-5979 .saturating_sub(path.in_flight.bytes)
-5980 }
+5965 /// Decrypts a packet, returning the packet number on success
+5966 fn decrypt_packet(
+5967 &mut self,
+5968 now: Instant,
+5969 path_id: PathId,
+5970 packet: &mut Packet,
+5971 ) -> Result<Option<u64>, Option<TransportError>> {
+5972 let result = packet_crypto::decrypt_packet_body(
+5973 packet,
+5974 path_id,
+5975 &self.spaces,
+5976 self.zero_rtt_crypto.as_ref(),
+5977 self.key_phase,
+5978 self.prev_crypto.as_ref(),
+5979 self.next_crypto.as_ref(),
+5980 )?;
5981
-5982 /// Whether no timers but keepalive, idle, rtt, pushnewcid, and key discard are running
-5983 #[cfg(test)]
-5984 pub(crate) fn is_idle(&self) -> bool {
-5985 let current_timers = self.timers.values();
-5986 current_timers
-5987 .into_iter()
-5988 .filter(|(timer, _)| {
-5989 !matches!(
-5990 timer,
-5991 Timer::Conn(ConnTimer::KeepAlive)
-5992 | Timer::PerPath(_, PathTimer::PathKeepAlive)
-5993 | Timer::Conn(ConnTimer::PushNewCid)
-5994 | Timer::Conn(ConnTimer::KeyDiscard)
-5995 )
-5996 })
-5997 .min_by_key(|(_, time)| *time)
-5998 .is_none_or(|(timer, _)| timer == Timer::Conn(ConnTimer::Idle))
-5999 }
-6000
-6001 /// Whether explicit congestion notification is in use on outgoing packets.
-6002 #[cfg(test)]
-6003 pub(crate) fn using_ecn(&self) -> bool {
-6004 self.path_data(PathId::ZERO).sending_ecn
-6005 }
-6006
-6007 /// The number of received bytes in the current path
-6008 #[cfg(test)]
-6009 pub(crate) fn total_recvd(&self) -> u64 {
-6010 self.path_data(PathId::ZERO).total_recvd
-6011 }
-6012
-6013 #[cfg(test)]
-6014 pub(crate) fn active_local_cid_seq(&self) -> (u64, u64) {
-6015 self.local_cid_state
-6016 .get(&PathId::ZERO)
-6017 .unwrap()
-6018 .active_seq()
-6019 }
-6020
-6021 #[cfg(test)]
-6022 #[track_caller]
-6023 pub(crate) fn active_local_path_cid_seq(&self, path_id: u32) -> (u64, u64) {
-6024 self.local_cid_state
-6025 .get(&PathId(path_id))
-6026 .unwrap()
-6027 .active_seq()
-6028 }
-6029
-6030 /// Instruct the peer to replace previously issued CIDs by sending a NEW_CONNECTION_ID frame
-6031 /// with updated `retire_prior_to` field set to `v`
-6032 #[cfg(test)]
-6033 pub(crate) fn rotate_local_cid(&mut self, v: u64, now: Instant) {
-6034 let n = self
-6035 .local_cid_state
-6036 .get_mut(&PathId::ZERO)
-6037 .unwrap()
-6038 .assign_retire_seq(v);
-6039 self.endpoint_events
-6040 .push_back(EndpointEventInner::NeedIdentifiers(PathId::ZERO, now, n));
-6041 }
-6042
-6043 /// Check the current active remote CID sequence for `PathId::ZERO`
-6044 #[cfg(test)]
-6045 pub(crate) fn active_rem_cid_seq(&self) -> u64 {
-6046 self.rem_cids.get(&PathId::ZERO).unwrap().active_seq()
-6047 }
-6048
-6049 /// Returns the detected maximum udp payload size for the current path
-6050 #[cfg(test)]
-6051 pub(crate) fn path_mtu(&self, path_id: PathId) -> u16 {
-6052 self.path_data(path_id).current_mtu()
-6053 }
-6054
-6055 /// Triggers path validation on all paths
-6056 #[cfg(test)]
-6057 pub(crate) fn trigger_path_validation(&mut self) {
-6058 for path in self.paths.values_mut() {
-6059 path.data.send_new_challenge = true;
-6060 }
-6061 }
-6062
-6063 /// Whether we have 1-RTT data to send
-6064 ///
-6065 /// This checks for frames that can only be sent in the data space (1-RTT):
-6066 /// - Pending PATH_CHALLENGE frames on the active and previous path if just migrated.
-6067 /// - Pending PATH_RESPONSE frames.
-6068 /// - Pending data to send in STREAM frames.
-6069 /// - Pending DATAGRAM frames to send.
-6070 ///
-6071 /// See also [`PacketSpace::can_send`] which keeps track of all other frame types that
-6072 /// may need to be sent.
-6073 fn can_send_1rtt(&self, path_id: PathId, max_size: usize) -> SendableFrames {
-6074 let path_exclusive = self.paths.get(&path_id).is_some_and(|path| {
-6075 path.data.send_new_challenge
-6076 || path
-6077 .prev
-6078 .as_ref()
-6079 .is_some_and(|(_, path)| path.send_new_challenge)
-6080 || !path.data.path_responses.is_empty()
-6081 });
-6082 let other = self.streams.can_send_stream_data()
-6083 || self
-6084 .datagrams
-6085 .outgoing
-6086 .front()
-6087 .is_some_and(|x| x.size(true) <= max_size);
-6088 SendableFrames {
-6089 acks: false,
-6090 other,
-6091 close: false,
-6092 path_exclusive,
-6093 }
-6094 }
-6095
-6096 /// Terminate the connection instantly, without sending a close packet
-6097 fn kill(&mut self, reason: ConnectionError) {
-6098 self.close_common();
-6099 self.state.move_to_drained(Some(reason));
-6100 self.endpoint_events.push_back(EndpointEventInner::Drained);
-6101 }
-6102
-6103 /// Storage size required for the largest packet that can be transmitted on all currently
-6104 /// available paths
-6105 ///
-6106 /// Buffers passed to [`Connection::poll_transmit`] should be at least this large.
-6107 ///
-6108 /// When multipath is enabled, this value is the minimum MTU across all available paths.
-6109 pub fn current_mtu(&self) -> u16 {
-6110 self.paths
-6111 .iter()
-6112 .filter(|&(path_id, _path_state)| !self.abandoned_paths.contains(path_id))
-6113 .map(|(_path_id, path_state)| path_state.data.current_mtu())
-6114 .min()
-6115 .expect("There is always at least one available path")
-6116 }
-6117
-6118 /// Size of non-frame data for a 1-RTT packet
-6119 ///
-6120 /// Quantifies space consumed by the QUIC header and AEAD tag. All other bytes in a packet are
-6121 /// frames. Changes if the length of the remote connection ID changes, which is expected to be
-6122 /// rare. If `pn` is specified, may additionally change unpredictably due to variations in
-6123 /// latency and packet loss.
-6124 fn predict_1rtt_overhead(&mut self, pn: u64, path: PathId) -> usize {
-6125 let pn_len = PacketNumber::new(
-6126 pn,
-6127 self.spaces[SpaceId::Data]
-6128 .for_path(path)
-6129 .largest_acked_packet
-6130 .unwrap_or(0),
-6131 )
-6132 .len();
-6133
-6134 // 1 byte for flags
-6135 1 + self
-6136 .rem_cids
-6137 .get(&path)
-6138 .map(|cids| cids.active().len())
-6139 .unwrap_or(20) // Max CID len in QUIC v1
-6140 + pn_len
-6141 + self.tag_len_1rtt()
-6142 }
-6143
-6144 fn predict_1rtt_overhead_no_pn(&self) -> usize {
-6145 let pn_len = 4;
-6146
-6147 let cid_len = self
-6148 .rem_cids
-6149 .values()
-6150 .map(|cids| cids.active().len())
-6151 .max()
-6152 .unwrap_or(20); // Max CID len in QUIC v1
-6153
-6154 // 1 byte for flags
-6155 1 + cid_len + pn_len + self.tag_len_1rtt()
+5982 let result = match result {
+5983 Some(r) => r,
+5984 None => return Ok(None),
+5985 };
+5986
+5987 if result.outgoing_key_update_acked {
+5988 if let Some(prev) = self.prev_crypto.as_mut() {
+5989 prev.end_packet = Some((result.number, now));
+5990 self.set_key_discard_timer(now, packet.header.space());
+5991 }
+5992 }
+5993
+5994 if result.incoming_key_update {
+5995 trace!("key update authenticated");
+5996 self.update_keys(Some((result.number, now)), true);
+5997 self.set_key_discard_timer(now, packet.header.space());
+5998 }
+5999
+6000 Ok(Some(result.number))
+6001 }
+6002
+6003 fn update_keys(&mut self, end_packet: Option<(u64, Instant)>, remote: bool) {
+6004 trace!("executing key update");
+6005 // Generate keys for the key phase after the one we're switching to, store them in
+6006 // `next_crypto`, make the contents of `next_crypto` current, and move the current keys into
+6007 // `prev_crypto`.
+6008 let new = self
+6009 .crypto
+6010 .next_1rtt_keys()
+6011 .expect("only called for `Data` packets");
+6012 self.key_phase_size = new
+6013 .local
+6014 .confidentiality_limit()
+6015 .saturating_sub(KEY_UPDATE_MARGIN);
+6016 let old = mem::replace(
+6017 &mut self.spaces[SpaceId::Data]
+6018 .crypto
+6019 .as_mut()
+6020 .unwrap() // safe because update_keys() can only be triggered by short packets
+6021 .packet,
+6022 mem::replace(self.next_crypto.as_mut().unwrap(), new),
+6023 );
+6024 self.spaces[SpaceId::Data]
+6025 .iter_paths_mut()
+6026 .for_each(|s| s.sent_with_keys = 0);
+6027 self.prev_crypto = Some(PrevCrypto {
+6028 crypto: old,
+6029 end_packet,
+6030 update_unacked: remote,
+6031 });
+6032 self.key_phase = !self.key_phase;
+6033 }
+6034
+6035 fn peer_supports_ack_frequency(&self) -> bool {
+6036 self.peer_params.min_ack_delay.is_some()
+6037 }
+6038
+6039 /// Send an IMMEDIATE_ACK frame to the remote endpoint
+6040 ///
+6041 /// According to the spec, this will result in an error if the remote endpoint does not support
+6042 /// the Acknowledgement Frequency extension
+6043 pub(crate) fn immediate_ack(&mut self, path_id: PathId) {
+6044 debug_assert_eq!(
+6045 self.highest_space,
+6046 SpaceId::Data,
+6047 "immediate ack must be written in the data space"
+6048 );
+6049 self.spaces[self.highest_space]
+6050 .for_path(path_id)
+6051 .immediate_ack_pending = true;
+6052 }
+6053
+6054 /// Decodes a packet, returning its decrypted payload, so it can be inspected in tests
+6055 #[cfg(test)]
+6056 pub(crate) fn decode_packet(&self, event: &ConnectionEvent) -> Option<Vec<u8>> {
+6057 let (path_id, first_decode, remaining) = match &event.0 {
+6058 ConnectionEventInner::Datagram(DatagramConnectionEvent {
+6059 path_id,
+6060 first_decode,
+6061 remaining,
+6062 ..
+6063 }) => (path_id, first_decode, remaining),
+6064 _ => return None,
+6065 };
+6066
+6067 if remaining.is_some() {
+6068 panic!("Packets should never be coalesced in tests");
+6069 }
+6070
+6071 let decrypted_header = packet_crypto::unprotect_header(
+6072 first_decode.clone(),
+6073 &self.spaces,
+6074 self.zero_rtt_crypto.as_ref(),
+6075 self.peer_params.stateless_reset_token,
+6076 )?;
+6077
+6078 let mut packet = decrypted_header.packet?;
+6079 packet_crypto::decrypt_packet_body(
+6080 &mut packet,
+6081 *path_id,
+6082 &self.spaces,
+6083 self.zero_rtt_crypto.as_ref(),
+6084 self.key_phase,
+6085 self.prev_crypto.as_ref(),
+6086 self.next_crypto.as_ref(),
+6087 )
+6088 .ok()?;
+6089
+6090 Some(packet.payload.to_vec())
+6091 }
+6092
+6093 /// The number of bytes of packets containing retransmittable frames that have not been
+6094 /// acknowledged or declared lost.
+6095 #[cfg(test)]
+6096 pub(crate) fn bytes_in_flight(&self) -> u64 {
+6097 // TODO(@divma): consider including for multipath?
+6098 self.path_data(PathId::ZERO).in_flight.bytes
+6099 }
+6100
+6101 /// Number of bytes worth of non-ack-only packets that may be sent
+6102 #[cfg(test)]
+6103 pub(crate) fn congestion_window(&self) -> u64 {
+6104 let path = self.path_data(PathId::ZERO);
+6105 path.congestion
+6106 .window()
+6107 .saturating_sub(path.in_flight.bytes)
+6108 }
+6109
+6110 /// Whether no timers but keepalive, idle, rtt, pushnewcid, and key discard are running
+6111 #[cfg(test)]
+6112 pub(crate) fn is_idle(&self) -> bool {
+6113 let current_timers = self.timers.values();
+6114 current_timers
+6115 .into_iter()
+6116 .filter(|(timer, _)| {
+6117 !matches!(
+6118 timer,
+6119 Timer::Conn(ConnTimer::KeepAlive)
+6120 | Timer::PerPath(_, PathTimer::PathKeepAlive)
+6121 | Timer::Conn(ConnTimer::PushNewCid)
+6122 | Timer::Conn(ConnTimer::KeyDiscard)
+6123 )
+6124 })
+6125 .min_by_key(|(_, time)| *time)
+6126 .is_none_or(|(timer, _)| timer == Timer::Conn(ConnTimer::Idle))
+6127 }
+6128
+6129 /// Whether explicit congestion notification is in use on outgoing packets.
+6130 #[cfg(test)]
+6131 pub(crate) fn using_ecn(&self) -> bool {
+6132 self.path_data(PathId::ZERO).sending_ecn
+6133 }
+6134
+6135 /// The number of received bytes in the current path
+6136 #[cfg(test)]
+6137 pub(crate) fn total_recvd(&self) -> u64 {
+6138 self.path_data(PathId::ZERO).total_recvd
+6139 }
+6140
+6141 #[cfg(test)]
+6142 pub(crate) fn active_local_cid_seq(&self) -> (u64, u64) {
+6143 self.local_cid_state
+6144 .get(&PathId::ZERO)
+6145 .unwrap()
+6146 .active_seq()
+6147 }
+6148
+6149 #[cfg(test)]
+6150 #[track_caller]
+6151 pub(crate) fn active_local_path_cid_seq(&self, path_id: u32) -> (u64, u64) {
+6152 self.local_cid_state
+6153 .get(&PathId(path_id))
+6154 .unwrap()
+6155 .active_seq()
6156 }
6157
-6158 fn tag_len_1rtt(&self) -> usize {
-6159 let key = match self.spaces[SpaceId::Data].crypto.as_ref() {
-6160 Some(crypto) => Some(&*crypto.packet.local),
-6161 None => self.zero_rtt_crypto.as_ref().map(|x| &*x.packet),
-6162 };
-6163 // If neither Data nor 0-RTT keys are available, make a reasonable tag length guess. As of
-6164 // this writing, all QUIC cipher suites use 16-byte tags. We could return `None` instead,
-6165 // but that would needlessly prevent sending datagrams during 0-RTT.
-6166 key.map_or(16, |x| x.tag_len())
-6167 }
-6168
-6169 /// Mark the path as validated, and enqueue NEW_TOKEN frames to be sent as appropriate
-6170 fn on_path_validated(&mut self, path_id: PathId) {
-6171 self.path_data_mut(path_id).validated = true;
-6172 let ConnectionSide::Server { server_config } = &self.side else {
-6173 return;
-6174 };
-6175 let network_path = self.path_data(path_id).network_path;
-6176 let new_tokens = &mut self.spaces[SpaceId::Data as usize].pending.new_tokens;
-6177 new_tokens.clear();
-6178 for _ in 0..server_config.validation_token.sent {
-6179 new_tokens.push(network_path);
-6180 }
+6158 /// Instruct the peer to replace previously issued CIDs by sending a NEW_CONNECTION_ID frame
+6159 /// with updated `retire_prior_to` field set to `v`
+6160 #[cfg(test)]
+6161 pub(crate) fn rotate_local_cid(&mut self, v: u64, now: Instant) {
+6162 let n = self
+6163 .local_cid_state
+6164 .get_mut(&PathId::ZERO)
+6165 .unwrap()
+6166 .assign_retire_seq(v);
+6167 self.endpoint_events
+6168 .push_back(EndpointEventInner::NeedIdentifiers(PathId::ZERO, now, n));
+6169 }
+6170
+6171 /// Check the current active remote CID sequence for `PathId::ZERO`
+6172 #[cfg(test)]
+6173 pub(crate) fn active_rem_cid_seq(&self) -> u64 {
+6174 self.rem_cids.get(&PathId::ZERO).unwrap().active_seq()
+6175 }
+6176
+6177 /// Returns the detected maximum udp payload size for the current path
+6178 #[cfg(test)]
+6179 pub(crate) fn path_mtu(&self, path_id: PathId) -> u16 {
+6180 self.path_data(path_id).current_mtu()
6181 }
6182
-6183 /// Handle new path status information: PATH_STATUS_AVAILABLE, PATH_STATUS_BACKUP
-6184 fn on_path_status(&mut self, path_id: PathId, status: PathStatus, status_seq_no: VarInt) {
-6185 if let Some(path) = self.paths.get_mut(&path_id) {
-6186 path.data.status.remote_update(status, status_seq_no);
-6187 } else {
-6188 debug!("PATH_STATUS_AVAILABLE received unknown path {:?}", path_id);
-6189 }
-6190 self.events.push_back(
-6191 PathEvent::RemoteStatus {
-6192 id: path_id,
-6193 status,
-6194 }
-6195 .into(),
-6196 );
-6197 }
-6198
-6199 /// Returns the maximum [`PathId`] to be used for sending in this connection.
-6200 ///
-6201 /// This is calculated as minimum between the local and remote's maximums when multipath is
-6202 /// enabled, or `None` when disabled.
-6203 ///
-6204 /// For data that's received, we should use [`Self::local_max_path_id`] instead.
-6205 /// The reasoning is that the remote might already have updated to its own newer
-6206 /// [`Self::max_path_id`] after sending out a `MAX_PATH_ID` frame, but it got re-ordered.
-6207 fn max_path_id(&self) -> Option<PathId> {
-6208 if self.is_multipath_negotiated() {
-6209 Some(self.remote_max_path_id.min(self.local_max_path_id))
-6210 } else {
-6211 None
-6212 }
-6213 }
-6214
-6215 /// Add addresses the local endpoint considers are reachable for nat traversal
-6216 pub fn add_nat_traversal_address(&mut self, address: SocketAddr) -> Result<(), iroh_hp::Error> {
-6217 if let Some(added) = self.iroh_hp.add_local_address(address)? {
-6218 self.spaces[SpaceId::Data].pending.add_address.insert(added);
-6219 };
-6220 Ok(())
-6221 }
-6222
-6223 /// Removes an address the endpoing no longer considers reachable for nat traversal
-6224 ///
-6225 /// Addresses not present in the set will be silently ignored.
-6226 pub fn remove_nat_traversal_address(
-6227 &mut self,
-6228 address: SocketAddr,
-6229 ) -> Result<(), iroh_hp::Error> {
-6230 if let Some(removed) = self.iroh_hp.remove_local_address(address)? {
-6231 self.spaces[SpaceId::Data]
-6232 .pending
-6233 .remove_address
-6234 .insert(removed);
-6235 }
-6236 Ok(())
-6237 }
-6238
-6239 /// Get the current local nat traversal addresses
-6240 pub fn get_local_nat_traversal_addresses(&self) -> Result<Vec<SocketAddr>, iroh_hp::Error> {
-6241 self.iroh_hp.get_local_nat_traversal_addresses()
-6242 }
-6243
-6244 /// Get the currently advertised nat traversal addresses by the server
-6245 pub fn get_remote_nat_traversal_addresses(&self) -> Result<Vec<SocketAddr>, iroh_hp::Error> {
-6246 Ok(self
-6247 .iroh_hp
-6248 .client_side()?
-6249 .get_remote_nat_traversal_addresses())
-6250 }
-6251
-6252 /// Attempts to open a path for nat traversal.
-6253 ///
-6254 /// `ipv6` indicates if the path should be opened using an IPV6 remote. If the address is
-6255 /// ignored, it will return `None`.
-6256 ///
-6257 /// On success returns the [`PathId`] and remote address of the path, as well as whether the path
-6258 /// existed for the adjusted remote.
-6259 fn open_nat_traversal_path(
-6260 &mut self,
-6261 now: Instant,
-6262 (ip, port): (IpAddr, u16),
-6263 ipv6: bool,
-6264 ) -> Result<Option<(PathId, SocketAddr, bool)>, PathError> {
-6265 // If this endpoint is an IPv6 endpoint we use IPv6 addresses for all remotes.
-6266 let remote = match ip {
-6267 IpAddr::V4(addr) if ipv6 => SocketAddr::new(addr.to_ipv6_mapped().into(), port),
-6268 IpAddr::V4(addr) => SocketAddr::new(addr.into(), port),
-6269 IpAddr::V6(_) if ipv6 => SocketAddr::new(ip, port),
-6270 IpAddr::V6(_) => {
-6271 trace!("not using IPv6 nat candidate for IPv4 socket");
-6272 return Ok(None);
-6273 }
-6274 };
-6275 // TODO(matheus23): Probe the correct 4-tuple, instead of only a remote address?
-6276 // By specifying None, we do two things: 1. open_path_ensure won't generate two
-6277 // paths to the same remote and 2. we let the OS choose which interface to use for
-6278 // sending on that path.
-6279 let network_path = FourTuple {
-6280 remote,
-6281 local_ip: None,
-6282 };
-6283 match self.open_path_ensure(network_path, PathStatus::Backup, now) {
-6284 Ok((path_id, path_was_known)) => {
-6285 if path_was_known {
-6286 trace!(%path_id, %remote, "nat traversal: path existed for remote");
-6287 }
-6288 Ok(Some((path_id, remote, path_was_known)))
-6289 }
-6290 Err(e) => {
-6291 debug!(%remote, %e, "nat traversal: failed to probe remote");
-6292 Err(e)
-6293 }
-6294 }
+6183 /// Triggers path validation on all paths
+6184 #[cfg(test)]
+6185 pub(crate) fn trigger_path_validation(&mut self) {
+6186 for path in self.paths.values_mut() {
+6187 path.data.send_new_challenge = true;
+6188 }
+6189 }
+6190
+6191 /// Whether we have 1-RTT data to send
+6192 ///
+6193 /// This checks for frames that can only be sent in the data space (1-RTT):
+6194 /// - Pending PATH_CHALLENGE frames on the active and previous path if just migrated.
+6195 /// - Pending PATH_RESPONSE frames.
+6196 /// - Pending data to send in STREAM frames.
+6197 /// - Pending DATAGRAM frames to send.
+6198 ///
+6199 /// See also [`PacketSpace::can_send`] which keeps track of all other frame types that
+6200 /// may need to be sent.
+6201 fn can_send_1rtt(&self, path_id: PathId, max_size: usize) -> SendableFrames {
+6202 let path_exclusive = self.paths.get(&path_id).is_some_and(|path| {
+6203 path.data.send_new_challenge
+6204 || path
+6205 .prev
+6206 .as_ref()
+6207 .is_some_and(|(_, path)| path.send_new_challenge)
+6208 || !path.data.path_responses.is_empty()
+6209 });
+6210 let other = self.streams.can_send_stream_data()
+6211 || self
+6212 .datagrams
+6213 .outgoing
+6214 .front()
+6215 .is_some_and(|x| x.size(true) <= max_size);
+6216 SendableFrames {
+6217 acks: false,
+6218 other,
+6219 close: false,
+6220 path_exclusive,
+6221 }
+6222 }
+6223
+6224 /// Terminate the connection instantly, without sending a close packet
+6225 fn kill(&mut self, reason: ConnectionError) {
+6226 self.close_common();
+6227 self.state.move_to_drained(Some(reason));
+6228 self.endpoint_events.push_back(EndpointEventInner::Drained);
+6229 }
+6230
+6231 /// Storage size required for the largest packet that can be transmitted on all currently
+6232 /// available paths
+6233 ///
+6234 /// Buffers passed to [`Connection::poll_transmit`] should be at least this large.
+6235 ///
+6236 /// When multipath is enabled, this value is the minimum MTU across all available paths.
+6237 pub fn current_mtu(&self) -> u16 {
+6238 self.paths
+6239 .iter()
+6240 .filter(|&(path_id, _path_state)| !self.abandoned_paths.contains(path_id))
+6241 .map(|(_path_id, path_state)| path_state.data.current_mtu())
+6242 .min()
+6243 .expect("There is always at least one available path")
+6244 }
+6245
+6246 /// Size of non-frame data for a 1-RTT packet
+6247 ///
+6248 /// Quantifies space consumed by the QUIC header and AEAD tag. All other bytes in a packet are
+6249 /// frames. Changes if the length of the remote connection ID changes, which is expected to be
+6250 /// rare. If `pn` is specified, may additionally change unpredictably due to variations in
+6251 /// latency and packet loss.
+6252 fn predict_1rtt_overhead(&mut self, pn: u64, path: PathId) -> usize {
+6253 let pn_len = PacketNumber::new(
+6254 pn,
+6255 self.spaces[SpaceId::Data]
+6256 .for_path(path)
+6257 .largest_acked_packet
+6258 .unwrap_or(0),
+6259 )
+6260 .len();
+6261
+6262 // 1 byte for flags
+6263 1 + self
+6264 .rem_cids
+6265 .get(&path)
+6266 .map(|cids| cids.active().len())
+6267 .unwrap_or(20) // Max CID len in QUIC v1
+6268 + pn_len
+6269 + self.tag_len_1rtt()
+6270 }
+6271
+6272 fn predict_1rtt_overhead_no_pn(&self) -> usize {
+6273 let pn_len = 4;
+6274
+6275 let cid_len = self
+6276 .rem_cids
+6277 .values()
+6278 .map(|cids| cids.active().len())
+6279 .max()
+6280 .unwrap_or(20); // Max CID len in QUIC v1
+6281
+6282 // 1 byte for flags
+6283 1 + cid_len + pn_len + self.tag_len_1rtt()
+6284 }
+6285
+6286 fn tag_len_1rtt(&self) -> usize {
+6287 let key = match self.spaces[SpaceId::Data].crypto.as_ref() {
+6288 Some(crypto) => Some(&*crypto.packet.local),
+6289 None => self.zero_rtt_crypto.as_ref().map(|x| &*x.packet),
+6290 };
+6291 // If neither Data nor 0-RTT keys are available, make a reasonable tag length guess. As of
+6292 // this writing, all QUIC cipher suites use 16-byte tags. We could return `None` instead,
+6293 // but that would needlessly prevent sending datagrams during 0-RTT.
+6294 key.map_or(16, |x| x.tag_len())
6295 }
6296
-6297 /// Initiates a new nat traversal round
-6298 ///
-6299 /// A nat traversal round involves advertising the client's local addresses in `REACH_OUT`
-6300 /// frames, and initiating probing of the known remote addresses. When a new round is
-6301 /// initiated, the previous one is cancelled, and paths that have not been opened are closed.
-6302 ///
-6303 /// Returns the server addresses that are now being probed.
-6304 /// If addresses fail due to spurious errors, these might succeed later and not be returned in
-6305 /// this set.
-6306 pub fn initiate_nat_traversal_round(
-6307 &mut self,
-6308 now: Instant,
-6309 ) -> Result<Vec<SocketAddr>, iroh_hp::Error> {
-6310 if self.state.is_closed() {
-6311 return Err(iroh_hp::Error::Closed);
-6312 }
-6313
-6314 let client_state = self.iroh_hp.client_side_mut()?;
-6315 let iroh_hp::NatTraversalRound {
-6316 new_round,
-6317 reach_out_at,
-6318 addresses_to_probe,
-6319 prev_round_path_ids,
-6320 } = client_state.initiate_nat_traversal_round()?;
-6321
-6322 self.spaces[SpaceId::Data].pending.reach_out = Some((new_round, reach_out_at));
-6323
-6324 for path_id in prev_round_path_ids {
-6325 // TODO(@divma): this sounds reasonable but we need if this actually works for the
-6326 // purposes of the protocol
-6327 let validated = self
-6328 .path(path_id)
-6329 .map(|path| path.validated)
-6330 .unwrap_or(false);
-6331
-6332 if !validated {
-6333 let _ = self.close_path(
-6334 now,
-6335 path_id,
-6336 TransportErrorCode::APPLICATION_ABANDON_PATH.into(),
-6337 );
-6338 }
-6339 }
-6340
-6341 let mut err = None;
+6297 /// Mark the path as validated, and enqueue NEW_TOKEN frames to be sent as appropriate
+6298 fn on_path_validated(&mut self, path_id: PathId) {
+6299 self.path_data_mut(path_id).validated = true;
+6300 let ConnectionSide::Server { server_config } = &self.side else {
+6301 return;
+6302 };
+6303 let network_path = self.path_data(path_id).network_path;
+6304 let new_tokens = &mut self.spaces[SpaceId::Data as usize].pending.new_tokens;
+6305 new_tokens.clear();
+6306 for _ in 0..server_config.validation_token.sent {
+6307 new_tokens.push(network_path);
+6308 }
+6309 }
+6310
+6311 /// Handle new path status information: PATH_STATUS_AVAILABLE, PATH_STATUS_BACKUP
+6312 fn on_path_status(&mut self, path_id: PathId, status: PathStatus, status_seq_no: VarInt) {
+6313 if let Some(path) = self.paths.get_mut(&path_id) {
+6314 path.data.status.remote_update(status, status_seq_no);
+6315 } else {
+6316 debug!("PATH_STATUS_AVAILABLE received unknown path {:?}", path_id);
+6317 }
+6318 self.events.push_back(
+6319 PathEvent::RemoteStatus {
+6320 id: path_id,
+6321 status,
+6322 }
+6323 .into(),
+6324 );
+6325 }
+6326
+6327 /// Returns the maximum [`PathId`] to be used for sending in this connection.
+6328 ///
+6329 /// This is calculated as minimum between the local and remote's maximums when multipath is
+6330 /// enabled, or `None` when disabled.
+6331 ///
+6332 /// For data that's received, we should use [`Self::local_max_path_id`] instead.
+6333 /// The reasoning is that the remote might already have updated to its own newer
+6334 /// [`Self::max_path_id`] after sending out a `MAX_PATH_ID` frame, but it got re-ordered.
+6335 fn max_path_id(&self) -> Option<PathId> {
+6336 if self.is_multipath_negotiated() {
+6337 Some(self.remote_max_path_id.min(self.local_max_path_id))
+6338 } else {
+6339 None
+6340 }
+6341 }
6342
-6343 let mut path_ids = Vec::with_capacity(addresses_to_probe.len());
-6344 let mut probed_addresses = Vec::with_capacity(addresses_to_probe.len());
-6345 let ipv6 = self
-6346 .paths
-6347 .values()
-6348 .any(|p| p.data.network_path.remote.is_ipv6());
-6349
-6350 for (id, address) in addresses_to_probe {
-6351 match self.open_nat_traversal_path(now, address, ipv6) {
-6352 Ok(None) => {}
-6353 Ok(Some((path_id, remote, path_was_known))) => {
-6354 if !path_was_known {
-6355 path_ids.push(path_id);
-6356 probed_addresses.push(remote);
-6357 }
-6358 }
-6359 Err(e) => {
-6360 self.iroh_hp
-6361 .client_side_mut()
-6362 .expect("validated")
-6363 .report_in_continuation(id, e);
-6364 err.get_or_insert(e);
-6365 }
-6366 }
-6367 }
-6368
-6369 if let Some(err) = err {
-6370 // We failed to probe any addresses, bail out
-6371 if probed_addresses.is_empty() {
-6372 return Err(iroh_hp::Error::Multipath(err));
-6373 }
-6374 }
-6375
-6376 self.iroh_hp
-6377 .client_side_mut()
-6378 .expect("connection side validated")
-6379 .set_round_path_ids(path_ids);
-6380
-6381 Ok(probed_addresses)
-6382 }
-6383
-6384 /// Attempts to continue a nat traversal round by trying to open paths for pending client probes.
-6385 ///
-6386 /// If there was nothing to do, it returns `None`. Otherwise it returns whether the path was
-6387 /// successfully open.
-6388 fn continue_nat_traversal_round(&mut self, now: Instant) -> Option<bool> {
-6389 let client_state = self.iroh_hp.client_side_mut().ok()?;
-6390 let (id, address) = client_state.continue_nat_traversal_round()?;
-6391 let ipv6 = self
-6392 .paths
-6393 .values()
-6394 .any(|p| p.data.network_path.remote.is_ipv6());
-6395 let open_result = self.open_nat_traversal_path(now, address, ipv6);
-6396 let client_state = self.iroh_hp.client_side_mut().expect("validated");
-6397 match open_result {
-6398 Ok(None) => Some(true),
-6399 Ok(Some((path_id, _remote, path_was_known))) => {
-6400 if !path_was_known {
-6401 client_state.add_round_path_id(path_id);
-6402 }
-6403 Some(true)
-6404 }
-6405 Err(e) => {
-6406 client_state.report_in_continuation(id, e);
-6407 Some(false)
-6408 }
-6409 }
-6410 }
-6411}
-6412
-6413impl fmt::Debug for Connection {
-6414 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
-6415 f.debug_struct("Connection")
-6416 .field("handshake_cid", &self.handshake_cid)
-6417 .finish()
-6418 }
-6419}
-6420
-6421#[derive(Debug, Copy, Clone, PartialEq, Eq)]
-6422enum PathBlocked {
-6423 No,
-6424 AntiAmplification,
-6425 Congestion,
-6426 Pacing,
-6427}
-6428
-6429/// Fields of `Connection` specific to it being client-side or server-side
-6430enum ConnectionSide {
-6431 Client {
-6432 /// Sent in every outgoing Initial packet. Always empty after Initial keys are discarded
-6433 token: Bytes,
-6434 token_store: Arc<dyn TokenStore>,
-6435 server_name: String,
-6436 },
-6437 Server {
-6438 server_config: Arc<ServerConfig>,
-6439 },
-6440}
+6343 /// Add addresses the local endpoint considers are reachable for nat traversal
+6344 pub fn add_nat_traversal_address(&mut self, address: SocketAddr) -> Result<(), iroh_hp::Error> {
+6345 if let Some(added) = self.iroh_hp.add_local_address(address)? {
+6346 self.spaces[SpaceId::Data].pending.add_address.insert(added);
+6347 };
+6348 Ok(())
+6349 }
+6350
+6351 /// Removes an address the endpoing no longer considers reachable for nat traversal
+6352 ///
+6353 /// Addresses not present in the set will be silently ignored.
+6354 pub fn remove_nat_traversal_address(
+6355 &mut self,
+6356 address: SocketAddr,
+6357 ) -> Result<(), iroh_hp::Error> {
+6358 if let Some(removed) = self.iroh_hp.remove_local_address(address)? {
+6359 self.spaces[SpaceId::Data]
+6360 .pending
+6361 .remove_address
+6362 .insert(removed);
+6363 }
+6364 Ok(())
+6365 }
+6366
+6367 /// Get the current local nat traversal addresses
+6368 pub fn get_local_nat_traversal_addresses(&self) -> Result<Vec<SocketAddr>, iroh_hp::Error> {
+6369 self.iroh_hp.get_local_nat_traversal_addresses()
+6370 }
+6371
+6372 /// Get the currently advertised nat traversal addresses by the server
+6373 pub fn get_remote_nat_traversal_addresses(&self) -> Result<Vec<SocketAddr>, iroh_hp::Error> {
+6374 Ok(self
+6375 .iroh_hp
+6376 .client_side()?
+6377 .get_remote_nat_traversal_addresses())
+6378 }
+6379
+6380 /// Attempts to open a path for nat traversal.
+6381 ///
+6382 /// `ipv6` indicates if the path should be opened using an IPV6 remote. If the address is
+6383 /// ignored, it will return `None`.
+6384 ///
+6385 /// On success returns the [`PathId`] and remote address of the path, as well as whether the path
+6386 /// existed for the adjusted remote.
+6387 fn open_nat_traversal_path(
+6388 &mut self,
+6389 now: Instant,
+6390 (ip, port): (IpAddr, u16),
+6391 ipv6: bool,
+6392 ) -> Result<Option<(PathId, SocketAddr, bool)>, PathError> {
+6393 // If this endpoint is an IPv6 endpoint we use IPv6 addresses for all remotes.
+6394 let remote = match ip {
+6395 IpAddr::V4(addr) if ipv6 => SocketAddr::new(addr.to_ipv6_mapped().into(), port),
+6396 IpAddr::V4(addr) => SocketAddr::new(addr.into(), port),
+6397 IpAddr::V6(_) if ipv6 => SocketAddr::new(ip, port),
+6398 IpAddr::V6(_) => {
+6399 trace!("not using IPv6 nat candidate for IPv4 socket");
+6400 return Ok(None);
+6401 }
+6402 };
+6403 // TODO(matheus23): Probe the correct 4-tuple, instead of only a remote address?
+6404 // By specifying None, we do two things: 1. open_path_ensure won't generate two
+6405 // paths to the same remote and 2. we let the OS choose which interface to use for
+6406 // sending on that path.
+6407 let network_path = FourTuple {
+6408 remote,
+6409 local_ip: None,
+6410 };
+6411 match self.open_path_ensure(network_path, PathStatus::Backup, now) {
+6412 Ok((path_id, path_was_known)) => {
+6413 if path_was_known {
+6414 trace!(%path_id, %remote, "nat traversal: path existed for remote");
+6415 }
+6416 Ok(Some((path_id, remote, path_was_known)))
+6417 }
+6418 Err(e) => {
+6419 debug!(%remote, %e, "nat traversal: failed to probe remote");
+6420 Err(e)
+6421 }
+6422 }
+6423 }
+6424
+6425 /// Initiates a new nat traversal round
+6426 ///
+6427 /// A nat traversal round involves advertising the client's local addresses in `REACH_OUT`
+6428 /// frames, and initiating probing of the known remote addresses. When a new round is
+6429 /// initiated, the previous one is cancelled, and paths that have not been opened are closed.
+6430 ///
+6431 /// Returns the server addresses that are now being probed.
+6432 /// If addresses fail due to spurious errors, these might succeed later and not be returned in
+6433 /// this set.
+6434 pub fn initiate_nat_traversal_round(
+6435 &mut self,
+6436 now: Instant,
+6437 ) -> Result<Vec<SocketAddr>, iroh_hp::Error> {
+6438 if self.state.is_closed() {
+6439 return Err(iroh_hp::Error::Closed);
+6440 }
6441
-6442impl ConnectionSide {
-6443 fn remote_may_migrate(&self, state: &State) -> bool {
-6444 match self {
-6445 Self::Server { server_config } => server_config.migration,
-6446 Self::Client { .. } => {
-6447 if let Some(hs) = state.as_handshake() {
-6448 hs.allow_server_migration
-6449 } else {
-6450 false
-6451 }
-6452 }
-6453 }
-6454 }
-6455
-6456 fn is_client(&self) -> bool {
-6457 self.side().is_client()
-6458 }
+6442 let client_state = self.iroh_hp.client_side_mut()?;
+6443 let iroh_hp::NatTraversalRound {
+6444 new_round,
+6445 reach_out_at,
+6446 addresses_to_probe,
+6447 prev_round_path_ids,
+6448 } = client_state.initiate_nat_traversal_round()?;
+6449
+6450 self.spaces[SpaceId::Data].pending.reach_out = Some((new_round, reach_out_at));
+6451
+6452 for path_id in prev_round_path_ids {
+6453 // TODO(@divma): this sounds reasonable but we need if this actually works for the
+6454 // purposes of the protocol
+6455 let validated = self
+6456 .path(path_id)
+6457 .map(|path| path.validated)
+6458 .unwrap_or(false);
6459
-6460 fn is_server(&self) -> bool {
-6461 self.side().is_server()
-6462 }
-6463
-6464 fn side(&self) -> Side {
-6465 match *self {
-6466 Self::Client { .. } => Side::Client,
-6467 Self::Server { .. } => Side::Server,
-6468 }
-6469 }
-6470}
-6471
-6472impl From<SideArgs> for ConnectionSide {
-6473 fn from(side: SideArgs) -> Self {
-6474 match side {
-6475 SideArgs::Client {
-6476 token_store,
-6477 server_name,
-6478 } => Self::Client {
-6479 token: token_store.take(&server_name).unwrap_or_default(),
-6480 token_store,
-6481 server_name,
-6482 },
-6483 SideArgs::Server {
-6484 server_config,
-6485 pref_addr_cid: _,
-6486 path_validated: _,
-6487 } => Self::Server { server_config },
-6488 }
-6489 }
-6490}
-6491
-6492/// Parameters to `Connection::new` specific to it being client-side or server-side
-6493pub(crate) enum SideArgs {
-6494 Client {
-6495 token_store: Arc<dyn TokenStore>,
-6496 server_name: String,
-6497 },
-6498 Server {
-6499 server_config: Arc<ServerConfig>,
-6500 pref_addr_cid: Option<ConnectionId>,
-6501 path_validated: bool,
-6502 },
-6503}
-6504
-6505impl SideArgs {
-6506 pub(crate) fn pref_addr_cid(&self) -> Option<ConnectionId> {
-6507 match *self {
-6508 Self::Client { .. } => None,
-6509 Self::Server { pref_addr_cid, .. } => pref_addr_cid,
-6510 }
-6511 }
-6512
-6513 pub(crate) fn path_validated(&self) -> bool {
-6514 match *self {
-6515 Self::Client { .. } => true,
-6516 Self::Server { path_validated, .. } => path_validated,
-6517 }
-6518 }
-6519
-6520 pub(crate) fn side(&self) -> Side {
-6521 match *self {
-6522 Self::Client { .. } => Side::Client,
-6523 Self::Server { .. } => Side::Server,
-6524 }
-6525 }
-6526}
-6527
-6528/// Reasons why a connection might be lost
-6529#[derive(Debug, Error, Clone, PartialEq, Eq)]
-6530pub enum ConnectionError {
-6531 /// The peer doesn't implement any supported version
-6532 #[error("peer doesn't implement any supported version")]
-6533 VersionMismatch,
-6534 /// The peer violated the QUIC specification as understood by this implementation
-6535 #[error(transparent)]
-6536 TransportError(#[from] TransportError),
-6537 /// The peer's QUIC stack aborted the connection automatically
-6538 #[error("aborted by peer: {0}")]
-6539 ConnectionClosed(frame::ConnectionClose),
-6540 /// The peer closed the connection
-6541 #[error("closed by peer: {0}")]
-6542 ApplicationClosed(frame::ApplicationClose),
-6543 /// The peer is unable to continue processing this connection, usually due to having restarted
-6544 #[error("reset by peer")]
-6545 Reset,
-6546 /// Communication with the peer has lapsed for longer than the negotiated idle timeout
-6547 ///
-6548 /// If neither side is sending keep-alives, a connection will time out after a long enough idle
-6549 /// period even if the peer is still reachable. See also [`TransportConfig::max_idle_timeout()`]
-6550 /// and [`TransportConfig::keep_alive_interval()`].
-6551 #[error("timed out")]
-6552 TimedOut,
-6553 /// The local application closed the connection
-6554 #[error("closed")]
-6555 LocallyClosed,
-6556 /// The connection could not be created because not enough of the CID space is available
-6557 ///
-6558 /// Try using longer connection IDs.
-6559 #[error("CIDs exhausted")]
-6560 CidsExhausted,
-6561}
-6562
-6563impl From<Close> for ConnectionError {
-6564 fn from(x: Close) -> Self {
-6565 match x {
-6566 Close::Connection(reason) => Self::ConnectionClosed(reason),
-6567 Close::Application(reason) => Self::ApplicationClosed(reason),
-6568 }
-6569 }
-6570}
-6571
-6572// For compatibility with API consumers
-6573impl From<ConnectionError> for io::Error {
-6574 fn from(x: ConnectionError) -> Self {
-6575 use ConnectionError::*;
-6576 let kind = match x {
-6577 TimedOut => io::ErrorKind::TimedOut,
-6578 Reset => io::ErrorKind::ConnectionReset,
-6579 ApplicationClosed(_) | ConnectionClosed(_) => io::ErrorKind::ConnectionAborted,
-6580 TransportError(_) | VersionMismatch | LocallyClosed | CidsExhausted => {
-6581 io::ErrorKind::Other
-6582 }
-6583 };
-6584 Self::new(kind, x)
-6585 }
-6586}
+6460 if !validated {
+6461 let _ = self.close_path(
+6462 now,
+6463 path_id,
+6464 TransportErrorCode::APPLICATION_ABANDON_PATH.into(),
+6465 );
+6466 }
+6467 }
+6468
+6469 let mut err = None;
+6470
+6471 let mut path_ids = Vec::with_capacity(addresses_to_probe.len());
+6472 let mut probed_addresses = Vec::with_capacity(addresses_to_probe.len());
+6473 let ipv6 = self
+6474 .paths
+6475 .values()
+6476 .any(|p| p.data.network_path.remote.is_ipv6());
+6477
+6478 for (id, address) in addresses_to_probe {
+6479 match self.open_nat_traversal_path(now, address, ipv6) {
+6480 Ok(None) => {}
+6481 Ok(Some((path_id, remote, path_was_known))) => {
+6482 if !path_was_known {
+6483 path_ids.push(path_id);
+6484 probed_addresses.push(remote);
+6485 }
+6486 }
+6487 Err(e) => {
+6488 self.iroh_hp
+6489 .client_side_mut()
+6490 .expect("validated")
+6491 .report_in_continuation(id, e);
+6492 err.get_or_insert(e);
+6493 }
+6494 }
+6495 }
+6496
+6497 if let Some(err) = err {
+6498 // We failed to probe any addresses, bail out
+6499 if probed_addresses.is_empty() {
+6500 return Err(iroh_hp::Error::Multipath(err));
+6501 }
+6502 }
+6503
+6504 self.iroh_hp
+6505 .client_side_mut()
+6506 .expect("connection side validated")
+6507 .set_round_path_ids(path_ids);
+6508
+6509 Ok(probed_addresses)
+6510 }
+6511
+6512 /// Attempts to continue a nat traversal round by trying to open paths for pending client probes.
+6513 ///
+6514 /// If there was nothing to do, it returns `None`. Otherwise it returns whether the path was
+6515 /// successfully open.
+6516 fn continue_nat_traversal_round(&mut self, now: Instant) -> Option<bool> {
+6517 let client_state = self.iroh_hp.client_side_mut().ok()?;
+6518 let (id, address) = client_state.continue_nat_traversal_round()?;
+6519 let ipv6 = self
+6520 .paths
+6521 .values()
+6522 .any(|p| p.data.network_path.remote.is_ipv6());
+6523 let open_result = self.open_nat_traversal_path(now, address, ipv6);
+6524 let client_state = self.iroh_hp.client_side_mut().expect("validated");
+6525 match open_result {
+6526 Ok(None) => Some(true),
+6527 Ok(Some((path_id, _remote, path_was_known))) => {
+6528 if !path_was_known {
+6529 client_state.add_round_path_id(path_id);
+6530 }
+6531 Some(true)
+6532 }
+6533 Err(e) => {
+6534 client_state.report_in_continuation(id, e);
+6535 Some(false)
+6536 }
+6537 }
+6538 }
+6539}
+6540
+6541impl fmt::Debug for Connection {
+6542 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+6543 f.debug_struct("Connection")
+6544 .field("handshake_cid", &self.handshake_cid)
+6545 .finish()
+6546 }
+6547}
+6548
+6549#[derive(Debug, Copy, Clone, PartialEq, Eq)]
+6550enum PathBlocked {
+6551 No,
+6552 AntiAmplification,
+6553 Congestion,
+6554 Pacing,
+6555}
+6556
+6557/// Fields of `Connection` specific to it being client-side or server-side
+6558enum ConnectionSide {
+6559 Client {
+6560 /// Sent in every outgoing Initial packet. Always empty after Initial keys are discarded
+6561 token: Bytes,
+6562 token_store: Arc<dyn TokenStore>,
+6563 server_name: String,
+6564 },
+6565 Server {
+6566 server_config: Arc<ServerConfig>,
+6567 },
+6568}
+6569
+6570impl ConnectionSide {
+6571 fn remote_may_migrate(&self, state: &State) -> bool {
+6572 match self {
+6573 Self::Server { server_config } => server_config.migration,
+6574 Self::Client { .. } => {
+6575 if let Some(hs) = state.as_handshake() {
+6576 hs.allow_server_migration
+6577 } else {
+6578 false
+6579 }
+6580 }
+6581 }
+6582 }
+6583
+6584 fn is_client(&self) -> bool {
+6585 self.side().is_client()
+6586 }
6587
-6588/// Errors that might trigger a path being closed
-6589// TODO(@divma): maybe needs to be reworked based on what we want to do with the public API
-6590#[derive(Debug, Error, PartialEq, Eq, Clone, Copy)]
-6591pub enum PathError {
-6592 /// The extension was not negotiated with the peer
-6593 #[error("multipath extension not negotiated")]
-6594 MultipathNotNegotiated,
-6595 /// Paths can only be opened client-side
-6596 #[error("the server side may not open a path")]
-6597 ServerSideNotAllowed,
-6598 /// Current limits do not allow us to open more paths
-6599 #[error("maximum number of concurrent paths reached")]
-6600 MaxPathIdReached,
-6601 /// No remote CIDs available to open a new path
-6602 #[error("remoted CIDs exhausted")]
-6603 RemoteCidsExhausted,
-6604 /// Path could not be validated and will be abandoned
-6605 #[error("path validation failed")]
-6606 ValidationFailed,
-6607 /// The remote address for the path is not supported by the endpoint
-6608 #[error("invalid remote address")]
-6609 InvalidRemoteAddress(SocketAddr),
-6610}
-6611
-6612/// Errors triggered when abandoning a path
-6613#[derive(Debug, Error, Clone, Eq, PartialEq)]
-6614pub enum ClosePathError {
-6615 /// The path is already closed or was never opened
-6616 #[error("closed path")]
-6617 ClosedPath,
-6618 /// This is the last path, which can not be abandoned
-6619 #[error("last open path")]
-6620 LastOpenPath,
-6621}
-6622
-6623/// Error when the multipath extension was not negotiated, but attempted to be used.
-6624#[derive(Debug, Error, Clone, Copy)]
-6625#[error("Multipath extension not negotiated")]
-6626pub struct MultipathNotNegotiated {
-6627 _private: (),
-6628}
-6629
-6630/// Events of interest to the application
-6631#[derive(Debug)]
-6632pub enum Event {
-6633 /// The connection's handshake data is ready
-6634 HandshakeDataReady,
-6635 /// The connection was successfully established
-6636 Connected,
-6637 /// The TLS handshake was confirmed
-6638 HandshakeConfirmed,
-6639 /// The connection was lost
-6640 ///
-6641 /// Emitted if the peer closes the connection or an error is encountered.
-6642 ConnectionLost {
-6643 /// Reason that the connection was closed
-6644 reason: ConnectionError,
-6645 },
-6646 /// Stream events
-6647 Stream(StreamEvent),
-6648 /// One or more application datagrams have been received
-6649 DatagramReceived,
-6650 /// One or more application datagrams have been sent after blocking
-6651 DatagramsUnblocked,
-6652 /// (Multi)Path events
-6653 Path(PathEvent),
-6654 /// Iroh's nat traversal events
-6655 NatTraversal(iroh_hp::Event),
-6656}
-6657
-6658impl From<PathEvent> for Event {
-6659 fn from(source: PathEvent) -> Self {
-6660 Self::Path(source)
-6661 }
-6662}
-6663
-6664fn get_max_ack_delay(params: &TransportParameters) -> Duration {
-6665 Duration::from_micros(params.max_ack_delay.0 * 1000)
-6666}
-6667
-6668// Prevents overflow and improves behavior in extreme circumstances
-6669const MAX_BACKOFF_EXPONENT: u32 = 16;
-6670
-6671/// Minimal remaining size to allow packet coalescing, excluding cryptographic tag
-6672///
-6673/// This must be at least as large as the header for a well-formed empty packet to be coalesced,
-6674/// plus some space for frames. We only care about handshake headers because short header packets
-6675/// necessarily have smaller headers, and initial packets are only ever the first packet in a
-6676/// datagram (because we coalesce in ascending packet space order and the only reason to split a
-6677/// packet is when packet space changes).
-6678const MIN_PACKET_SPACE: usize = MAX_HANDSHAKE_OR_0RTT_HEADER_SIZE + 32;
-6679
-6680/// Largest amount of space that could be occupied by a Handshake or 0-RTT packet's header
-6681///
-6682/// Excludes packet-type-specific fields such as packet number or Initial token
-6683// https://www.rfc-editor.org/rfc/rfc9000.html#name-0-rtt: flags + version + dcid len + dcid +
-6684// scid len + scid + length + pn
-6685const MAX_HANDSHAKE_OR_0RTT_HEADER_SIZE: usize =
-6686 1 + 4 + 1 + MAX_CID_SIZE + 1 + MAX_CID_SIZE + VarInt::from_u32(u16::MAX as u32).size() + 4;
-6687
-6688/// Perform key updates this many packets before the AEAD confidentiality limit.
-6689///
-6690/// Chosen arbitrarily, intended to be large enough to prevent spurious connection loss.
-6691const KEY_UPDATE_MARGIN: u64 = 10_000;
-6692
-6693#[derive(Default)]
-6694struct SentFrames {
-6695 retransmits: ThinRetransmits,
-6696 /// The packet number of the largest acknowledged packet for each path
-6697 largest_acked: FxHashMap<PathId, u64>,
-6698 stream_frames: StreamMetaVec,
-6699 /// Whether the packet contains non-retransmittable frames (like datagrams)
-6700 non_retransmits: bool,
-6701 /// If the datagram containing these frames should be padded to the min MTU
-6702 requires_padding: bool,
-6703}
-6704
-6705impl SentFrames {
-6706 /// Returns whether the packet contains only ACKs
-6707 fn is_ack_only(&self, streams: &StreamsState) -> bool {
-6708 !self.largest_acked.is_empty()
-6709 && !self.non_retransmits
-6710 && self.stream_frames.is_empty()
-6711 && self.retransmits.is_empty(streams)
-6712 }
-6713
-6714 fn retransmits_mut(&mut self) -> &mut Retransmits {
-6715 self.retransmits.get_or_create()
-6716 }
-6717
-6718 fn record_sent_frame(&mut self, frame: frame::EncodableFrame<'_>) {
-6719 use frame::EncodableFrame::*;
-6720 match frame {
-6721 PathAck(path_ack_encoder) => {
-6722 if let Some(max) = path_ack_encoder.ranges.max() {
-6723 self.largest_acked.insert(path_ack_encoder.path_id, max);
-6724 }
-6725 }
-6726 Ack(ack_encoder) => {
-6727 if let Some(max) = ack_encoder.ranges.max() {
-6728 self.largest_acked.insert(PathId::ZERO, max);
-6729 }
-6730 }
-6731 Close(_) => { /* non retransmittable, but after this we don't really care */ }
-6732 PathResponse(_) => self.non_retransmits = true,
-6733 HandshakeDone(_) => self.retransmits_mut().handshake_done = true,
-6734 ReachOut(frame::ReachOut { round, ip, port }) => self
-6735 .retransmits_mut()
-6736 .reach_out
-6737 .get_or_insert_with(|| (round, Vec::new()))
-6738 .1
-6739 .push((ip, port)),
-6740 ObservedAddr(_) => self.retransmits_mut().observed_addr = true,
-6741 Ping(_) => self.non_retransmits = true,
-6742 ImmediateAck(_) => self.non_retransmits = true,
-6743 AckFrequency(_) => self.retransmits_mut().ack_frequency = true,
-6744 PathChallenge(_) => self.non_retransmits = true,
-6745 Crypto(crypto) => self.retransmits_mut().crypto.push_back(crypto),
-6746 PathAbandon(path_abandon) => {
-6747 self.retransmits_mut()
-6748 .path_abandon
-6749 .entry(path_abandon.path_id)
-6750 .or_insert(path_abandon.error_code);
-6751 }
-6752 PathStatusAvailable(frame::PathStatusAvailable { path_id, .. })
-6753 | PathStatusBackup(frame::PathStatusBackup { path_id, .. }) => {
-6754 self.retransmits_mut().path_status.insert(path_id);
-6755 }
-6756 MaxPathId(_) => self.retransmits_mut().max_path_id = true,
-6757 PathsBlocked(_) => self.retransmits_mut().paths_blocked = true,
-6758 PathCidsBlocked(path_cids_blocked) => {
-6759 self.retransmits_mut()
-6760 .path_cids_blocked
-6761 .insert(path_cids_blocked.path_id);
-6762 }
-6763 ResetStream(reset) => self
-6764 .retransmits_mut()
-6765 .reset_stream
-6766 .push((reset.id, reset.error_code)),
-6767 StopSending(stop_sending) => self.retransmits_mut().stop_sending.push(stop_sending),
-6768 NewConnectionId(new_cid) => self.retransmits_mut().new_cids.push(new_cid.issued()),
-6769 RetireConnectionId(retire_cid) => self
-6770 .retransmits_mut()
-6771 .retire_cids
-6772 .push((retire_cid.path_id.unwrap_or_default(), retire_cid.sequence)),
-6773 Datagram(_) => self.non_retransmits = true,
-6774 NewToken(_) => {}
-6775 AddAddress(add_address) => {
-6776 self.retransmits_mut().add_address.insert(add_address);
-6777 }
-6778 RemoveAddress(remove_address) => {
-6779 self.retransmits_mut().remove_address.insert(remove_address);
-6780 }
-6781 StreamMeta(stream_meta_encoder) => self.stream_frames.push(stream_meta_encoder.meta),
-6782 MaxData(_) => self.retransmits_mut().max_data = true,
-6783 MaxStreamData(max) => {
-6784 self.retransmits_mut().max_stream_data.insert(max.id);
-6785 }
-6786 MaxStreams(max_streams) => {
-6787 self.retransmits_mut().max_stream_id[max_streams.dir as usize] = true
-6788 }
-6789 }
-6790 }
-6791}
-6792
-6793/// Compute the negotiated idle timeout based on local and remote max_idle_timeout transport parameters.
-6794///
-6795/// According to the definition of max_idle_timeout, a value of `0` means the timeout is disabled; see <https://www.rfc-editor.org/rfc/rfc9000#section-18.2-4.4.1.>
-6796///
-6797/// According to the negotiation procedure, either the minimum of the timeouts or one specified is used as the negotiated value; see <https://www.rfc-editor.org/rfc/rfc9000#section-10.1-2.>
-6798///
-6799/// Returns the negotiated idle timeout as a `Duration`, or `None` when both endpoints have opted out of idle timeout.
-6800fn negotiate_max_idle_timeout(x: Option<VarInt>, y: Option<VarInt>) -> Option<Duration> {
-6801 match (x, y) {
-6802 (Some(VarInt(0)) | None, Some(VarInt(0)) | None) => None,
-6803 (Some(VarInt(0)) | None, Some(y)) => Some(Duration::from_millis(y.0)),
-6804 (Some(x), Some(VarInt(0)) | None) => Some(Duration::from_millis(x.0)),
-6805 (Some(x), Some(y)) => Some(Duration::from_millis(cmp::min(x, y).0)),
-6806 }
-6807}
-6808
-6809#[cfg(test)]
-6810mod tests {
-6811 use super::*;
-6812
-6813 #[test]
-6814 fn negotiate_max_idle_timeout_commutative() {
-6815 let test_params = [
-6816 (None, None, None),
-6817 (None, Some(VarInt(0)), None),
-6818 (None, Some(VarInt(2)), Some(Duration::from_millis(2))),
-6819 (Some(VarInt(0)), Some(VarInt(0)), None),
-6820 (
-6821 Some(VarInt(2)),
-6822 Some(VarInt(0)),
-6823 Some(Duration::from_millis(2)),
-6824 ),
-6825 (
-6826 Some(VarInt(1)),
-6827 Some(VarInt(4)),
-6828 Some(Duration::from_millis(1)),
-6829 ),
-6830 ];
-6831
-6832 for (left, right, result) in test_params {
-6833 assert_eq!(negotiate_max_idle_timeout(left, right), result);
-6834 assert_eq!(negotiate_max_idle_timeout(right, left), result);
-6835 }
-6836 }
-6837}
\ No newline at end of file
+6588 fn is_server(&self) -> bool {
+6589 self.side().is_server()
+6590 }
+6591
+6592 fn side(&self) -> Side {
+6593 match *self {
+6594 Self::Client { .. } => Side::Client,
+6595 Self::Server { .. } => Side::Server,
+6596 }
+6597 }
+6598}
+6599
+6600impl From<SideArgs> for ConnectionSide {
+6601 fn from(side: SideArgs) -> Self {
+6602 match side {
+6603 SideArgs::Client {
+6604 token_store,
+6605 server_name,
+6606 } => Self::Client {
+6607 token: token_store.take(&server_name).unwrap_or_default(),
+6608 token_store,
+6609 server_name,
+6610 },
+6611 SideArgs::Server {
+6612 server_config,
+6613 pref_addr_cid: _,
+6614 path_validated: _,
+6615 } => Self::Server { server_config },
+6616 }
+6617 }
+6618}
+6619
+6620/// Parameters to `Connection::new` specific to it being client-side or server-side
+6621pub(crate) enum SideArgs {
+6622 Client {
+6623 token_store: Arc<dyn TokenStore>,
+6624 server_name: String,
+6625 },
+6626 Server {
+6627 server_config: Arc<ServerConfig>,
+6628 pref_addr_cid: Option<ConnectionId>,
+6629 path_validated: bool,
+6630 },
+6631}
+6632
+6633impl SideArgs {
+6634 pub(crate) fn pref_addr_cid(&self) -> Option<ConnectionId> {
+6635 match *self {
+6636 Self::Client { .. } => None,
+6637 Self::Server { pref_addr_cid, .. } => pref_addr_cid,
+6638 }
+6639 }
+6640
+6641 pub(crate) fn path_validated(&self) -> bool {
+6642 match *self {
+6643 Self::Client { .. } => true,
+6644 Self::Server { path_validated, .. } => path_validated,
+6645 }
+6646 }
+6647
+6648 pub(crate) fn side(&self) -> Side {
+6649 match *self {
+6650 Self::Client { .. } => Side::Client,
+6651 Self::Server { .. } => Side::Server,
+6652 }
+6653 }
+6654}
+6655
+6656/// Reasons why a connection might be lost
+6657#[derive(Debug, Error, Clone, PartialEq, Eq)]
+6658pub enum ConnectionError {
+6659 /// The peer doesn't implement any supported version
+6660 #[error("peer doesn't implement any supported version")]
+6661 VersionMismatch,
+6662 /// The peer violated the QUIC specification as understood by this implementation
+6663 #[error(transparent)]
+6664 TransportError(#[from] TransportError),
+6665 /// The peer's QUIC stack aborted the connection automatically
+6666 #[error("aborted by peer: {0}")]
+6667 ConnectionClosed(frame::ConnectionClose),
+6668 /// The peer closed the connection
+6669 #[error("closed by peer: {0}")]
+6670 ApplicationClosed(frame::ApplicationClose),
+6671 /// The peer is unable to continue processing this connection, usually due to having restarted
+6672 #[error("reset by peer")]
+6673 Reset,
+6674 /// Communication with the peer has lapsed for longer than the negotiated idle timeout
+6675 ///
+6676 /// If neither side is sending keep-alives, a connection will time out after a long enough idle
+6677 /// period even if the peer is still reachable. See also [`TransportConfig::max_idle_timeout()`]
+6678 /// and [`TransportConfig::keep_alive_interval()`].
+6679 #[error("timed out")]
+6680 TimedOut,
+6681 /// The local application closed the connection
+6682 #[error("closed")]
+6683 LocallyClosed,
+6684 /// The connection could not be created because not enough of the CID space is available
+6685 ///
+6686 /// Try using longer connection IDs.
+6687 #[error("CIDs exhausted")]
+6688 CidsExhausted,
+6689}
+6690
+6691impl From<Close> for ConnectionError {
+6692 fn from(x: Close) -> Self {
+6693 match x {
+6694 Close::Connection(reason) => Self::ConnectionClosed(reason),
+6695 Close::Application(reason) => Self::ApplicationClosed(reason),
+6696 }
+6697 }
+6698}
+6699
+6700// For compatibility with API consumers
+6701impl From<ConnectionError> for io::Error {
+6702 fn from(x: ConnectionError) -> Self {
+6703 use ConnectionError::*;
+6704 let kind = match x {
+6705 TimedOut => io::ErrorKind::TimedOut,
+6706 Reset => io::ErrorKind::ConnectionReset,
+6707 ApplicationClosed(_) | ConnectionClosed(_) => io::ErrorKind::ConnectionAborted,
+6708 TransportError(_) | VersionMismatch | LocallyClosed | CidsExhausted => {
+6709 io::ErrorKind::Other
+6710 }
+6711 };
+6712 Self::new(kind, x)
+6713 }
+6714}
+6715
+6716/// Errors that might trigger a path being closed
+6717// TODO(@divma): maybe needs to be reworked based on what we want to do with the public API
+6718#[derive(Debug, Error, PartialEq, Eq, Clone, Copy)]
+6719pub enum PathError {
+6720 /// The extension was not negotiated with the peer
+6721 #[error("multipath extension not negotiated")]
+6722 MultipathNotNegotiated,
+6723 /// Paths can only be opened client-side
+6724 #[error("the server side may not open a path")]
+6725 ServerSideNotAllowed,
+6726 /// Current limits do not allow us to open more paths
+6727 #[error("maximum number of concurrent paths reached")]
+6728 MaxPathIdReached,
+6729 /// No remote CIDs available to open a new path
+6730 #[error("remoted CIDs exhausted")]
+6731 RemoteCidsExhausted,
+6732 /// Path could not be validated and will be abandoned
+6733 #[error("path validation failed")]
+6734 ValidationFailed,
+6735 /// The remote address for the path is not supported by the endpoint
+6736 #[error("invalid remote address")]
+6737 InvalidRemoteAddress(SocketAddr),
+6738}
+6739
+6740/// Errors triggered when abandoning a path
+6741#[derive(Debug, Error, Clone, Eq, PartialEq)]
+6742pub enum ClosePathError {
+6743 /// The path is already closed or was never opened
+6744 #[error("closed path")]
+6745 ClosedPath,
+6746 /// This is the last path, which can not be abandoned
+6747 #[error("last open path")]
+6748 LastOpenPath,
+6749}
+6750
+6751/// Error when the multipath extension was not negotiated, but attempted to be used.
+6752#[derive(Debug, Error, Clone, Copy)]
+6753#[error("Multipath extension not negotiated")]
+6754pub struct MultipathNotNegotiated {
+6755 _private: (),
+6756}
+6757
+6758/// Events of interest to the application
+6759#[derive(Debug)]
+6760pub enum Event {
+6761 /// The connection's handshake data is ready
+6762 HandshakeDataReady,
+6763 /// The connection was successfully established
+6764 Connected,
+6765 /// The TLS handshake was confirmed
+6766 HandshakeConfirmed,
+6767 /// The connection was lost
+6768 ///
+6769 /// Emitted if the peer closes the connection or an error is encountered.
+6770 ConnectionLost {
+6771 /// Reason that the connection was closed
+6772 reason: ConnectionError,
+6773 },
+6774 /// Stream events
+6775 Stream(StreamEvent),
+6776 /// One or more application datagrams have been received
+6777 DatagramReceived,
+6778 /// One or more application datagrams have been sent after blocking
+6779 DatagramsUnblocked,
+6780 /// (Multi)Path events
+6781 Path(PathEvent),
+6782 /// Iroh's nat traversal events
+6783 NatTraversal(iroh_hp::Event),
+6784}
+6785
+6786impl From<PathEvent> for Event {
+6787 fn from(source: PathEvent) -> Self {
+6788 Self::Path(source)
+6789 }
+6790}
+6791
+6792fn get_max_ack_delay(params: &TransportParameters) -> Duration {
+6793 Duration::from_micros(params.max_ack_delay.0 * 1000)
+6794}
+6795
+6796// Prevents overflow and improves behavior in extreme circumstances
+6797const MAX_BACKOFF_EXPONENT: u32 = 16;
+6798
+6799/// Minimal remaining size to allow packet coalescing, excluding cryptographic tag
+6800///
+6801/// This must be at least as large as the header for a well-formed empty packet to be coalesced,
+6802/// plus some space for frames. We only care about handshake headers because short header packets
+6803/// necessarily have smaller headers, and initial packets are only ever the first packet in a
+6804/// datagram (because we coalesce in ascending packet space order and the only reason to split a
+6805/// packet is when packet space changes).
+6806const MIN_PACKET_SPACE: usize = MAX_HANDSHAKE_OR_0RTT_HEADER_SIZE + 32;
+6807
+6808/// Largest amount of space that could be occupied by a Handshake or 0-RTT packet's header
+6809///
+6810/// Excludes packet-type-specific fields such as packet number or Initial token
+6811// https://www.rfc-editor.org/rfc/rfc9000.html#name-0-rtt: flags + version + dcid len + dcid +
+6812// scid len + scid + length + pn
+6813const MAX_HANDSHAKE_OR_0RTT_HEADER_SIZE: usize =
+6814 1 + 4 + 1 + MAX_CID_SIZE + 1 + MAX_CID_SIZE + VarInt::from_u32(u16::MAX as u32).size() + 4;
+6815
+6816/// Perform key updates this many packets before the AEAD confidentiality limit.
+6817///
+6818/// Chosen arbitrarily, intended to be large enough to prevent spurious connection loss.
+6819const KEY_UPDATE_MARGIN: u64 = 10_000;
+6820
+6821#[derive(Default)]
+6822struct SentFrames {
+6823 retransmits: ThinRetransmits,
+6824 /// The packet number of the largest acknowledged packet for each path
+6825 largest_acked: FxHashMap<PathId, u64>,
+6826 stream_frames: StreamMetaVec,
+6827 /// Whether the packet contains non-retransmittable frames (like datagrams)
+6828 non_retransmits: bool,
+6829 /// If the datagram containing these frames should be padded to the min MTU
+6830 requires_padding: bool,
+6831}
+6832
+6833impl SentFrames {
+6834 /// Returns whether the packet contains only ACKs
+6835 fn is_ack_only(&self, streams: &StreamsState) -> bool {
+6836 !self.largest_acked.is_empty()
+6837 && !self.non_retransmits
+6838 && self.stream_frames.is_empty()
+6839 && self.retransmits.is_empty(streams)
+6840 }
+6841
+6842 fn retransmits_mut(&mut self) -> &mut Retransmits {
+6843 self.retransmits.get_or_create()
+6844 }
+6845
+6846 fn record_sent_frame(&mut self, frame: frame::EncodableFrame<'_>) {
+6847 use frame::EncodableFrame::*;
+6848 match frame {
+6849 PathAck(path_ack_encoder) => {
+6850 if let Some(max) = path_ack_encoder.ranges.max() {
+6851 self.largest_acked.insert(path_ack_encoder.path_id, max);
+6852 }
+6853 }
+6854 Ack(ack_encoder) => {
+6855 if let Some(max) = ack_encoder.ranges.max() {
+6856 self.largest_acked.insert(PathId::ZERO, max);
+6857 }
+6858 }
+6859 Close(_) => { /* non retransmittable, but after this we don't really care */ }
+6860 PathResponse(_) => self.non_retransmits = true,
+6861 HandshakeDone(_) => self.retransmits_mut().handshake_done = true,
+6862 ReachOut(frame::ReachOut { round, ip, port }) => self
+6863 .retransmits_mut()
+6864 .reach_out
+6865 .get_or_insert_with(|| (round, Vec::new()))
+6866 .1
+6867 .push((ip, port)),
+6868 ObservedAddr(_) => self.retransmits_mut().observed_addr = true,
+6869 Ping(_) => self.non_retransmits = true,
+6870 ImmediateAck(_) => self.non_retransmits = true,
+6871 AckFrequency(_) => self.retransmits_mut().ack_frequency = true,
+6872 PathChallenge(_) => self.non_retransmits = true,
+6873 Crypto(crypto) => self.retransmits_mut().crypto.push_back(crypto),
+6874 PathAbandon(path_abandon) => {
+6875 self.retransmits_mut()
+6876 .path_abandon
+6877 .entry(path_abandon.path_id)
+6878 .or_insert(path_abandon.error_code);
+6879 }
+6880 PathStatusAvailable(frame::PathStatusAvailable { path_id, .. })
+6881 | PathStatusBackup(frame::PathStatusBackup { path_id, .. }) => {
+6882 self.retransmits_mut().path_status.insert(path_id);
+6883 }
+6884 MaxPathId(_) => self.retransmits_mut().max_path_id = true,
+6885 PathsBlocked(_) => self.retransmits_mut().paths_blocked = true,
+6886 PathCidsBlocked(path_cids_blocked) => {
+6887 self.retransmits_mut()
+6888 .path_cids_blocked
+6889 .insert(path_cids_blocked.path_id);
+6890 }
+6891 ResetStream(reset) => self
+6892 .retransmits_mut()
+6893 .reset_stream
+6894 .push((reset.id, reset.error_code)),
+6895 StopSending(stop_sending) => self.retransmits_mut().stop_sending.push(stop_sending),
+6896 NewConnectionId(new_cid) => self.retransmits_mut().new_cids.push(new_cid.issued()),
+6897 RetireConnectionId(retire_cid) => self
+6898 .retransmits_mut()
+6899 .retire_cids
+6900 .push((retire_cid.path_id.unwrap_or_default(), retire_cid.sequence)),
+6901 Datagram(_) => self.non_retransmits = true,
+6902 NewToken(_) => {}
+6903 AddAddress(add_address) => {
+6904 self.retransmits_mut().add_address.insert(add_address);
+6905 }
+6906 RemoveAddress(remove_address) => {
+6907 self.retransmits_mut().remove_address.insert(remove_address);
+6908 }
+6909 StreamMeta(stream_meta_encoder) => self.stream_frames.push(stream_meta_encoder.meta),
+6910 MaxData(_) => self.retransmits_mut().max_data = true,
+6911 MaxStreamData(max) => {
+6912 self.retransmits_mut().max_stream_data.insert(max.id);
+6913 }
+6914 MaxStreams(max_streams) => {
+6915 self.retransmits_mut().max_stream_id[max_streams.dir as usize] = true
+6916 }
+6917 }
+6918 }
+6919}
+6920
+6921/// Compute the negotiated idle timeout based on local and remote max_idle_timeout transport parameters.
+6922///
+6923/// According to the definition of max_idle_timeout, a value of `0` means the timeout is disabled; see <https://www.rfc-editor.org/rfc/rfc9000#section-18.2-4.4.1.>
+6924///
+6925/// According to the negotiation procedure, either the minimum of the timeouts or one specified is used as the negotiated value; see <https://www.rfc-editor.org/rfc/rfc9000#section-10.1-2.>
+6926///
+6927/// Returns the negotiated idle timeout as a `Duration`, or `None` when both endpoints have opted out of idle timeout.
+6928fn negotiate_max_idle_timeout(x: Option<VarInt>, y: Option<VarInt>) -> Option<Duration> {
+6929 match (x, y) {
+6930 (Some(VarInt(0)) | None, Some(VarInt(0)) | None) => None,
+6931 (Some(VarInt(0)) | None, Some(y)) => Some(Duration::from_millis(y.0)),
+6932 (Some(x), Some(VarInt(0)) | None) => Some(Duration::from_millis(x.0)),
+6933 (Some(x), Some(y)) => Some(Duration::from_millis(cmp::min(x, y).0)),
+6934 }
+6935}
+6936
+6937#[cfg(test)]
+6938mod tests {
+6939 use super::*;
+6940
+6941 #[test]
+6942 fn negotiate_max_idle_timeout_commutative() {
+6943 let test_params = [
+6944 (None, None, None),
+6945 (None, Some(VarInt(0)), None),
+6946 (None, Some(VarInt(2)), Some(Duration::from_millis(2))),
+6947 (Some(VarInt(0)), Some(VarInt(0)), None),
+6948 (
+6949 Some(VarInt(2)),
+6950 Some(VarInt(0)),
+6951 Some(Duration::from_millis(2)),
+6952 ),
+6953 (
+6954 Some(VarInt(1)),
+6955 Some(VarInt(4)),
+6956 Some(Duration::from_millis(1)),
+6957 ),
+6958 ];
+6959
+6960 for (left, right, result) in test_params {
+6961 assert_eq!(negotiate_max_idle_timeout(left, right), result);
+6962 assert_eq!(negotiate_max_idle_timeout(right, left), result);
+6963 }
+6964 }
+6965}
Human-readable explanation of the reason
An underlying crypto (e.g. TLS) layer error
Source§impl Error for Error
impl Error for Error
1.30.0 · Source§fn source(&self) -> Option<&(dyn Error + 'static)>
fn source(&self) -> Option<&(dyn Error + 'static)>
Returns the lower-level source of this error, if any. Read more
1.0.0 · Source§fn description(&self) -> &str
fn description(&self) -> &str
👎Deprecated since 1.42.0: use the Display impl or to_string()
Source§impl From<Error> for ConnectionClose
impl From<Error> for ConnectionClose
Source§fn from(x: TransportError) -> Self
fn from(x: TransportError) -> Self
Converts to this type from the input type.
Source§impl From<Error> for ConnectionError
impl From<Error> for ConnectionError
Source§fn from(source: TransportError) -> Self
fn from(source: TransportError) -> Self
Converts to this type from the input type.
Source§impl From<Error> for TransportError
impl From<Error> for TransportError
Source§impl Error for Error
impl Error for Error
1.30.0 · Source§fn source(&self) -> Option<&(dyn Error + 'static)>
fn source(&self) -> Option<&(dyn Error + 'static)>
Returns the lower-level source of this error, if any. Read more
1.0.0 · Source§fn description(&self) -> &str
fn description(&self) -> &str
👎Deprecated since 1.42.0: use the Display impl or to_string()
Source§impl From<Error> for ConnectionClose
impl From<Error> for ConnectionClose
Source§fn from(x: TransportError) -> Self
fn from(x: TransportError) -> Self
Converts to this type from the input type.
Source§impl From<Error> for ConnectionError
impl From<Error> for ConnectionError
Source§fn from(source: TransportError) -> Self
fn from(source: TransportError) -> Self
Converts to this type from the input type.