This allows to specify a "--client" parameter in the benchmark,
which specifies the amount of clients which perform a test concurrently.
Each client uses its own thread and single-threaded runtime.
Stats are currently printed for each individual client and not aggregated.
This changes the bulk benchmark to support server->client
in addition to client->server transfers.
The default direction is set to server->client, since
this is more typically used.
In order to extend the bulk benchmark further, this change moves most
common functions out of the bulk.rs binary and into a lib.rs and stats.rs file.
Functionally there are no relevant changes.
For most setters this was just a case of changing the signature to take
a `VarInt`.
`max_idle_timeout` was a bit more involved in order to maintain a usable
conversion from `Duration`. The internal representation was changed from
`Duration` from `VarInt`, and an `IdleTimeout` newtype was introduced
for the setter signature. `IdleTimeout` contains a `VarInt`-encoded
millisecond value, and has a fallible conversion from `Duration`. The
use of a newtype is preferable to a direct `Duration` -> `VarInt`
encoding since there could be other `VarInt`-encoded values in future
that use a different resolution than milliseconds.
These changes allow callers to avoid the type-level possibility of error
if they have a `VarInt` (or a value that can be converted infallibly
into one). Fallible conversion from `VarInt` is still convenient with
`TryInto`, and callers can handle the conversion errors without the
indirection of `ConfigError`.
Closes#1176.
BREAKING CHANGE: The `max_concurrent_bidi_streams`,
`max_concurrent_uni_streams`, `stream_receive_window`, and
`receive_window` methods on `TransportConfig` now take `VarInt`
arguments and return `&mut Self`. `TransportConfig::max_idle_timeout`
now takes an `IdleTimeout` argument and returns `&mut Self`.
rustls defaults to CHACHA20, which is rather slow on desktop hardware.
This change allows to specify a single cipher suite for this test, and
sets the default to AES128.
CHACHA20 (default):
> Sent 4294967296 bytes on 1 streams in 14.86s (275.68 MiB/s)
AES128:
> Sent 4294967296 bytes on 1 streams in 12.21s (335.59 MiB/s)
This causes nom to be pulled in, which currently causes the CI error and build
conflict between `funty` and `bitvec`: https://github.com/myrrlyn/funty/issues/3
We don't need the feature in this scenario anyway.
This extends the benchmark to make the consumer either utilize the
ordered or unordered read API.
The ordered read API is now set to the default, since it might be
applicable to more applications.
Previously, we limited the number of streams that could be opened by
the peer but not `accept`ed by the application. However, to guarantee
bounded resource use, applications will typically want to limit the
number of streams they process concurrently. While this could
be *approximately* implemented at the application layer by controlling
calls to accept, that approach has a significant drawback: If streams
are slow to process, the worst-case per-stream latency observed by a
peer who opens the maximum number of streams can be arbitrarily bad,
because streams may be opened above the limit the local application is
willing to process. Reducing the number of unaccepted streams
tolerated can reduce the proportion of streams affected, but reducing
it too low will increase the number of round trips required to open
any given number of streams, increasing average latency significantly.
As a side benefit, this reduces the effort needed for applications to
limit concurrency to a fixed quantity, which is expected to be the
overwhelmingly common case. Should a use case for dynamic concurrency
limits arise, we can expose a setter.
The sending data on the benchmark failed it panicked due
to an unwrap and thereby showed no result. This propagates
the first client-side error instead of panicking - which will also
preserve the statistics.
This change improves the bulk benchmark:
- The total amount of requests the benchmark performs can be configured
- The amount of concurrent requests on a given connection can be
configured
- The amount of data to transfer per stream can be configured
E.g.
```
bulk --streams 300 --max_streams 100 --stream_size 10
```
will create 300 streams - with a maximum of 100 active streams at a time,
and send 10MB of data on each stream.
The "short" cli flags are taken form h2load where applicable.
The benchmark now also shows metrics which indicate the performance
of individual streams, which allows to judge fairness. Example output:
```
Overall stats:
Sent 3145728000 bytes on 300 streams in 12.11s (247.81 MiB/s)
Stream metrics:
│ Throughput │ Duration
──────┼───────────────┼──────────
AVG │ 149.17 MiB/s │ 396.00ms
P0 │ 0.83 MiB/s │ 39.00ms
P10 │ 51.19 MiB/s │ 40.00ms
P50 │ 129.75 MiB/s │ 77.00ms
P90 │ 247.00 MiB/s │ 195.00ms
P100 │ 254.75 MiB/s │ 12.11s
```
Dirkjan Ochtman