Commit Graph

28 Commits

Author SHA1 Message Date
Matthias Einwag 80376ff305 Allow to specify the cipher suite in bulk test
rustls defaults to CHACHA20, which is rather slow on desktop hardware.
This change allows to specify a single cipher suite for this test, and
sets the default to AES128.

CHACHA20 (default):
> Sent 4294967296 bytes on 1 streams in 14.86s (275.68 MiB/s)

AES128:
> Sent 4294967296 bytes on 1 streams in 12.21s (335.59 MiB/s)
2021-02-16 19:14:36 +01:00
Matthias Einwag cd974a7945 Disable hdrhistogram features
This causes nom to be pulled in, which currently causes the CI error and build
conflict between `funty` and `bitvec`: https://github.com/myrrlyn/funty/issues/3

We don't need the feature in this scenario anyway.
2021-02-15 07:13:59 +01:00
Matthias Einwag ff542299c4 Allow to use the vectored ordered read API in benchmark
This extends the benchmark to make the consumer either utilize the
ordered or unordered read API.
The ordered read API is now set to the default, since it might be
applicable to more applications.
2021-02-05 10:14:57 +01:00
Dirkjan Ochtman a280b7770f quinn: unify ordered and unordered read APIs 2021-01-30 10:09:09 +01:00
Dirkjan Ochtman 51685fb760 Upgrade to tokio 1, bytes 1 and rustls 0.19 2021-01-25 08:53:00 +01:00
Benjamin Saunders 1e5a538221 Limit concurrent streams rather than accept queue size
Previously, we limited the number of streams that could be opened by
the peer but not `accept`ed by the application. However, to guarantee
bounded resource use, applications will typically want to limit the
number of streams they process concurrently. While this could
be *approximately* implemented at the application layer by controlling
calls to accept, that approach has a significant drawback: If streams
are slow to process, the worst-case per-stream latency observed by a
peer who opens the maximum number of streams can be arbitrarily bad,
because streams may be opened above the limit the local application is
willing to process. Reducing the number of unaccepted streams
tolerated can reduce the proportion of streams affected, but reducing
it too low will increase the number of round trips required to open
any given number of streams, increasing average latency significantly.

As a side benefit, this reduces the effort needed for applications to
limit concurrency to a fixed quantity, which is expected to be the
overwhelmingly common case. Should a use case for dynamic concurrency
limits arise, we can expose a setter.
2021-01-22 11:39:00 -08:00
Matthias Einwag 6f1687c5fb Fix bulk benchmark
The sending data on the benchmark failed it panicked due
to an unwrap and thereby showed no result. This propagates
the first client-side error instead of panicking - which will also
preserve the statistics.
2021-01-19 08:16:47 +01:00
Dirkjan Ochtman 17fe82b598 Apply clippy fixes for 1.49 2021-01-01 11:16:05 -08:00
Matthias Einwag 62ef3cc1f2 Allow to show connection stats in bulk benchmark
This allows to determine the impact of particular changes a
bit easier.

Example output:

```
Overall stats:

Sent 1073741824 bytes on 1 streams in 4.04s (253.61 MiB/s)

Stream metrics:

      │  Throughput   │ Duration
──────┼───────────────┼──────────
 AVG  │  253.56 MiB/s │     4.04s
 P0   │  253.50 MiB/s │     4.04s
 P10  │  253.62 MiB/s │     4.04s
 P50  │  253.62 MiB/s │     4.04s
 P90  │  253.62 MiB/s │     4.04s
 P100 │  253.62 MiB/s │     4.04s

Server connection stats:
ConnectionStats {
    udp_tx: UdpStats {
        datagrams: 26391,
        bytes: 1110961,
    },
    udp_rx: UdpStats {
        datagrams: 918524,
        bytes: 1116003172,
    },
    frame_tx: FrameStats {
        ACK: 26388,
        CRYPTO: 6,
        DATAGRAM: 0,
        HANDSHAKE_DONE: 4,
        MAX_DATA: 0,
        MAX_STREAM_DATA: 4984,
        MAX_STREAMS_BIDI: 0,
        MAX_STREAMS_UNI: 1,
        NEW_CONNECTION_ID: 16,
        PATH_CHALLENGE: 0,
        PATH_RESPONSE: 0,
        PING: 0,
        RESET_STREAM: 0,
        RETIRE_CONNECTION_ID: 3,
        STOP_SENDING: 0,
        STREAM: 0,
    },
}

Client connection stats:
ConnectionStats {
    udp_tx: UdpStats {
        datagrams: 918556,
        bytes: 1116042037,
    },
    udp_rx: UdpStats {
        datagrams: 26391,
        bytes: 1110961,
    },
    frame_tx: FrameStats {
        ACK: 918554,
        CRYPTO: 2,
        DATAGRAM: 0,
        HANDSHAKE_DONE: 0,
        MAX_DATA: 0,
        MAX_STREAM_DATA: 0,
        MAX_STREAMS_BIDI: 0,
        MAX_STREAMS_UNI: 0,
        NEW_CONNECTION_ID: 5,
        PATH_CHALLENGE: 0,
        PATH_RESPONSE: 0,
        PING: 0,
        RESET_STREAM: 0,
        RETIRE_CONNECTION_ID: 0,
        STOP_SENDING: 0,
        STREAM: 918552,
    },
}
```
2021-01-01 14:23:40 +01:00
Matthias Einwag 1aaff731ab Extend bulk benchmark
This change improves the bulk benchmark:
- The total amount of requests the benchmark performs can be configured
- The amount of concurrent requests on a given connection can be
  configured
- The amount of data to transfer per stream can be configured

E.g.

```
bulk --streams 300 --max_streams 100 --stream_size 10
```

will create 300 streams - with a maximum of 100 active streams at a time,
and send 10MB of data on each stream.

The "short" cli flags are taken form h2load where applicable.

The benchmark now also shows metrics which indicate the performance
of individual streams, which allows to judge fairness. Example output:

```
Overall stats:

Sent 3145728000 bytes on 300 streams in 12.11s (247.81 MiB/s)

Stream metrics:

      │  Throughput   │ Duration
──────┼───────────────┼──────────
 AVG  │  149.17 MiB/s │  396.00ms
 P0   │    0.83 MiB/s │   39.00ms
 P10  │   51.19 MiB/s │   40.00ms
 P50  │  129.75 MiB/s │   77.00ms
 P90  │  247.00 MiB/s │  195.00ms
 P100 │  254.75 MiB/s │    12.11s
```
2020-12-29 15:15:43 +01:00
Matthias Einwag 6d32b64d4c Extract data sending in bulk benchmark
This change moves establishing a stream and sending data into
a separate method.
2020-12-29 15:15:43 +01:00
Dirkjan Ochtman 120c8157ae Update futures-* to 0.3.8
Prevents security issues in futures older than 0.3.5.

Specifically RUSTSEC-2020-0059 and RUSTSEC-2020-0060.
2020-12-21 14:22:18 -08:00
Benjamin Saunders f17eeca4fa Print rate in standalone bulk benchmark 2020-11-09 08:04:01 +01:00
kwantam ee6f7d01f2 pin all crates with rustls as transitive dep
this is necessary for export_keying_material to work;
can be undone once rustls pushes a new version of the crates.
2020-09-29 09:35:14 +02:00
Dirkjan Ochtman 5a4d18f837 Upgrade to rustls-0.18 2020-07-05 23:05:33 +02:00
Dirkjan Ochtman 73fbb9bdee tracing-subscriber 0.2.5 is required since 77aeb5544 2020-06-04 22:36:09 +02:00
Dirkjan Ochtman eeac018bd9 Update license to proper SPDX expression
From the Cargo docs:

"Previously multiple licenses could be separated with a /,
but that usage is deprecated."

https://doc.rust-lang.org/cargo/reference/manifest.html#the-license-and-license-file-fields
2020-05-04 09:14:22 -07:00
Benjamin Saunders e0319215b7 Lighten tracing-subscriber
Omitting json logging gets serde out of our dev-dependency closure
2020-03-31 13:20:30 +02:00
Dirkjan Ochtman 67f448c553 Bump rcgen to 0.8 to avoid duplicate dependency 2020-03-12 19:23:52 +01:00
stammw 0ff3d8f51d H3: set FrameDecoder initial buffersize to UDP size 2020-03-10 13:23:00 +01:00
Dirkjan Ochtman 33990db1d5 Upgrade to rustls 0.17 2020-02-24 22:30:07 +01:00
dependabot-preview[bot] 147588db31 Update tracing-subscriber requirement from 0.1.5 to 0.2.0
Updates the requirements on [tracing-subscriber](https://github.com/tokio-rs/tracing) to permit the latest version.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.1.5...tracing-subscriber-0.2.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-05 22:00:55 -08:00
Benjamin Saunders 5774071b48 Spawn driver tasks implicitly 2020-01-13 09:54:26 +01:00
Timon 73b1011ba0 Fixed Clippy Warnings (#531)
* Fixed clippy errors

* removed statement
2019-12-02 19:31:32 +01:00
Dirkjan Ochtman a72ab73614 Sort dependencies again 2019-12-02 08:00:26 -08:00
daxpedda 7233455078 Update tokio, futures and bytes. (#528) 2019-12-02 10:42:05 +01:00
Benjamin Saunders 8d178f94da Remove subcrate-specific profile config
Cargo ignores these.
2019-11-21 21:26:07 +01:00
Benjamin Saunders 3ffa250b35 Add minimal self-contained benchmark for easy profiling 2019-11-21 09:07:36 +01:00