iperf3
(2024-11-12)
Most of the time I'm running iperf3 to understand the capability of a long distance link (think cross-continental or intercontinental), some of the tricks listed below won't apply or will be misleading if applied to a short distance link.
- Run the server:
# iperf3 -s
- Run the Client (testing upload to server):
# iperf3 -c 10.20.5.227
Connecting to host 10.20.5.227, port 5201
[ 5] local 10.0.1.245 port 39788 connected to 10.20.5.227 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 4.89 MBytes 41.0 Mbits/sec 0 2.37 MBytes
[ 5] 1.00-2.01 sec 12.4 MBytes 103 Mbits/sec 336 1.72 MBytes
[ 5] 2.01-3.00 sec 10.7 MBytes 90.2 Mbits/sec 14 1.28 MBytes
[ 5] 3.00-4.00 sec 9.14 MBytes 77.0 Mbits/sec 14 976 KBytes
[ 5] 4.00-5.00 sec 8.25 MBytes 69.2 Mbits/sec 32 801 KBytes
[ 5] 5.00-6.00 sec 4.91 MBytes 41.2 Mbits/sec 35 547 KBytes
[ 5] 6.00-7.00 sec 4.74 MBytes 39.8 Mbits/sec 0 586 KBytes
[ 5] 7.00-8.00 sec 4.91 MBytes 41.2 Mbits/sec 0 612 KBytes
[ 5] 8.00-9.00 sec 5.13 MBytes 43.1 Mbits/sec 0 625 KBytes
[ 5] 9.00-10.00 sec 5.24 MBytes 44.0 Mbits/sec 0 632 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 70.3 MBytes 59.0 Mbits/sec 431 sender
[ 5] 0.00-10.12 sec 70.1 MBytes 58.2 Mbits/sec receiver
iperf Done.
- Run the client in reverse mode (testing download from server):
# iperf3 -c 10.20.5.227 -R
Connecting to host 10.20.5.227, port 5201
Reverse mode, remote host 10.20.5.227 is sending
[ 5] local 10.0.1.245 port 35842 connected to 10.20.5.227 port 5201
[ ID] Interval Transfer Bitrate
[ 5] 0.00-2.94 sec 256 KBytes 714 Kbits/sec
[ 5] 2.94-5.83 sec 128 KBytes 362 Kbits/sec
[ 5] 5.83-8.54 sec 128 KBytes 387 Kbits/sec
[ 5] 8.54-11.05 sec 128 KBytes 418 Kbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-11.17 sec 744 KBytes 546 Kbits/sec 27 sender
[ 5] 0.00-11.05 sec 640 KBytes 474 Kbits/sec receiver
iperf Done.
- Run the client with parallel streams (you will want to learn about tcp and latency to understand why you can usually jam more traffic in total through a given pipe than a single transfer can):
# iperf3 -c 10.20.5.227 -P 5
Connecting to host 10.20.5.227, port 5201
[ 5] local 10.0.1.245 port 49732 connected to 10.20.5.227 port 5201
[ 7] local 10.0.1.245 port 49738 connected to 10.20.5.227 port 5201
[ 9] local 10.0.1.245 port 49750 connected to 10.20.5.227 port 5201
[ 11] local 10.0.1.245 port 49762 connected to 10.20.5.227 port 5201
[ 13] local 10.0.1.245 port 49778 connected to 10.20.5.227 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 903 KBytes 7.39 Mbits/sec 3 143 KBytes
[ 7] 0.00-1.00 sec 803 KBytes 6.57 Mbits/sec 13 120 KBytes
[ 9] 0.00-1.00 sec 854 KBytes 7.00 Mbits/sec 11 133 KBytes
[ 11] 0.00-1.00 sec 1.01 MBytes 8.49 Mbits/sec 36 169 KBytes
[ 13] 0.00-1.00 sec 1.17 MBytes 9.85 Mbits/sec 35 188 KBytes
[SUM] 0.00-1.00 sec 4.69 MBytes 39.3 Mbits/sec 98
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 1.00-2.00 sec 1.37 MBytes 11.5 Mbits/sec 0 171 KBytes
[ 7] 1.00-2.00 sec 1.12 MBytes 9.38 Mbits/sec 0 143 KBytes
[ 9] 1.00-2.00 sec 1.26 MBytes 10.5 Mbits/sec 0 158 KBytes
[ 11] 1.00-2.00 sec 1.56 MBytes 13.1 Mbits/sec 0 200 KBytes
[ 13] 1.00-2.00 sec 1.69 MBytes 14.2 Mbits/sec 0 222 KBytes
[SUM] 1.00-2.00 sec 7.00 MBytes 58.7 Mbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 2.00-3.00 sec 1.44 MBytes 12.1 Mbits/sec 0 184 KBytes
[ 7] 2.00-3.00 sec 1.24 MBytes 10.4 Mbits/sec 0 156 KBytes
[ 9] 2.00-3.00 sec 1.37 MBytes 11.5 Mbits/sec 0 171 KBytes
[ 11] 2.00-3.00 sec 1.77 MBytes 14.8 Mbits/sec 0 219 KBytes
[ 13] 2.00-3.00 sec 1.97 MBytes 16.6 Mbits/sec 0 244 KBytes
[SUM] 2.00-3.00 sec 7.80 MBytes 65.4 Mbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 3.00-4.00 sec 1.63 MBytes 13.6 Mbits/sec 0 192 KBytes
[ 7] 3.00-4.00 sec 1.37 MBytes 11.5 Mbits/sec 0 162 KBytes
[ 9] 3.00-4.00 sec 1.50 MBytes 12.6 Mbits/sec 0 177 KBytes
[ 11] 3.00-4.00 sec 1.90 MBytes 16.0 Mbits/sec 0 229 KBytes
[ 13] 3.00-4.00 sec 1.65 MBytes 13.8 Mbits/sec 6 187 KBytes
[SUM] 3.00-4.00 sec 8.05 MBytes 67.5 Mbits/sec 6
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 4.00-5.00 sec 1.30 MBytes 10.9 Mbits/sec 42 139 KBytes
[ 7] 4.00-5.00 sec 1.13 MBytes 9.51 Mbits/sec 21 119 KBytes
[ 9] 4.00-5.00 sec 1.20 MBytes 10.0 Mbits/sec 24 129 KBytes
[ 11] 4.00-5.00 sec 1.64 MBytes 13.8 Mbits/sec 31 168 KBytes
[ 13] 4.00-5.00 sec 1.45 MBytes 12.2 Mbits/sec 27 142 KBytes
[SUM] 4.00-5.00 sec 6.73 MBytes 56.4 Mbits/sec 145
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 5.00-6.00 sec 895 KBytes 7.33 Mbits/sec 4 111 KBytes
[ 7] 5.00-6.00 sec 772 KBytes 6.32 Mbits/sec 16 94.7 KBytes
[ 9] 5.00-6.00 sec 827 KBytes 6.77 Mbits/sec 3 103 KBytes
[ 11] 5.00-6.00 sec 1021 KBytes 8.36 Mbits/sec 14 131 KBytes
[ 13] 5.00-6.00 sec 836 KBytes 6.85 Mbits/sec 7 112 KBytes
[SUM] 5.00-6.00 sec 4.25 MBytes 35.6 Mbits/sec 44
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 6.00-7.00 sec 966 KBytes 7.92 Mbits/sec 0 118 KBytes
[ 7] 6.00-7.00 sec 835 KBytes 6.84 Mbits/sec 0 102 KBytes
[ 9] 6.00-7.00 sec 891 KBytes 7.30 Mbits/sec 0 110 KBytes
[ 11] 6.00-7.00 sec 1.13 MBytes 9.49 Mbits/sec 0 141 KBytes
[ 13] 6.00-7.00 sec 1.01 MBytes 8.49 Mbits/sec 0 120 KBytes
[SUM] 6.00-7.00 sec 4.77 MBytes 40.0 Mbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 7.00-8.00 sec 899 KBytes 7.36 Mbits/sec 2 96.1 KBytes
[ 7] 7.00-8.00 sec 763 KBytes 6.25 Mbits/sec 3 82.6 KBytes
[ 9] 7.00-8.00 sec 960 KBytes 7.86 Mbits/sec 0 111 KBytes
[ 11] 7.00-8.00 sec 1.21 MBytes 10.1 Mbits/sec 0 143 KBytes
[ 13] 7.00-8.00 sec 1.02 MBytes 8.54 Mbits/sec 0 122 KBytes
[SUM] 7.00-8.00 sec 4.78 MBytes 40.1 Mbits/sec 5
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 8.00-9.00 sec 830 KBytes 6.80 Mbits/sec 0 110 KBytes
[ 7] 8.00-9.00 sec 767 KBytes 6.29 Mbits/sec 0 96.1 KBytes
[ 9] 8.00-9.00 sec 960 KBytes 7.86 Mbits/sec 0 114 KBytes
[ 11] 8.00-9.00 sec 1.12 MBytes 9.41 Mbits/sec 0 143 KBytes
[ 13] 8.00-9.00 sec 1.01 MBytes 8.49 Mbits/sec 0 123 KBytes
[SUM] 8.00-9.00 sec 4.63 MBytes 38.9 Mbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 9.00-10.00 sec 1.00 MBytes 8.39 Mbits/sec 0 118 KBytes
[ 7] 9.00-10.00 sec 838 KBytes 6.86 Mbits/sec 0 103 KBytes
[ 9] 9.00-10.00 sec 1018 KBytes 8.34 Mbits/sec 0 119 KBytes
[ 11] 9.00-10.00 sec 1.27 MBytes 10.6 Mbits/sec 0 146 KBytes
[ 13] 9.00-10.00 sec 1.00 MBytes 8.42 Mbits/sec 0 129 KBytes
[SUM] 9.00-10.00 sec 5.08 MBytes 42.6 Mbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 11.1 MBytes 9.34 Mbits/sec 51 sender
[ 5] 0.00-10.12 sec 11.0 MBytes 9.10 Mbits/sec receiver
[ 7] 0.00-10.00 sec 9.53 MBytes 7.99 Mbits/sec 53 sender
[ 7] 0.00-10.12 sec 9.39 MBytes 7.78 Mbits/sec receiver
[ 9] 0.00-10.00 sec 10.7 MBytes 8.98 Mbits/sec 38 sender
[ 9] 0.00-10.12 sec 10.5 MBytes 8.73 Mbits/sec receiver
[ 11] 0.00-10.00 sec 13.6 MBytes 11.4 Mbits/sec 81 sender
[ 11] 0.00-10.12 sec 13.5 MBytes 11.2 Mbits/sec receiver
[ 13] 0.00-10.00 sec 12.8 MBytes 10.7 Mbits/sec 75 sender
[ 13] 0.00-10.12 sec 12.7 MBytes 10.5 Mbits/sec receiver
[SUM] 0.00-10.00 sec 57.8 MBytes 48.5 Mbits/sec 298 sender
[SUM] 0.00-10.12 sec 57.0 MBytes 47.3 Mbits/sec receiver
iperf Done.
- Run the client in udp mode (don't care about packet sequencing or robustness, just jam packets into the wire):
# iperf3 -c 10.20.5.227 -u
Connecting to host 10.20.5.227, port 5201
[ 5] local 10.0.1.245 port 34819 connected to 10.20.5.227 port 5201
[ ID] Interval Transfer Bitrate Total Datagrams
[ 5] 0.00-1.00 sec 129 KBytes 1.05 Mbits/sec 95
[ 5] 1.00-2.00 sec 129 KBytes 1.05 Mbits/sec 95
[ 5] 2.00-3.00 sec 127 KBytes 1.04 Mbits/sec 94
[ 5] 3.00-4.00 sec 129 KBytes 1.05 Mbits/sec 95
[ 5] 4.00-5.00 sec 127 KBytes 1.04 Mbits/sec 94
[ 5] 5.00-6.00 sec 129 KBytes 1.05 Mbits/sec 95
[ 5] 6.00-7.00 sec 127 KBytes 1.04 Mbits/sec 94
[ 5] 7.00-8.00 sec 129 KBytes 1.05 Mbits/sec 95
[ 5] 8.00-9.00 sec 129 KBytes 1.05 Mbits/sec 95
[ 5] 9.00-10.00 sec 127 KBytes 1.04 Mbits/sec 94
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-10.00 sec 1.25 MBytes 1.05 Mbits/sec 0.000 ms 0/946 (0%) sender
[ 5] 0.00-10.12 sec 1.25 MBytes 1.04 Mbits/sec 0.022 ms 0/946 (0%) receiver
iperf Done.
Note that in most cases, -u will pick a default datarate of 1Mb/s. Since this is useless, you probably want to...
- Run the client in udp mode, forcing packets to be emitted at a given rate:
# iperf3 -c 10.20.5.227 -u -b 750m
Connecting to host 10.20.5.227, port 5201
[ 5] local 10.0.1.245 port 44574 connected to 10.20.5.227 port 5201
[ ID] Interval Transfer Bitrate Total Datagrams
[ 5] 0.00-1.00 sec 89.3 MBytes 749 Mbits/sec 67593
[ 5] 1.00-2.00 sec 89.4 MBytes 750 Mbits/sec 67638
[ 5] 2.00-3.00 sec 89.4 MBytes 750 Mbits/sec 67644
[ 5] 3.00-4.00 sec 89.4 MBytes 750 Mbits/sec 67641
[ 5] 4.00-5.00 sec 89.4 MBytes 750 Mbits/sec 67640
[ 5] 5.00-6.00 sec 89.4 MBytes 750 Mbits/sec 67641
[ 5] 6.00-7.00 sec 89.4 MBytes 750 Mbits/sec 67641
[ 5] 7.00-8.00 sec 89.4 MBytes 750 Mbits/sec 67641
[ 5] 8.00-9.00 sec 89.4 MBytes 750 Mbits/sec 67641
[ 5] 9.00-10.00 sec 89.4 MBytes 750 Mbits/sec 67645
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-10.00 sec 894 MBytes 750 Mbits/sec 0.000 ms 0/676365 (0%) sender
[ 5] 0.00-10.12 sec 824 MBytes 683 Mbits/sec 0.013 ms 52749/676358 (7.8%) receiver
iperf Done.
Note that in this case, the sender will continue to jam packets into the wire even if the wire and/or the server can't deal with them. The difference will be listed as the count of Lost Datagrams. Once you exceed the connection's end-to-end capacity, this can give you an idea of the maximum possible data rate through a given link. In the example above, we're jamming in 750Mb/s but the server is only getting 683Mb/s.
Something to remember
If you are dealing with intersite links you are quite probably dealing with firewalls and ipsec tunnels. Remember that for tunnels in particular, the firewall can potentially treat the traffic differently depending on which "direction" the connection is going -- think security policies, tunnelling parameters, route selection, etc. And the "direction" of the connection is determined by the direction of the client's initial SYN packet. That is, regardless of the actual data flow, the firewall will tread the connection's "direction" as "from the client to the server".
You also have to consider the platforms involved -- virtual platforms may give worse performance than can be otherwise realized,