Hello,
i tried to verify our Network performance between two specific hosts and i found a huge gap between UDP and TCP.
UDP
Control connection MSS 8948
Setting UDP block size to 8948
Test Complete. Summary Results: [ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams [ 4] 0.00-10.00 sec 1.25 MBytes 1.04 Mbits/sec 0.040 ms 0/146 (0%) [ 4] Sent 146 datagrams [ 6] 0.00-10.00 sec 1.25 MBytes 1.04 Mbits/sec 0.037 ms 0/146 (0%) [ 6] Sent 146 datagrams [ 8] 0.00-10.00 sec 1.25 MBytes 1.04 Mbits/sec 0.041 ms 0/146 (0%) [ 8] Sent 146 datagrams [ 10] 0.00-10.00 sec 1.25 MBytes 1.04 Mbits/sec 0.040 ms 0/146 (0%) [ 10] Sent 146 datagrams [ 12] 0.00-10.00 sec 1.25 MBytes 1.04 Mbits/sec 0.040 ms 0/146 (0%) [ 12] Sent 146 datagrams [ 14] 0.00-10.00 sec 1.25 MBytes 1.04 Mbits/sec 0.041 ms 0/146 (0%) [ 14] Sent 146 datagrams [ 16] 0.00-10.00 sec 1.25 MBytes 1.04 Mbits/sec 0.042 ms 0/146 (0%) [ 16] Sent 146 datagrams [ 18] 0.00-10.00 sec 1.25 MBytes 1.04 Mbits/sec 0.040 ms 0/146 (0%) [ 18] Sent 146 datagrams [ 20] 0.00-10.00 sec 1.25 MBytes 1.04 Mbits/sec 0.040 ms 0/146 (0%) [ 20] Sent 146 datagrams [ 22] 0.00-10.00 sec 1.25 MBytes 1.04 Mbits/sec 0.039 ms 0/146 (0%) [ 22] Sent 146 datagrams [SUM] 0.00-10.00 sec 12.5 MBytes 10.4 Mbits/sec 0.040 ms 0/1460 (0%) CPU Utilization: local/sender 1.2% (1.2%u/0.0%s), remote/receiver 0.0% (0.0%u/0.0%s)
TCP
Control connection MSS 8948
Test Complete. Summary Results: [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 1.15 GBytes 987 Mbits/sec 0 sender [ 4] 0.00-10.00 sec 1.15 GBytes 987 Mbits/sec receiver [ 6] 0.00-10.00 sec 1.15 GBytes 987 Mbits/sec 0 sender [ 6] 0.00-10.00 sec 1.15 GBytes 987 Mbits/sec receiver [ 8] 0.00-10.00 sec 1.15 GBytes 986 Mbits/sec 0 sender [ 8] 0.00-10.00 sec 1.15 GBytes 986 Mbits/sec receiver [ 10] 0.00-10.00 sec 1.14 GBytes 982 Mbits/sec 0 sender [ 10] 0.00-10.00 sec 1.14 GBytes 982 Mbits/sec receiver [ 12] 0.00-10.00 sec 1.15 GBytes 986 Mbits/sec 0 sender [ 12] 0.00-10.00 sec 1.15 GBytes 986 Mbits/sec receiver [ 14] 0.00-10.00 sec 1.15 GBytes 987 Mbits/sec 0 sender [ 14] 0.00-10.00 sec 1.15 GBytes 987 Mbits/sec receiver [ 16] 0.00-10.00 sec 1.15 GBytes 984 Mbits/sec 0 sender [ 16] 0.00-10.00 sec 1.15 GBytes 984 Mbits/sec receiver [ 18] 0.00-10.00 sec 1.15 GBytes 986 Mbits/sec 0 sender [ 18] 0.00-10.00 sec 1.15 GBytes 986 Mbits/sec receiver [ 20] 0.00-10.00 sec 1.15 GBytes 986 Mbits/sec 0 sender [ 20] 0.00-10.00 sec 1.15 GBytes 986 Mbits/sec receiver [ 22] 0.00-10.00 sec 1.15 GBytes 987 Mbits/sec 0 sender [ 22] 0.00-10.00 sec 1.15 GBytes 987 Mbits/sec receiver [SUM] 0.00-10.00 sec 11.5 GBytes 9.86 Gbits/sec 0 sender [SUM] 0.00-10.00 sec 11.5 GBytes 9.86 Gbits/sec receiver CPU Utilization: local/sender 108.7% (108.7%u/0.0%s), remote/receiver 14.3% (14.3%u/0.0%s)
My Setup
Client: ./iperf3-Copy -c 10.139.205.20 -P 10 -i 60 -u
Server: ./iperf3-Copy -s -B 10.139.205.20
iPerf Version: iperf 3.1.6
VMkernel: 6.5.0 #1 SMP Release build-9298722 Jul 25 2018 18:56:07 x86_64
NIC:
Name PCI Device Driver Admin Status Link Status Speed Duplex MAC Address MTU Description
------ ------------ ------ ------------ ----------- ----- ------ ----------------- ---- -------------------------------------------------------------
vmnic0 0000:04:00.0 qfle3 Up Up 10000 Full 14:02:ec:44:ab:a0 9000 Broadcom Corporation QLogic 57810 10 Gigabit Ethernet Adapter
Advertised Auto Negotiation: false
Advertised Link Modes: 10000BaseTwinax/Full
Auto Negotiation: false
Cable Type: DA
Current Message Level: 0
Driver Info:
Bus Info: 0000:04:00:0
Driver: qfle3
Firmware Version: FW: 7.13.109.0 BC: 7.15.3
Version: 1.0.60.3
Link Detected: true
Link Status: Up
Name: vmnic0
PHYAddress: 0
Pause Autonegotiate: false
Pause RX: true
Pause TX: true
Supported Ports: FIBRE
Supports Auto Negotiation: false
Supports Pause: true
Supports Wakeon: true
Transceiver: external
Virtual Address: 00:50:56:54:ab:a0
Wakeon: MagicPacket(tm)
Any hints whats going on here (I don't think that is a physical network performance problem)?
Best regards,
Markus