中科云 3A6000 机器 dwmac 网卡存在掉速问题

felixonmars

走对应网络接口运行 iperf3 --bidir 双向测试比较容易复现，但不是每次都能复现（复现成功率较高）。

在同样的测试条件下，另一张 yt6801 网卡无法复现此问题。龙芯 2K2000 开发板的网口在同内核版本的同样驱动下也无法复现。

当前测试内核版本为 6.12.4 主线内核。

felixonmars

另有一个好像更稳定复现的相关问题，虽然我不确定是不是同一个问题：

同样运行 iperf3 --bidir 测试 sctp 协议，这台机器的两块网卡（dwmac 和 yt6801）均在几秒后速度归零。此问题同样在 2K2000 无法复现，在我这里其他设备（主要是 RISC-V 设备）上也都无法复现。

随便看看

刚刚测了一下龙芯3A6000的XA61200主板和铭凡BD790i AMD 锐龙R9-7945HX主板，iperf3 --bidir 双向测试都没有问题，测试 sctp 协议都在几秒后速度归零。系统Debian sid，内核版本6.12.5。

龙芯3A6000 XA61200主板【网卡检测为02:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8211/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 15)】：

root@debian13:~# iperf3 -c 192.168.1.167 --bidir Connecting to host 192.168.1.167, port 5201
[ 5] local 192.168.1.40 port 42150 connected to [ 7] local 192.168.1.40 port 42160 connected to [ ID][Role] Interval Transfer Bitrate [ 5][TX-C] 0.00-1.00 sec 113 MBytes 950 Mbits/sec [ 7][RX-C] 0.00-1.00 sec 112 MBytes 937 Mbits/sec [ 5][TX-C] 1.00-2.00 sec 113 MBytes 951 Mbits/sec [ 7][RX-C] 1.00-2.00 sec 112 MBytes 940 Mbits/sec [ 5][TX-C] 2.00-3.00 sec 111 MBytes 933 Mbits/sec [ 7][RX-C] 2.00-3.00 sec 112 MBytes 937 Mbits/sec [ 5][TX-C] 3.00-4.00 sec 112 MBytes 940 Mbits/sec [ 7][RX-C] 3.00-4.00 sec 112 MBytes 935 Mbits/sec [ 5][TX-C] 4.00-5.00 sec 112 MBytes 942 Mbits/sec [ 7][RX-C] 4.00-5.00 sec 111 MBytes 934 Mbits/sec [ 5][TX-C] 5.00-6.00 sec 112 MBytes 943 Mbits/sec [ 7][RX-C] 5.00-6.00 sec 111 MBytes 934 Mbits/sec [ 5][TX-C] 6.00-7.00 sec 112 MBytes 942 Mbits/sec [ 7][RX-C] 6.00-7.00 sec 111 MBytes 934 Mbits/sec [ 5][TX-C] 7.00-8.00 sec 112 MBytes 942 Mbits/sec [ 7][RX-C] 7.00-8.00 sec 111 MBytes 934 Mbits/sec [ 5][TX-C] 8.00-9.00 sec 112 MBytes 940 Mbits/sec [ 7][RX-C] 8.00-9.00 sec 111 MBytes 934 Mbits/sec [ 5][TX-C] 9.00-10.00 sec 111 MBytes 930 Mbits/sec [ 7][RX-C] 9.00-10.00 sec 112 MBytes 935 Mbits/sec --bind-dev enp2s0
192.168.1.167 port 5201
192.168.1.167 port 5201
Retr Cwnd
0 628 KBytes

0 690 KBytes

0 721 KBytes

0 721 KBytes

0 721 KBytes

0 721 KBytes

0 721 KBytes

0 721 KBytes

0 721 KBytes

0 721 KBytes

[ ID][Role] Interval Transfer Bitrate Retr
[ 5][TX-C] 0.00-10.00 sec 1.10 GBytes 941 Mbits/sec 0 sender
[ 5][TX-C] 0.00-10.00 sec 1.09 GBytes 939 Mbits/sec receiver
[ 7][RX-C] 0.00-10.00 sec 1.09 GBytes 937 Mbits/sec 116 sender
[ 7][RX-C] 0.00-10.00 sec 1.09 GBytes 936 Mbits/sec receiver

iperf Done.

root@debian13:~# iperf3 -c 192.168.1.167 --bidir --bind-dev enp2s0 --sctp
Connecting to host 192.168.1.167, port 5201
[ 5] local 192.168.1.40 port 50063 connected to 192.168.1.167 port 5201
[ 7] local 192.168.1.40 port 55636 connected to 192.168.1.167 port 5201
[ ID][Role] Interval Transfer Bitrate
[ 5][TX-C] 0.00-1.00 sec 69.0 MBytes 578 Mbits/sec
[ 7][RX-C] 0.00-1.00 sec 76.1 MBytes 638 Mbits/sec
[ 5][TX-C] 1.00-2.00 sec 77.7 MBytes 652 Mbits/sec
[ 7][RX-C] 1.00-2.00 sec 77.8 MBytes 653 Mbits/sec
[ 5][TX-C] 2.00-3.00 sec 74.2 MBytes 622 Mbits/sec
[ 7][RX-C] 2.00-3.00 sec 74.1 MBytes 622 Mbits/sec
[ 5][TX-C] 3.00-4.00 sec 75.8 MBytes 636 Mbits/sec
[ 7][RX-C] 3.00-4.00 sec 56.7 MBytes 476 Mbits/sec
[ 5][TX-C] 4.00-5.00 sec 39.6 MBytes 332 Mbits/sec
[ 7][RX-C] 4.00-5.00 sec 0.00 Bytes 0.00 bits/sec
[ 5][TX-C] 5.00-6.00 sec 0.00 Bytes 0.00 bits/sec
[ 7][RX-C] 5.00-6.00 sec 0.00 Bytes 0.00 bits/sec
[ 5][TX-C] 6.00-7.00 sec 0.00 Bytes 0.00 bits/sec
[ 7][RX-C] 6.00-7.00 sec 0.00 Bytes 0.00 bits/sec
[ 5][TX-C] 7.00-8.00 sec 0.00 Bytes 0.00 bits/sec
[ 7][RX-C] 7.00-8.00 sec 0.00 Bytes 0.00 bits/sec
[ 5][TX-C] 8.00-9.00 sec 0.00 Bytes 0.00 bits/sec
[ 7][RX-C] 8.00-9.00 sec 0.00 Bytes 0.00 bits/sec
[ 5][TX-C] 9.00-10.00 sec 0.00 Bytes 0.00 bits/sec
[ 7][RX-C] 9.00-10.00 sec 0.00 Bytes 0.00 bits/sec

[ ID][Role] Interval Transfer Bitrate
[ 5][TX-C] 0.00-10.00 sec 336 MBytes 282 Mbits/sec sender
[ 5][TX-C] 0.00-10.00 sec 336 MBytes 282 Mbits/sec receiver
[ 7][RX-C] 0.00-10.00 sec 285 MBytes 239 Mbits/sec sender
[ 7][RX-C] 0.00-10.00 sec 285 MBytes 239 Mbits/sec receiver

iperf Done.

%%%%%%%%%%%%%%%%%%%%%%

铭凡BD790i主板【网卡检测为04:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)】：

root@mf7945hx:~# iperf3 -c 192.168.1.167 --bidir --bind-dev enp4s0
Connecting to host 192.168.1.167, port 5201
[ 5] local 192.168.1.44 port 58118 connected to 192.168.1.167 port 5201
[ 7] local 192.168.1.44 port 58120 connected to 192.168.1.167 port 5201
[ ID][Role] Interval Transfer Bitrate Retr Cwnd
[ 5][TX-C] 0.00-1.00 sec 115 MBytes 962 Mbits/sec 0 742 KBytes
[ 7][RX-C] 0.00-1.00 sec 112 MBytes 936 Mbits/sec
[ 5][TX-C] 1.00-2.00 sec 112 MBytes 937 Mbits/sec 0 742 KBytes
[ 7][RX-C] 1.00-2.00 sec 112 MBytes 940 Mbits/sec
[ 5][TX-C] 2.00-3.00 sec 113 MBytes 949 Mbits/sec 0 839 KBytes
[ 7][RX-C] 2.00-3.00 sec 112 MBytes 940 Mbits/sec
[ 5][TX-C] 3.00-4.00 sec 111 MBytes 933 Mbits/sec 0 839 KBytes
[ 7][RX-C] 3.00-4.00 sec 112 MBytes 937 Mbits/sec
[ 5][TX-C] 4.00-5.00 sec 112 MBytes 942 Mbits/sec 0 839 KBytes
[ 7][RX-C] 4.00-5.00 sec 111 MBytes 934 Mbits/sec
[ 5][TX-C] 5.00-6.00 sec 112 MBytes 942 Mbits/sec 0 839 KBytes
[ 7][RX-C] 5.00-6.00 sec 111 MBytes 934 Mbits/sec
[ 5][TX-C] 6.00-7.00 sec 112 MBytes 942 Mbits/sec 0 839 KBytes
[ 7][RX-C] 6.00-7.00 sec 111 MBytes 934 Mbits/sec
[ 5][TX-C] 7.00-8.00 sec 112 MBytes 942 Mbits/sec 0 839 KBytes
[ 7][RX-C] 7.00-8.00 sec 111 MBytes 934 Mbits/sec
[ 5][TX-C] 8.00-9.00 sec 112 MBytes 941 Mbits/sec 0 839 KBytes
[ 7][RX-C] 8.00-9.00 sec 111 MBytes 934 Mbits/sec
[ 5][TX-C] 9.00-10.00 sec 111 MBytes 932 Mbits/sec 0 839 KBytes
[ 7][RX-C] 9.00-10.00 sec 111 MBytes 934 Mbits/sec

[ ID][Role] Interval Transfer Bitrate Retr
[ 5][TX-C] 0.00-10.00 sec 1.10 GBytes 942 Mbits/sec 0 sender
[ 5][TX-C] 0.00-10.00 sec 1.09 GBytes 940 Mbits/sec receiver
[ 7][RX-C] 0.00-10.00 sec 1.09 GBytes 938 Mbits/sec 176 sender
[ 7][RX-C] 0.00-10.00 sec 1.09 GBytes 936 Mbits/sec receiver

iperf Done.

root@mf7945hx:~# iperf3 -c 192.168.1.167 --bidir --bind-dev enp4s0 --sctp
Connecting to host 192.168.1.167, port 5201
[ 5] local 192.168.1.44 port 46869 connected to 192.168.1.167 port 5201
[ 7] local 192.168.1.44 port 38000 connected to 192.168.1.167 port 5201
[ ID][Role] Interval Transfer Bitrate
[ 5][TX-C] 0.00-1.00 sec 70.3 MBytes 589 Mbits/sec
[ 7][RX-C] 0.00-1.00 sec 70.2 MBytes 589 Mbits/sec
[ 5][TX-C] 1.00-2.00 sec 72.2 MBytes 606 Mbits/sec
[ 7][RX-C] 1.00-2.00 sec 73.4 MBytes 616 Mbits/sec
[ 5][TX-C] 2.00-3.00 sec 72.6 MBytes 609 Mbits/sec
[ 7][RX-C] 2.00-3.00 sec 79.0 MBytes 663 Mbits/sec
[ 5][TX-C] 3.00-4.00 sec 87.1 MBytes 731 Mbits/sec
[ 7][RX-C] 3.00-4.00 sec 16.9 MBytes 142 Mbits/sec
[ 5][TX-C] 4.00-5.00 sec 0.00 Bytes 0.00 bits/sec
[ 7][RX-C] 4.00-5.00 sec 0.00 Bytes 0.00 bits/sec
[ 5][TX-C] 5.00-6.00 sec 0.00 Bytes 0.00 bits/sec
[ 7][RX-C] 5.00-6.00 sec 0.00 Bytes 0.00 bits/sec
[ 5][TX-C] 6.00-7.00 sec 0.00 Bytes 0.00 bits/sec
[ 7][RX-C] 6.00-7.00 sec 0.00 Bytes 0.00 bits/sec
[ 5][TX-C] 7.00-8.00 sec 0.00 Bytes 0.00 bits/sec
[ 7][RX-C] 7.00-8.00 sec 0.00 Bytes 0.00 bits/sec
[ 5][TX-C] 8.00-9.00 sec 0.00 Bytes 0.00 bits/sec
[ 7][RX-C] 8.00-9.00 sec 0.00 Bytes 0.00 bits/sec
[ 5][TX-C] 9.00-10.00 sec 0.00 Bytes 0.00 bits/sec
[ 7][RX-C] 9.00-10.00 sec 0.00 Bytes 0.00 bits/sec

[ ID][Role] Interval Transfer Bitrate
[ 5][TX-C] 0.00-10.00 sec 302 MBytes 254 Mbits/sec sender
[ 5][TX-C] 0.00-10.00 sec 302 MBytes 253 Mbits/sec receiver
[ 7][RX-C] 0.00-10.00 sec 240 MBytes 201 Mbits/sec sender
[ 7][RX-C] 0.00-10.00 sec 240 MBytes 201 Mbits/sec receiver

iperf Done.

随便看看

随便看看测试 sctp 协议都在几秒后速度归零是wifi的影响。把wifi关闭后两台机器都没问题

3A6000:

root@debian13:~# iperf3 -c 192.168.1.44 --bidir --bind-dev enp2s0 --sctp
Connecting to host 192.168.1.44, port 5201
[ 5] local 192.168.1.40 port 36045 connected to 192.168.1.44 port 5201
[ 7] local 192.168.1.40 port 56662 connected to 192.168.1.44 port 5201
[ ID][Role] Interval Transfer Bitrate
[ 5][TX-C] 0.00-1.00 sec 86.1 MBytes 722 Mbits/sec
[ 7][RX-C] 0.00-1.00 sec 86.1 MBytes 721 Mbits/sec
[ 5][TX-C] 1.00-2.00 sec 94.6 MBytes 793 Mbits/sec
[ 7][RX-C] 1.00-2.00 sec 94.6 MBytes 793 Mbits/sec
[ 5][TX-C] 2.00-3.00 sec 98.4 MBytes 826 Mbits/sec
[ 7][RX-C] 2.00-3.00 sec 98.4 MBytes 826 Mbits/sec
[ 5][TX-C] 3.00-4.00 sec 98.1 MBytes 823 Mbits/sec
[ 7][RX-C] 3.00-4.00 sec 98.1 MBytes 823 Mbits/sec
[ 5][TX-C] 4.00-5.00 sec 96.0 MBytes 805 Mbits/sec
[ 7][RX-C] 4.00-5.00 sec 96.0 MBytes 805 Mbits/sec
[ 5][TX-C] 5.00-6.00 sec 95.5 MBytes 801 Mbits/sec
[ 7][RX-C] 5.00-6.00 sec 95.5 MBytes 801 Mbits/sec
[ 5][TX-C] 6.00-7.00 sec 94.7 MBytes 794 Mbits/sec
[ 7][RX-C] 6.00-7.00 sec 94.6 MBytes 794 Mbits/sec
[ 5][TX-C] 7.00-8.00 sec 95.1 MBytes 798 Mbits/sec
[ 7][RX-C] 7.00-8.00 sec 95.1 MBytes 798 Mbits/sec
[ 5][TX-C] 8.00-9.00 sec 88.6 MBytes 743 Mbits/sec
[ 7][RX-C] 8.00-9.00 sec 88.6 MBytes 743 Mbits/sec
[ 5][TX-C] 9.00-10.00 sec 85.8 MBytes 719 Mbits/sec
[ 7][RX-C] 9.00-10.00 sec 85.8 MBytes 719 Mbits/sec

[ ID][Role] Interval Transfer Bitrate
[ 5][TX-C] 0.00-10.00 sec 933 MBytes 782 Mbits/sec sender
[ 5][TX-C] 0.00-10.00 sec 933 MBytes 782 Mbits/sec receiver
[ 7][RX-C] 0.00-10.00 sec 933 MBytes 782 Mbits/sec sender
[ 7][RX-C] 0.00-10.00 sec 933 MBytes 782 Mbits/sec receiver

iperf Done.

%%%%%%%%%%%

AMD 7945HX:

root@mf7945hx:~# iperf3 -c 192.168.1.40 --bidir --bind-dev enp4s0 --sctp
Connecting to host 192.168.1.40, port 5201
[ 5] local 192.168.1.44 port 49972 connected to 192.168.1.40 port 5201
[ 7] local 192.168.1.44 port 45595 connected to 192.168.1.40 port 5201
[ ID][Role] Interval Transfer Bitrate
[ 5][TX-C] 0.00-1.00 sec 87.6 MBytes 734 Mbits/sec
[ 7][RX-C] 0.00-1.00 sec 87.5 MBytes 733 Mbits/sec
[ 5][TX-C] 1.00-2.00 sec 87.8 MBytes 736 Mbits/sec
[ 7][RX-C] 1.00-2.00 sec 87.8 MBytes 736 Mbits/sec
[ 5][TX-C] 2.00-3.00 sec 87.6 MBytes 735 Mbits/sec
[ 7][RX-C] 2.00-3.00 sec 87.5 MBytes 734 Mbits/sec
[ 5][TX-C] 3.00-4.00 sec 87.9 MBytes 738 Mbits/sec
[ 7][RX-C] 3.00-4.00 sec 87.9 MBytes 738 Mbits/sec
[ 5][TX-C] 4.00-5.00 sec 87.8 MBytes 736 Mbits/sec
[ 7][RX-C] 4.00-5.00 sec 87.7 MBytes 736 Mbits/sec
[ 5][TX-C] 5.00-6.00 sec 88.2 MBytes 740 Mbits/sec
[ 7][RX-C] 5.00-6.00 sec 88.1 MBytes 739 Mbits/sec
[ 5][TX-C] 6.00-7.00 sec 89.7 MBytes 752 Mbits/sec
[ 7][RX-C] 6.00-7.00 sec 90.3 MBytes 758 Mbits/sec
[ 5][TX-C] 7.00-8.00 sec 87.8 MBytes 736 Mbits/sec
[ 7][RX-C] 7.00-8.00 sec 87.8 MBytes 737 Mbits/sec
[ 5][TX-C] 8.00-9.00 sec 87.9 MBytes 737 Mbits/sec
[ 7][RX-C] 8.00-9.00 sec 87.8 MBytes 736 Mbits/sec
[ 5][TX-C] 9.00-10.00 sec 87.6 MBytes 735 Mbits/sec
[ 7][RX-C] 9.00-10.00 sec 87.6 MBytes 735 Mbits/sec

[ ID][Role] Interval Transfer Bitrate
[ 5][TX-C] 0.00-10.00 sec 880 MBytes 738 Mbits/sec sender
[ 5][TX-C] 0.00-10.00 sec 880 MBytes 738 Mbits/sec receiver
[ 7][RX-C] 0.00-10.00 sec 880 MBytes 738 Mbits/sec sender
[ 7][RX-C] 0.00-10.00 sec 880 MBytes 738 Mbits/sec receiver

iperf Done.

广高菜鸟科技

你的问题我已转发给中科云的龚老板了。

felixonmars

sctp 问题似乎和具体交换机/网络环境有关，虽然我只能用这台机器复现……

tcp 掉速问题在我这里不同环境都可以复现出来，不过不是第一次跑就可以复现，有时候要跑几次之后才行。

鄂ICP备2022017735号

此论坛为纯技术类论坛, 且为"独立运作，风险自控", 请用户在论坛中不要讨论无关话题, 以免影响论坛正常运行