CCR2004-1G-2XS-PCIe strange behavior in operation.

Ejik · Wed Jan 15, 2025 12:04 pm

Good day.
We recently bought CCR2004-1G-2XS-PCIe and installed it in a server with Ubuntu Linux 22.04 kernel 6.8.0-51-generic.
Current Firmware 7.16.2
Package 7.16.2

ifconfig eth1
gives the following data
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet x.x.x.x netmask x.x.x.x broadcast x.x.x.x
ether 78:9a:18:b6:70:f0 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 1 collisions 0

No statistics on packets and bytes.

also ethool gives

ethtool -l eth1
netlink error: Operation not supported
ethtool -g eth1
netlink error: Operation not supported

In /proc/interrupts it is clear that on eth1 there is a load only on 1 CPU and there is no way to increase the number of queues, etc.
As on network cards of another vendor, for example.

ethtool -g eth3
Ring parameters for eth3:
Pre-set maximums:
RX: 4096
RX Mini: n/a
RX Jumbo: n/a
TX: 4096
Current hardware settings:
RX: 256
RX Mini: n/a
RX Jumbo: n/a
TX: 256
ethtool -l eth3
Channel parameters for eth3:
Pre-set maximums:
RX: n/a
TX: n/a
Other: 1
Combined: 8
Current hardware settings:
RX: n/a
TX: n/a
Other: 1
Combined: 8

Working in Passthrough mode or in router mode does not change the situation with counters and settings.

What can be done to be able to manage network card parameters and view statistics?

pts0 · Wed Jan 15, 2025 7:06 pm

Hi,
how did you setup the card ? is in standard config as in https://cdn.mikrotik.com/web-assets/rb_ ... 2155_m.png ?
Did you let card bootup before you ubuntu bootet, try to restart networking services.

i have it running on debian and it use driver atl1c

ethtool -i ensfp1
driver: atl1c
version: 6.8.12-5-pve
firmware-version:
expansion-rom-version:
bus-info: 0000:01:00.0
supports-statistics: no
supports-test: no
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: no

statistics are not show

ensfp1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP mode DEFAULT group default qlen 1000
    link/ether 78:9a:18:8e:77:69 brd ff:ff:ff:ff:ff:ff
    RX:  bytes packets errors dropped  missed   mcast
             0       0      0  103630       0       0
    RX errors:  length    crc   frame    fifo overrun
                     0      0       0       0       0
    TX:  bytes packets errors dropped carrier collsns
             0       0      0       0       5       0
    TX errors: aborted   fifo  window heartbt transns
                     0      0       0       0       6

use the winbox, there you get all statistics.

Ejik · Thu Jan 16, 2025 9:09 am

Good day.

Yes, the default settings.
Set a password to access the device and update to the current version.
The server takes a long time to load, so re-initialize is not necessary.
For testing, I did it on all 4 ports with the driver removed

echo "1" > /sys/bus/pci/devices/0000\:03\:00.0/remove
sleep 2
echo "1" > /sys/bus/pci/rescan

no difference.

Adding a network card to bond also does not work.

Jan 15 11:55:56 testsrv kernel: [866.310239] bond0: (slave eth1): dev_set_mac_address on slave failed! ALB mode requires that the base driver support setting the hw address also when the network device's interface is open

There are counters through winbox and in snmp.
When working in router mode, the counters are also not displayed.
When under load, the card falls off after a while.
The logs say something about watchdog and transmission timeouts.

Jan 15 15:01:11 testsrv kernel: [ 1034.859476] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 8: transmit queue 2 timed out 7283 ms
Jan 15 15:01:18 testsrv kernel: [ 1041.515633] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 25: transmit queue 1 timed out 739840 ms
Jan 15 15:01:24 testsrv kernel: [ 1047.659716] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 24: transmit queue 1 timed out 745984 ms
Jan 15 15:01:31 testsrv kernel: [ 1054.827851] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 12: transmit queue 1 timed out 753152 ms
Jan 15 15:01:38 testsrv kernel: [ 1061.484005] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 26: transmit queue 1 timed out 5632 ms
Jan 15 15:01:44 testsrv kernel: [ 1067.628139] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 25: transmit queue 2 timed out 765952 ms
Jan 15 15:01:51 testsrv kernel: [ 1074.796260] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 25: transmit queue 2 timed out 773120 ms
Jan 15 15:01:58 testsrv kernel: [ 1081.452401] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 24: transmit queue 2 timed out 779776 ms
Jan 15 15:02:04 testsrv kernel: [ 1087.596518] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 28: transmit queue 1 timed out 785920 ms
Jan 15 15:02:11 testsrv kernel: [ 1094.764631] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 31: transmit queue 2 timed out 793088 ms
Jan 15 15:02:18 testsrv kernel: [ 1101.420759] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 24: transmit queue 0 timed out 5632 ms
Jan 15 15:02:24 testsrv kernel: [ 1107.564871] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 25: transmit queue 2 timed out 805888 ms
Jan 15 15:02:31 testsrv kernel: [ 1114.732998] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 28: transmit queue 0 timed out 5812 ms
Jan 15 15:02:38 testsrv kernel: [ 1121.389149] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 10: transmit queue 1 timed out 819712 ms
Jan 15 15:02:44 testsrv kernel: [ 1127.533245] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 8: transmit queue 1 timed out 825856 ms
Jan 15 15:02:50 testsrv kernel: [ 1133.677375] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 24: transmit queue 1 timed out 832000 ms
Jan 15 15:02:57 testsrv kernel: [ 1140.333521] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 31: transmit queue 1 timed out 5391 ms
Jan 15 15:03:03 testsrv kernel: [ 1146.477632] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 25: transmit queue 0 timed out 5120 ms
Jan 15 15:03:09 testsrv kernel: [ 1152.621734] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 27: transmit queue 1 timed out 850944 ms
Jan 15 15:03:16 testsrv kernel: [ 1159.789889] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 13: transmit queue 1 timed out 858112 ms
Jan 15 15:03:23 testsrv kernel: [ 1166.445999] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 28: transmit queue 1 timed out 864768 ms
Jan 15 15:03:29 testsrv kernel: [ 1172.590133] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 29: transmit queue 0 timed out 5120 ms
Jan 15 15:03:36 testsrv kernel: [ 1179.758271] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 12: transmit queue 1 timed out 878080 ms
Jan 15 15:03:43 testsrv kernel: [ 1186.414373] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 12: transmit queue 1 timed out 884736 ms
Jan 15 15:03:49 testsrv kernel: [ 1192.558499] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 26: transmit queue 0 timed out 5120 ms
Jan 15 15:03:55 testsrv kernel: [ 1198.702622] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 26: transmit queue 2 timed out 897024 ms
Jan 15 15:04:02 testsrv kernel: [ 1205.358766] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 26: transmit queue 1 timed out 5461 ms
Jan 15 15:04:08 testsrv kernel: [ 1211.502901] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 24: transmit queue 1 timed out 909824 ms
Jan 15 15:04:14 testsrv kernel: [ 1217.646970] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 24: transmit queue 1 timed out 915968 ms
Jan 15 15:04:21 testsrv kernel: [ 1224.815121] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 25: transmit queue 1 timed out 923136 ms
Jan 15 15:04:28 testsrv kernel: [ 1231.471264] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 24: transmit queue 0 timed out 5585 ms
Jan 15 15:04:34 testsrv kernel: [ 1237.615369] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 24: transmit queue 0 timed out 5120 ms
Jan 15 15:04:41 testsrv kernel: [ 1244.783519] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 30: transmit queue 0 timed out 6144 ms
Jan 15 15:04:48 testsrv kernel: [ 1251.439650] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 14: transmit queue 0 timed out 5632 ms
Jan 15 15:04:54 testsrv kernel: [ 1257.583769] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 27: transmit queue 0 timed out 5120 ms
Jan 15 15:05:01 testsrv kernel: [ 1264.751891] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 14: transmit queue 1 timed out 963072 ms
Jan 15 15:05:08 testsrv kernel: [ 1271.409031] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 14: transmit queue 0 timed out 5633 ms
Jan 15 15:05:14 testsrv kernel: [ 1277.552137] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 24: transmit queue 0 timed out 5120 ms
Jan 15 15:05:21 testsrv kernel: [ 1284.720261] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 28: transmit queue 0 timed out 6143 ms
Jan 15 15:13:45 testsrv kernel: [  247.405323] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 25: transmit queue 1 timed out 4294913024 ms
Jan 15 15:13:51 testsrv kernel: [  253.549446] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 26: transmit queue 2 timed out 4294919168 ms
Jan 15 15:13:59 testsrv kernel: [  260.717602] atl1c 0000:83:00.1 eth1: NETDEV WATCHDOG: CPU: 12: transmit queue 1 timed out 4294926336 ms

Ejik · Fri Jan 17, 2025 7:53 am

Good day.
Upgrade mikrotik to 7.17
Current Firmware 7.17
Package 7.17

The situation with the counters has not changed.
Working in router mode began to work well, without the network card failing.
Nothing has changed with ethtool.
The transition to kernel 5.15.0-130 did not change the behavior of the network card.

CCR2004-1G-2XS-PCIe strange behavior in operation.

CCR2004-1G-2XS-PCIe strange behavior in operation.

Re: CCR2004-1G-2XS-PCIe strange behavior in operation.

Re: CCR2004-1G-2XS-PCIe strange behavior in operation.

Re: CCR2004-1G-2XS-PCIe strange behavior in operation.