Community discussions

MikroTik App
  4. RouterOS
  5. Beginner Basics

CCR2216-1G-12XS-2XQ QSPF28 connection  [SOLVED]

Post Reply
 
Litdex
just joined
Topic Author
Posts: 5
Joined: Sun May 31, 2020 8:21 pm

CCR2216-1G-12XS-2XQ QSPF28 connection

Tue Mar 14, 2023 9:48 am

Good day,
The problem:
Get 100G connectivity between CCR2216-1G-12XS-2XQ and DELL EMS S4148F-ON (Profile 1).

Equipment:
Code: Select all
MIKROTIK
# jan/02/1970 00:23:15 by RouterOS 7.8
# software id = BWS2-FISS
#
# model = CCR2216-1G-12XS-2XQ
# serial number = HE308SWXXXX
================================
DELL
Dell EMC Networking OS10 Enterprise
Copyright (c) 1999-2020 by Dell Inc. All Rights Reserved.
OS Version: 10.5.0.4
Build Version: 10.5.0.4.638
Build Time: 2020-01-30T21:08:56+0000
System Type: S4148F-ON
Architecture: x86_64
Couple DAC QSFP28 100G cables 3 and 0.5 meters.

Road so far:
On dell side by default I have interface 100G-1x. No luck to connect to Mikrotik using different DAC QSFP28 100G cables. Played with different MTU on Mikrotik side up to 9570, auto-negotiation=no.
On dell side I can breakout the interface so I played with than too:
Code: Select all
interface breakout 1/1/29 map 100g-1x 50g-2x  25g-4x  10g-4x
breakout to 50g-2x no point, CCR2216 do not support it. So I tried 25g-4x and 10g-4x.
No luck with breakout 25g-4x, tried different DAC QSFP28 cables, configurations, MTU.
I get connectivity with breakout on dell side 10g-4x only (with pretty much any MTU on Mikrotik side):
Code: Select all
/interface ethernet
set [ find default-name=qsfp28-1-1 ] auto-negotiation=no speed=10Gbps
set [ find default-name=qsfp28-1-2 ] auto-negotiation=no speed=10Gbps
set [ find default-name=qsfp28-1-3 ] auto-negotiation=no speed=10Gbps
set [ find default-name=qsfp28-1-4 ] auto-negotiation=no speed=10Gbps
The questions:
1. Can I somehow to combine qsfp28-1-1,2,3,4 to one 100G interface like it is on dell switch? Maybe then it will work..
2. How actually I should make this connection?
I searched the forums for answers without success. I would appreciate any ideas or advice. Spent a lot of time with this problem.

Thanks for your time.
Top
 
emunt6
Member Candidate
Member Candidate
Posts: 114
Joined: Fri Feb 02, 2018 7:00 pm

Re: CCR2216-1G-12XS-2XQ QSPF28 connection

Sat Mar 18, 2023 2:00 am

Hi!

When you connected the qsfp-to-qsfp cable, did you to set the speed manually on both sides ? ( DELL/Mikrotik: 1x100G duplex, autoneg off, )
( Static settings maybe work to get link )

Cable test:
-When you connect the 2xQSFP ports together ( loopback ) locally on Mikrotik, did you get any link?
-When you connect the 2xQSFP ports together ( loopback ) locally on DELL, did you get any link?
-What kind of DAC do you using? ( active or passive )
Top
 
Litdex
just joined
Topic Author
Posts: 5
Joined: Sun May 31, 2020 8:21 pm

Re: CCR2216-1G-12XS-2XQ QSPF28 connection

Sat Mar 18, 2023 10:42 am

Hi!

When you connected the qsfp-to-qsfp cable, did you to set the speed manually on both sides ? ( DELL/Mikrotik: 1x100G duplex, autoneg off, )
( Static settings maybe work to get link )

Cable test:
-When you connect the 2xQSFP ports together ( loopback ) locally on Mikrotik, did you get any link?
-When you connect the 2xQSFP ports together ( loopback ) locally on DELL, did you get any link?
-What kind of DAC do you using? ( active or passive )

Thanks for your replay! I already thought that no one would even answer anything..
That's what I did on Dell switch for 100G and late for 25Gx4:
Code: Select all
DELL-SW2# configure terminal
DELL-SW2(config)# interface breakout  1/1/29 map 100g-1x
DELL-SW2(config)# interface ethernet 1/1/29
DELL-SW2(conf-if-eth1/1/29)# show configuration
!
interface ethernet1/1/29
 no shutdown
 switchport access vlan 1
 flowcontrol receive on
DELL-SW2(conf-if-eth1/1/29)# negotiation off
DELL-SW2(conf-if-eth1/1/29)# show configuration
!
interface ethernet1/1/29
 no shutdown
 switchport access vlan 1
 negotiation off
 flowcontrol receive on
DELL-SW2(conf-if-eth1/1/29)# exit
DELL-SW2(config)# interface breakout 1/1/29 map 25g-4x
DELL-SW2(config)# interface ethernet 1/1/29:1
DELL-SW2(conf-if-eth1/1/29:1)# show configuration
!
interface ethernet1/1/29:1
 no shutdown
 switchport access vlan 1
 flowcontrol receive on
DELL-SW2(conf-if-eth1/1/29:1)# negotiation off
DELL-SW2(conf-if-eth1/1/29:1)# exit
DELL-SW2(config)# interface ethernet 1/1/29:2
DELL-SW2(conf-if-eth1/1/29:2)# negotiation off
DELL-SW2(conf-if-eth1/1/29:2)# exit
DELL-SW2(config)# interface ethernet 1/1/29:3
DELL-SW2(conf-if-eth1/1/29:3)# negotiation off
DELL-SW2(conf-if-eth1/1/29:3)# exit
DELL-SW2(config)# interface ethernet 1/1/29:4
DELL-SW2(conf-if-eth1/1/29:4)# negotiation off
DELL-SW2(conf-if-eth1/1/29:4)# show configuration
!
interface ethernet1/1/29:4
 no shutdown
 switchport access vlan 1
 negotiation off
 flowcontrol receive on

On Mikrotik side I just gonna show the print screen:
Image

this is on all qsfp28-1-x interfaces for 100G.
Later I changed it to 25G when I did "interface breakout 1/1/29 map 25g-4x".
As you see no effect at all.

And It works only when I do "interface breakout 1/1/29 map 10g-4x" on Dell and on Mikrotik I set 10G as I mention before:
Image

Regarding to DAC. One is Dell EMC DAC-QSFP28-100G-3M NDAAFJ-D102 03CC35 example https://www.ebay.com/itm/334727534821 and it is passive. The second should be the same just 0.5M but I can't check it until Monday. I will let you know on Monday.
Top
 
Litdex
just joined
Topic Author
Posts: 5
Joined: Sun May 31, 2020 8:21 pm

Re: CCR2216-1G-12XS-2XQ QSPF28 connection

Mon Mar 20, 2023 12:44 pm

The second one is not Dell DAC https://www.fs.com/de-en/products/104215.html 0.5m (2ft) 100G QSFP28 Passive Direct Attach Copper Twinax Cable for FS Switches
Top
 
Litdex
just joined
Topic Author
Posts: 5
Joined: Sun May 31, 2020 8:21 pm

Re: CCR2216-1G-12XS-2XQ QSPF28 connection  [SOLVED]

Mon Mar 20, 2023 2:25 pm

Finally I did it!
What you need to get 100G:
On dell switch disable negotiation
Code: Select all
DELL-SW2(conf-if-eth1/1/25)# show configuration
!
interface ethernet1/1/25
 no shutdown
 switchport access vlan 1
 negotiation off
 flowcontrol receive on

on MikroTik router (in my case) to set:
Code: Select all
/interface ethernet set [ find default-name=qsfp28-2-1 ] auto-negotiation=no speed=100Gbps  fec-mode=fec91

The issue was Forward Error Correction in Mikrotik. By default it is auto but in Dell switch by default it is off for breakout map 25g-4x and fec91 for breakout map 100g-1x. For same reason auto didn't help to make connection at all. I faced that kind of problem for the first time.
When I set fec-mode=fec91 on interface I immediately get my 100G.

I hope it will help some one. That cost me almost two weeks...
Top
 
ericsooter
Member Candidate
Member Candidate
Posts: 285
Joined: Mon Mar 07, 2005 6:16 pm
Location: Oklahoma USA

Re: CCR2216-1G-12XS-2XQ QSPF28 connection

Thu Jun 22, 2023 6:17 pm

Thanks for the information. Linking to a Fiberstore switch with 100G QSFP28's and also had to use fec-mode=fec91. "Auto" was a fail.
Top
 
User avatar
simcity
just joined
Posts: 3
Joined: Mon Mar 08, 2021 9:41 pm

Re: CCR2216-1G-12XS-2XQ QSPF28 connection

Tue Sep 10, 2024 4:24 pm

Same issue here linking to a Cisco C9300 switch with 100G uplink.

The only possible way to connect was to change the FEC mode from 'auto' to 'fec91'. Changing the connection speed from auto to a manual setting made no difference. Tested using both a Mikrotik brand 100G DAC cable and an fs.com branded 100G DAC cable.
Top
Post Reply
  4. RouterOS
  5. Beginner Basics