MikroTik

Posted: **Sun Nov 17, 2013 11:03 pm**

When I read: "fully functional Layer 3 switch" and "full wire speed smart switch" I assumed that I could use the CS125 to link, at full wire speed, via 802.3ad to another device (perhaps an RB951G-2HnD), assuming the linked device supports 802.3ad. I have yet to find a way of doing this without bridging which sends all packets to RouterOS (CPU) for processing effectively nullifying the increased throughput benefits of 802.3ad. I don't intend on using the CRS125 for DHCP, Firewall, NAT or anything other than switching at this point in time. This is for a home setup and I have no intended use VLANs, unless what I want to accomplish can't be done without VLANs. Any suggestions are more than welcome.

I've reset to a blank config on the CRS125 and configured things as seen below, where ether1 is my uplink from a Ubiquiti EdgeRouter, and then the rest of the ports are all switched. In other words ether2-ether24 are slaves to ether1, and ether1 and wlan1 are bridged.

Flags: X - disabled, R - running, S - slave 
 #    NAME          MTU MAC-ADDRESS       ARP        MASTER-PORT      SWITCH     
 0 RS ether1       1500 D4:CA:6D:FA:6F:1E enabled    none             switch1    
 1  S ether2       1500 D4:CA:6D:FA:6F:1F enabled    ether1           switch1    
 2  S ether3       1500 D4:CA:6D:FA:6F:20 enabled    ether1           switch1    
 3  S ether4       1500 D4:CA:6D:FA:6F:21 enabled    ether1           switch1    
 4 RS ether5       1500 D4:CA:6D:FA:6F:22 enabled    ether1           switch1    
 5 RS ether6       1500 D4:CA:6D:FA:6F:23 enabled    ether1           switch1    
 6 RS ether7       1500 D4:CA:6D:FA:6F:24 enabled    ether1           switch1    
 7 RS ether8       1500 D4:CA:6D:FA:6F:25 enabled    ether1           switch1    
 8 RS ether9       1500 D4:CA:6D:FA:6F:26 enabled    ether1           switch1    
 9 RS ether10      1500 D4:CA:6D:FA:6F:27 enabled    ether1           switch1    
10 RS ether11      1500 D4:CA:6D:FA:6F:28 enabled    ether1           switch1    
11 RS ether12      1500 D4:CA:6D:FA:6F:29 enabled    ether1           switch1    
12  S ether13      1500 D4:CA:6D:FA:6F:2A enabled    ether1           switch1    
13  S ether14      1500 D4:CA:6D:FA:6F:2B enabled    ether1           switch1    
14  S ether15      1500 D4:CA:6D:FA:6F:2C enabled    ether1           switch1    
15  S ether16      1500 D4:CA:6D:FA:6F:2D enabled    ether1           switch1    
16  S ether17      1500 D4:CA:6D:FA:6F:2E enabled    ether1           switch1    
17  S ether18      1500 D4:CA:6D:FA:6F:2F enabled    ether1           switch1    
18 RS ether19      1500 D4:CA:6D:FA:6F:30 enabled    ether1           switch1    
19  S ether20      1500 D4:CA:6D:FA:6F:31 enabled    ether1           switch1    
20 RS ether21      1500 D4:CA:6D:FA:6F:32 enabled    ether1           switch1    
21 XS ether22      1500 D4:CA:6D:FA:6F:33 enabled    ether1           switch1    
22 XS ether23      1500 D4:CA:6D:FA:6F:34 enabled    ether1           switch1    
23 XS ether24      1500 D4:CA:6D:FA:6F:35 enabled    ether1           switch1    
24    sfp1         1500 D4:CA:6D:FA:6F:36 enabled    none             switch1

The issues I'm struggling with are that I cannot bond ports while they are slaves, but I want all ports to be switched except the main uplink from the EdgeRouter (ether1) and the wireless (wlan1) which are bridged. I'm mainly wanting to keep everything on the same broadcast domain instead of segmenting it with VLANs. I am totally open to what suggestions you might have. Please see the diagram I've drawn to see what I'm trying to accomplish, although I don't know if it is plausible. Also feel free to educate me on anything I seem to be doing wrong or could be doing better a different way.

Network layout.png

Posted: **Tue Nov 19, 2013 5:14 pm**

This device is very new and most of us have not had time to play with it yet. I've ordered two for a project I'm working on but I won't have access to them for maybe a week or more.

Posted: **Tue Nov 19, 2013 6:22 pm**

Since ethernets are already configured as switch slaves, you cannot use them in bonding anymore.

If you want bonding remove them from switch, bond them together and bridge switch master port with bonding interface.

Posted: **Wed Nov 20, 2013 3:06 am**

Since ethernets are already configured as switch slaves, you cannot use them in bonding anymore.

If you want bonding remove them from switch, bond them together and bridge switch master port with bonding interface.

Perhaps I'm missing something here. Doesn't that defeat the purpose of having a wirespeed switch that's capable of 802.3ad? Won't all packets get processed by the CPU then? This would defeat the purpose of using 802.3ad because now the CPU has to process everything. At best the CPU can probably handle 1Gbps and then what happens to the remainder of the bandwidth I need?

Am I wrong here?

Posted: **Wed Nov 20, 2013 3:22 am**

Since ethernets are already configured as switch slaves, you cannot use them in bonding anymore.

If you want bonding remove them from switch, bond them together and bridge switch master port with bonding interface.
Perhaps I'm missing something here. Doesn't that defeat the purpose of having a wirespeed switch that's capable of 802.3ad? Won't all packets get processed by the CPU then? This would defeat the purpose of using 802.3ad because now the CPU has to process everything. At best the CPU can probably handle 1Gbps and then what happens to the remainder of the bandwidth I need?

Am I wrong here?

Curious about this myself.

Posted: **Wed Nov 20, 2013 8:48 am**

802.3ad implementaion using CRS switch-chip is not ready, yet.
The first available features from CRS switch-chip will be:
* VLANs
* Q-in-Q VLANs
* Mirroring
* QoS
* Queuing
* Port isolation

Posted: **Wed Nov 20, 2013 10:24 pm**

Oh well at least I'm not crazy, although that would have been nice to know a week ago

Thank you for the fast response!

Is there any ETA on when 802.3ad will be working as it should with the CRS switch chip? I assume that this is a RouterOS wide issue because it is the same implementation on the RB951G-2HnD, with the exception of the switch chip being different.

Posted: **Thu Nov 21, 2013 11:04 am**

@ anyone from Mikrotik

Are we able to ask please for you to add notes as you add basic implementations / improve implementations / think they are finished for all these switch chip features (and future new features on MikroTik products.

This would probably be best suited to the Pre Release download pages so we can see the things you have added in or have improved so we can test them in our labs (or our live networks if we are very gutsy) and provide additional feedback.

Would also be great on these pre release pages if a new build fixes any regressions / other glitches from pre release builds as its helpful for me when looking over as sometimes the new build times on the site don't actually have any new changes in them, I don't mind at all having short things or oddities that I may not experience as I run the pre release builds in my home network, on my test builds etc and only occasionally try them in the field so it would also help me out with focussing testing on things that have moved since the last build so I can do extra testing for production.

Regards
Alexander

p.s. I have submitted this as a bug in the sense of "enhancement" bugs in open source software

Posted: **Thu Nov 28, 2013 11:24 am**

I have same problem:
http://forum.mikrotik.com/viewtopic.php ... 34&p=0&e=0

Posted: **Wed Dec 11, 2013 10:38 pm**

Mikrotik, FYI - I am having issues with the CRS I have been testing. The issue started after upgrading to routerOS 6.7

I have lost ethernet links on all interfaces 3 times. Each time, the router will not get a link on any interface. Once, this happened immediately after upgrading software. Once it happened immediately after disabling an interface. If, while logged in through serial, I type /interface print, it just hangs on that command.

When this happens, I login to the router through the serial port and reset. When it resets, it takes a long time and shows these errors:

[admin@MikroTik] /system> reset
Dangerous! Reset anyway? [y/N]:
y
system configuration will be reset

Rebooting...
Stopping services...
failed to stop racoon: std failure: timeout (13)
failed to stop ppp: std failure: timeout (13)
ar7100_wdt_disable
failed to stop route: std failure: timeout (13)

Posted: **Thu Dec 12, 2013 10:17 am**

pkelly1603,
Write to MikroTik support if you need RouterOS v6.8rc1 for upgrading your CRS125.

Posted: **Tue Jan 07, 2014 2:40 am**

I have the same problem on two brand new CRS125's.

I even factory reset both and started over, but after simple configuration, I get spurious hangs.

I noticed an OOPS when having the console port connected during bootup.

Also running v6.7.

Posted: **Tue Jan 07, 2014 9:27 am**

The issue exists for v6.5, v6.6 and v6.7 on a clean upgrade on a brand new CRS125.

Remove pretty much all of the default configuration, change the IP to ether2-master, reboot.

The following OOPS is displayed. Any attempt to display or change configuration apart from a router reset, results in a hard hang.

Oops[#1]:
Cpu 0
$ 0 : 00000000 7fb26170 00000002 000001f0
$ 4 : c6f88b80 00000006 00000006 00000004
$ 8 : 00000006 c75e9380 00000000 00000000
$12 : 7fb26168 00000200 00000002 00000000
$16 : c75e9380 c7611e28 c7611e28 c6f88b80
$20 : 00000000 00000000 004dacb4 20010000
$24 : 00000000 c6f41d98
$28 : c7610000 c7611d60 00000001 c6e6a08c
Hi : 11cf4d00
: 0000000b
pc : c6f41de0 0xc6f41de0 [music_switch@0xc6f40000]

Cause : 00800008
BadVA : 000001f0
PrId : 0001974c (MIPS 74Kc)
Process net (pid: 245, threadinfo=c7610000, task=c7456f70, tls=77e06770)
Stack : 004dacb4 c0142628 00190400 00000047 c6f88000 c75c1240 c6f88000 00000001
00000000 c6f88000 00000000 c7456f70 c7cba700 c7456f70 c6f88000 c7611e28
c7611e28 00000000 7fb26160 00000000 004dacb4 20010000 00000001 c6e6a08c
00000000 00000000 004d614c c7f78d48 00000006 00000000 c7cba6d0 000089f4
c6f88000 c02b549c c7cba6d0 c01459a8 00008946 c7611e28 c0422440 c02b33c8
...
Call Trace:
[<c6f41de0>] 0xc6f41de0 [music_switch@0xc6f40000]

Posted: **Fri Jan 24, 2014 8:10 pm**

Mikrotik, are you working on a fix?

Also, will there be any multicast filtering options in the switch chips? Or IGMP handled by the switch?

Posted: **Sun Feb 09, 2014 11:09 pm**

pkelly1603,
Write to MikroTik support if you need RouterOS v6.8rc1 for upgrading your CRS125.

I'm still experiencing the same problem as others after upgrading to this version. The router works, then all of the Ethernet ports stop working. Sometimes it will work for a week and other times for a day. I have eliminated all of the configuration using it only as a dumb switch as well as running it with the default configuration. Results are always the same; it works, then it doesn't. Doing a factory reset always resolves the problem, however the switch is useless because it is only a matter of time before it stops working.

Posted: **Fri Feb 28, 2014 4:26 pm**

Mikrotik, are you working on a fix?

Also, will there be any multicast filtering options in the switch chips? Or IGMP handled by the switch?

?

Posted: **Sun Mar 02, 2014 6:31 am**

I deployed a CRS125 and I'm afraid I'm going to have to pull it out. It's behaving extremely erratically. Not all ports will negotiate. A second IP address assigned to the switch group does not respond to pings. I've reset the software once and it's still acting unlike a RouterOS device should.

@Mikrotik what's up with this?

Posted: **Thu Apr 03, 2014 4:06 pm**

Any update?

Posted: **Wed Apr 23, 2014 11:15 pm**

Posted: **Sun May 04, 2014 6:55 pm**

Hello guys,

Any update on the hardware LACP/port aggregation on the CRS125? Any approximates/timeframes?

I was thinking of it as an inexpensive alternative to other wirespeed L2 GigE switches and potentially as an all-in-one hardware L2/software L3 swouter but it is disappointingly not fit yet.

Posted: **Mon May 19, 2014 11:10 pm**

Are any of the above issues in this thread resolved with RouterOS 6.12?

What's new in 6.12 (2014-Apr-14 09:27):

*) many fixes for CRS managed switch functionality -
particularly improved VLAN support, port isolation, defaults;
*) added trunk support for CRS switches;
*) added policing support for CRS switches;

We spent $30,000 on our last batch of Dlink switches. I wish Mikrotik could make theirs work because if it did we would be buying them.

Posted: **Mon Jun 02, 2014 11:03 pm**

I tested the new RouterOS and the switch is much more stable now. Unfortunately it is missing some of the features I want. Is there documentation on the hardware capabilities of the switch chip (so we know what these switches will potentially be able to do when they grow up)?

IGMP snooping or some way to filter multicast and IGMP requests. There is a multicast FDB menu but it doesn't make much sense to me and is very lightly documented.

Posted: **Tue Jun 03, 2014 11:58 am**

I tested the new RouterOS and the switch is much more stable now. Unfortunately it is missing some of the features I want. Is there documentation on the hardware capabilities of the switch chip (so we know what these switches will potentially be able to do when they grow up)?

IGMP snooping or some way to filter multicast and IGMP requests. There is a multicast FDB menu but it doesn't make much sense to me and is very lightly documented.

Cloud Router Switch Multicast FDB entries set to which ports multicast traffic is sent if its destination address (MAC or IPv4) matches the specified one. In IVL mode VLAN id is also taken into account.

Posted: **Thu Jun 12, 2014 8:20 am**

After the port flapping fiasco I put all my mikrotik gear on the shelf. I'm assuming that we still aren't close to an implementation of 802.3ad on the CRS. I haven't seen note of 802.3ad in any of the RouterOS release notes. I am fairly disappointed that Mikrotik advertised hardware capable of features that have never actually been implemented at the software level. It's been more than 7 months after the CRS was released. Looks like I'm going to buy something else that will do what i need it to.

Posted: **Thu Jun 12, 2014 6:22 pm**

bridging which sends all packets to RouterOS (CPU) for processing effectively nullifying the increased throughput benefits of 802.3ad

I am net to Mikrotik, I have a CRS226, and I have placed it into bridge mode. I did this because I didn't need any layer3, no routing, DHCP, LVANS etc. I assumed bridged mode simple disabled the router features and disabling the routing functionality would be a performance boost, not a performance killer. Have I made an error?

The only functionality I need at this point is switching.

Posted: **Fri Jun 13, 2014 10:48 pm**

I tested the new RouterOS and the switch is much more stable now. Unfortunately it is missing some of the features I want. Is there documentation on the hardware capabilities of the switch chip (so we know what these switches will potentially be able to do when they grow up)?

IGMP snooping or some way to filter multicast and IGMP requests. There is a multicast FDB menu but it doesn't make much sense to me and is very lightly documented.
Cloud Router Switch Multicast FDB entries set to which ports multicast traffic is sent if its destination address (MAC or IPv4) matches the specified one. In IVL mode VLAN id is also taken into account.

I tried to use the Multicast FDB and could not make it work. Here is my configuration.
- VLAN 100 has 3 ports (ether1,2,3) and receives the multicast
- VLAN 200 has ether10 and I am trying to forward the multicast to this.

/interface ethernet
set [ find default-name=ether2 ] master-port=ether1
set [ find default-name=ether3 ] master-port=ether1
/interface ethernet switch
set drop-if-invalid-or-src-port-not-member-of-vlan-on-ports=ether1,ether2,ether3,ether10
/interface ethernet switch egress-vlan-tag
add tagged-ports=switch1-cpu vlan-id=100
add tagged-ports=switch1-cpu vlan-id=200
/interface ethernet switch ingress-vlan-translation
add customer-vlan-format=untagged-or-tagged new-customer-vid=100 ports=ether1,ether2,ether3 \
sa-learning=yes service-vlan-format=untagged-or-tagged
add customer-vid=0 customer-vlan-format=untagged-or-tagged new-customer-vid=200 ports=ether10 \
service-vlan-format=untagged-or-tagged
/interface ethernet switch multicast-fdb
add address=239.1.1.2 ports=ether10 vlan-id=0
/interface ethernet switch vlan
add ports=ether1,ether2,ether3,switch1-cpu vlan-id=100
add ports=ether10,switch1-cpu vlan-id=200

I also tried setting the VLAN ID on the multicast FDB entry to 200. Mutlicast traffic is currently flooded to all interfaces on VLAN 100 so I know it is being received. I also tried turning off "drop-if-invalid-or-src-port-not-member-of-vlan-on-ports". Any ideas, becs?

Posted: **Mon Jun 16, 2014 11:05 am**

Any ideas, becs?

First of all, multicast forwarding occurs between switched ports which means ether10 also has to be slave port of ether1.
Secondly, ingress vlan translation rule assigns frames from ether1,ether2,ether3 to VLAN 100 therefore MFDB entry should have VLAN id 100 to match this multicast traffic.
And VLAN 100 in VLAN table also needs to be allowed on ether10, otherwise this multicast traffic is considered as invalid.

Posted: **Mon Jun 16, 2014 5:20 pm**

Multicast FBD is working correctly for me, it does filter which ports are allowed to receive multicast. Not dynamic and not as useful as IGMP Snooping, but could be enough for some usage scenarios:

/interface ethernet switch multicast-fdb
add address=239.0.1.1 ports=ether5 vlan-id=1
add address=239.0.1.2 ports=ether5 vlan-id=1
add address=239.0.1.3 ports=ether5 vlan-id=1

Meaning that if any port in VLAN1 sends multicast to any of those addresses traffic will be sent out only to ether5. You can put more ports in each rule, of course. I don't really understand how to use MAC address in these rules, because I don't know if they refer to source or destination of the multicast stream.

I tested the new RouterOS and the switch is much more stable now. Unfortunately it is missing some of the features I want. Is there documentation on the hardware capabilities of the switch chip (so we know what these switches will potentially be able to do when they grow up)?

IGMP snooping or some way to filter multicast and IGMP requests. There is a multicast FDB menu but it doesn't make much sense to me and is very lightly documented.
Cloud Router Switch Multicast FDB entries set to which ports multicast traffic is sent if its destination address (MAC or IPv4) matches the specified one. In IVL mode VLAN id is also taken into account.

Posted: **Tue Jun 17, 2014 4:04 pm**

First of all, multicast forwarding occurs between switched ports which means ether10 also has to be slave port of ether1.
Secondly, ingress vlan translation rule assigns frames from ether1,ether2,ether3 to VLAN 100 therefore MFDB entry should have VLAN id 100 to match this multicast traffic.
And VLAN 100 in VLAN table also needs to be allowed on ether10, otherwise this multicast traffic is considered as invalid.

Thanks that worked for me. Is it possible though to route a multicast from one VLAN to another using only switch hardware features? And is it possible to turn on a 'default' multicast filter, where all multicasts are not forwarded unless manually added to the FDB?

Posted: **Tue Jun 17, 2014 5:15 pm**

Don't have access to my CSR right now, but try to put something like 224.0.0.0/8 in Multicast FBD and assign to a port you control (say, main router, a server...). That would filter out, but I don't know if its possible to put ranges in FBD. Anyway, IGMP snooping would solve your problem a lot more efficiently...

Regarding forwarding multicast, I don't think its possible. Better use IGMP proxy, it will use few cpu resources and you can even use a different device for the task if you have a lot of mcast traffic. Oh well, maybe this switch chip also has hardware IGMP proxy capabilities but Mikrotik hasn't implemented it yet

PS: thanks for karma!

Posted: **Wed Jun 18, 2014 4:44 pm**

Thanks for the suggestions.

1. The multicast FDB does not allow ranges or networks (only single IPs or MAC addresses)
2. Yes, IGMP snooping would be very helpful. I believe it would also be very helpful to get an ACL working with this too so that IGMP requests can be allowed or blocked along with snooping.
3. I will give IGMP proxy a try. I am afraid I might have CPU issues, it's worth a try though

Posted: **Wed Jun 18, 2014 7:54 pm**

I tried IGMP proxy, unfortunately I need more than one upstream interface which is not supported by IGMP proxy. Also, the CRS CPU is too slow for PIM.

Posted: **Mon Jul 28, 2014 3:58 pm**

Mikrotik, will either of these features be implemented in a future version?

IGMP snooping
PIM routing (supported in hardware)

Posted: **Thu Sep 25, 2014 4:14 pm**

Mikrotik, please post your roadmap for future features of the CRS series switches.

MikroTik

CRS125 Configuration Help

CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help

Re: CRS125 Configuration Help