MikroTik

We're an urban WISP with a dense mesh of wireless links and a router per building. I am concerned that, without paying attention, we have grown to 600+ routers and ~2550 routes in one OSPF domain. This network has a diverse mix of routers from CCR1036s down to RB750UPs. We're not having any OSPF problems at this time and I have plenty of other things to worry about, but I'd hate to hit some limit and have the whole thing blow up.

Does anyone have experience (positive or negative) with large flat OSPF networks based on MikroTik routers?
And, if you have had problems, what were the problems?

In a single ospf area must be not more than 100 routers I believe. I hope you using lsa type 3 between areas in order to save processors time to recalculate shortest path. With OSPF help you have to announce only tech nets. For large scale better use BGP.

In a single ospf area must be not more than 100 routers I believe. I hope you using lsa type 3 between areas in order to save processors time to recalculate shortest path. With OSPF help you have to announce only tech nets. For large scale better use BGP.

Well we have 600+ routers in our largest network and it is working and it is stable. Are you quoting a rule of thumb from the 1990s or do you have experience with a 100+ router network that had problems?

The only measurable issue I can see is the time for routes to propagate. For example if I add a route or I throw traffic in one part of the network onto a backup path, the route change can take 2-3 seconds to appear on a router that is far removed from where the change happened. But that's the only issue I can see today.

Do you use bfd protocol between peers?

Yes we use BFD, with MikroTik default timers.

I'm assuming BFD is entirely local between two peers and doesn't relate to stability of the whole network. In any event, we've never had occasion to look at other than BFD defaults.

In a WISP covering a very extensive area with 30 PPPoE servers, more than 300 PtMP and more than 2000 CPE we had problem with MPLS: sometimes MPLS forwarding table doesn't follow OSPF.
We decided to split the very big OSPF domain into several little ones using iBGP. Routing tables have diminished from almost 800 routes to 30.
Troubleshooting is now easier than before.

In a WISP covering a very extensive area with 30 PPPoE servers, more than 300 PtMP and more than 2000 CPE we had problem with MPLS: sometimes MPLS forwarding table doesn't follow OSPF.
We decided to split the very big OSPF domain into several little ones using iBGP. Routing tables have diminished from almost 800 routes to 30.
Troubleshooting is now easier than before.

Yes. First of all need to understand how big you need to create mpls cloud with ospf area 0.0.0.0, then you can delegate diff ospf areas with none Dijkstra's algorithm routing(lsa type 3). After it you can create bgp relationships between ospf loopbacks. With this design, the network will feel better.

Hi!! the craziest network that I've seen is about 700subs in a flat bridged/switched one... but we really recommend to everyone move into a routed network instead of a switched one, by using OSPF with BGP, you can configure OSPF as an IGP which can handle everything related to convergence, load balancing and loopbacks while iBGP can be used for managing the rest of your subnets. It's really a most effective way of building a WISP/ISP because it's scalable and easy to manage, once that you get the idea of how everything works.

Hi!! the craziest network that I've seen is about 700subs in a flat bridged/switched one... but we really recommend to everyone move into a routed network instead of a switched one, by using OSPF with BGP, you can configure OSPF as an IGP which can handle everything related to convergence, load balancing and loopbacks while iBGP can be used for managing the rest of your subnets. It's really a most effective way of building a WISP/ISP because it's scalable and easy to manage, once that you get the idea of how everything works.

That's true! I conform this solution. Besides of routing protocols for your distribution network, I recomend implenting BGP in PPPoE servers which connect with you central connnection node when you provide Public addressing to your customers.