Long time mikrotik user, but long hyatus from the fold in using it in anger; finally found availability on the CCR2004-1G-12S+2XS so I could get one on order along with a CRS328-24P-4S+RM POE switch to power my AP's. Going to work towards migrating to this over the next week or two to replace the buggy netgear l3 switches; and anemic single processor performance of the er-4.
I'm currently running a ubiquity edgerouter 4 (er-4) with dot1x (802.1x) authentication via using wpa_supplicant directly to the (Alcatel-Lucent G-010G-A) ONT (no ATT BGW210-700 bridged or tethered; just off in a box as backup). Current edgerouter OS doesn't include wpa_supplicant and supportability long term is questionable for a variety of reasons; not the least of which is ubiquity has completely abandoned the CLI in their unifi line.
(Also for background on why I'm primarily migrating, ubiquity has basically stopped development on their VyOS / Vyatta core fork due to a variety of reasons; and all of their current embedded hardware has a kernel related bug with a very poorly written ethernet packet routing offload driver causing out of order UDP packets. This in turn causes hiccups in nearly all video / audio realtime communications that don't rely on TCP. The only workaround is to limit the ethernet driver to 1 cpu core only; which neuters performance quite a bit).
I'm curious to see if anyone has dot1x ATT fiber auth setup on an CR2004 currently; or has attempted and or run into issues / success.
Previously I've run a CCR1009-7G-1C-1S+PC setup where the ONT wasn't authenticated on fiber and had great success in the past for years of solid operations.
Assuming nobody has attempted this yet; I'll post my trials and tribulations as I work through the testing efforts.
-
-
SatireWolf
just joined
- Posts: 16
- Joined:
Re: CCR2004-1G-12S+2XS - ATT Residential Fiber Termination via ONT
A discussion with interested individuals is occurring here.
-
-
SatireWolf
just joined
- Posts: 16
- Joined:
Re: CCR2004-1G-12S+2XS - ATT Residential Fiber Termination via ONT
Thanks. I had read through that post previously, just didn't see anyone attempting this with a CCR2004 yet, just the RB4011, CCR1009 and some other lower end unpowered gear.
That thread is actually why I pursued this path in the first place; now that there's a fully supportable dot1x client implementation in mikrotik router os.
I started a new thread for the CCR2004-1G-12S+2XS specific implementation.
That thread is actually why I pursued this path in the first place; now that there's a fully supportable dot1x client implementation in mikrotik router os.
I started a new thread for the CCR2004-1G-12S+2XS specific implementation.
-
-
SatireWolf
just joined
- Posts: 16
- Joined:
Re: CCR2004-1G-12S+2XS - ATT Residential Fiber Termination via ONT
I was finally able to get a working config going this last week and have had several days of uninterrupted networking bliss.
I did find two oddities I'm having to work around. Due to the lack of a dedicated switch chip a software bridge is necessary to decode the VLAN 0 packets and strip the VLAN header.
Additionally, on reboot, I've had to cycle the interface more than once to get dot1x / dhcp to sync up without one or both having issues. Most of the time on reboot dot1x comes up Authenticated, no server instead of Authenticated; which is a problem probably with the version of RouterOS I'm on I'm guessing since dot1x with vlan 0 support is brand spanking new.
Notes on this specific configuration, primary router address is 10.x.x.254 (replace your X's with your preferred subnets). My primary L3 router is on 10.x.x.1 which happens to be a CRS328-24P-4S+ r2 currently running the exact same Mikrotik RouterOS Version : v6.48beta48
sfp-sfpplus1 has a Mikrotik S+RJ10 version 2.16 connected directly to the Alcatel Fiber ONT for ATT
sfp-sfpplus10 has a S+DA0001 1m 10g passive coax SFP connected directly to the Mikrotik C328-24P-4S+ POE L3 switch.
I plan to use sfp-sfpplus11/12 to LAG two S+DA0001 DAC cables together eventually, just haven't gotten around to it yet; planning on changing that when I figure out how to get IPv6 working in this configuration like I had it provisioned on my edgerouter-4 previously.
DNS is set to use Cloudflare, and Cloudflare DNS over HTTPS. You'll need to load certs locally to enable verification of the HTTPS cert Cloudflare uses. Also, this config uses PIM for multicast, master being the L3 switch PIM router / IGMP snoop hostmaster.
The router and l3 list each other as tertiary DNS backups. NTP is configured to 10.X.X.10 which is a pi-hole with unbound, stubby, and ntpsec for NTP servers. Cloudflare has an ntpsec supporting NTP server now. Currently not using the pi-hole until I get 3 intel based SBC's imported from Taiwan to handle redundant DNS service. ATT doesn't have great connectivity to Quad9 DNS currently which is who I have setup for Pi-Hole (would rather not use Cloudflare for DNS after all their recent outages).
/etc/ntpsec/ntp.conf
I'm open to config suggestions. If anyone knows how to get ipv6 working with ATT properly on a config like this I'm all ears!!! Also, if there's a cron script to handle the dot1x/dhcp race condition we seem to have with 'Authenticated, no server' I'd love to know what that solution looks like!
Hope this helps someone else trying to set this up! Note I used the stickied forum thread to configure the dot1x certs that I pulled from a BGW210 with the backdoor firmware mod.
CCR2004 Config
I did find two oddities I'm having to work around. Due to the lack of a dedicated switch chip a software bridge is necessary to decode the VLAN 0 packets and strip the VLAN header.
Additionally, on reboot, I've had to cycle the interface more than once to get dot1x / dhcp to sync up without one or both having issues. Most of the time on reboot dot1x comes up Authenticated, no server instead of Authenticated; which is a problem probably with the version of RouterOS I'm on I'm guessing since dot1x with vlan 0 support is brand spanking new.
Notes on this specific configuration, primary router address is 10.x.x.254 (replace your X's with your preferred subnets). My primary L3 router is on 10.x.x.1 which happens to be a CRS328-24P-4S+ r2 currently running the exact same Mikrotik RouterOS Version : v6.48beta48
sfp-sfpplus1 has a Mikrotik S+RJ10 version 2.16 connected directly to the Alcatel Fiber ONT for ATT
sfp-sfpplus10 has a S+DA0001 1m 10g passive coax SFP connected directly to the Mikrotik C328-24P-4S+ POE L3 switch.
I plan to use sfp-sfpplus11/12 to LAG two S+DA0001 DAC cables together eventually, just haven't gotten around to it yet; planning on changing that when I figure out how to get IPv6 working in this configuration like I had it provisioned on my edgerouter-4 previously.
DNS is set to use Cloudflare, and Cloudflare DNS over HTTPS. You'll need to load certs locally to enable verification of the HTTPS cert Cloudflare uses. Also, this config uses PIM for multicast, master being the L3 switch PIM router / IGMP snoop hostmaster.
The router and l3 list each other as tertiary DNS backups. NTP is configured to 10.X.X.10 which is a pi-hole with unbound, stubby, and ntpsec for NTP servers. Cloudflare has an ntpsec supporting NTP server now. Currently not using the pi-hole until I get 3 intel based SBC's imported from Taiwan to handle redundant DNS service. ATT doesn't have great connectivity to Quad9 DNS currently which is who I have setup for Pi-Hole (would rather not use Cloudflare for DNS after all their recent outages).
Code: Select all
/tool fetch url=https://curl.haxx.se/ca/cacert.pem
/certificate import file-name=cacert.pem passphrase=""
/etc/ntpsec/ntp.conf
Code: Select all
server time.cloudflare.com:1234 nts
server time.cloudflare.com iburst
server time1.google.com iburst
server time2.google.com iburst
server time3.google.com iburst
server time4.google.com iburst
I'm open to config suggestions. If anyone knows how to get ipv6 working with ATT properly on a config like this I'm all ears!!! Also, if there's a cron script to handle the dot1x/dhcp race condition we seem to have with 'Authenticated, no server' I'd love to know what that solution looks like!
Hope this helps someone else trying to set this up! Note I used the stickied forum thread to configure the dot1x certs that I pulled from a BGW210 with the backdoor firmware mod.
CCR2004 Config
Code: Select all
# nov/06/2020 12:40:43 by RouterOS 6.48beta48
# software id = SomeSecretNumberHere
#
# model = CCR2004-1G-12S+2XS
# serial number = SomeSecretNumberHere
/interface bridge
add admin-mac=MyATTRouterMacAddressHere auto-mac=no igmp-snooping=yes name=bridge_wan0 protocol-mode=none vlan-filtering=yes
/interface ethernet
set [ find default-name=sfp-sfpplus1 ] advertise=10M-half,10M-full,100M-half,100M-full,1000M-half,1000M-full,2500M-full,5000M-full,10000M-full loop-protect=off mac-address=MyATTRouterMacAddressHere
set [ find default-name=sfp-sfpplus10 ] advertise=10M-half,10M-full,100M-half,100M-full,1000M-half,1000M-full,2500M-full,5000M-full,10000M-full
/interface list
add name=WAN
add name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=dhcp ranges=10.X.X.101-10.X.X.201
add name=pool0 ranges=10.X.X.101-10.X.X.249
/ip dhcp-server
add add-arp=yes address-pool=pool0 always-broadcast=yes disabled=no interface=sfp-sfpplus10 lease-time=1w1d name=dhcp0 relay=10.X.X.1
/lora servers
add address=eu.mikrotik.thethings.industries down-port=1700 name=TTN-EU up-port=1700
add address=us.mikrotik.thethings.industries down-port=1700 name=TTN-US up-port=1700
/tool user-manager customer
set admin access=own-routers,own-users,own-profiles,own-limits,config-payment-gw
/interface bridge port
add bridge=bridge_wan0 ingress-filtering=yes interface=sfp-sfpplus1
/ip neighbor discovery-settings
set discover-interface-list=!dynamic
/interface dot1x client
add certificate=Client_PartialClientAddressHere-SomeSerialNumberHere.pem_0 eap-methods=eap-tls identity=MyATTRouterMacAddressHere interface=sfp-sfpplus1
/interface list member
add interface=sfp-sfpplus1 list=WAN
add interface=sfp-sfpplus10 list=LAN
add interface=ether1 list=LAN
/ip address
add address=192.168.88.1/24 interface=ether1 network=192.168.88.0
add address=10.X.X.254/24 interface=sfp-sfpplus10 network=10.X.X.0
/ip dhcp-client
add disabled=no interface=bridge_wan0 use-peer-ntp=no
/ip dhcp-server network
add address=10.X.X.0/24 dns-server=10.X.X.254,10.X.X.1 domain=int.ninsystems.com gateway=10.X.X.254 netmask=24 ntp-server=10.X.X.10
/ip dns
set allow-remote-requests=yes cache-size=131072KiB max-concurrent-queries=2048 max-concurrent-tcp-sessions=1024 query-server-timeout=1s servers=\
104.16.248.249,104.16.249.249,1.1.1.1,1.0.0.1,10.X.X.1 use-doh-server=https://cloudflare-dns.com/dns-query verify-doh-cert=yes
/ip firewall filter
add action=accept chain=input comment="Allow established related" connection-state=established,related
add action=accept chain=input comment="Allow LAN" in-interface-list=LAN
add action=accept chain=input comment="Allow Ping" protocol=icmp
add action=drop chain=input comment="Drop all other input"
add action=accept chain=forward comment="Allow established related" connection-state=established,related
add action=accept chain=forward comment="Allow LAN" connection-state=new in-interface-list=LAN
add action=accept chain=forward comment="Allow port forwards" connection-nat-state=dstnat in-interface-list=WAN
add action=drop chain=forward comment="Drop all other forward"
/ip firewall nat
add action=masquerade chain=srcnat out-interface=bridge_wan0
/ip upnp interfaces
add interface=sfp-sfpplus1 type=external
add interface=sfp-sfpplus10 type=internal
/ipv6 dhcp-client
add add-default-route=yes interface=bridge_wan0 pool-name=dhcp_wan0_ipv6_0 pool-prefix-length=60 request=prefix use-peer-dns=no
/ipv6 nd
set [ find default=yes ] interface=sfp-sfpplus10
/routing pim interface
add igmp-version=IGMPv3 interface=sfp-sfpplus10
/system clock
set time-zone-name=UTC
/system identity
set name=fw0-100-city-state-country
/system ntp client
set enabled=yes primary-ntp=10.X.X.10
/system package update
set channel=testing
/tool graphing interface
add allow-address=10.X.X.0/24
/tool user-manager database
set db-path=user-manager