Community discussions

MikroTik App
 
globalmedia
newbie
Topic Author
Posts: 30
Joined: Mon Mar 20, 2023 11:09 pm

Regex Format in Conditional DNS forwarding

Sun Jun 23, 2024 8:39 pm

I have created a this REGEXP below:
^(?![\w]*[-][\d]{2})(.*[\.]?ad\.example\.com)$
But doesn't work on Mikrotik Layer7 RegExp. Can anyone help me to convert it to mikrotik?

Here is a little run down on how to do this, the only thing left to do is to check what janisk says but im pretty sure this handles it decently enough.
/ip firewall layer7-protocol
add name=testdns regexp=lantest.mindlesstux.com

/ip firewall nat
add action=dst-nat chain=dstnat disabled=no dst-address=4.2.2.2 dst-port=53 layer7-protocol=testdns protocol=udp to-addresses=8.8.8.8 to-ports=53
add action=dst-nat chain=dstnat disabled=no dst-address=4.2.2.2 dst-port=53 layer7-protocol=testdns protocol=tcp to-addresses=8.8.8.8 to-ports=53
1. Change the regex to match your domain.
2. Change 4.2.2.2 to be your RB DNS server
3. Change 8.8.8.8 to be your AD DNS server

*edit*
/me applies this little craft for his work domain on his home RB
 
hapoo
Frequent Visitor
Frequent Visitor
Posts: 60
Joined: Wed Apr 24, 2019 1:35 am

Re: Conditional DNS forwarding

Sun Jun 23, 2024 8:48 pm

Why would you use this in Layer7 when dns already allows for regex and forwarding requests
 
globalmedia
newbie
Topic Author
Posts: 30
Joined: Mon Mar 20, 2023 11:09 pm

Re: Conditional DNS forwarding

Sun Jun 23, 2024 9:35 pm

Because I want to forward DNS requests to different servers based on hostname request.

Using this rules:
/ip firewall layer7-protocol
add name=testdns regexp=lantest.mindlesstux.com

/ip firewall nat
add action=dst-nat chain=dstnat disabled=no dst-address=4.2.2.2 dst-port=53 layer7-protocol=testdns protocol=udp to-addresses=8.8.8.8 to-ports=53
add action=dst-nat chain=dstnat disabled=no dst-address=4.2.2.2 dst-port=53 layer7-protocol=testdns protocol=tcp to-addresses=8.8.8.8 to-ports=53
Apparently you can do it for a host in regexp. But using the expression that I post above I can filter requests based on my needs.

Why would you use this in Layer7 when dns already allows for regex and forwarding requests
 
globalmedia
newbie
Topic Author
Posts: 30
Joined: Mon Mar 20, 2023 11:09 pm

Re: Conditional DNS forwarding

Sun Jun 23, 2024 9:50 pm

Sorry, I just see that is possible to add regexp directly to DNS static. But even doing that, it's not working:
/ip/dns/static> add regexp="^(?![\\w]*[-][\\d]{2})(.*[\\.]?ad\\.localdomain)$" forward-to=10.10.10.10
failure: name or regexp required
RegExp is not working correctly for me.
Because I want to forward DNS requests to different servers based on hostname request.

Using this rules:
/ip firewall layer7-protocol
add name=testdns regexp=lantest.mindlesstux.com

/ip firewall nat
add action=dst-nat chain=dstnat disabled=no dst-address=4.2.2.2 dst-port=53 layer7-protocol=testdns protocol=udp to-addresses=8.8.8.8 to-ports=53
add action=dst-nat chain=dstnat disabled=no dst-address=4.2.2.2 dst-port=53 layer7-protocol=testdns protocol=tcp to-addresses=8.8.8.8 to-ports=53
Apparently you can do it for a host in regexp. But using the expression that I post above I can filter requests based on my needs.

Why would you use this in Layer7 when dns already allows for regex and forwarding requests
 
hapoo
Frequent Visitor
Frequent Visitor
Posts: 60
Joined: Wed Apr 24, 2019 1:35 am

Re: Conditional DNS forwarding

Sun Jun 23, 2024 9:51 pm

Maybe I misunderstand what you are trying to do, but everything you said can be done by dns.

/ip dns static add forward-to=8.8.8.8 match-subdomain=yes name=example.com type=FWD
 
globalmedia
newbie
Topic Author
Posts: 30
Joined: Mon Mar 20, 2023 11:09 pm

Re: Conditional DNS forwarding

Sun Jun 23, 2024 10:07 pm

Hello,

I did it, look:
> /ip dns static add regexp="^(?![\\w]*[-][\\d]{2})(.*[\\.]?ad\\.localdomain)$" match-subdomain=yes type=FWD forward-to=10.10.10.10             
failure: name or regexp required
My regexp doesn't work. Don't know why. I have checked it several times in https://regex101.com/.
Maybe I misunderstand what you are trying to do, but everything you said can be done by dns.

/ip dns static add forward-to=8.8.8.8 match-subdomain=yes name=example.com type=FWD
 
hapoo
Frequent Visitor
Frequent Visitor
Posts: 60
Joined: Wed Apr 24, 2019 1:35 am

Re: Conditional DNS forwarding

Sun Jun 23, 2024 11:25 pm

Can you explain in plain language what you are trying to match?
 
User avatar
tangent
Forum Guru
Forum Guru
Posts: 1672
Joined: Thu Jul 01, 2021 3:15 pm
Contact:

Re: Conditional DNS forwarding

Mon Jun 24, 2024 5:49 am

/ip dns static add regexp="^(?![\\w]*[-][\\d]{2})(.*[\\.]?ad\\.localdomain)$"…
failure: name or regexp required

That isn't a "POSIX basic regular expression" (BRE) that this setting is documented as taking. It's vaguely PCRE style, though with odd variations like with the double-escape of things like "\d". It might be a valid POSIX extended regex (ERE), but I can't be bothered to validate it for you given that RouterOS doesn't take ERE, leaving me little incentive to bother.

I have checked it several times in https://regex101.com/.

The 8 "flavors" of RE offered by that site include neither POSIX BRE nor ERE. You're using the wrong tool to convince yourself that this should work.

Test with something that speaks BRE natively instead, something like GNU grep, which assumes BRE when not given the -E flag. A good guide to the differences is Linux's regex(7) man page.

If you're using BSD grep — as on modern macOS — you might need to give -G for full BRE compatibility.

Always know which flavor your RE processor expects. The 10 considered here aren't even all there is. I'd count Vim's RE format as an eleventh distinct flavor, for example.
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 4467
Joined: Sun May 01, 2016 7:12 pm
Location: California
Contact:

Re: Regex Format in Conditional DNS forwarding

Mon Jun 24, 2024 8:04 am

@tanget's right, RouterOS is closer to the C-runtime's more limited subset. But I'm not sure your doing anything tricky in the regex...

You can run a simple test on RouterOS CLI to check a regex too:
:put ("aaa"~"[a]{3}")      
# true
:put ("aaa"~"[a]{4}") 
# false

But often what is more problematic is RouterOS's string interpolation rules - since regex's can have variables (not in /ip/dns/static, but generally, regardless string rules still apply). So the final $" is at least one problem, since that need to be escaped for RouterOS since $ is reserved for variable substitution, like \$.

RouterOS will process the string BEFORE regex evaluation, so stuff like " and \ also need to be escaped, \" or \\. The "?" is a special character in RouterOS v6, so it need to be escaped - but NOT in V7.
 
globalmedia
newbie
Topic Author
Posts: 30
Joined: Mon Mar 20, 2023 11:09 pm

Re: Regex Format in Conditional DNS forwarding

Mon Jun 24, 2024 8:44 pm

:put doesn't work for me. How do you debug scripts placed inside system / scripts? Every :put inside the script just is not being displayed.

I am using v7.
@tanget's right, RouterOS is closer to the C-runtime's more limited subset. But I'm not sure your doing anything tricky in the regex...

You can run a simple test on RouterOS CLI to check a regex too:
:put ("aaa"~"[a]{3}")      
# true
:put ("aaa"~"[a]{4}") 
# false

But often what is more problematic is RouterOS's string interpolation rules - since regex's can have variables (not in /ip/dns/static, but generally, regardless string rules still apply). So the final $" is at least one problem, since that need to be escaped for RouterOS since $ is reserved for variable substitution, like \$.

RouterOS will process the string BEFORE regex evaluation, so stuff like " and \ also need to be escaped, \" or \\. The "?" is a special character in RouterOS v6, so it need to be escaped - but NOT in V7.
 
globalmedia
newbie
Topic Author
Posts: 30
Joined: Mon Mar 20, 2023 11:09 pm

Re: Regex Format in Conditional DNS forwarding

Mon Jun 24, 2024 8:46 pm

Could you help me to rewrite this regexp ^(?![\\w]*[-][\\d]{2})(.*[\\.]?ad\\.localdomain)$ to work correctly in RouteOS V7?

@tanget's right, RouterOS is closer to the C-runtime's more limited subset. But I'm not sure your doing anything tricky in the regex...

You can run a simple test on RouterOS CLI to check a regex too:
:put ("aaa"~"[a]{3}")      
# true
:put ("aaa"~"[a]{4}") 
# false

But often what is more problematic is RouterOS's string interpolation rules - since regex's can have variables (not in /ip/dns/static, but generally, regardless string rules still apply). So the final $" is at least one problem, since that need to be escaped for RouterOS since $ is reserved for variable substitution, like \$.

RouterOS will process the string BEFORE regex evaluation, so stuff like " and \ also need to be escaped, \" or \\. The "?" is a special character in RouterOS v6, so it need to be escaped - but NOT in V7.
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 4467
Joined: Sun May 01, 2016 7:12 pm
Location: California
Contact:

Re: Regex Format in Conditional DNS forwarding

Mon Jun 24, 2024 9:56 pm

If using :put with your regex does not work to match true/false correct, it ain't going to work as a /system/script. As I said the $ is WRONG and needs escaping. A :put at the CLI would catch that.

You've never said what it should match, so hard to know what's right/wrong (and the ?! may not be allowed). Assuming the $ becomes a \$ - you can use the :put ("matchtest"~"[m][a][t][c][h].*") like you're using regex101 to test it.
:put ("test-01.ad.localdomain"~"^(?![\\w]*[-][\\d]{2})(.*[\\.]?ad\\.localdomain)\$")
# false
 :put ("matchtest"~"[m][a][t][c][h].*")
# true
:put ("matchtest"~"^[t][e][s][t].*") 
# false
But yeah in a /system/script, you need to use /log/info "msg", instead of :put to "see" output from /system/script or schedule.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 12652
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Regex Format in Conditional DNS forwarding

Tue Jun 25, 2024 1:30 pm

Simply RouterOS for RegEx use only POSIX standard without metadata (Character classes) like [:digit:] (stay for [0-9] ) or \d (again is equal to [0-9] ) from other languages
https://en.wikibooks.org/wiki/Regular_E ... xpressions

So... no \w (but can be replaced with [a-zA-Z0-9_]), no \d (but can be replaced with [0-9]), no ?! (completely unsupported the negative lookahead), etc., etc., etc.
 
User avatar
eworm
Forum Guru
Forum Guru
Posts: 1092
Joined: Wed Oct 22, 2014 9:23 am
Location: Oberhausen, Germany
Contact:

Re: Regex Format in Conditional DNS forwarding

Tue Jun 25, 2024 1:33 pm

So... no \w, no \d, no ?!, etc., etc., etc.
I am pretty sure I have code that uses "\w"...

Edit... Scratch that... It was "\b". Confusing these two all the time. 😝
 
User avatar
eworm
Forum Guru
Forum Guru
Posts: 1092
Joined: Wed Oct 22, 2014 9:23 am
Location: Oberhausen, Germany
Contact:

Re: Regex Format in Conditional DNS forwarding

Tue Jun 25, 2024 1:39 pm

So... no \w, no \d, no ?!, etc., etc., etc.
I am pretty sure I have code that uses "\w"...

Edit... Scratch that... It was "\b". Confusing these two all the time. 😝
Still... Looks like "\w" does work:
> :put ("abc" ~ "^\\w\\w\\w\$")    
true
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 12652
Joined: Tue Feb 25, 2014 12:49 pm
Location: Italy
Contact:

Re: Regex Format in Conditional DNS forwarding

Tue Jun 25, 2024 2:04 pm

Yes, the obsolete help from MikroTik must be updated...

\w and \W work on both v6 and v7 (from what version? on 6.48.7 work),
\d and \D do NOT work on v6 (tested on 6.48.7) but work on v7 (tested on 7.15.1),

(?!) is not supported from both:
v7] > :put ("test0"~"test(?!\\d)")       
false

v7] > :put ("testX"~"test(?!\\d)") 
false
# must be true, ?! is unsupported

# but can be writed on this way, for same result...
v7] > :put ("testX"~"test\\D") 
true

Probably the wrong RegEx provided from the OP can be replaced with working one, if the OP provide some examples for what must match and what not....


Notice: Some section, like DNS, layer7, or script, apparently have different rules for RegEx syntax............
 
globalmedia
newbie
Topic Author
Posts: 30
Joined: Mon Mar 20, 2023 11:09 pm

Re: Regex Format in Conditional DNS forwarding

Tue Jun 25, 2024 2:47 pm

Thank you for helping me.

Sorry for that. I want to match anything [\.]?ad.localdomain (with or without dot), including subdomains, except for subdomain where you have word-XX (where XX is a numeric value).

Using https://regex101.com/ you can easy debug the regex. With your tips now I will try to debug correctly the tests.
If using :put with your regex does not work to match true/false correct, it ain't going to work as a /system/script. As I said the $ is WRONG and needs escaping. A :put at the CLI would catch that.

You've never said what it should match, so hard to know what's right/wrong (and the ?! may not be allowed). Assuming the $ becomes a \$ - you can use the :put ("matchtest"~"[m][a][t][c][h].*") like you're using regex101 to test it.
:put ("test-01.ad.localdomain"~"^(?![\\w]*[-][\\d]{2})(.*[\\.]?ad\\.localdomain)\$")
# false
 :put ("matchtest"~"[m][a][t][c][h].*")
# true
:put ("matchtest"~"^[t][e][s][t].*") 
# false
But yeah in a /system/script, you need to use /log/info "msg", instead of :put to "see" output from /system/script or schedule.
 
globalmedia
newbie
Topic Author
Posts: 30
Joined: Mon Mar 20, 2023 11:09 pm

Re: Regex Format in Conditional DNS forwarding

Tue Jun 25, 2024 2:59 pm

I have tried different ways:
 > :put ("test.ad.localdomain"~"^(?![a-zA-z0-9]*[-][0-9]{2})(.*[\\.]?ad\\.localdomain)\$")   
false
 > :put ("test.ad.localdomain"~"^([\\W]*[-][\\D]{2})(.*[\\.]?ad\\.localdomain)\$")   
false
 > :put ("test.ad.localdomain"~"^([\\W]*[-][\\D]{2})(\\.*[\\.]?ad\\.localdomain)\$")
false
Using this :put will help me to try to debug it... I will keep trying.
Yes, the obsolete help from MikroTik must be updated...

\w and \W work on both v6 and v7 (from what version? on 6.48.7 work),
\d and \D do NOT work on v6 (tested on 6.48.7) but work on v7 (tested on 7.15.1),

(?!) is not supported from both:
v7] > :put ("test0"~"test(?!\\d)")       
false

v7] > :put ("testX"~"test(?!\\d)") 
false
# must be true, ?! is unsupported

# but can be writed on this way, for same result...
v7] > :put ("testX"~"test\\D") 
true

Probably the wrong RegEx provided from the OP can be replaced with working one, if the OP provide some examples for what must match and what not....


Notice: Some section, like DNS, layer7, or script, apparently have different rules for RegEx syntax............
 
globalmedia
newbie
Topic Author
Posts: 30
Joined: Mon Mar 20, 2023 11:09 pm

Re: Regex Format in Conditional DNS forwarding

Tue Jun 25, 2024 3:33 pm

Ok!

Now I am matching, but the first group I want to negative it. ?! just doesn't work.
> :put ("test-01.ad.localdomain"~"^([a-zA-Z0-9]*[-][0-9]{2})(\\.*[\\.]?ad\\.localdomain)\$")
true
> :put ("test.ad.localdomain"~"^(?![a-zA-Z0-9]*[-][0-9]{2})(\\.*[\\.]?ad\\.localdomain)\$")
false
I'm almost giving up... LOL!
 
globalmedia
newbie
Topic Author
Posts: 30
Joined: Mon Mar 20, 2023 11:09 pm

Re: Regex Format in Conditional DNS forwarding

Tue Jun 25, 2024 4:48 pm

Thank you for everybody that participate to this thread.

I finally did it:

For who want to do something similar, follow the working regex:
:put ("test-01.ad.localdomain"~"([^a-zA-Z0-9]*[^-][^0-9]{2})(\\.*[\\.]?ad\\.localdomain)\$")
:put ("test.ad.localdomain"~"([^a-zA-Z0-9]*[^-][^0-9]{2})(\\.*[\\.]?ad\\.localdomain)\$")
My DHCP update my static DNS to the standard [machine|print]-XX where XX is a numeric value, so I have to resolve DNS using RB. For all others records ad.localdomain, I want to check the AD DNS.

It's working almost perfectly now.

I don't know why, but the ad.localdomain should match the rule too, but it's not.
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 4467
Joined: Sun May 01, 2016 7:12 pm
Location: California
Contact:

Re: Regex Format in Conditional DNS forwarding

Tue Jun 25, 2024 6:16 pm

Good to hear. You do kinda have to test them, since RouterOS has its own flavor. I kinda knew the ?! was likely not going work here.

But \\D \\w etc stuff I'm not sure about... and seemingly the 3rd problem here (1st being the lack of escaped "\$", 2nd the "?!" does not work) since OP using the [A-z] style...

Still... Looks like "\w" does work:
> :put ("abc" ~ "^\\w\\w\\w\$")    
true
While those work "standalone", RouterOS does not seem to process them in a character class. Maybe this right in some flavor, but these should be "true" IMO:
:put ("a"~"[\\D]")   
# false
:put ("a"~"[\\w]") 
# false
Matching without the char-class [], and just char-class-escape, they do work:
:put ("a"~"\\w")  
true
:put ("a"~"\\D") 
true

I generally use the [A-z] or [^A-z] style - since I think it's more readable. But I would have thought "[\\w]" would work...
 
User avatar
infabo
Forum Guru
Forum Guru
Posts: 1495
Joined: Thu Nov 12, 2020 12:07 pm

Re: Regex Format in Conditional DNS forwarding

Tue Jun 25, 2024 6:42 pm

I already pulled my hair out because of this limited POSIX regex implemention of ROS. In my case, make it very hard to impossible to write a regex for FWD for any subdomain except not matching 2 explicit 3rd Level Domains. Easy with negative lookahead, but Mikrotik likes us to suffer.
Last edited by infabo on Tue Jun 25, 2024 6:43 pm, edited 1 time in total.
 
globalmedia
newbie
Topic Author
Posts: 30
Joined: Mon Mar 20, 2023 11:09 pm

Re: Regex Format in Conditional DNS forwarding

Tue Jun 25, 2024 6:42 pm

Do you think is it a bug in RB implementation?


([^a-zA-Z0-9]*[^-][^0-9]{2})(\\.*[\\.]?ad\\.localdomain\$)

I'm exhausted from testing different possibilities. Can you see the error of not being able to validate just the domain?

:put ("ad.localdomain"~"([^a-zA-Z0-9]*[^-][^0-9]{2})(\\.*[\\.]?ad\\.localdomain\$)")
false

The above rule works in others testers that I made, just RB doesn't recognize it.

But it's working with: test.ad.localdomain and test-01.ad.localdomain


Good to hear. You do kinda have to test them, since RouterOS has its own flavor. I kinda knew the ?! was likely not going work here.

But \\D \\w etc stuff I'm not sure about... and seemingly the 3rd problem here (1st being the lack of escaped "\$", 2nd the "?!" does not work) since OP using the [A-z] style...

Still... Looks like "\w" does work:
> :put ("abc" ~ "^\\w\\w\\w\$")    
true
While those work "standalone", RouterOS does not seem to process them in a character class. Maybe this right in some flavor, but these should be "true" IMO:
:put ("a"~"[\\D]")   
# false
:put ("a"~"[\\w]") 
# false
Matching without the char-class [], and just char-class-escape, they do work:
:put ("a"~"\\w")  
true
:put ("a"~"\\D") 
true

I generally use the [A-z] or [^A-z] style - since I think it's more readable. But I would have thought "[\\w]" would work...
 
globalmedia
newbie
Topic Author
Posts: 30
Joined: Mon Mar 20, 2023 11:09 pm

Re: Regex Format in Conditional DNS forwarding

Wed Jun 26, 2024 5:30 am

This is my first time using MK regex... I think I finally solve this puzzle.

As I have a script updating dns static on dhcp leasing, I need to make a small change. Previously I has DNS static using this domain: WORD-XX.ad.localdomain. The machine/printer name still the same, for example: SALES-01, SALES-02, etc., but I was made a change in the domain. Now all my lan computers will following this new standard: WORD-XX.LAN.AD.LOCALDOMAIN.

So I have changed the FWD regex to: (\\.*[^lan|^wlan]\\.ad\\.localdomain\$|^ad\\.localdomain\$)

Important: to debug, I have to use \\ on terminal... but on regex you have to remove ALL \\ and \ to get it working.

So, just use: .*[^lan|^wlan].ad.localdomain$|^ad.localdomain$


Any request to ad.localdomain or subdomains.ad.localdomain will be forwarded to AD DNS, except records with lan and wlan in the host, that in my case, represents clients using lan (cabled and wifi) that must be forwarded to MK.

I also implemented a DNS check in netwatch that, in case of timeout of my AD, I just disable the static DNS regex.

Apparently, now everything is running as a sharm.
I already pulled my hair out because of this limited POSIX regex implemention of ROS. In my case, make it very hard to impossible to write a regex for FWD for any subdomain except not matching 2 explicit 3rd Level Domains. Easy with negative lookahead, but Mikrotik likes us to suffer.
 
User avatar
Amm0
Forum Guru
Forum Guru
Posts: 4467
Joined: Sun May 01, 2016 7:12 pm
Location: California
Contact:

Re: Regex Format in Conditional DNS forwarding

Wed Jun 26, 2024 6:20 am

So, just use: .*[^lan|^wlan].ad.localdomain$|^ad.localdomain$
Well, that does make it easy and likely okay. But "." still means any character to the regex. While improbable, lan1ad.localdomain would match the domain parts. Not say it's critical, but \\. is there for a reason, since you want match the literal ".".
:put ("a1com"~"a\\.com") 
# false
:put ("a1com"~"a.com")  
# true

Now the side-effect of [\\w] not working... is this does work to match the literal . dot:
:put ("a.com"~"a[.]com") 
# true
:put ("a1com"~"a[.]com") 
# false
To the "ripping your hair out" commentary, apparently inside the [character class], the . is literal.. which might be right, IDK. But that avoid needing escaping things between RouterOS and some other regex parser, like regex101.com.