Community discussions

MikroTik App
 
dg3feh
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 68
Joined: Mon Jan 30, 2017 10:52 am

Exclude domains from DNS cache!?

Thu Jul 13, 2017 1:08 pm

Hello!

I have quite often regsitration losts on my VoIP. I figured out that the provider is changing all the time the IP Address of the VoIP Server. Thus I need to exclude the Provider Domain from the DNS cache. How can I do this? I need to use the mikrotik as DNS server due to the fact, that the DNS Servers are pushed dynamicaly by the Provider.

BR Holger
 
pe1chl
Forum Guru
Forum Guru
Posts: 10542
Joined: Mon Jun 08, 2015 12:09 pm

Re: Exclude domains from DNS cache!?

Thu Jul 13, 2017 2:33 pm

When your provider changes the IP address frequently he should set a low TTL value on the DNS records,
so the MikroTik will expire the entry from the cache quickly and no issue occurs.

When the provider is clueless and does not want/know to do this, you can set a low cache max TTL in your
DNS service. Unfortunately, only global for all domains. However, in todays internet it does not make much
sense to do long DNS caching anymore, it should be no problem to set it to 00:30:00 or so.
 
dg3feh
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 68
Joined: Mon Jan 30, 2017 10:52 am

Re: Exclude domains from DNS cache!?

Thu Jul 13, 2017 2:42 pm

Hello!

I know all this. The IPs TTLs and weightings seems to be wrong. I want to send all DNS queries direct to the ISP DNS and all the rest first to the caching name server. The normal TTL is 1h.
Is there no possibility to force this?

BR Holger
 
dg3feh
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 68
Joined: Mon Jan 30, 2017 10:52 am

Re: Exclude domains from DNS cache!?

Thu Jul 13, 2017 2:49 pm

Is it possible to see the priority and the weighting of the records?
 
pe1chl
Forum Guru
Forum Guru
Posts: 10542
Joined: Mon Jun 08, 2015 12:09 pm

Re: Exclude domains from DNS cache!?

Thu Jul 13, 2017 2:53 pm

When you set the max cache TTL to 00:00:10 or so there should be no issue with caching
and you still have some optimization for stupid software that requests the same DNS name over and over again.

You can see the (remaining) TTL in the cache overview. I do not know what you mean by "weighting" of a DNS record.
 
dg3feh
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 68
Joined: Mon Jan 30, 2017 10:52 am

Re: Exclude domains from DNS cache!?

Thu Jul 13, 2017 4:51 pm

I am talking about the SRV record. If there are two or more entries for a canocial name the descission which is taken is made by pritority and weight. -> https://en.wikipedia.org/wiki/SRV_record
 
pe1chl
Forum Guru
Forum Guru
Posts: 10542
Joined: Mon Jun 08, 2015 12:09 pm

Re: Exclude domains from DNS cache!?

Thu Jul 13, 2017 5:05 pm

Ok, but that is not relevant for the problem you brought up. When the TTL is low enough the entry
will disappear from the cache quickly and will be looked up again. Then it should be correct.
 
dg3feh
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 68
Joined: Mon Jan 30, 2017 10:52 am

Re: Exclude domains from DNS cache!?

Thu Jul 13, 2017 5:14 pm

Nope! I know that my VoIP Service is some how currupted by wrong DNS entries, The work around is to ask not the mikrotik DNS cache, but the ISP DNS directly. Thus I want a directive to do that only for the VoIP domains. The rest works fine with the "normal" DNS relay. Only if this is not possible with mikrotik, it is necassary to reduce the TTL for all DNS entries dramatically.

As fare as I learned mikrotik lacks this possiblility......
 
User avatar
Cha0s
Forum Guru
Forum Guru
Posts: 1162
Joined: Tue Oct 11, 2005 4:53 pm

Re: Exclude domains from DNS cache!?

Thu Jul 13, 2017 5:16 pm

Regardless of the weight, if the provider changes IPs all the time and they don't have a low enough TTL on their records, it's not a problem of MikroTik but a problem on their side.
If they change IP so often that will cause you problems due to DNS caching, then everyone would have the same problem regardless of what DNS server they use.
Which lead me to believe that either the ISP is an amateur one that don't know how to setup their DNS (in which case there's nothing you can really do except report to them the problem to fix it) or most likely your problem is not DNS related.

Mikrotik DNS cache (no matter how basic or bad it is) will follow whatever TTL they have configured on their records.
SRV weights is irrelevant to Mikrotik. The weight is used by your Voip software/client to do failover essentially.

Are you sure that the problem is DNS related?
Is it possible your VoIP/SIP client/software/device does not properly follow the IP change?
Can you confirm in IP > DNS > Cache that the record does not expire/update itself properly?

I use a custom DDNS setup on Mikrotik with a TTL of 1second for immediate update of my dynamic IPs.
It works flawlessly for over 7 years now. What I mean is that I've never had problems with Mikrotik not obeying the TTL as you insinuating.
It used to have issues with static dns records edit/removal not being removed properly from the cache but that's completely different to your issue.


Either way, what you ask cannot be done in the DNS of Mikrotik directly. The only way I know of in Mikrotik is by using L7 filters. Search the forum, there is a post somewhere explaining in detail how to do it.

I wonder, even if you exclude the domain, and you use another DNS server for it, what will change? If the TTL is not low enough, any DNS server you use will follow whatever the TTL is.
 
pe1chl
Forum Guru
Forum Guru
Posts: 10542
Joined: Mon Jun 08, 2015 12:09 pm

Re: Exclude domains from DNS cache!?

Thu Jul 13, 2017 5:23 pm

I agree with Cha0s. If it is a DNS problem, it is caused by improper TTL settings on the DNS records by your provider. Let them fix it.
More likely it is a problem with NAT that results from this IP change. NAT is always tricky with VoIP. Use IPv6 whenever possible.
What is the domain name of the records that you think are bad? I would like to do some lookups to see if they really are wrongly
configured. (changing more often than the TTL suggests)
 
msatter
Forum Guru
Forum Guru
Posts: 2942
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Exclude domains from DNS cache!?

Thu Jul 13, 2017 7:12 pm

If you add the domain name of the VOIP server to the IP - Firewall - Address-list it will drive your ISP DNS server MAD and the Mikrotik cache only hold that resolved IP for a very very short time.

Just type domain name in the Address field and lets the refreshing begins.
 
idlemind
Forum Guru
Forum Guru
Posts: 1146
Joined: Fri Mar 24, 2017 11:15 pm
Location: USA

Re: Exclude domains from DNS cache!?

Thu Jul 13, 2017 7:29 pm

I need to use the mikrotik as DNS server due to the fact, that the DNS Servers are pushed dynamically by the Provider.
Like others have said, remove or lower the caching feature if you feel the problem is caused by cached DNS records on the MikroTik side.

Additionally you could play with connection tracking timeouts but this is likely going to wreak havoc on your network.

A safer option to prove or disprove it's a problem with the MikroTik DNS cache would be to set the DNS Server value in /ip dhcp-server network to the upstream DNS server provided by your ISP.

Initially for testing this could be done manually, if it proves to be a working solution create a script that updates the value of the DNS Server on each lease change. I find it hard to believe that your ISP is constantly changing their DNS server IP address though so a static entry likely will suffice. Regardless, the lease script functionality is in 6.39+ so make sure you're running current code.
 
User avatar
ZeroByte
Forum Guru
Forum Guru
Posts: 4047
Joined: Wed May 11, 2011 6:08 pm

Re: Exclude domains from DNS cache!?

Thu Jul 13, 2017 10:58 pm

Short answer: Mikrotik can't do what you want. It's only a simple proxy resolver.

Medium answer: I agree 100% with all of the others who've responded. If the ISP is changing their VoIP address in DNS in such a way that it breaks things, then they need to learn what they're doing and fix it. Caching DNS happens at pretty much every single level of the DNS system, all the way into your own computer. When changing their DNS pointer, they should make sure that the previous IP remains available for at least TTL seconds after they make their change. Period. If it's something they change frequently, then they should make their TTL something short like 3 minutes.

Any workarounds done on the client side are kludges and should not be necessary.
 
dg3feh
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 68
Joined: Mon Jan 30, 2017 10:52 am

Re: Exclude domains from DNS cache!?

Sat Jul 15, 2017 4:41 pm

In the last 24h I forced a TTL of 1sec to all DNS entries and there where no registration losts on my VoIP. So I am pretty sure that the ISPs DNS SRV Records are corrupt regarding TTL, weight, priority.

I am already in discussion with my ISP, but as usual: To get someone on the phone, who is able to follow a discussion like this hard ;) ...and, if u don't us their hardware they are very suspisious :) Nerverthe less I send them logs from the packet sniffer, we will c...

And yes, it is a server-side problem and need to be fixed there, BUT until that I have to use this line ;) and I don't want to drive the ISPs DNS mad, therefore I asked how to send DNS requests for certain domains directly to the DNS of the ISP. I can't do that on the VoiP client, because the DNS Servers are dynamic (pushed by PPPoE). Ok, that won't work within the Mikrotik, so i use a TTL of 1sec....
 
dg3feh
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 68
Joined: Mon Jan 30, 2017 10:52 am

Re: Exclude domains from DNS cache!?

Sat Jul 15, 2017 5:02 pm

...beside that I tried to write the sniffer log to a Memorystick, without success.
[admin@Router-HH] /disk> print
 # NAME                                 LABEL                                TYPE    DISK                                      FREE       SIZE
 0 disk1                                1GB-MEMSTIC                          fat32   PDU01_1G 71G2.0                       968.6MiB   984.0MiB
...that seems ok for me.

I found the disk1 in the files:
[admin@Router-HH] /file> print
 # NAME                                                  TYPE                                                        SIZE CREATION-TIME       
 0 skins                                                 directory                                                        jan/01/1970 02:00:01
 1 autosupout.rif                                        .rif file                                               558.5KiB feb/11/2017 20:01:08
 2 console-dump.txt                                      .txt file                                                17.6KiB jul/13/2017 13:51:15
 3 disk1                                                 disk                                                             jul/15/2017 15:22:39
 4 autosupout.old.rif                                    .rif file                                               477.8KiB jan/30/2017 01:26:03
 5 Router-HH-20170314-0153.backup                        backup                                                  343.5KiB mar/14/2017 02:53:33
 6 pub                                                   directory                                                        jan/25/2017 21:36:16
Within the sniffer i set the file-name to
file-name: /disk1/voip-sniff-ttl-5sec
and
file-name: disk1/voip-sniff-ttl-5sec
without any success. Any hints on that?

BR Holger
 
User avatar
ZeroByte
Forum Guru
Forum Guru
Posts: 4047
Joined: Wed May 11, 2011 6:08 pm

Re: Exclude domains from DNS cache!?

Mon Jul 17, 2017 6:19 pm

Well, there's a middle-way solution for you, and it's probably a good step to achieve anyway....

If all of your Mikrotiks are pointing at some centralized caching resolver DNS host that you control, you could implement cache flushes of the SRV records on your DNS host (assuming that it's BIND or some other fully-featured server).

Having your own centralized on-net caching resolvers would actually be a good thing to implement anyway. Speedy DNS is a key ingredient of a "fast" user experience. Caching DNS at the CPE as a secondary cache behind your own centralized DNS cache will accelerate things because the centralized cache gets filled by requests from users throughout your network, increasing the chance for cache hits. On-net cache hits will be much faster than off-net cache hits (in the case where you're forwarding cache misses to some public DNS server such as 8.8.8.8 or OpenDNS). These "off-net" caches are going to at least be faster than fully traversing the DNS heirarchy starting at the root would be, but you're still subject to whatever caching policy is present on those public servers, whereas running a fully independent resolver yourself means that you can control the cache behavior for your network.
 
User avatar
Cha0s
Forum Guru
Forum Guru
Posts: 1162
Joined: Tue Oct 11, 2005 4:53 pm

Re: Exclude domains from DNS cache!?

Mon Jul 17, 2017 6:27 pm

I agree with ZeroByte,

Since RouterOS DNS is rather basic (at least) I use a raspberry-pi just for BIND.
It can handle many many DNS requests before exhausting its resources.

This way I have full control (local domains, forwarders, slave zones, etc) of all DNS on my LAN.
It's not ideal for everyone (you need an extra device) but at least you don't need a full fledged server box for it (ie: no noise, low power consumption, cheap hardware).


Since I do many DNS transfers etc I need to be able traverse the whole DNS tree to make sure that changes propagate properly.
So I can flush my whole or specific DNS cache on demand to be able to check things right away and not wait for Google or OpenDNS to have their caches expire before I see the changes.

In any case, having a full DNS server brings only positive things to your network :)
 
pe1chl
Forum Guru
Forum Guru
Posts: 10542
Joined: Mon Jun 08, 2015 12:09 pm

Re: Exclude domains from DNS cache!?

Mon Jul 17, 2017 7:04 pm

Sure I agree with that, but with the problem posed by the original poster (of which we have seen no evidence and no information allowing us to research it further), it can be helpful to reduce the amount of caching.
Also, in some home routers there are quite serious bugs in the caching resolver implementation. I have not seen examples of that in the MikroTik routers, but for many other home routers it is best not to use the built-in resolver.
 
User avatar
ZeroByte
Forum Guru
Forum Guru
Posts: 4047
Joined: Wed May 11, 2011 6:08 pm

Re: Exclude domains from DNS cache!?

Mon Jul 17, 2017 9:38 pm

Just to be clear - I envisioned the full-featured caching resolver being somewhere at the ISP network core, leaving the ROS local caching resolver doing the grunt work at each site. They're all pointed at the central cache so that all sites can get speedy DNS for commonly requested RRs, with more chances of cache hits due to the larger number of clients going through the central cache.

Although one might make the argument that in today's world of geo-distributed load balancing being so ultra-common, that many of the most frequently accessed URLs are in fact going to be short-lived TTL things. (e.g: www.google.com comes back with TTL=300 from ns1.google.com) - thus caching is not going to be as helpful as it once would have been.
 
pe1chl
Forum Guru
Forum Guru
Posts: 10542
Joined: Mon Jun 08, 2015 12:09 pm

Re: Exclude domains from DNS cache!?

Mon Jul 17, 2017 10:02 pm

Although one might make the argument that in today's world of geo-distributed load balancing being so ultra-common, that many of the most frequently accessed URLs are in fact going to be short-lived TTL things. (e.g: http://www.google.com comes back with TTL=300 from ns1.google.com) - thus caching is not going to be as helpful as it once would have been.
Yes, that is what I wanted to hint at in reply #2.
Also, the network is now so fast that trying to save a couple of DNS queries is now no longer as helpful as when we used 28.8 modems and ISDN-2.
 
User avatar
ZeroByte
Forum Guru
Forum Guru
Posts: 4047
Joined: Wed May 11, 2011 6:08 pm

Re: Exclude domains from DNS cache!?

Tue Jul 18, 2017 12:04 am

Also, the network is now so fast that trying to save a couple of DNS queries is now no longer as helpful as when we used 28.8 modems and ISDN-2.
Yes and no - RTT can still add up quickly, especially if some website / connection requires several RR lookups. Suppose it takes 5 of them which must happen in sequence, and suppose that the response time for queries to/from 8.8.8.8 take ~20ms for you on average - or even ~10ms... That would mean that site views, etc could take on the order of 0.5 to 1.0 seconds before they start actually opening. This is a noticeable time frame on the human scale.

We have a client whose point-of-sale system was designed with wire speed LAN in mind. They put points-of-sale all over the campus, many locations were fed via a wireless mesh system which had 10ms first-hop-latency, and 20ms for each additional hop. Thus some of their POS were behind 30 or 50ms latency built right in from the topology when it was working at peak performance as specified by the manufacturer... i.e. it can't get any better than that. They started opening tickets with us because their POS was slow at these locations. Long story short, I ended up wiresharking the terminal and the server at the same time to prove the responsiveness of things (I thought it was the server's fault) and learned that each key press on the terminal resulted in 8 query/response pairs before the POS terminal would update the order in question. 8x30ms = 0.25 seconds - and when the clerk is trying to tap the "popcorn" button 3 times in a row, they would get errors.

Point is - things can add up in interesting ways one wouldn't think of, and thus having a 1ms cache hit at the customer's device is 10 to 20 times faster than going out to the Internet for that same cache hit. This is why I still believe in on-site caching.

Granted: In my previous post I admit that cache hits are probably going down in recent years due to the way CDNs' DNS works... it would be interesting to see some statistics from real-world deployment the rate of cache hits vs. recursive lookups required.
 
dg3feh
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 68
Joined: Mon Jan 30, 2017 10:52 am

Re: Exclude domains from DNS cache!?

Thu Jul 20, 2017 2:47 pm

Hello all,

thanks for u comments. I have a small unix server in my LAN doing mostly nothing ;) (beside family file serving) which is able do act as DNS. Problem is the dynamic push of the ISPs nameserver to the mikrotik. How do I transfer this information to the server?


Beside that: Some hints for the file handling problem to store the logs on the USB Stick?


BR Holger
 
msatter
Forum Guru
Forum Guru
Posts: 2942
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Exclude domains from DNS cache!?

Thu Jul 20, 2017 3:26 pm

Have a look at PIhole which also accelerates you connection by not dowloading advertisements.

You say everty time "push" don't you mean "use peer DNS"?

If you don't have your ISP "push" it as you writ then you put the DNS server of the ISP in your own DNS server. Any unknown domains go upstream to the DNS server of your ISP.
 
dg3feh
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 68
Joined: Mon Jan 30, 2017 10:52 am

Re: Exclude domains from DNS cache!?

Thu Jul 20, 2017 9:14 pm

"use peer DNS" results in a "push". These DNS are dynamical set by the ISP.

No hints regarding the file problem? :(
 
pe1chl
Forum Guru
Forum Guru
Posts: 10542
Joined: Mon Jun 08, 2015 12:09 pm

Re: Exclude domains from DNS cache!?

Fri Jul 21, 2017 11:32 am

Your ISP sounds like one I would say goodbye to very quickly...
Dynamic address of VoIP servers, dynamic address of DNS servers, likely a dynamic address for your own connection as well, what a mess...
Unfortunately it appears to be common practice in Germany.
 
msatter
Forum Guru
Forum Guru
Posts: 2942
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Exclude domains from DNS cache!?

Fri Jul 21, 2017 2:42 pm

If you add the domain name of the VOIP server to the IP - Firewall - Address-list it will drive your ISP DNS server MAD and the Mikrotik cache only hold that resolved IP for a very very short time.

Just type domain name in the Address field and lets the refreshing begins.
I have to take it back what I wrote and this behavior was because I had set the TTL of local addresses to 0. Now the DNS cache reflects the TTL set be the administrator of the domain.

Mikrotik has to look at the refreshing of cached statics domains because that seem to also be set too zero lifetime.
 
msatter
Forum Guru
Forum Guru
Posts: 2942
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Exclude domains from DNS cache!?

Fri Jul 21, 2017 3:03 pm

Your ISP sounds like one I would say goodbye to very quickly...
Dynamic address of VoIP servers, dynamic address of DNS servers, likely a dynamic address for your own connection as well, what a mess...
Unfortunately it appears to be common practice in Germany.
It is not all bad because privacy is increased because you get each time a new IP. Partitioning of the of the network can work also but is a PITA to sync and the max TTL has to modified dynamicly, to be the same as the left connect time of client.

On the side of router you could flush the DNS cache if the WAN is restarted or when the IP changes. The SIP looses his connection and how that works I don't know.

I just tried it and I put in the Profile for the PPPoE when it goes up: /ip dns cache flush and the DNS cache is flushed and every first request is now not cached anymore until the TTL ends or the PPPoE connection is renewed.

If you have activated you private linux DNS server in the meantime that will mask the new DNS sever from your ISP so remove it have active data.

Next test was pulling the cable out of the WAN port and reconnect it. The cache was flushed so I think this problem is solved or your ISP has an other way to change you dynamic address.
 
dg3feh
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 68
Joined: Mon Jan 30, 2017 10:52 am

Re: Exclude domains from DNS cache!?

Wed Jul 26, 2017 9:27 am

In /ip firewall service-port there is the possibility to reduce the TTL for SIP connections:
SIP helper. Additional options:
sip-direct-media allows redirect the RTP media stream to go directly from the caller to the callee. Default value is yes.
sip-timeout allows adjust TTL of SIP UDP connections. Default: 1 hour. In some setups you have to reduce that.
So my problem seems to be quite common :)

BR Holger
 
msatter
Forum Guru
Forum Guru
Posts: 2942
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Exclude domains from DNS cache!?

Wed Jul 26, 2017 1:30 pm

It states SIP UDP connections and that is the VOIP traffic self not the protocol to make and break VOIP connections.

To make/break the connection you go over TCP 5060-5061.

You can just try if that the solution for you....this is the way I also learn a lot. If you seek with Google you find 99.9% of the hits on how to disable this service.