The third tool out of the DNSDiag toolkit from Babak is dnseval. “dnseval is a bulk ping utility that sends an arbitrary DNS query to a given list of DNS servers. This script is meant for comparing response times of multiple DNS servers at once”. It is not only listing the response times but also further information about the DNS responses such as the TTL and the flags. Really great for comparison and troubleshooting different DNS forwarders as well as own authoritative DNS server responses as seen by others.
At first I want to thank Babak for adding so many feature requests I have sent to him. (More than ten!) Also note the two other tools from him, dnsping and dnstraceroute which I covered in other blog posts.
./dnseval with own server list
When called without any options dnseval displays a short info page:
weberjoh@jw-nb12-lx:~/dnsdiag$ ./dnseval.py dnseval.py version 1.5 usage: dnseval.py [-h] [-f server-list] [-c count] [-t type] [-w wait] hostname -h --help show this help -f --file dns server list to use (default: system resolvers) -c --count number of requests to send (default: 10) -w --wait maximum wait time for a reply (default: 2) -t --type DNS request record type (default: A) -T --tcp Use TCP instead of UDP -e --edns Disable EDNS0 (Default: Enabled)
Note the
-f <file>option that uses a file which lists all the servers dnseval should query. I am always using an own list with the following entries, of course with both, IPv6 and legacy IP addresses:
- the DNS forwarders from my ISP (Deutsche Telekom)
- common public servers (Google Public DNS, OpenDNS)
- my own internal recursive DNS servers (BIND, Unbound)
- some other router/firewall/CPE DNS forwarders (Palo Alto DNS Proxy, FRITZ!Box)
- own authoritative DNS servers (BIND).
With this big list I can check many different DNS problems as shown below:
A Picture is worth a Thousand Words
Have a look at the following sample output from dnseval and all the information you can gather out of it. (If you are not familiar with the DNS header flags, have a look here.) I queried the FQDN
fg.weberdns.dewhich I have on my own authoritative DNS servers. That is I can check whether all of these DNS servers are able to reach out my own authoritative ones:
weberjoh@jw-nb12-lx:~/dnsdiag$ ./dnseval.py -f ../dns-servers fg.weberdns.de server avg(ms) min(ms) max(ms) stddev(ms) lost(%) ttl flags ------------------------------------------------------------------------------------------------------------------ 194.25.0.68 10.998 4.044 16.210 4.496 %0 3600 QR -- -- RD RA -- -- 194.25.0.60 21.514 8.169 32.698 6.527 %0 3600 QR -- -- RD RA -- -- 2003:40:2000::53 8.531 3.488 21.811 6.498 %0 3600 QR -- -- RD RA -- -- 2003:56::53 19.255 8.444 31.983 9.074 %0 3600 QR -- -- RD RA -- -- 8.8.8.8 44.967 12.262 78.750 22.474 %0 3599 QR -- -- RD RA AD -- 8.8.4.4 130.931 14.610 1059.192 326.610 %0 3598 QR -- -- RD RA AD -- 2001:4860:4860::8888 17.181 13.751 20.350 3.152 %0 3598 QR -- -- RD RA AD -- 2001:4860:4860::8844 22.530 14.902 58.615 12.892 %0 3599 QR -- -- RD RA AD -- resolver1.opendns.com 14.336 3.790 39.841 14.940 %0 3600 QR -- -- RD RA -- -- resolver2.opendns.com 4.347 3.686 7.650 1.173 %0 3600 QR -- -- RD RA -- -- ns1-v4.weberdns.de 6.710 5.903 13.215 2.287 %0 3600 QR AA -- RD -- -- -- ns1-v6.weberdns.de 4.490 4.101 5.710 0.645 %0 3600 QR AA -- RD -- -- -- ns2.weberdns.de 10.220 9.644 12.352 0.862 %0 3600 QR AA -- RD -- -- -- ns3.weberdns.de 24.978 24.110 26.742 0.834 %0 3600 QR AA -- RD -- -- -- int-dns.webernetz.net 6.559 5.929 9.449 1.098 %0 3599 QR -- -- RD RA AD -- 192.168.110.1 2.233 2.048 2.594 0.164 %0 3463 QR -- -- -- -- -- -- 192.168.7.1 8.359 6.431 21.811 4.739 %0 3600 QR -- -- RD RA AD -- 192.168.7.5 15.352 11.708 46.910 11.091 %0 3600 QR -- -- RD RA AD --
You can see the following:
- every server was reachable (0 % lost) and answered (QR = query response flag)
- the OpenDNS server were faster than the Google Public DNS servers
- the TTL seems to be correct by all servers (3600 seconds are configured), while the query was not in the cache in any of them because the TTLs just started decreasing from 3600
- some servers are validating DNSSEC (AD = authentic data flag), which are the Google Public DNS servers as well as my own recursive ones
- my own authoritative servers are correctly answering with the AA = authoritative answer flags, while they have no RA = recursion available, which is correct, too
Perfect! It seems that I have neither a problem on my own authoritative servers nor on any recursive ones.
Time Matters: TTL
From the DDoS attacks against Dyn we have learned that the TTL should not be too short. But what happens if the TTL is really high, e.g., 30 days = 2592000 seconds? Let’s have a look at the DNS forwarders. I am querying
ttl-long.weberdns.de:
weberjoh@jw-nb12-lx:~/dnsdiag$ ./dnseval.py -f ../dns-servers ttl-long.weberdns.de server avg(ms) min(ms) max(ms) stddev(ms) lost(%) ttl flags ------------------------------------------------------------------------------------------------------------------ 194.25.0.68 5.002 3.228 6.897 1.574 %0 86373 QR -- -- RD RA -- -- 194.25.0.60 12.684 7.832 33.517 8.405 %0 86374 QR -- -- RD RA -- -- 2003:40:2000::53 4.812 3.640 7.118 1.394 %0 86400 QR -- -- RD RA -- -- 2003:56::53 8.945 8.061 15.702 2.377 %0 86373 QR -- -- RD RA -- -- 8.8.8.8 67.043 13.463 507.222 154.708 %0 86372 QR -- -- RD RA AD -- 8.8.4.4 16.761 14.265 20.786 2.744 %0 86372 QR -- -- RD RA AD -- 2001:4860:4860::8888 17.270 13.400 20.694 3.253 %0 86371 QR -- -- RD RA AD -- 2001:4860:4860::8844 17.578 15.262 21.558 2.766 %0 86371 QR -- -- RD RA AD -- resolver1.opendns.com 7.229 3.769 33.654 9.300 %0 604773 QR -- -- RD RA -- -- resolver2.opendns.com 3.865 3.814 3.978 0.047 %0 604773 QR -- -- RD RA -- -- ns1-v4.weberdns.de 6.444 5.929 9.226 1.030 %0 2592000 QR AA -- RD -- -- -- ns1-v6.weberdns.de 4.450 4.114 4.940 0.355 %0 2592000 QR AA -- RD -- -- -- ns2.weberdns.de 9.648 8.444 9.974 0.482 %0 2592000 QR AA -- RD -- -- -- ns3.weberdns.de 25.328 24.242 26.882 0.992 %0 2592000 QR AA -- RD -- -- -- int-dns.webernetz.net 6.338 6.073 7.794 0.525 %0 604774 QR -- -- RD RA AD -- 192.168.110.1 0.000 0.000 0.000 0.000 %100 N/A -- -- -- -- -- -- -- 192.168.7.1 7.110 6.692 8.120 0.412 %0 86373 QR -- -- RD RA AD -- 192.168.7.5 12.250 11.992 13.065 0.307 %0 2591973 QR -- -- RD RA AD --
In fact only one reply delivered/cached the correct TTL, namely the Unbound server. All other servers have limited the TTL, e.g., my ISP and Google to 1 day or OpenDNS to 7 days.
Short TTL values such as 60 seconds seem to be ok:
weberjoh@jw-nb12-lx:~/dnsdiag$ ./dnseval.py -f ../dns-servers ttl-short.weberdns.de server avg(ms) min(ms) max(ms) stddev(ms) lost(%) ttl flags ------------------------------------------------------------------------------------------------------------------ 194.25.0.68 13.184 4.054 18.359 4.314 %0 60 QR -- -- RD RA -- -- 194.25.0.60 21.242 8.219 32.522 7.828 %0 60 QR -- -- RD RA -- -- 2003:40:2000::53 11.680 4.148 18.468 4.958 %0 60 QR -- -- RD RA -- -- 2003:56::53 16.581 8.108 33.683 9.342 %0 60 QR -- -- RD RA -- -- 8.8.8.8 44.196 19.879 64.991 17.435 %0 59 QR -- -- RD RA AD -- 8.8.4.4 23.051 14.727 49.934 12.653 %0 58 QR -- -- RD RA AD -- 2001:4860:4860::8888 20.924 13.493 63.579 15.233 %0 58 QR -- -- RD RA AD -- 2001:4860:4860::8844 17.840 15.058 24.503 3.243 %0 58 QR -- -- RD RA AD -- resolver1.opendns.com 11.289 3.878 35.229 12.658 %0 60 QR -- -- RD RA -- -- resolver2.opendns.com 4.231 3.755 5.842 0.621 %0 59 QR -- -- RD RA -- -- ns1-v4.weberdns.de 6.471 5.975 9.055 0.971 %0 60 QR AA -- RD -- -- -- ns1-v6.weberdns.de 4.516 4.207 4.954 0.340 %0 60 QR AA -- RD -- -- -- ns2.weberdns.de 9.721 9.007 10.255 0.380 %0 60 QR AA -- RD -- -- -- ns3.weberdns.de 61.182 54.789 71.092 4.264 %0 60 QR AA -- RD -- -- -- int-dns.webernetz.net 6.785 6.132 10.287 1.280 %0 60 QR -- -- RD RA AD -- 192.168.110.1 0.000 0.000 0.000 0.000 %100 N/A -- -- -- -- -- -- -- 192.168.7.1 8.128 6.683 18.702 3.721 %0 60 QR -- -- RD RA AD -- 192.168.7.5 13.107 11.854 22.336 3.256 %0 60 QR -- -- RD RA AD --
Validating DNSSEC
As already mentioned not all public DNS servers are validating DNSSEC. Google does but OpenDNS or my ISP don’t. When querying
sigfail.verteiltesysteme.net, a false DNSSEC FQDN, no server should reply. But those do:
weberjoh@jw-nb12-lx:~/dnsdiag$ ./dnseval.py -f ../dns-servers sigfail.verteiltesysteme.net server avg(ms) min(ms) max(ms) stddev(ms) lost(%) ttl flags ------------------------------------------------------------------------------------------------------------------ 194.25.0.68 47.211 5.013 168.361 55.172 %0 60 QR -- -- RD RA -- -- 194.25.0.60 72.734 8.806 305.321 90.659 %0 60 QR -- -- RD RA -- -- 2003:40:2000::53 14.458 3.440 46.282 14.409 %0 59 QR -- -- RD RA -- -- 2003:56::53 59.879 7.961 310.540 94.759 %0 60 QR -- -- RD RA -- -- 8.8.8.8 0.000 0.000 0.000 0.000 %100 N/A -- -- -- -- -- -- -- 8.8.4.4 0.000 0.000 0.000 0.000 %100 N/A -- -- -- -- -- -- -- 2001:4860:4860::8888 0.000 0.000 0.000 0.000 %100 N/A -- -- -- -- -- -- -- 2001:4860:4860::8844 0.000 0.000 0.000 0.000 %100 N/A -- -- -- -- -- -- -- resolver1.opendns.com 39.926 4.185 134.027 51.552 %0 60 QR -- -- RD RA -- -- resolver2.opendns.com 9.903 3.830 33.534 11.996 %0 60 QR -- -- RD RA -- -- ns1-v4.weberdns.de 0.000 0.000 0.000 0.000 %100 N/A -- -- -- -- -- -- -- ns1-v6.weberdns.de 0.000 0.000 0.000 0.000 %100 N/A -- -- -- -- -- -- -- ns2.weberdns.de 0.000 0.000 0.000 0.000 %100 N/A -- -- -- -- -- -- -- ns3.weberdns.de 0.000 0.000 0.000 0.000 %100 N/A -- -- -- -- -- -- -- int-dns.webernetz.net 0.000 0.000 0.000 0.000 %100 N/A -- -- -- -- -- -- -- 192.168.110.1 0.000 0.000 0.000 0.000 %100 N/A -- -- -- -- -- -- -- 192.168.7.1 0.000 0.000 0.000 0.000 %100 N/A -- -- -- -- -- -- -- 192.168.7.5 0.000 0.000 0.000 0.000 %100 N/A -- -- -- -- -- -- --
That’s it for now. Did I miss some interesting test cases? Please write a comment if so.
Featured image: “Werkzeug” by Andreas Issleib is licensed under CC BY-NC-ND 2.0.