Jostle logic seems to randomly stop working

Dmitri Kourennyi dkour at mykolab.com
Tue May 15 23:55:16 UTC 2018


Hello,

I have setup unbound as a caching DNS server on my home router, and I 
have been
having intermittent problems with it. In my attempt to fix it, I 
recently
switched my desktop DNS resolver to systemd-resolved, and now the 
problem
appears a little more clear cut:

Most of the time, unbound works great. However, it seems that every day 
when I
come back home and fire up my PC, a vast majority of queries no longer 
work.
Restarting unbound solves the issue.
Turning on maximum logging, I notice that the syslog is filling up with:
"Too many queries. dropping incoming query."

Running "unbound-control stats_noreset" also shows the
`total.requestlist.exceeded` stat at several hundred within minutes, and
actively climbing as systemd-resolved keeps trying to resolve names.

Ok, so I looked in the code a bit, and I'm guessing that I'm hitting the
`num-queries-per-thread` limit, which I have configured at 225. However, 
based
on the documentation and the code, stale queries should be jostled out 
after a
delay (I have it configured as 500ms, due to DNS over TLS being a bit 
slower).
Yet I can sit for several minutes and watch as a vast majority of 
queries fail.
It appears like stale queries aren't being jostled out like they should.
Interestingly, every once in a while a query makes it through, so it's 
not a
100% failure rate.

I'm not sure how to proceed in order to debug this issue further. I 
confess I
don't have intimate knowledge of the details of DNS, and my setup is 
somewhat
experimental so I can play with stuff (router is a custom Linux box with
manually written firewall rules, aggressive QoS settings, things like 
that), so
it's possible the issue may lie outside of my unbound configuration.

-Dmitri

Snippets of my logs and config file(s):

### Snippet from syslog during failing condition:
...
May 15 18:50:55 homebrew unbound[575]: [575:0] debug: answer from the 
cache failed
May 15 18:50:55 homebrew unbound[575]: [575:0] debug: udp request from 
ip4 192.168.0.2 port 41884 (len 16)
May 15 18:50:55 homebrew unbound[575]: [575:0] debug: Too many queries. 
dropping incoming query.
May 15 18:50:55 homebrew unbound[575]: [575:0] debug: cache memory 
msg=495151 rrset=506194 infra=7600 val=109330
May 15 18:50:55 homebrew unbound[575]: [575:0] debug: answer from the 
cache failed
May 15 18:50:55 homebrew unbound[575]: [575:0] debug: udp request from 
ip4 192.168.0.2 port 33122 (len 16)
May 15 18:50:55 homebrew unbound[575]: [575:0] debug: Too many queries. 
dropping incoming query.
May 15 18:50:55 homebrew unbound[575]: [575:0] debug: cache memory 
msg=495151 rrset=506194 infra=7600 val=109330
...

### Info from stats_noreset:
...
total.num.queries=4666
total.num.queries_ip_ratelimited=0
total.num.cachehits=844
total.num.cachemiss=3822
total.num.prefetch=8
total.num.zero_ttl=0
total.num.recursivereplies=3064
total.requestlist.avg=1.11906
total.requestlist.max=14
total.requestlist.overwritten=0
total.requestlist.exceeded=758
total.requestlist.current.all=0
total.requestlist.current.user=700
total.recursion.time.avg=0.247922
total.recursion.time.median=0.13798
total.tcpusage=0
time.now=1526424323.220624
time.up=85257.239793
time.elapsed=85257.239793

### unbound.conf:
server:
   aggressive-nsec: yes
   jostle-timeout: 500
   key-cache-slabs: 2
   low-rtt: 160
   low-rtt-pct: 900
   msg-cache-slabs: 2
   num-queries-per-thread: 225
   num-threads: 2
   outgoing-range: 450
   rrset-roundrobin: yes
   so-reuseport: yes
   trust-anchor-file: trusted-key.key
   use-caps-for-id: yes
   use-syslog: yes
   username: "unbound"
   verbosity: 2
   directory: "/etc/unbound"
   root-hints: root.hints

   interface: 127.0.0.1
   interface: 192.168.0.1
   access-control: 192.168.0.0/24 allow

   msg-cache-size: 256m
   rrset-cache-size: 512m

   neg-cache-size: 32m
   prefetch: yes
   prefetch-key: yes
   qname-minimisation: yes
   tls-cert-bundle: /etc/ca-certificates/extracted/ca-bundle.trust.crt
   tls-upstream: yes

auth-zone:
   name: "."
   fallback-enabled: yes
   for-downstream: no
   for-upstream: yes
   zonefile: root.keys
   master: f.root-servers.net
   master: k.root-servers.net
   master: g.root-servers.net
   master: xfr.lax.dns.icann.org
   master: c.root-servers.net
   master: xfr.cjr.dns.icann.org
   master: b.root-servers.net

forward-zone:
   name: "."
   forward-addr: 199.58.81.218 at 853#dns.cmrg.net
   forward-addr: 1.0.0.1 at 853#cloudflare-dns.com
   forward-addr: 1.1.1.1 at 853#cloudflare-dns.com
   forward-addr: 2606:4700:4700::1111 at 853#cloudflare-dns.com
   forward-addr: 2606:4700:4700::1001 at 853#cloudflare-dns.com
   forward-addr: 9.9.9.9 at 853#dns.quad9.net
   forward-addr: 149.112.112.112 at 853#dns.quad9.net
   forward-addr: 2620:fe::fe at 853#dns.quad9.net

remote-control:
   control-enable: yes



More information about the Unbound-users mailing list