Jostle logic seems to randomly stop working
Dmitri Kourennyi
dkour at mykolab.com
Tue May 15 23:55:16 UTC 2018
Hello,
I have setup unbound as a caching DNS server on my home router, and I
have been
having intermittent problems with it. In my attempt to fix it, I
recently
switched my desktop DNS resolver to systemd-resolved, and now the
problem
appears a little more clear cut:
Most of the time, unbound works great. However, it seems that every day
when I
come back home and fire up my PC, a vast majority of queries no longer
work.
Restarting unbound solves the issue.
Turning on maximum logging, I notice that the syslog is filling up with:
"Too many queries. dropping incoming query."
Running "unbound-control stats_noreset" also shows the
`total.requestlist.exceeded` stat at several hundred within minutes, and
actively climbing as systemd-resolved keeps trying to resolve names.
Ok, so I looked in the code a bit, and I'm guessing that I'm hitting the
`num-queries-per-thread` limit, which I have configured at 225. However,
based
on the documentation and the code, stale queries should be jostled out
after a
delay (I have it configured as 500ms, due to DNS over TLS being a bit
slower).
Yet I can sit for several minutes and watch as a vast majority of
queries fail.
It appears like stale queries aren't being jostled out like they should.
Interestingly, every once in a while a query makes it through, so it's
not a
100% failure rate.
I'm not sure how to proceed in order to debug this issue further. I
confess I
don't have intimate knowledge of the details of DNS, and my setup is
somewhat
experimental so I can play with stuff (router is a custom Linux box with
manually written firewall rules, aggressive QoS settings, things like
that), so
it's possible the issue may lie outside of my unbound configuration.
-Dmitri
Snippets of my logs and config file(s):
### Snippet from syslog during failing condition:
...
May 15 18:50:55 homebrew unbound[575]: [575:0] debug: answer from the
cache failed
May 15 18:50:55 homebrew unbound[575]: [575:0] debug: udp request from
ip4 192.168.0.2 port 41884 (len 16)
May 15 18:50:55 homebrew unbound[575]: [575:0] debug: Too many queries.
dropping incoming query.
May 15 18:50:55 homebrew unbound[575]: [575:0] debug: cache memory
msg=495151 rrset=506194 infra=7600 val=109330
May 15 18:50:55 homebrew unbound[575]: [575:0] debug: answer from the
cache failed
May 15 18:50:55 homebrew unbound[575]: [575:0] debug: udp request from
ip4 192.168.0.2 port 33122 (len 16)
May 15 18:50:55 homebrew unbound[575]: [575:0] debug: Too many queries.
dropping incoming query.
May 15 18:50:55 homebrew unbound[575]: [575:0] debug: cache memory
msg=495151 rrset=506194 infra=7600 val=109330
...
### Info from stats_noreset:
...
total.num.queries=4666
total.num.queries_ip_ratelimited=0
total.num.cachehits=844
total.num.cachemiss=3822
total.num.prefetch=8
total.num.zero_ttl=0
total.num.recursivereplies=3064
total.requestlist.avg=1.11906
total.requestlist.max=14
total.requestlist.overwritten=0
total.requestlist.exceeded=758
total.requestlist.current.all=0
total.requestlist.current.user=700
total.recursion.time.avg=0.247922
total.recursion.time.median=0.13798
total.tcpusage=0
time.now=1526424323.220624
time.up=85257.239793
time.elapsed=85257.239793
### unbound.conf:
server:
aggressive-nsec: yes
jostle-timeout: 500
key-cache-slabs: 2
low-rtt: 160
low-rtt-pct: 900
msg-cache-slabs: 2
num-queries-per-thread: 225
num-threads: 2
outgoing-range: 450
rrset-roundrobin: yes
so-reuseport: yes
trust-anchor-file: trusted-key.key
use-caps-for-id: yes
use-syslog: yes
username: "unbound"
verbosity: 2
directory: "/etc/unbound"
root-hints: root.hints
interface: 127.0.0.1
interface: 192.168.0.1
access-control: 192.168.0.0/24 allow
msg-cache-size: 256m
rrset-cache-size: 512m
neg-cache-size: 32m
prefetch: yes
prefetch-key: yes
qname-minimisation: yes
tls-cert-bundle: /etc/ca-certificates/extracted/ca-bundle.trust.crt
tls-upstream: yes
auth-zone:
name: "."
fallback-enabled: yes
for-downstream: no
for-upstream: yes
zonefile: root.keys
master: f.root-servers.net
master: k.root-servers.net
master: g.root-servers.net
master: xfr.lax.dns.icann.org
master: c.root-servers.net
master: xfr.cjr.dns.icann.org
master: b.root-servers.net
forward-zone:
name: "."
forward-addr: 199.58.81.218 at 853#dns.cmrg.net
forward-addr: 1.0.0.1 at 853#cloudflare-dns.com
forward-addr: 1.1.1.1 at 853#cloudflare-dns.com
forward-addr: 2606:4700:4700::1111 at 853#cloudflare-dns.com
forward-addr: 2606:4700:4700::1001 at 853#cloudflare-dns.com
forward-addr: 9.9.9.9 at 853#dns.quad9.net
forward-addr: 149.112.112.112 at 853#dns.quad9.net
forward-addr: 2620:fe::fe at 853#dns.quad9.net
remote-control:
control-enable: yes
More information about the Unbound-users
mailing list