[Unbound-users] High number of system context switches

[Jan-Frode Myklebust]

Hi, 

I'm considering switching from bind to unbound, and have been testing it
on one of our recursive dns servers. Our servers are KVM virtual machines,
running RHEL6.5, 12GB memory, 8 cpu cores, with unbound-1.4.21-1.el6.x86_64
from EPEL.

We typically have high time periods of 12-14k qps on our DNS-servers, and
it's been working fairly well on bind, but with unbound we seem to get into
trouble when the qps exceed 7k. We then see a clear drop in the request
rate, and clients move over to the secondary dns server.

The only problem we've noticed on the unbound server is that context
switches/s is very high. pidstat for unbound doesn't report high
cswch/s, but the system does..

Here's "sar -w" from yesterday evening running unbound:

kl. 19.00 +0200    proc/s   cswch/s
kl. 19.10 +0200      3,70 116480,30
kl. 19.20 +0200      3,47 123118,48
kl. 19.30 +0200      3,67 128948,60
kl. 19.40 +0200      3,45 125471,32
kl. 19.50 +0200      3,69 132641,76
kl. 20.00 +0200      3,48 140126,75

while the day before on bind:

kl. 19.00 +0200      1,90  64801,51
kl. 19.10 +0200      2,15  64550,78
kl. 19.20 +0200      1,94  64389,23
kl. 19.30 +0200      2,19  64369,56
kl. 19.40 +0200      1,92  64211,15
kl. 19.50 +0200      2,09  64087,84
kl. 20.00 +0200      1,91  63691,33


Any ideas for what we should try to improve this?

Full unbound.conf stripped for comments:
-------------------------------------------------------------------
server:
verbosity: 1
statistics-interval: 60
statistics-cumulative: yes
extended-statistics: yes
num-threads: 8
interface: 0.0.0.0
interface: ::0
interface-automatic: yes
outgoing-range: 4096
outgoing-port-permit: 32768-65535
outgoing-port-avoid: 0-32767
max-udp-size: 3072
msg-cache-size: 4G
num-queries-per-thread: 4096
rrset-cache-size: 8G
cache-min-ttl: 2
do-ip4: yes
do-ip6: yes
do-udp: yes
do-tcp: yes
access-control: 0.0.0.0/0 allow
access-control: ::0/0 allow
chroot: ""
username: "unbound"
directory: "/etc/unbound"
log-time-ascii: yes
pidfile: "/var/run/unbound/unbound.pid"
harden-glue: yes
harden-dnssec-stripped: yes
harden-below-nxdomain: yes
harden-referral-path: yes
use-caps-for-id: no
unwanted-reply-threshold: 10000000
prefetch: yes
prefetch-key: yes
rrset-roundrobin: yes
minimal-responses: no
dlv-anchor-file: "/etc/unbound/dlv.isc.org.key"
trusted-keys-file: /etc/unbound/keys.d/*.key
auto-trust-anchor-file: "/var/lib/unbound/root.anchor"
val-clean-additional: yes
val-permissive-mode: no
val-log-level: 2
include: /etc/unbound/local.d/*.conf
remote-control:
control-enable: yes
control-interface: 127.0.0.1
control-interface: ::1
server-key-file: "/etc/unbound/unbound_server.key"
server-cert-file: "/etc/unbound/unbound_server.pem"
control-key-file: "/etc/unbound/unbound_control.key"
control-cert-file: "/etc/unbound/unbound_control.pem"
include: /etc/unbound/conf.d/*.conf
-------------------------------------------------------------------




   -jf