High Latency During Peak Periods on Unbound Recursive DNS

[sir izake]

Hi Everyone

I am running a recursive anycast DNS for  over 15m subscribers in a
telecom/isp environment. I have 7 HP servers running Freebsd 13.2, Unbound
DNS 1.17.1 & Quagga.

I have  tuned the OS  and applied Unbound DNS optimization recommendations
for busy servers.
NLnet Labs Documentation - Unbound - Howto Optimise
<https://nlnetlabs.nl/documentation/unbound/howto-optimise/>

additions to /etc/sysctl.conf
# set to at least 25MB for 10GE hosts
   kern.ipc.maxsockbuf=26214400
   # set autotuning maximum to at least 25MB too
   net.inet.tcp.sendbuf_max=26214400
   net.inet.tcp.recvbuf_max=26214400
   # enable send/recv autotuning
   net.inet.tcp.sendbuf_auto=1
   net.inet.tcp.recvbuf_auto=1
   # increase autotuning step size
   net.inet.tcp.sendbuf_inc=16384


Servers performed very well (peak/off peak) with latency under 50ms until
recently  when I enabled dnssec validation.
https://nlnetlabs.nl/documentation/unbound/howto-anchor/

Since then performance has been erratic with latency spiking to over 600ms
during peak hours.

I have disabled dnnsec validation and restarted unbound service / server
several times. But the performance is not as good as before.

Can anyone suggest any tuning/optimization parameter to implement?

Hardware specs

hw.machine: amd64
hw.model: Intel(R) Xeon(R) Gold 6334 CPU @ 3.60GHz
hw.ncpu: 32
hw.byteorder: 1234
hw.physmem: 549395005440
hw.usermem: 546043224064
hw.pagesize: 4096

Regards,
Isaac
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nlnetlabs.nl/pipermail/unbound-users/attachments/20230811/e89e1d58/attachment.htm>