Latency jump after upgrade from 1.5.8 to 1.9.0

[Matthew Ghali]

Hi folks- we're working on an internal upgrade of our Unbound instances from the Ubuntu xenial default of 1.5.8 to 1.9.0 (backported from eoan) so we can start using dnstap and prefetching.

Our initial test upgrades (without either dnstap or prefetching enabled yet) consistently show a large jump in response latency though. Adjacent un-updated Unbounds left as controls do not show the same changes.


While the query types and response codes show no corresponding change:




And overall cache efficiency looks unchanged:




We do see a dramatic change in latencies:



These results do not change over time as the upgraded instances become "warmed".

We've looked at the Ubuntu maintainer scripts for the packages and don't see any red flags in build option differences between versions. Were there perhaps build defaults that changed between versions? We're puzzled since a performance change like this surely would produce some mailing list traffic at the least, which we also couldn't find- so that must mean there is some difference local to us somehow. Are there suggestions on where we can look?

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nlnetlabs.nl/pipermail/unbound-users/attachments/20191126/9fb88900/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screen Shot 2019-11-26 at 10.02.22 AM.png
Type: image/png
Size: 408163 bytes
Desc: not available
URL: <http://lists.nlnetlabs.nl/pipermail/unbound-users/attachments/20191126/9fb88900/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screen Shot 2019-11-26 at 10.01.46 AM.png
Type: image/png
Size: 229310 bytes
Desc: not available
URL: <http://lists.nlnetlabs.nl/pipermail/unbound-users/attachments/20191126/9fb88900/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screen Shot 2019-11-26 at 10.01.14 AM.png
Type: image/png
Size: 398239 bytes
Desc: not available
URL: <http://lists.nlnetlabs.nl/pipermail/unbound-users/attachments/20191126/9fb88900/attachment-0002.png>