Tuning for survey workloads

[W.C.A. Wijngaards]

Hi Viktor,

On 23/05/18 16:50, Viktor Dukhovni via Unbound-users wrote:
> My workload sends lots of queries to various TLDs and public suffix
> 2LDs (.co.uk, ...), but non-infrastructure queries to leaf domains
> are almost not repeated sufficiently often to be found in the cache.

I don't know about separate survey caches.  But unbound has an option
for moving around, the target fetch policy.  It fetches the nameservers
referenced by other nameservers.  In advance, anticipating future
queries to them.  It can fetch based on depth (recursion depth) and
number.  Default is "3 2 1 0 0", but for you maybe "3 3 3 2 1 1" or so?
That would fetch more nameserver names, and that could help with speed,
but may not actually increase cache hits.

If you do a lot of DNSKEY queries, the prefetch-key: yes option
prefetches the DNSKEY query when a referral is followed.

RFC7706 root zone copy would speed up all the TLD referrals that unbound
wants from walking all the root referrals, by fetching the root zone
data in bulk.

Best regards, Wouter

> 
> How should I tune the cache?  Ideally, (but unbound likely can't
> do this), the NS/A/AAAA/DNSKEY records of domains that have delegated
> sub-domains would be cached in a separate cache (maybe even a
> separate max-ttl) from the cache that handles "leaf" domains.
> 
> In the mean time I probably need a medium-sized infra cache and a
> small data cache?  Not quite sure how to tune a nameserver whose
> dominant client walks ~6 or more million domains scattered across
> various TLDs and 2LDs, getting a few records from each domain
> (DNSKEY, MX, A + TLSA for each MX stopping early if same MX already
> seen at some other domain or is in an unsigned zone) and moves on
> to the next domain, visiting each delegated domain "once" (a few
> related queries in quick succession, in parallel with a few hundred
> similar domains).
> 
> As mentioned originally, 3.5 billion queries, 1.4 million cache
> hits!  Any advice on tuning for "surveys"?
> 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.nlnetlabs.nl/pipermail/unbound-users/attachments/20180523/8f15df8b/attachment.bin>