DNS-0x20 encoding reduces cache hit count

[Robert Edmonds]

John Todd via Unbound-users wrote:
> I have not yet looked to see if new insertions are happening on new 0x20
> variants, but based on Peter's comments and evidence above it looks like that
> may be happening. We see many queries from downstream unbound systems with 0x20
> turned on, and if pihole is going to start doing that as a default that seems
> like it would cause quite a bit of un-necessary cache churn if it is indeed
> causing new entries to be saved on each 0x20 variation. For our installations,
> this probably isn't catastrophic since we have dnsdist in front of unbound and
> I suspect the dnsdist packet cache (which ignores 0x20 uniqueness when
> answering an existing in-memory item) minimizes the leakage of 0x20 issues
> through to the back-end recursive resolvers. However, if we have N front-end
> dnsdist instances pointing at a single unbound instance (I'm simplifying this
> for clarity) with clients sending 0x20 unique requests, then it would be the
> case that up to N requests with different 0x20 values would cause lookup events
> even within the cache timeout period in unbound. That would be sub-optimal.

As far as I can tell from using dig against a quiescent Unbound
instance and watching the msg.cache.count and rrset.cache.count metrics,
Unbound's RRset and message caches are case insensitive but appear to
be case retentive.

In any case, Unbound's hash functions internally coerce the name key
to lowercase before hashing, rather than requiring domain names to be
lowercased before calling the hash function:

https://github.com/NLnetLabs/unbound/blob/1894c0a1505c6791d6c9f6e77b7ff47cfc1f1545/util/data/dname.c#L300

https://github.com/NLnetLabs/unbound/blob/1894c0a1505c6791d6c9f6e77b7ff47cfc1f1545/util/data/dname.c#L336

So it would be physically impossible for Unbound's in-memory caches to
store multiple cache entries whose keys differ only by case, because
they would hash to the same value.

I have only skimmed the pi-hole discussion thread but I noticed "TTL:
does not expire" in the redis screenshots. I would guess the pi-hole
users are getting confused by the redis cache entries simply not
expiring, as documented in the unbound.conf manpage:

https://unbound.docs.nlnetlabs.nl/en/latest/manpages/unbound.conf.html#cache-db-module-options

	Note: Unbound never removes data stored in the Redis server, even
	if some data have expired in terms of DNS TTL or the Redis server
	has cached too much data; if necessary the Redis server must be
	configured to limit the cache size, preferably with some kind of
	least-recently-used eviction policy.

	Additionally, the redis-expire-records: option can be used in order to
	set the relative DNS TTL of the message as timeout to the Redis records;
	keep in mind that some additional memory is used per key and that
	the expire information is stored as absolute Unix timestamps in Redis
	(computer time must be stable).

So if a "docs.pi-hole.net" entry of whatever caps perturbation ever
expires from Unbound's in-memory caches (which of course would happen
frequently due to DNS TTL-based expiration), or Unbound is restarted,
additional cache entries for the same name but likely of different caps
perturbations would be sent to the redis cache when Unbound resolves the
name again, without overwriting the previous entries in the redis cache.
(Redis cache keys are case-sensitive/case-retentive.) And then when
you go and snoop in the redis cache, you'd find all the different caps
perturbations of the same name that had ever been stored in the redis
cache.

Probably it makes sense for Unbound's cachedb module to coerce the cache
key to lowercase when communicating with the redis server, though.

-- 
Robert Edmonds
edmonds at debian.org