Very strange DNSSEC validation failure affecting Unbound

[max at nummer378.de]

Hello to all,

I've exhausted most of my options at this point, so I'm now asking here. 
I've encountered one of the strangest DNSSEC issues I've ever seen.

Let's get straight to the point. One of the two affected FQDNs is:

home.local.magisystems.de

the other one is

koenigsberg.local.magisystems.de

If you try to resolve that using Unbound, with the validator module 
enabled & trust anchors configured, you will get a SERVFAIL from 
Unbound. If you also have EDE enabled, you will see:

EDE: 10 (RRSIGs Missing): (validation failure 
<home.local.magisystems.de. A IN>: no signatures from <...>)

However, if you ask one of the nameservers directly, you will see that 
the FQDN in question does have a proper RRSig:

> dig home.local.magisystems.de +dnssec @ns1.hosting.de
>
> ; <<>> DiG 9.18.16-1~deb12u1-Debian <<>> home.local.magisystems.de 
> +dnssec @ns1.hosting.de
> ;; global options: +cmd
> ;; Got answer:
> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 42931
> ;; flags: qr aa rd; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
> ;; WARNING: recursion requested but not available
>
> ;; OPT PSEUDOSECTION:
> ; EDNS: version: 0, flags: do; udp: 1232
> ;; QUESTION SECTION:
> ;home.local.magisystems.de.     IN      A
>
> ;; ANSWER SECTION:
> home.local.magisystems.de. 3600 IN      A       172.22.22.27
> home.local.magisystems.de. 3600 IN      RRSIG   A 8 4 3600 
> 20230808083531 20230718083031 62664 magisystems.de. 
> RSoBY/8Nqt/2iATHt2rW98bTGOAaF1l7j0ACMJW5ezTLo9zCpMJOa0mt 
> nbZApJ78hK92dvp3kk2n545YNQtyRbidGg6Yo8J1hg2ZNqltuIwFdQQm 
> B3Aoq7xemueX78xVGgaBIUjAi6HiJOggz3Ty/AxzvOMOLqx1p+woK3aL 7+w=

Now, let's make this even more strange: Try to resolve this FQDN using 
any other public resolver not running Unbound: Cloudflare, Google Public 
DNS, Quad9, you name it: If it's not running Unbound, it will have zero 
trouble resolving the FQDN.

Some facts about the issue:

  * The zone in question is, to my best knowledge, properly DNSSEC signed
  * Only Unbound has trouble resolving this FQDN: All other resolvers
    I've tried can resolve it just fine
  * All other FQDNs on the same zone work without any issue: For
    example, try out local.magisystems.de or just magisystems.de:
    Unbound can resolve them just fine
  * I've already spoken to the DNS hosting provider (hosting.de). Just
    like me, they're clueless. IIRC, they're running PowerDNS and we
    couldn't identify any other zone that has the same issue
  * We tried regenerating the RRSig, without any change in the behaviour
  * We have reproduced this using 5 different Unbound installs in 2
    different countries. We tried older and recent versions (up to the
    current 1.17.1)
  * The issue has persisted over multiple weeks now and is most
    certainly not related to caching

I don't own the domain in question, though I do know the person owning 
it, so I can request changes to the zone. I'm absolutely clueless as to 
what is going wrong here: DNSViz.net doesn't see anything wrong with the 
DNSSEC. I myself run dozens of domains using the exact same 
configuration: *All* of them resolve properly using Unbound. Only this 
FQDN has trouble. It uses the same key type/size, signature algorithm, 
everything is identical to how the other zones are configured.

Does anyone have an idea? At this point I'm inclined to believe we've 
hit some bug in Unbound, but I honestly don't know what.

Kind regards,
Max
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nlnetlabs.nl/pipermail/unbound-users/attachments/20230720/c8bc74dc/attachment.htm>