wildcard dnssec test fails

[Viktor Dukhovni]

On Tue, Dec 19, 2017 at 06:08:50AM +0000, Viktor Dukhovni wrote:

> The original coded uses non-portable undefined overflow behaviour
> for signed integer arithmetic.  The compiler is free to replace
> "incep - expi > 0" with "incep > expi".  The intermediate "var"
> may in some cases avoid the problem, but this is still brittle
> under optimization.  To avoid non-deterministic behaviour unsigned
> arithmetic must be used:
> 
>     uint32_t incep;
>     uint32_t expi;
> 
>     /*
>      * In serial number arithmetic a > b iff as unsigned integers mod 2^32
>      * we have (a - b) < (b - a)
>      */
>     if ((incep - expi) < (expi - incep)) {
> 	... fail ...
>     }
> 
> The same code should be used for SOA comparisons.

I should perhaps note that in the RFC1982 definition of sequence
space arithmetic, two points that are diametrically opposite on
the circle are not comparable.

Since such ambiguity should be a failure case, a more precisely
correct condition is

    if ((incep - expi) <= (expi - incep)) {
	... fail ...
    }

Note that this now also includes incep == expi, which should never
be the case for RRSIGs, and so for RRSIG failure makes sense for
both equal and diametrically opposite values.  When comparing SOA
serials for AXFR (perhaps not something unbound ever needs to do),
a pair of equal values would of course be treated differently than
a pair or diametrically opposite values.

-- 
	Viktor.