[RPKI] transcient differences between rpki-client and routinator

[Claudio Jeker]

On Mon, Dec 09, 2019 at 12:06:53PM +0100, Martin Hoffmann wrote:
> Hi Job!
> 
> Job Snijders wrote:
> > 
> > The rsync data was fetched around Thu 05 Dec 2019 13:30:12 UTC
> > rpki-client (first) and routinator (second) were done at Thu 05 Dec
> > 2019 13:34:54 UTC
> 
> [...]
> > 
> > a snapshot of the data of that run is available here
> > http://instituut.net/~job/rpki-repository.6E17IG9pm.tar.gz
> 
> The snapshot seems to be from around 2019-12-05 22:30 UTC. There still
> is a failure to decode one AFRINIC manifest but that doesn’t seem to be
> time related. Seems to be a ROA for AS0 on, among other things,
> 196.10.140.0/24, if I translated the BIT STRING correctly.
> 
> > the script that runs tools one after the other and compares the output
> > is available here:
> > https://gist.github.com/job/ea11fc59b2411e042eaad1c1b0213c74
> > 
> > Now, what is very curious to me is that based on the same data input,
> > rpki-client and routinator don't /always/ produce the same output. I'd
> > say that it seems that 80% of the time they have the same output, and
> > 20% of the time there are minute differences such as below:
> > 
> >     hanna:~ job$ diff rpki-repository.6E17IG9pm/export-routinator.cvs
> > rpki-repository.6E17IG9pm/export-rpki-client.cvs 34792d34791
> >     < AS207036,200.1.154.0/24,24,lacnic
> > 
> > Does any one have an idea what can explain these differences? Is there
> > perhaps some timestamp difference in an intermediate certificate where
> > routinator decides that the ROA for 200.1.154.0/24 is not valid, or is
> > there some check that rpki-client is maybe skipping over? What made
> > '200.1.154.0/24,24,AS207036' valid in the eyes of rpki-client, but not
> > in the eyes of routinator?
> 
> Isn’t it the other way around? At least I thought the "<" points to the
> side where the line actually is?
> 
> In any case, Routinator does accept it:
> 
> | m at glaurung:/tmp/foo$ faketime "2019-12-05 23:30" \
> | > routinator --disable-rrdp -t ~/.rpki-cache/newtals/ -r /tmp/foo/ \
> | > vrps -nf csv | grep 200.1.154.0
> | AS207036,200.1.154.0/24,24,lacnic
> 

The reason why rpki-client did not accept this ROA is because the CRL is
bad:

rpki-client: repository.lacnic.net/rpki/lacnic/8bb5ab6a-b91f-439b-bdfb-eb3c43470d36/17ccf2e528990c3d8b0172b3372656083e3ddf4f.crl: bad message digest

and therefore the certificate for the roa could not be verified:

rpki-client: repository.lacnic.net/rpki/lacnic/8bb5ab6a-b91f-439b-bdfb-eb3c43470d36/0c2f7644106123110d3619cf85dd4440bdc67119.roa: unable to get certificate CRL

In dir repository.lacnic.net/rpki/lacnic/8bb5ab6a-b91f-439b-bdfb-eb3c43470d36/
there is the mft, crl and roa we are interested in.
Looking at 17ccf2e528990c3d8b0172b3372656083e3ddf4f.mft we see it
references the CRL and ROA with the following hashes:
    1: 0c2f7644106123110d3619cf85dd4440bdc67119.roa
        hash g6bCSNza52glKOZfjbxZHZ/FwgYMJSVnFRLtEklyH7Q=
    2: 17ccf2e528990c3d8b0172b3372656083e3ddf4f.crl
        hash 1OOBfzIFTAUAsD7LTLVFKi9WxdTy2d+ht+wXIo4TOuw=

But sha256 tells us this:
SHA256 (0c2f7644106123110d3619cf85dd4440bdc67119.roa) =
			g6bCSNza52glKOZfjbxZHZ/FwgYMJSVnFRLtEklyH7Q=
SHA256 (17ccf2e528990c3d8b0172b3372656083e3ddf4f.crl) =
			XSmoaab95/UuLUEbokJahsRSiMU6QIrwuLUNDV/giIM=

As you can see the hash for the CRL is totally wrong and this is why
rpki-client is dropping the ROA because the certificate validation fails
since the CRL could not be loaded.

I have seen this happen on a few occasions that lacnic publishes MFTs with
bad checksums. I guess somebody should talk to them to understand why this
happens.

Also why does routinator not fail for this ROA? I find it dangerous to fail
open in such a case. It mostly disables revocation.

-- 
:wq Claudio