[RPKI] APNIC had an unexpected drop in VRP 00:00 - 02:00
Chris Caputo
ccaputo at alt.net
Wed Dec 2 03:32:50 UTC 2020
On Wed, 2 Dec 2020, George Michaelson via RPKI wrote:
> On Wed, Dec 2, 2020 at 3:45 AM Job Snijders <job at ntt.net> wrote:
> >
> > On Tue, Dec 01, 2020 at 01:29:58PM +1000, George Michaelson wrote:
> > > We have received reports that our RPKI repository was producing zero
> > > VRP from 00:00 to 02:00 today, Tuesday 01 December. This was visible
> > > in Seattle and may have been seen elsewhere.
> > >
> > > We are looking into what happened and will report back as soon as possible.
> >
> > Some preliminary analysis on my side suggests this event might have been
> > RRDP-specific.
> >
> > On (multiple) rsync-only RPKI collectors I did not observe a drop in
> > VRPs in the 00:00-02:00 UTC time frame. Hope this helps debugging.
> >
> > Kind regards,
> >
> > Job
>
> We continue to investigate.
>
> Not all RPs saw this, and it appears that the problem was due to
> recent updates to some relying party software.
With Routinator 0.8.1 I observed:
rsync://rpki.apnic.net/repository/B527EF581D6611E2BB468F7C72FD1FF2/ZwTFeTEC0uxi4JpTfGQbsyoqqhM.cer: CA certificate failed to validate.
CA for rsync://rpki.apnic.net/repository/B527EF581D6611E2BB468F7C72FD1FF2/ rejected, resources marked as unsafe:
1.0.0.0/8
14.0.0.0/8
27.0.0.0/8
36.0.0.0/8
39.0.0.0/8
42.0.0.0/7
45.64.0.0-45.65.63.255
45.112.0.0/12
45.248.0.0/13
[...]
and:
rsync://rpki.apnic.net/repository/B3A24F201D6611E28AC8837C72FD1FF2/0I2GgcK-TUfCopBV9m5olVhGF_c.cer: CA certificate failed to validate.
CA for rsync://rpki.apnic.net/repository/B3A24F201D6611E28AC8837C72FD1FF2/ rejected, resources marked as unsafe:
8.128.0.0/10
8.208.0.0/12
23.106.120.0/21
23.106.248.0/21
23.108.96.0/21
23.111.12.0/22
23.226.0.0/20
23.232.128.0/17
24.41.112.0/20
[...]
Decoded versions of those certificates are below. Both expired at Dec 1
00:00:00 2020 GMT and then the problem ensued. Also at:
http://console.rpki-client.org/rpki.apnic.net/repository/B527EF581D6611E2BB468F7C72FD1FF2/ZwTFeTEC0uxi4JpTfGQbsyoqqhM.cer.html
http://console.rpki-client.org/rpki.apnic.net/repository/B3A24F201D6611E28AC8837C72FD1FF2/0I2GgcK-TUfCopBV9m5olVhGF_c.cer.html
Is there an explanation for how these expired certs contributed to a wide
impact?
RIPE Validator reported "Not valid after time is in the past:
2020-12-01T00:00:00.000Z" in regards to the APNIC trust anchor, at the
time. I'm guessing that is not Routinator based, or is it?
Thanks,
Chris
---------------------------------------------------------------
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 47483 (0xb97b)
Signature Algorithm: sha256WithRSAEncryption
Issuer: CN = A90DC5BE, serialNumber = 0E65A4F5FD36B5BD68EB3C923408978C907AA79F
Validity
Not Before: Oct 23 10:14:32 2019 GMT
Not After : Dec 1 00:00:00 2020 GMT
Subject: CN = A91CFAC8, serialNumber = 6704C5793102D2EC62E09A537C641BB32A2AAA13
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public-Key: (2048 bit)
Modulus:
00:b9:42:87:00:97:d1:22:10:d4:0a:04:6d:17:d0:
a0:4c:85:a7:c1:8b:2f:94:cd:ab:fa:2d:d2:ce:4c:
82:65:98:03:b9:66:6e:a5:f2:35:d6:a3:df:89:42:
63:e4:9a:8a:92:64:fd:06:b1:fb:48:5d:72:11:03:
98:71:f3:30:1b:87:5d:b1:fa:00:a9:d9:b2:45:f3:
05:8e:4b:45:7c:42:cf:9b:cf:38:2c:d1:5e:fa:df:
b1:bb:15:30:57:37:9d:f2:b5:21:1f:bf:97:d8:3e:
ad:ba:86:62:88:8f:7a:54:b4:10:f4:d2:db:46:76:
79:34:93:ee:c4:88:da:2d:68:18:55:b7:f7:06:6c:
3f:63:87:7c:9b:76:ff:77:99:2f:39:59:b5:77:c5:
cb:07:d1:7e:45:f4:ed:e1:0a:d3:a0:76:90:ee:6d:
98:d3:20:d9:d1:67:79:12:25:09:bc:e4:2f:15:06:
38:54:79:84:77:a4:83:56:28:14:7f:b6:21:62:c4:
92:e7:ad:67:90:1b:da:94:17:b1:2f:20:f1:a4:9f:
9d:38:72:6c:4e:f8:9b:b7:b6:48:43:5b:38:16:89:
a0:1b:27:6a:02:3a:78:bd:3d:0b:8c:75:15:6e:41:
23:7d:b9:a4:c3:ea:08:92:a6:ce:c8:76:07:30:19:
41:c1
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Subject Key Identifier:
67:04:C5:79:31:02:D2:EC:62:E0:9A:53:7C:64:1B:B3:2A:2A:AA:13
X509v3 Authority Key Identifier:
keyid:0E:65:A4:F5:FD:36:B5:BD:68:EB:3C:92:34:08:97:8C:90:7A:A7:9F
X509v3 Key Usage: critical
Certificate Sign, CRL Sign
X509v3 Basic Constraints: critical
CA:TRUE
X509v3 CRL Distribution Points:
Full Name:
URI:rsync://rpki.apnic.net/repository/B527EF581D6611E2BB468F7C72FD1FF2/DmWk9f02tb1o6zySNAiXjJB6p58.crl
Authority Information Access:
CA Issuers - URI:rsync://rpki.apnic.net/repository/980652E0B77E11E7A96A39521A4F4FB4/DmWk9f02tb1o6zySNAiXjJB6p58.cer
X509v3 Certificate Policies: critical
Policy: 1.3.6.1.5.5.7.14.2
CPS: https://www.apnic.net/RPKI/CPS.pdf
Subject Information Access:
CA Repository - URI:rsync://rpki.apnic.net/member_repository/A91CFAC8/3E167A58292711E692DAA117C4F9AE02/
1.3.6.1.5.5.7.48.10 - URI:rsync://rpki.apnic.net/member_repository/A91CFAC8/3E167A58292711E692DAA117C4F9AE02/ZwTFeTEC0uxi4JpTfGQbsyoqqhM.mft
1.3.6.1.5.5.7.48.13 - URI:https://rrdp.apnic.net/notification.xml
sbgp-autonomousSysNum: critical
0...0......
sbgp-ipAddrBlock: critical
0.0.....0......p0....0....$...
Signature Algorithm: sha256WithRSAEncryption
a7:93:36:5e:6f:51:35:71:09:52:a1:d7:58:5b:09:fd:41:bb:
39:ee:a9:8f:77:93:94:cf:6e:0c:8d:f5:75:c7:6c:d3:70:95:
ea:72:af:13:94:f5:d7:41:62:24:26:dd:1e:08:8d:d1:e3:cb:
fe:e4:be:12:29:4a:ca:7f:f9:8f:98:f1:b4:0c:49:c9:12:8f:
f7:18:f6:90:61:9e:da:fd:75:35:bf:5b:55:a6:39:24:8d:82:
d4:cd:72:39:4d:03:c4:8f:e2:8f:bc:dd:48:c4:09:6e:61:6e:
13:28:7b:58:bf:43:0b:58:b3:b7:fc:4d:93:90:05:15:10:fe:
e9:7d:3c:17:7f:41:f4:5d:8b:62:27:77:f8:5f:d4:9e:e4:e7:
8c:e0:96:d0:42:4a:e5:73:6f:dd:3d:47:77:be:0e:69:96:c1:
ef:74:ef:e4:cb:df:63:81:35:b1:cb:73:c6:8f:ad:b2:c8:cb:
c1:a0:f6:c4:ed:9f:a5:9f:f8:2b:3a:06:5b:cb:1e:5f:93:38:
b5:e5:57:e0:05:f0:ee:e3:14:d2:7d:73:c4:29:f4:5d:87:71:
1a:87:8a:e4:57:18:f3:79:02:50:0f:be:66:e5:f9:5c:c8:42:
e4:6d:3c:37:33:47:a6:26:f8:68:37:a0:fa:3b:0d:dc:63:b3:
f7:56:6b:0d
---------------------------------------------------------------
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 11713 (0x2dc1)
Signature Algorithm: sha256WithRSAEncryption
Issuer: CN = A90DC5BE, serialNumber = 740165A80D1071970ABC09C02B71C1AC7C1D6E0E
Validity
Not Before: Sep 9 04:12:15 2019 GMT
Not After : Dec 1 00:00:00 2020 GMT
Subject: CN = A91FCEB1, serialNumber = D08D8681C2BE4D47C2A29055F66E6895584617F7
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public-Key: (2048 bit)
Modulus:
00:a8:b4:0e:5f:0e:c6:db:84:62:b1:5b:a3:23:36:
a4:7a:c2:91:ae:e1:35:8d:39:77:0c:83:46:c2:7b:
f4:aa:b3:7d:d2:60:d6:de:88:06:19:76:79:98:20:
ce:52:56:52:ee:88:09:9b:0c:54:33:79:84:09:ce:
4a:62:14:08:f5:ca:8d:ee:b6:64:4a:1d:fa:76:48:
22:29:36:7c:1e:5d:79:86:e7:df:da:70:cc:fd:72:
75:76:43:ad:c0:17:69:fa:b3:db:32:77:81:70:8b:
1f:b7:a2:0a:8b:61:96:f1:1e:88:e7:4a:8a:44:e6:
20:1a:25:63:ac:5d:7b:b8:4a:8e:bc:3c:ff:66:49:
20:8b:49:bd:5e:4e:6d:dc:f5:79:55:e4:f3:79:ee:
b6:c6:c3:e7:79:18:c8:46:da:ae:b5:c1:ad:71:8c:
57:c7:4a:e9:70:88:8d:f1:ab:92:cb:75:f0:51:d1:
df:f3:81:2e:84:f6:7a:0e:93:46:c5:84:99:29:cc:
0b:86:b6:4f:00:0a:24:4a:7d:b3:45:bd:2d:72:f6:
57:31:22:2c:a7:8e:30:9e:2c:0c:6f:cc:da:ca:c4:
72:48:42:55:92:54:f2:eb:e0:9b:33:33:c1:b0:82:
8a:76:3d:54:06:a2:24:99:72:61:99:de:2e:e5:7f:
4f:4f
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Subject Key Identifier:
D0:8D:86:81:C2:BE:4D:47:C2:A2:90:55:F6:6E:68:95:58:46:17:F7
X509v3 Authority Key Identifier:
keyid:74:01:65:A8:0D:10:71:97:0A:BC:09:C0:2B:71:C1:AC:7C:1D:6E:0E
X509v3 Key Usage: critical
Certificate Sign, CRL Sign
X509v3 Basic Constraints: critical
CA:TRUE
X509v3 CRL Distribution Points:
Full Name:
URI:rsync://rpki.apnic.net/repository/B3A24F201D6611E28AC8837C72FD1FF2/dAFlqA0QcZcKvAnAK3HBrHwdbg4.crl
Authority Information Access:
CA Issuers - URI:rsync://rpki.apnic.net/repository/980652E0B77E11E7A96A39521A4F4FB4/dAFlqA0QcZcKvAnAK3HBrHwdbg4.cer
X509v3 Certificate Policies: critical
Policy: 1.3.6.1.5.5.7.14.2
CPS: https://www.apnic.net/RPKI/CPS.pdf
Subject Information Access:
CA Repository - URI:rsync://rpki.apnic.net/member_repository/A91FCEB1/9C14B4063A2111E79AA0D51CC4F9AE02/
1.3.6.1.5.5.7.48.10 - URI:rsync://rpki.apnic.net/member_repository/A91FCEB1/9C14B4063A2111E79AA0D51CC4F9AE02/0I2GgcK-TUfCopBV9m5olVhGF_c.mft
1.3.6.1.5.5.7.48.13 - URI:https://rrdp.apnic.net/notification.xml
sbgp-ipAddrBlock: critical
0.0.....0......*
Signature Algorithm: sha256WithRSAEncryption
b0:af:f5:85:0f:4c:0d:75:08:b1:1a:56:62:3b:74:85:a0:ea:
06:6f:d6:de:1d:f5:04:b0:59:bd:80:e0:9d:ae:78:2b:23:c3:
78:6d:30:f2:f4:af:96:f8:dc:eb:3e:4a:b4:a1:4f:46:33:ca:
25:06:ba:31:3c:82:bc:09:59:15:a1:47:51:98:a5:57:17:82:
1b:de:16:b4:58:41:d5:32:80:e2:55:78:21:66:5f:8f:b6:fb:
0c:96:3d:d9:3b:58:bf:57:ad:cc:c1:af:e7:3c:71:9d:81:e1:
98:23:24:77:a9:c6:59:1e:8f:1e:fe:a9:d2:0c:84:64:6d:a4:
87:b0:65:bf:03:b2:18:be:9a:d1:48:2b:b1:1f:97:98:a4:ec:
06:0a:ab:c2:e3:3f:5a:84:bd:01:00:29:b9:e5:fe:3f:cc:e1:
be:5c:dc:76:a9:0f:21:13:45:7a:e2:06:5f:eb:98:c9:55:16:
6c:31:19:64:78:2d:b6:df:c7:e3:3d:30:9b:ec:8c:8f:fe:39:
04:29:48:c2:d8:b2:07:e6:41:e6:f8:15:f1:d8:8d:46:7c:95:
d9:b8:51:53:67:0a:f9:88:8e:87:56:66:a9:df:fd:95:2f:01:
17:c5:84:e2:ee:af:5c:36:c8:b2:4f:89:48:ec:50:7a:ae:17:
a3:d6:dd:a1
More information about the RPKI
mailing list