From me at cynthia.re Wed Feb 5 09:54:14 2020 From: me at cynthia.re (=?UTF-8?Q?Cynthia_Revstr=C3=B6m?=) Date: Wed, 5 Feb 2020 10:54:14 +0100 Subject: [RPKI] ARIN Identity.xml format Message-ID: Hello, So there is a bit of a lack of documentation when it comes to the format of the "Up/Down Identity XML" file that ARIN wants when you are setting up Delegated RPKI and the format of the parent response from ARIN. ARIN's XML files are not RFC8183 but rather something a bit odd that I had to craft by hand based on example files I found in old git repositories. I mainly posted this in the case that someone else has a similar issue since well as I said, docs are lacking atm. (I have also brought it up with NLNetLabs so they can discuss it with ARIN) When running `krillc parents myid` I got something in the format of: MII... Where as ARIN seemingly wanted (and accepted) the format of: MII... And the parent response had a similar issue, this is what I got from ARIN: MII... MII... Where as krill/RFC8183 wants the format of: MII... - Cynthia -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim at nlnetlabs.nl Wed Feb 5 10:04:22 2020 From: tim at nlnetlabs.nl (Tim Bruijnzeels) Date: Wed, 5 Feb 2020 11:04:22 +0100 Subject: [RPKI] ARIN Identity.xml format In-Reply-To: References: Message-ID: <6C7473F6-9D01-4A58-9AEE-C896835F0B58@nlnetlabs.nl> Hi Cynthia, Thank you for sharing this, and for giving Krill a chance :) So, it looks like rpkid from rpki.net introduced a version 2 identity exchange after RFC 8183 had been published and that ARIN supports this. This is odd because RFC 8183 itself is based on the original rpki.net implementation. And, in fact, I saw a message to the nanog list from someone who had issues getting their, presumable older version of, rpkid to work with the exchange. I will take this up with ARIN. As chance will have it I will meet up with them this Friday. I will advise them to stick to the RFC. If an update to the format is needed then that should be discussed in the IETF first. In the meantime, thank you! We can document this, and perhaps do some trickery of sorts if we must. But, I prefer that ARIN changes things. They can also polish off their xslt skills ;) Tim > On 5 Feb 2020, at 10:54, Cynthia Revstr?m via RPKI wrote: > > Hello, > > So there is a bit of a lack of documentation when it comes to the format of the "Up/Down Identity XML" file that ARIN wants when you are setting up Delegated RPKI and the format of the parent response from ARIN. > ARIN's XML files are not RFC8183 but rather something a bit odd that I had to craft by hand based on example files I found in old git repositories. > I mainly posted this in the case that someone else has a similar issue since well as I said, docs are lacking atm. (I have also brought it up with NLNetLabs so they can discuss it with ARIN) > > When running `krillc parents myid` I got something in the format of: > > MII... > > > Where as ARIN seemingly wanted (and accepted) the format of: > > MII... > > > And the parent response had a similar issue, this is what I got from ARIN: > version="2" > valid_until="2120-02-05T09:01:23Z" > service_uri="http://updown.arin.net/ARIN/QUL-4 " > parent_handle="ARIN" > child_handle="QUL-4"> > > MII... > > > MII... > > > > > Where as krill/RFC8183 wants the format of: > version="1" > valid_until="2120-02-05T09:01:23Z" > service_uri="http://updown.arin.net/ARIN/QUL-4 " > parent_handle="ARIN" > child_handle="QUL-4"> > > MII... > > > > - Cynthia > -- > RPKI mailing list > RPKI at lists.nlnetlabs.nl > https://lists.nlnetlabs.nl/mailman/listinfo/rpki -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim at nlnetlabs.nl Wed Feb 5 12:51:28 2020 From: tim at nlnetlabs.nl (Tim Bruijnzeels) Date: Wed, 5 Feb 2020 13:51:28 +0100 Subject: [RPKI] ARIN Identity.xml format In-Reply-To: <6C7473F6-9D01-4A58-9AEE-C896835F0B58@nlnetlabs.nl> References: <6C7473F6-9D01-4A58-9AEE-C896835F0B58@nlnetlabs.nl> Message-ID: <53056466-D85E-4317-A8F3-3C410A013F68@nlnetlabs.nl> Hi all, It was pointed out to me that the format that uses '' actually pre-dates the RFC. I was thrown off by the version="2" in there, but presumably that only applied to that particular namespace and the version number got reset for the RFC. My mistake. Rob Austein who did most of the rpkid coding and author of RFC 8183 also published an XSL file that can help translating between the two formats: https://raw.githubusercontent.com/dragonresearch/rpki.net/master/potpourri/oob-translate.xsl I will still urge ARIN to update their portal to use the RFC style, but having this XSL available should help. Tim > On 5 Feb 2020, at 11:04, Tim Bruijnzeels wrote: > > Hi Cynthia, > > Thank you for sharing this, and for giving Krill a chance :) > > So, it looks like rpkid from rpki.net introduced a version 2 identity exchange after RFC 8183 had been published and that ARIN supports this. This is odd because RFC 8183 itself is based on the original rpki.net implementation. And, in fact, I saw a message to the nanog list from someone who had issues getting their, presumable older version of, rpkid to work with the exchange. > > I will take this up with ARIN. As chance will have it I will meet up with them this Friday. I will advise them to stick to the RFC. If an update to the format is needed then that should be discussed in the IETF first. > > In the meantime, thank you! We can document this, and perhaps do some trickery of sorts if we must. But, I prefer that ARIN changes things. They can also polish off their xslt skills ;) > > Tim > >> On 5 Feb 2020, at 10:54, Cynthia Revstr?m via RPKI > wrote: >> >> Hello, >> >> So there is a bit of a lack of documentation when it comes to the format of the "Up/Down Identity XML" file that ARIN wants when you are setting up Delegated RPKI and the format of the parent response from ARIN. >> ARIN's XML files are not RFC8183 but rather something a bit odd that I had to craft by hand based on example files I found in old git repositories. >> I mainly posted this in the case that someone else has a similar issue since well as I said, docs are lacking atm. (I have also brought it up with NLNetLabs so they can discuss it with ARIN) >> >> When running `krillc parents myid` I got something in the format of: >> >> MII... >> >> >> Where as ARIN seemingly wanted (and accepted) the format of: >> >> MII... >> >> >> And the parent response had a similar issue, this is what I got from ARIN: >> > version="2" >> valid_until="2120-02-05T09:01:23Z" >> service_uri="http://updown.arin.net/ARIN/QUL-4 " >> parent_handle="ARIN" >> child_handle="QUL-4"> >> >> MII... >> >> >> MII... >> >> >> >> >> Where as krill/RFC8183 wants the format of: >> > version="1" >> valid_until="2120-02-05T09:01:23Z" >> service_uri="http://updown.arin.net/ARIN/QUL-4 " >> parent_handle="ARIN" >> child_handle="QUL-4"> >> >> MII... >> >> >> >> - Cynthia >> -- >> RPKI mailing list >> RPKI at lists.nlnetlabs.nl >> https://lists.nlnetlabs.nl/mailman/listinfo/rpki > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aistis at heficed.com Mon Feb 17 10:42:40 2020 From: aistis at heficed.com (=?utf-8?B?QWlzdGlzIFplbmtldmnEjWl1cw==?=) Date: Mon, 17 Feb 2020 10:42:40 +0000 Subject: [RPKI] HA strategy Message-ID: Hi list Was wondering if you have tips on running Krill in redundant configuration. I've ran it for half a week out of a prebuilt docker instance with a couple of CAs with no issue and plan on moving forward with a lot of more. My preference is multi-DC redundance, in case one of the sites goes down. So for the frontend I'm thinking between the lines of DNS with minimal TTL (e.g. via Route53) or unicast, rsync served files can probably be sinced with some rudimentary script, but what about CA backend itself? Thanks Aistis From tim at nlnetlabs.nl Mon Feb 17 13:07:10 2020 From: tim at nlnetlabs.nl (Tim Bruijnzeels) Date: Mon, 17 Feb 2020 14:07:10 +0100 Subject: [RPKI] HA strategy In-Reply-To: References: Message-ID: Hi Aistis, First of all, thank you for using Krill and congratulations on being the first to run it in production under the RIPE NCC! I will just stick to the generic story here. My colleague Ximon is on holidays at the moment, but he may be able to provide some docker specific insights at some point. Krill does not yet support running in a cluster or multi-master setup. This is something we want to look at later this year. For now the best approach is that you have a stand-by node, and that you use some kind of redundant storage for your main krill instance's data directory. If your main krill node dies, you can then bring up a secondary node using the same data. Linux HA may be useful, and there are docker strategies, that I don't have hands-on experience with, but which can help here. When Krill is unavailable you will not be able to make updates to your ROAs, and Krill will not (re-)publish objects at a repository. However, this does not need to affect relying parties as long as the repository remains available, and Krill comes back before objects start to become stale (you have 16 hours with the current built-in defaults, we can think of stretching this). So, for example, users of Krill under nic.br have the option of running their CA and publishing with nic.br . So their uptime requirements for Krill are relaxed, because nic.br makes the repository available. If you need to run your own repository, then it is advisable to use multiple rsync servers and multiple http servers. You can also use a CDN in front of your http servers. Sadly, our documentation is still somewhat lacking on how to do this part of the setup. This will be fixed soon. In the meantime let me try to give an overview here. Krill can use an embedded repository if so configured. You will need to set the following directives in the config file: repo_enabled = false rsync_base = "rsync:///repo/" rrdp_service_uri = "https:///rrdp/" Krill will write the repository files under its data directory: $DATA_DIR/repo/rsync/current/ Contains the files for rsync $DATA_DIR/repo/rrdp/ Contains the files for https (RRDP as the protocol is called in the RFC) You can share the contents of these directories with your repository servers in various ways. You could have some kind of redundant shared filesystem where the krill CA can write, and your repository servers can read. But you could also synchronise the contents of these directories in another way, e.g. rsync them over every couple of minutes. If you are using a shared filesystem, then please note that the rsync 'current' directory cannot be the mountpoint. Krill tries to write the entire repo to a new folder under $DATA_DIR/repo/rsync and then renames it. This is done to minimise issues with files being updated while validators are fetching data. You will then need to configure your rsync deamons to expose a 'module' for your files. Make sure that the rsync URI including the 'module' matches the 'rsync_base' in Krill's configuration file. Basic configuration can then be as simple as: root at krillprod:~# cat /etc/rsyncd.conf uid = nobody gid = nogroup max connections = 50 socket options = SO_KEEPALIVE [repo] path = /var/lib/krill/data/repo/rsync/current/ comment = RPKI repository read only = yes For the http you will need to set up your favourite http server and make sure that it has a valid https certificate (eg use letsencrypt). Then you can make the files found under, or copied from, $DATA_DIR/repo/rrdp here. Make sure that the public URI to the rrdp base dir matches the value of 'rrdp_service_uri' in your krill.conf. If you want, you can also use a CDN in front of this server to further reduce your load and uptime requirements. If you do, be sure that the public URI matched the directive in the krill.conf because this will be used in certificates. Now, to make matters even more interesting.. it is very hard - almost impossible - to change any of this set up in Krill's embedded repository after it has been initialised. This is at least in part because we want to encourage users to use a repository provided by their NIR or RIR whenever possible. That said, we understand that you may not have that option today.. Therefore we recommend that, if you run your own repository, you do so by running a separate Krill instance for this purpose only. Do not configure any CAs under it. Just let your CA publish at this repository as is described here: https://rpki.readthedocs.io/en/latest/krill/remote-publishing.html#krill-as-repository-server If you find that there is an issue with your repository, e.g. your URIs are wrong, then you can set up a new Krill instance with embedded repository. When you are satisfied that this one is correct, you can migrate your CA to it by adding them as a publisher under the new repository server, and then updating your CA to use the new repository. Krill will then make sure that objects are moved properly, and that a new certificate is requested from your parent(s) to match the new location. Note, if/when your RIR starts providing a repository service you can also update your CA to start publishing there. As I said, I understand that this part of the documentation still needs some love. I will look at this as soon as I am done with the upcoming 0.5 release. For now, I hope this helps. Thanks Tim > On 17 Feb 2020, at 11:42, Aistis Zenkevi?ius via RPKI wrote: > > Hi list > > Was wondering if you have tips on running Krill in redundant configuration. I've ran it for half a week out of a prebuilt docker instance with a couple of CAs with no issue and plan on moving forward with a lot of more. > > My preference is multi-DC redundance, in case one of the sites goes down. So for the frontend I'm thinking between the lines of DNS with minimal TTL (e.g. via Route53) or unicast, rsync served files can probably be sinced with some rudimentary script, but what about CA backend itself? > > Thanks > Aistis > > > > -- > RPKI mailing list > RPKI at lists.nlnetlabs.nl > https://lists.nlnetlabs.nl/mailman/listinfo/rpki -------------- next part -------------- An HTML attachment was scrubbed... URL: From aistis at heficed.com Wed Feb 19 12:04:47 2020 From: aistis at heficed.com (=?utf-8?B?QWlzdGlzIFplbmtldmnEjWl1cw==?=) Date: Wed, 19 Feb 2020 12:04:47 +0000 Subject: [RPKI] HA strategy In-Reply-To: References:

Message-ID: Hi Tim Thanks a lot for an extensive reply. So in essense, I have 16 hours to bring my repo back up one way or another. Good idea regarding Krill instance dedicated solely for CA purposes and then configuring it to publish into Repo Krill. I assume there's no way to configure it right now so it publishes into multiple repos, right? This, at least on paper, would simplify data dir synchronization between redundant repo servers. So in my case, setup should move from (single instance): 1. Add CA 2. Grab CA identity XML, hand it over to RIR 3. Configure embedded repo for 4. Add parent for with RIR response XML 5. Configure HTTPS proxy pass for RRDP 6. Configure rsyncd to serve files To: 1. Create CA Krill instance somewhere safe 2. Create Repo Krill instance 3. Add CA on CA Krill 4. Grab publisher request 5. Add CA Krill as publisher on Repo Krill, grab response 6. repo update rfc8183 on CA Krill 7. [???] Add parent for with RIR response XML on CA Krill [???] This can be HA'd to ones taste, either geo redundant or single site with tools that one likes: 8. Configure HTTPS proxy pass for RRDP 9. Configure rsyncd to serve files Something feels off above. Did I get it right? Thanks, Aistis From: Tim Bruijnzeels Sent: 2020 m. vasario 17 d., pirmadienis 15:07 To: Aistis Zenkevi?ius Cc: rpki at lists.nlnetlabs.nl Subject: Re: [RPKI] HA strategy Hi Aistis, First of all, thank you for using Krill and congratulations on being the first to run it in production under the RIPE NCC! I will just stick to the generic story here.?My colleague Ximon is on holidays at the moment, but he may be able to provide some docker specific insights at some point. Krill does not yet support running in a cluster or multi-master setup. This is something we want to look at later this year. For now the best approach is that you have a stand-by node, and that you use some kind of redundant storage for your main krill instance's data directory. If your main krill node dies, you can then bring up a secondary node using the same data. Linux HA may be useful, and there are docker strategies, that I don't have hands-on experience with, but which can help here. When Krill is unavailable you will not be able to make updates to your ROAs, and Krill will not (re-)publish objects at a repository. However, this does not need to affect relying parties as long as the repository remains available, and Krill comes back before objects start to become stale (you have 16 hours with the current built-in defaults, we can think of stretching this). So, for example, users of Krill under http://nic.br?have the option of running their CA and publishing with http://nic.br. So their uptime requirements for Krill are relaxed, because http://nic.br?makes the repository available. If you need to run your own repository, then it is advisable to use multiple rsync servers and multiple http servers. You can also use a CDN in front of your http servers. Sadly, our documentation is still somewhat lacking on how to do this part of the setup. This will be fixed soon. In the meantime let me try to give an overview here. Krill can use an embedded repository if so configured. You will need to set the following directives in the config file: repo_enabled = false rsync_base = "rsync:///repo/" rrdp_service_uri = "https:///rrdp/" Krill will write the repository files under its data directory: $DATA_DIR/repo/rsync/current/ ? ?Contains the files for rsync $DATA_DIR/repo/rrdp/ ? ? ? ? ? ? Contains the files for https (RRDP as the protocol is called in the RFC) You can share the contents of these directories with your repository servers in various ways. You could have some kind of redundant shared filesystem where the krill CA can write, and your repository servers can read. But you could also synchronise the contents of these directories in another way, e.g. rsync them over every couple of minutes. If you are using a shared filesystem, then please note that the rsync 'current' directory cannot be the mountpoint. Krill tries to write the entire repo to a new folder under $DATA_DIR/repo/rsync and then renames it. This is done to minimise issues with files being updated while validators are fetching data. You will then need to configure your rsync deamons to expose a 'module' for your files. Make sure that the rsync URI including the 'module' matches the 'rsync_base' in Krill's configuration file. Basic configuration can then be as simple as: root at krillprod:~# cat /etc/rsyncd.conf uid = nobody gid = nogroup max connections = 50 socket options = SO_KEEPALIVE ? [repo] path = /var/lib/krill/data/repo/rsync/current/ comment = RPKI repository read only = yes For the http you will need to set up your favourite http server and make sure that it has a valid https certificate (eg use letsencrypt). Then you can make the files found under, or copied from, $DATA_DIR/repo/rrdp here. Make sure that the public URI to the rrdp base dir matches the value of 'rrdp_service_uri' in your krill.conf. If you want, you can also use a CDN in front of this server to further reduce your load and uptime requirements. If you do, be sure that the public URI matched the directive in the krill.conf because this will be used in certificates. Now, to make matters even more interesting.. it is very hard - almost impossible - to change any of this set up in Krill's embedded repository after it has been initialised. This is at least in part because we want to encourage users to use a repository provided by their NIR or RIR whenever possible. That said, we understand that you may not have that option today.. Therefore we recommend that, if you run your own repository, you do so by running a separate Krill instance for this purpose only. Do not configure any CAs under it. Just let your CA publish at this repository as is described here:?https://rpki.readthedocs.io/en/latest/krill/remote-publishing.html#krill-as-repository-server If you find that there is an issue with your repository, e.g. your URIs are wrong, then you can set up a new Krill instance with embedded repository. When you are satisfied that this one is correct, you can migrate your CA to it by adding them as a publisher under the new repository server, and then updating your CA to use the new repository. Krill will then make sure that objects are moved properly, and that a new certificate is requested from your parent(s) to match the new location. Note, if/when your RIR starts providing a repository service you can also update your CA to start publishing there. As I said, I understand that this part of the documentation still needs some love. I will look at this as soon as I am done with the upcoming 0.5 release. For now, I hope this helps. Thanks Tim On 17 Feb 2020, at 11:42, Aistis Zenkevi?ius via RPKI wrote: Hi list Was wondering if you have tips on running Krill in redundant configuration. I've ran it for half a week out of a prebuilt docker instance with a couple of CAs with no issue and plan on moving forward with a lot of more. My preference is multi-DC redundance, in case one of the sites goes down. So for the frontend I'm thinking between the lines of DNS with minimal TTL (e.g. via Route53) or unicast, rsync served files can probably be sinced with some rudimentary script, but what about CA backend itself? Thanks Aistis -- RPKI mailing list mailto:RPKI at lists.nlnetlabs.nl https://lists.nlnetlabs.nl/mailman/listinfo/rpki From tim at nlnetlabs.nl Wed Feb 19 16:06:16 2020 From: tim at nlnetlabs.nl (Tim Bruijnzeels) Date: Wed, 19 Feb 2020 17:06:16 +0100 Subject: [RPKI] HA strategy In-Reply-To: References:

Message-ID: Hi Aistis, Re-sending because my email client (or I..) somehow messed up and my reply was not plain text as it should have been.. > On 19 Feb 2020, at 13:04, Aistis Zenkevi?ius wrote: > > Hi Tim > > Thanks a lot for an extensive reply. So in essense, I have 16 hours to bring my repo back up one way or another. You have 16 hours to fix your Krill CA and/or Repo instances. It's good to keep your repositories available as much as possible - i.e. the data. Most RPs will work with cached data, but it's best to be safe. > > Good idea regarding Krill instance dedicated solely for CA purposes and then configuring it to publish into Repo Krill. I assume there's no way to configure it right now so it publishes into multiple repos, right? This, at least on paper, would simplify data dir synchronization between redundant repo servers. > The RPKI standards only allow a single publication point in RPKI objects. So, you cannot get redundancy through secondary repositories, using other public URIs. Theoretically you could have multiple repository servers that share the same public URI - (DNS, anycast, or simply loadbalanced) - while they have separate back-end service ports - and then CAs could, theoretically, be configured to publish with each separately. However, this would be quite complicated of the CAs, and it could easily result in differences in the objects that are available at each repo instance, which would be confusing. All in all I think that sharing the data between the front-ends to the repo (httpd/rsyncd) is the easiest for now. Long term we could look into repo servers that are aware of each other, and that can synchronise somehow - though I am not sure that the benefit is worth the complexity just yet. Secondary servers is something that would have to go through the IETF and I think there would be some pushback there. > So in my case, setup should move from (single instance): > > 1. Add CA > 2. Grab CA identity XML, hand it over to RIR > 3. Configure embedded repo for > 4. Add parent for with RIR response XML > 5. Configure HTTPS proxy pass for RRDP > 6. Configure rsyncd to serve files > > To: > > 1. Create CA Krill instance somewhere safe > 2. Create Repo Krill instance > 3. Add CA on CA Krill > 4. Grab publisher request > 5. Add CA Krill as publisher on Repo Krill, grab response > 6. repo update rfc8183 on CA Krill > 7. [???] Add parent for with RIR response XML on CA Krill [???] yes > > This can be HA'd to ones taste, either geo redundant or single site with tools that one likes: > 8. Configure HTTPS proxy pass for RRDP > 9. Configure rsyncd to serve files > > Something feels off above. Did I get it right? > I would leave the Repo and CA Krills as single instances. Hot standby can work, provided that you keep their data dirs on reliable storage. I would then HA the https proxy for RRDP and rsyncd based on the datadir which is updated by a single instance Repo Krill. The RRDP content needs to be available to the world, so even though Krill can serve this data you are better off including the files on your proxy host. E.g. something like: root at krillprod:/etc/nginx/sites-enabled# cat rrdp.rpki.nlnetlabs.nl server { server_name rrdp.rpki.nlnetlabs.nl; location /rrdp/ { alias /var/lib/krill/repo/rrdp/; } listen 443 ssl; # managed by Certbot listen [::]:443 ssl; ssl_certificate /etc/letsencrypt/live/rrdp.rpki.nlnetlabs.nl/fullchain.pem; # managed by Certbot ssl_certificate_key /etc/letsencrypt/live/rrdp.rpki.nlnetlabs.nl/privkey.pem; # managed by Certbot include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot } server { if ($host = rrdp.rpki.nlnetlabs.nl) { return 301 https://$host$request_uri; } # managed by Certbot server_name rrdp.rpki.nlnetlabs.nl; listen 80; listen [::]:80; return 404; # managed by Certbot } If you want to make the API and UI (next week in 0.5!) available you can proxy everything else to Krill - you could then of course also choose to limit access in that config to your own network. Tim > > Thanks, > Aistis > > From: Tim Bruijnzeels > Sent: 2020 m. vasario 17 d., pirmadienis 15:07 > To: Aistis Zenkevi?ius > Cc: rpki at lists.nlnetlabs.nl > Subject: Re: [RPKI] HA strategy > > Hi Aistis, > > First of all, thank you for using Krill and congratulations on being the first to run it in production under the RIPE NCC! > > I will just stick to the generic story here. My colleague Ximon is on holidays at the moment, but he may be able to provide some docker specific insights at some point. > > Krill does not yet support running in a cluster or multi-master setup. This is something we want to look at later this year. For now the best approach is that you have a stand-by node, and that you use some kind of redundant storage for your main krill instance's data directory. If your main krill node dies, you can then bring up a secondary node using the same data. Linux HA may be useful, and there are docker strategies, that I don't have hands-on experience with, but which can help here. > > When Krill is unavailable you will not be able to make updates to your ROAs, and Krill will not (re-)publish objects at a repository. > > However, this does not need to affect relying parties as long as the repository remains available, and Krill comes back before objects start to become stale (you have 16 hours with the current built-in defaults, we can think of stretching this). So, for example, users of Krill under http://nic.br have the option of running their CA and publishing with http://nic.br. So their uptime requirements for Krill are relaxed, because http://nic.br makes the repository available. > > If you need to run your own repository, then it is advisable to use multiple rsync servers and multiple http servers. You can also use a CDN in front of your http servers. Sadly, our documentation is still somewhat lacking on how to do this part of the setup. This will be fixed soon. In the meantime let me try to give an overview here. > > Krill can use an embedded repository if so configured. You will need to set the following directives in the config file: > > repo_enabled = false > rsync_base = "rsync:///repo/" > rrdp_service_uri = "https:///rrdp/" > > Krill will write the repository files under its data directory: > > $DATA_DIR/repo/rsync/current/ Contains the files for rsync > $DATA_DIR/repo/rrdp/ Contains the files for https (RRDP as the protocol is called in the RFC) > > You can share the contents of these directories with your repository servers in various ways. You could have some kind of redundant shared filesystem where the krill CA can write, and your repository servers can read. But you could also synchronise the contents of these directories in another way, e.g. rsync them over every couple of minutes. If you are using a shared filesystem, then please note that the rsync 'current' directory cannot be the mountpoint. Krill tries to write the entire repo to a new folder under $DATA_DIR/repo/rsync and then renames it. This is done to minimise issues with files being updated while validators are fetching data. > > You will then need to configure your rsync deamons to expose a 'module' for your files. Make sure that the rsync URI including the 'module' matches the 'rsync_base' in Krill's configuration file. Basic configuration can then be as simple as: > > root at krillprod:~# cat /etc/rsyncd.conf > uid = nobody > gid = nogroup > max connections = 50 > socket options = SO_KEEPALIVE > > [repo] > path = /var/lib/krill/data/repo/rsync/current/ > comment = RPKI repository > read only = yes > > For the http you will need to set up your favourite http server and make sure that it has a valid https certificate (eg use letsencrypt). Then you can make the files found under, or copied from, $DATA_DIR/repo/rrdp here. Make sure that the public URI to the rrdp base dir matches the value of 'rrdp_service_uri' in your krill.conf. If you want, you can also use a CDN in front of this server to further reduce your load and uptime requirements. If you do, be sure that the public URI matched the directive in the krill.conf because this will be used in certificates. > > > Now, to make matters even more interesting.. it is very hard - almost impossible - to change any of this set up in Krill's embedded repository after it has been initialised. This is at least in part because we want to encourage users to use a repository provided by their NIR or RIR whenever possible. That said, we understand that you may not have that option today.. > > > Therefore we recommend that, if you run your own repository, you do so by running a separate Krill instance for this purpose only. Do not configure any CAs under it. Just let your CA publish at this repository as is described here: https://rpki.readthedocs.io/en/latest/krill/remote-publishing.html#krill-as-repository-server > > > If you find that there is an issue with your repository, e.g. your URIs are wrong, then you can set up a new Krill instance with embedded repository. When you are satisfied that this one is correct, you can migrate your CA to it by adding them as a publisher under the new repository server, and then updating your CA to use the new repository. Krill will then make sure that objects are moved properly, and that a new certificate is requested from your parent(s) to match the new location. > > > Note, if/when your RIR starts providing a repository service you can also update your CA to start publishing there. > > > > > As I said, I understand that this part of the documentation still needs some love. I will look at this as soon as I am done with the upcoming 0.5 release. For now, I hope this helps. > > > > Thanks > Tim > > > > > > > > > > > > On 17 Feb 2020, at 11:42, Aistis Zenkevi?ius via RPKI wrote: > > Hi list > > Was wondering if you have tips on running Krill in redundant configuration. I've ran it for half a week out of a prebuilt docker instance with a couple of CAs with no issue and plan on moving forward with a lot of more. > > My preference is multi-DC redundance, in case one of the sites goes down. So for the frontend I'm thinking between the lines of DNS with minimal TTL (e.g. via Route53) or unicast, rsync served files can probably be sinced with some rudimentary script, but what about CA backend itself? > > Thanks > Aistis > > > > -- > RPKI mailing list > mailto:RPKI at lists.nlnetlabs.nl > https://lists.nlnetlabs.nl/mailman/listinfo/rpki From christopher at ve7alb.ca Wed Feb 19 18:31:34 2020 From: christopher at ve7alb.ca (Christopher Munz-Michielin) Date: Wed, 19 Feb 2020 10:31:34 -0800 Subject: [RPKI] ARIN Compatible switch in GUI not working Message-ID: <627452de-609e-0eaf-6c86-2fbc2caca1e3@ve7alb.ca> Hi All, Just upgraded to the latest version of Krill with the WebUI (pretty slick!) and saw there was an ARIN-compatible switch when generating the identity file.? As I needed to resubmit my identity to ARIN anyway (due to an unrelated change) I figured I would try it out, however, ARIN's interface still doesn't accept the XML that Krill puts out.? Turns out that Krill outputs: ? .......... However,? ARIN's interface wants ? .......... Making the changes to the xml manually worked,? so not a big deal but something that should probably be fixed for a future release. Thanks again for the awesome software! From alex at nlnetlabs.nl Wed Feb 19 18:48:00 2020 From: alex at nlnetlabs.nl (Alex Band) Date: Wed, 19 Feb 2020 19:48:00 +0100 Subject: [RPKI] ARIN Compatible switch in GUI not working In-Reply-To: <627452de-609e-0eaf-6c86-2fbc2caca1e3@ve7alb.ca> References: <627452de-609e-0eaf-6c86-2fbc2caca1e3@ve7alb.ca> Message-ID: <1B48CF60-C47C-4D53-80CF-00171DF52BA7@nlnetlabs.nl> Hi Christopher, Thanks for the heads up! Let?s see if we can still fix this before the 0.5 release. I created a GitHub issue here: https://github.com/NLnetLabs/lagosta/issues/27 Cheers, Alex > On 19 Feb 2020, at 19:31, Christopher Munz-Michielin via RPKI wrote: > > Hi All, > > Just upgraded to the latest version of Krill with the WebUI (pretty slick!) and saw there was an ARIN-compatible switch when generating the identity file. As I needed to resubmit my identity to ARIN anyway (due to an unrelated change) I figured I would try it out, however, ARIN's interface still doesn't accept the XML that Krill puts out. Turns out that Krill outputs: > > > > .......... > > > > However, ARIN's interface wants > > > .......... > > > > Making the changes to the xml manually worked, so not a big deal but something that should probably be fixed for a future release. > > Thanks again for the awesome software! > -- > RPKI mailing list > RPKI at lists.nlnetlabs.nl > https://lists.nlnetlabs.nl/mailman/listinfo/rpki From christopher at ve7alb.ca Wed Feb 19 22:42:35 2020 From: christopher at ve7alb.ca (Christopher Munz-Michielin) Date: Wed, 19 Feb 2020 14:42:35 -0800 Subject: [RPKI] ARIN Compatible switch in GUI not working In-Reply-To: <1B48CF60-C47C-4D53-80CF-00171DF52BA7@nlnetlabs.nl> References: <627452de-609e-0eaf-6c86-2fbc2caca1e3@ve7alb.ca> <1B48CF60-C47C-4D53-80CF-00171DF52BA7@nlnetlabs.nl> Message-ID: Nice, looks like it's already been patched. Cheers, Chris On 2020-02-19 10:48 a.m., Alex Band wrote: > Hi Christopher, > > Thanks for the heads up! Let?s see if we can still fix this before the 0.5 release. I created a GitHub issue here: > > https://github.com/NLnetLabs/lagosta/issues/27 > > Cheers, > > Alex > >> On 19 Feb 2020, at 19:31, Christopher Munz-Michielin via RPKI wrote: >> >> Hi All, >> >> Just upgraded to the latest version of Krill with the WebUI (pretty slick!) and saw there was an ARIN-compatible switch when generating the identity file. As I needed to resubmit my identity to ARIN anyway (due to an unrelated change) I figured I would try it out, however, ARIN's interface still doesn't accept the XML that Krill puts out. Turns out that Krill outputs: >> >> >> >> .......... >> >> >> >> However, ARIN's interface wants >> >> >> .......... >> >> >> >> Making the changes to the xml manually worked, so not a big deal but something that should probably be fixed for a future release. >> >> Thanks again for the awesome software! >> -- >> RPKI mailing list >> RPKI at lists.nlnetlabs.nl >> https://lists.nlnetlabs.nl/mailman/listinfo/rpki From alex at nlnetlabs.nl Tue Feb 25 11:58:26 2020 From: alex at nlnetlabs.nl (Alex Band) Date: Tue, 25 Feb 2020 12:58:26 +0100 Subject: [RPKI] Krill 0.5.0 'Serve no Turf' Released Message-ID: <04E76ECD-F746-4C6B-837E-B9E092FA7A8C@nlnetlabs.nl> Dear mailing list, We are very excited about introducing Krill 0.5.0 'Serve No Turf'. This release is a major step forward for the project, improving the installation, onboarding, interoperability and usability of Krill. The most prominent change you will notice is that Krill now offers a multi-language user interface, allowing you to set up a Certificate Authority, perform the parent exchanges with one or more Regional and National Internet Registries, configure a publication server and manage Route Origin Authorisations (ROAs). The UI is managed as a separate open source project, named Lagosta. It uses Vue.js as Javascript framework and Element as UI framework. The first release, Lagosta 0.1.0 'Fritto Misto', offers everything to get started with Krill. You can expect the UI to evolve heavily over time, including ROA suggestions, tagging and alerts. The user interface is compiled as static HTML and JS, which is bundled in the Krill package. We're proud that this addition resulted in just an 8MB increase of memory usage, making Krill still completely capable of running on minimalist hardware such as a Raspberry Pi. But the UI is not the only major change in version 0.5.0. The documentation received an overhaul, clearly laying out every aspect of the software. Krill can now be installed from a new Linux machine in just seven steps, greatly simplifying the process. You can them immediately start using it using either the UI, CLI or API. Lastly, Krill now also offers a Prometheus monitoring end point. Finally, we ironed out some interoperability issues with ARIN and APNIC, ensuring Krill works reliably with all RIRs. We wish to thank Cynthia Revstr?m for the fantastic help she provided in ironing out some issues we found when setting up Krill under ARIN. You can find out more about the development and features in our blog post. On behalf of the NLnet Labs RPKI team, Alex Release notes: https://github.com/NLnetLabs/krill/releases/tag/v0.5.0 Evolving Krill - A blog post about the latest features in Krill 0.5.0: https://medium.com/nlnetlabs/evolving-krill-1313243437dd Krill Github repository: https://github.com/NLnetLabs/krill Lagosta Github repository: https://github.com/NLnetLabs/lagosta Krill Documentation: https://rpki.readthedocs.io/en/latest/krill/index.html From Rich.Compton at charter.com Fri Feb 28 17:27:12 2020 From: Rich.Compton at charter.com (Compton, Rich A) Date: Fri, 28 Feb 2020 17:27:12 +0000 Subject: [RPKI] "Permission denied (13)" rsync errors Message-ID: <22B25DA4-F70A-48D4-9759-62076C7272BB@contoso.com> Hi, I am trying to run routinator on a Centos7 VM. I created a systemd startup script and I'm running routinator at boot with it. I am getting "Permission denied (13)" errors on rsync. Also, the /metrics URL keeps saying "Initial validation ongoing. Please wait." I'm getting some different errors if I just start routinator from the command line (see below). When I start routinator this way, the /metrics URL shows stats so I'm assume it's working. Errors if I start routinator with the systemd script: Feb 28 15:42:41 rpki-validator3 routinator: rsync://repository.lacnic.net/rpki: rsync: failed to connect to repository.lacnic.net (2001:13c7:7002:4128::137): Permission denied (13) Feb 28 15:42:41 rpki-validator3 routinator: rsync://repository.lacnic.net/rpki: rsync: failed to connect to repository.lacnic.net (2001:13c7:7002:4128::185): Permission denied (13) Feb 28 15:42:41 rpki-validator3 routinator: rsync://repository.lacnic.net/rpki: rsync: failed to connect to repository.lacnic.net (2001:13c7:7002:4128::136): Permission denied (13) Feb 28 15:42:41 rpki-validator3 routinator: rsync://repository.lacnic.net/rpki: rsync: failed to connect to repository.lacnic.net (200.3.14.137): Permission denied (13) Feb 28 15:42:41 rpki-validator3 routinator: rsync://repository.lacnic.net/rpki: rsync: failed to connect to repository.lacnic.net (200.3.14.185): Permission denied (13) Feb 28 15:42:41 rpki-validator3 routinator: rsync://repository.lacnic.net/rpki: rsync: failed to connect to repository.lacnic.net (200.3.14.136): Permission denied (13) Feb 28 15:42:41 rpki-validator3 routinator: rsync://repository.lacnic.net/rpki: rsync error: error in socket IO (code 10) at clientserver.c(125) [Receiver=3.1.2] Feb 28 15:42:41 rpki-validator3 routinator: rsync://rpki.arin.net/repository: failed with status exit code: 10 Errors when I start routinator from the command line: Feb 28 16:54:16 rpki-validator3 routinator[2128]: rsync://repository.lacnic.net/rpki: failed with status exit code: 23 Feb 28 16:54:16 rpki-validator3 routinator[2128]: rsync://repository.lacnic.net/rpki: rsync: failed to set times on "/home/rcompton/routinator/.../rsync/repository.lacnic.net/rpki/lacnic/7eef83c3-8e97-4b54-a597-1036cb1dffae": Operation not permitted (1) Feb 28 16:54:16 rpki-validator3 routinator[2128]: rsync://repository.lacnic.net/rpki: rsync: opendir "/home/rcompton/routinator/.../rsync/repository.lacnic.net/rpki/lacnic/7eef83c3-8e97-4b54-a597-1036cb1dffae" failed: Permission denied (13) Feb 28 16:54:16 rpki-validator3 routinator[2128]: rsync://repository.lacnic.net/rpki: rsync: recv_generator: failed to stat "/home/rcompton/routinator/.../rsync/repository.lacnic.net/rpki/lacnic/7eef83c3-8e97-4b54-a597-1036cb1dffae/0f82d16e54d63e1fcbad2873ef9377e70690daad.roa": Permission denied (13) Feb 28 16:54:16 rpki-validator3 routinator[2128]: rsync://repository.lacnic.net/rpki: rsync: recv_generator: failed to stat "/home/rcompton/routinator/.../rsync/repository.lacnic.net/rpki/lacnic/7eef83c3-8e97-4b54-a597-1036cb1dffae/67fd53696c8aeee0f94a9db5d41967d282caea23.roa": Permission denied (13) Feb 28 16:54:16 rpki-validator3 routinator[2128]: rsync://repository.lacnic.net/rpki: rsync: recv_generator: failed to stat "/home/rcompton/routinator/.../rsync/repository.lacnic.net/rpki/lacnic/7eef83c3-8e97-4b54-a597-1036cb1dffae/b56daa0fd6210a6c89a9e60487c2c65e72218f5d.crl": Permission denied (13) Feb 28 16:54:16 rpki-validator3 routinator[2128]: rsync://repository.lacnic.net/rpki: rsync: recv_generator: failed to stat "/home/rcompton/routinator/.../rsync/repository.lacnic.net/rpki/lacnic/7eef83c3-8e97-4b54-a597-1036cb1dffae/b56daa0fd6210a6c89a9e60487c2c65e72218f5d.mft": Permission denied (13) Feb 28 16:54:16 rpki-validator3 routinator[2128]: rsync://repository.lacnic.net/rpki: rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1650) [generator=3.1.2] [signature_351058561] Rich Compton | Principal Eng | 314.596.2828 8560 Upland Drive, Suite B | Englewood, CO 80112 E-MAIL CONFIDENTIALITY NOTICE: The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 23501 bytes Desc: image001.png URL: