RPZ: is this config correct?

Thu Apr 22 14:40:41 UTC 2021

Hello,

I like to attach myself to this longer thread...

Am 10.11.20 um 17:53 schrieb RayG via Unbound-users:

> I have no issue getting the data using the URL in a browser but unbound flatly refuses to successfully retrieve it.
> 
>      name: "URLHaus"
>      zonefile: "C:\ProgramData\Unbound\Logs\urlhaus.zone"
>      url: https://urlhaus.abuse.ch/downloads/rpz
>      rpz-log: yes
>      rpz-log-name: "URLHausRPZ"
>      rpz-action-override: nxdomain
> 
> I have tried with an empty file, a populated file and no file but nothing works. No file is ever created.

My idea was to use RPZ and also start with urlhaus available as https://urlhaus.abuse.ch/downloads/rpz
Running 1.13.1 this failed partially. So here are my findings.
To be as reproducable as possible, I describe a docker setup and show some unnessesary exlicit options:

$ install -d /tmp/rpz/ && cd /tmp/rpz/

$ cat <<EOF > /tmp/rpz/unbound.conf
server:
 chroot: ""
 do-daemonize: no
 do-ip6: no
 logfile: ""
 log-replies: yes
 module-config: "respip validator iterator"
 tls-cert-bundle: /usr/share/ca-certificates/mozilla/GlobalSign_Root_CA_-_R3.crt
rpz:
 name: rpz.urlhaus.abuse.ch.
 zonefile: /tmp/spool/rpz.urlhaus.abuse.ch
 url: https://urlhaus.abuse.ch/downloads/rpz/
 rpz-log: yes
EOF

$ docker run --name unbound --rm -ti -v /tmp/rpz:/tmp/rpz:rw debian:testing-slim
root at f94664d2bfc7:/# apt-get update
...
root at f94664d2bfc7:/# apt-get -qq --no-install-recommends install ca-certificates knot-dnsutils unbound wget
...
root at f94664d2bfc7:/# install -d --owner unbound /tmp/spool/

root at f94664d2bfc7:/# unbound -c /tmp/rpz/unbound.conf
[1619098696] unbound[2749:0] notice: init module 0: respip
[1619098696] unbound[2749:0] notice: init module 1: validator
[1619098696] unbound[2749:0] notice: init module 2: iterator
[1619098696] unbound[2749:0] info: start of service (unbound 1.13.1).

According to unbound documentation I expext unbound to start downloading 
https://urlhaus.abuse.ch/downloads/rpz/ to /tmp/spool/rpz.urlhaus.abuse.ch
The directory /tmp/spool/ *is* writable for the unbound user.

That download does not happen.

Consequently dns queries are answered like no rpz is present at all.
Let's login from an other shell into the unbound container and ask for the test entry:

$ docker exec -ti unbound bash
root at f94664d2bfc7:/# kdig @127.0.0.1 testentry.rpz.urlhaus.abuse.ch.
;; ->>HEADER<<- opcode: QUERY; status: NXDOMAIN; id: 16488
;; Flags: qr rd ra; QUERY: 1; ANSWER: 0; AUTHORITY: 1; ADDITIONAL: 0

;; QUESTION SECTION:
;; testentry.rpz.urlhaus.abuse.ch.              IN      A

;; AUTHORITY SECTION:
abuse.ch.               1800    IN      SOA     ns1.p04.dynect.net. dnsadmin.abuse.ch. 2021042100 3600 600 604800 1800

;; Received 111 B
;; Time 2021-04-22 14:00:35 UTC
;; From 127.0.0.1 at 53(UDP) in 21.7 ms

the other terminal shows the log:
...
[1619098696] unbound[2749:0] info: start of service (unbound 1.13.1).
[1619100035] unbound[2874:0] info: 127.0.0.1 testentry.rpz.urlhaus.abuse.ch. A IN NXDOMAIN 0.021310 0 111

This is unexpected because there is an AUTHORITY SECTION

Now let's download https://urlhaus.abuse.ch/downloads/rpz/ manually.
We use 'wget' and explicit set ca-directory to an empty /opt/ and ca-certificate the the one matching CA so only this one download is 'permitted'
This is only to demonstrate there are *no* TLS certificate validatation issues.
Also, we run 'wget' as user unbound. This makes the resulting file owned an thus writable by unbound.

press CTRL+C in the unbound terminal

^C[1619099541] unbound[2856:0] info: service stopped (unbound 1.13.1).
[1619099541] unbound[2856:0] info: server stats for thread 0: 1 queries, 0 answers from cache, 1 recursions, 0 prefetch, 0 rejected by ip ratelimiting
[1619099541] unbound[2856:0] info: server stats for thread 0: requestlist max 0 avg 0 exceeded 0 jostled 0
[1619099541] unbound[2856:0] info: average recursion processing time 0.031167 sec
[1619099541] unbound[2856:0] info: histogram of recursion processing times
[1619099541] unbound[2856:0] info: [25%]=0 median[50%]=0 [75%]=0
[1619099541] unbound[2856:0] info: lower(secs) upper(secs) recursions
[1619099541] unbound[2856:0] info:    0.016384    0.032768 1

root at f94664d2bfc7:/#  su --shell /bin/sh --command 'wget --no-verbose -O /tmp/spool/rpz.urlhaus.abuse.ch --ca-directory=/opt --ca-certificate=/usr/share/ca-certificates/mozilla/GlobalSign_Root_CA_-_R3.crt https://urlhaus.abuse.ch/downloads/rpz/' unbound
2021-04-22 14:29:19 URL:https://urlhaus.abuse.ch/downloads/rpz/ [163814/163814] -> "/tmp/spool/rpz.urlhaus.abuse.ch" [1]

root at f94664d2bfc7:/# stat /tmp/spool/rpz.urlhaus.abuse.ch 
  File: /tmp/spool/rpz.urlhaus.abuse.ch
  Size: 163814          Blocks: 320        IO Block: 4096   regular file
Device: 34h/52d Inode: 123433959   Links: 1
Access: (0644/-rw-r--r--)  Uid: (  101/ unbound)   Gid: (  101/ unbound)
Access: 2021-04-22 14:29:19.000000000 +0000
Modify: 2021-04-22 14:25:03.000000000 +0000
Change: 2021-04-22 14:29:19.891804798 +0000
 Birth: 2021-04-22 14:29:19.619804800 +0000

again, start unbound:

root at f94664d2bfc7:/# unbound -c /tmp/rpz/unbound.conf
[1619099932] unbound[2866:0] notice: init module 0: respip
[1619099932] unbound[2866:0] notice: init module 1: validator
[1619099932] unbound[2866:0] notice: init module 2: iterator
[1619099932] unbound[2866:0] info: start of service (unbound 1.13.1).

in the other terminal now ask the *same* question:

root at f94664d2bfc7:/# kdig @127.0.0.1 testentry.rpz.urlhaus.abuse.ch. ns
;; ->>HEADER<<- opcode: QUERY; status: NXDOMAIN; id: 29041
;; Flags: qr aa rd ra; QUERY: 1; ANSWER: 0; AUTHORITY: 0; ADDITIONAL: 0

;; QUESTION SECTION:
;; testentry.rpz.urlhaus.abuse.ch.              IN      NS

;; Received 48 B
;; Time 2021-04-22 14:04:32 UTC
;; From 127.0.0.1 at 53(UDP) in 0.2 ms

and unbound logs this:
[1619100269] unbound[2880:0] info: start of service (unbound 1.13.1).
[1619100272] unbound[2880:0] info: RPZ applied testentry.rpz.urlhaus.abuse.ch. nxdomain 127.0.0.1 at 39411 testentry.rpz.urlhaus.abuse.ch. NS IN
[1619100272] unbound[2880:0] info: 127.0.0.1 testentry.rpz.urlhaus.abuse.ch. NS IN NXDOMAIN 0.000000 1 48

so we see, the same question is now handled by the rpz and is answered (in this case also) with NXDOMAIN
But this happen only if I manually provide the rpz data file.

I also setup a second nginx container that can serve http://local-nginx/rpz.urlhaus.abuse.ch to unbound
This eliminates https and allow running tcpdump on the local docker network: There is no communication from unbound to the local-nginx

But the documentation (https://github.com/NLnetLabs/unbound/blob/release-1.13.1/doc/unbound.conf.5.in#L2404)
promise somehow "If the file does not exist or is empty, unbound will attempt to fetch zone data (eg. from the primary servers)"

Conclusion: rpz with URL works but bootstrapping is unsure.
I've also not yet monitored, if and when updates happen.

Andreas