RPZ: is this config correct?

Tue Nov 10 15:46:55 UTC 2020

Hi RayG,

You don't have to create the file before starting unbound. If the file 
is there unbound will try to parse it.

You don't have to manually populate the file with anything unless the 
rpz source is only a file. Not for your case though.

I see in your log:
...
10/11/2020 15:00:14 C:\Program Files\Unbound\unbound.exe[15932:0] debug: 
read zonefile C:\ProgramData\Unbound\Logs\rpz.urlhaus.abuse.ch for 
rpz.urlhaus.abuse.ch.
...
10/11/2020 15:05:24 C:\Program Files\Unbound\unbound.exe[15932:0] debug: 
auth zone rpz.urlhaus.abuse.ch. transfer failed, wait
...

The first one shows unbound reading from the file it created from a 
previous run probably.

The second one shows that unbound could not complete the transfer and 
will try later.

Best regards,
-- George

On 10/11/2020 16:33, RayG wrote:
> OK so with log level set at 4 I don’t see in the log file lines like you list below
> 
> The log file is very large so I cannot attach it to this email even as pasted text.
> 
> Here is a link:
> https://1drv.ms/u/s!As73rPtzISrU4mTvPONvZCWVSCWD?e=MJ18Lx
> 
> A couple of points.
> 
> 1). Do I have to create the zone file before starting unbound?
> 2). Do I need to populate the file with anything?
> 
> I have tried all ways with no success.
> 
> Thanks
> 
> RayG
> 
> -----Original Message-----
> From: George Thessalonikefs <george at nlnetlabs.nl>
> Sent: 09 November 2020 17:35
> To: unbound-users at lists.nlnetlabs.nl
> Cc: RayG <rgsub1 at btinternet.com>
> Subject: Re: RPZ: is this config correct?
> 
> Hi RayG,
> 
> On verbosity >= 4 you could see the following entries that relate to rpz (from my own run where download and file creation succeed):
>       debug: auth zone rpz.urlhaus.abuse.ch. transfer next HTTP fetch from
>       debug: http download downloads/rpz of size
>       info: auth zone http downloaded content preview:
>       debug: auth zone rpz.urlhaus.abuse.ch. updated to serial
>       debug: write zonefile file.name for rpz.urlhaus.abuse.ch.
> 
> local-zone answers are before the rpz zones, so you will not see entries in the log file for those.
> 
> Best regards,
> -- George
> 
> On 09/11/2020 17:55, RayG wrote:
>> Hi George,
>>
>> Thanks for the reply.
>>    
>> I agree the XFR may not work but the URL should get the zone file.
>>
>> Given I have pointed the zone file at unbound's log directory and it
>> can write the log OK I think it should be able to write the zone file.
>> It writes the log OK.
>>
>> Again if I have understood correctly these two lines:
>>
>> rpz-log: yes
>> rpz-log-name: "URLHausRPZ"
>>
>> Make unbound write entries in the log file using the label "URLHausRPZ"
>>
>> I see none of those or a zone file in the log directory.
>>
>> What should I be looking for in unbound's log file to show it's working?
>>
>> One other question does it matter that some of the same entries may
>> have been entered in a block list file
>>
>> Which has entries looking like this:
>>
>> local-zone: "0.nextyourcontent.com" refuse # Source:
>> https://someonewhocares.org/hosts/
>> local-zone: "0.r.msn.com" refuse # Source:
>> http://winhelp2002.mvps.org/hosts.txt
>> local-zone: "0.start.bz" refuse # Source:
>> http://sysctl.org/cameleon/hosts.win
>> local-zone: "180clubrealestate.com" refuse # See:
>> https://urlhaus.abuse.ch/host/180clubrealestate.com
>>
>> RayG
>>
>> -----Original Message-----
>> From: George Thessalonikefs <george at nlnetlabs.nl>
>> Sent: 09 November 2020 11:07
>> To: unbound-users at lists.nlnetlabs.nl
>> Subject: Re: RPZ: is this config correct?
>>
>> Hi RayG,
>>
>> You are correct that the file should be written by unbound. Are you
>> sure that unbound has write permissions in that directory?
>>
>> You could also use IP addresses for XFRs and they will be probed for
>> the SOA value and also tried if the url does not work.
>>
>> However, I don't think that they offer the service over XFR. At least
>> they only advertise the url on their website.
>>
>> Best regards,
>> -- George
>>
>> On 07/11/2020 16:17, RayG via Unbound-users wrote:
>>> Hi,
>>>
>>> No response to this post as yet?
>>>
>>> Any help appreciated.
>>>
>>> RayG
>>>
>>> *From:*RayG <rgsub1 at btinternet.com>
>>> *Sent:* 14 October 2020 15:59
>>> *To:* 'unbound-users at lists.nlnetlabs.nl'
>>> <unbound-users at lists.nlnetlabs.nl>
>>> *Subject:* RPZ: is this config correct?
>>>
>>> I have created the following RPZ entry for unbound and added respip
>>> to the module configuration.
>>>
>>> rpz:
>>>
>>>         name: "rpz.urlhaus.abuse.ch."
>>>
>>>         zonefile: "c:\programdata\unbound\logs\URLHaus.rpz"
>>>
>>>         url: https://urlhaus.abuse.ch/downloads/rpz
>>> <https://urlhaus.abuse.ch/downloads/rpz>
>>>
>>>         rpz-log: yes
>>>
>>>         rpz-log-name: "URLHausRPZ"
>>>
>>> If I understand things correctly unbound should fetch the zone file
>>> using the URL and store the data in the zonefile. I created an empty
>>> zone file but it is not being populated by unbound. I cannot see any
>>> relevant issues in the log file.  I also do not (have not yet) seen
>>> any entries in the log file with the appended log name item.
>>>
>>> Do I have the correct configuration and understanding?
>>>
>>> Following on would it be correct to add these masters to the
>> configuration:
>>>
>>>         master: 151.101.130.49
>>>
>>>         master: 151.101.66.49
>>>
>>>         master: 151.101.194.49
>>>
>>>         master: 151.101.2.49
>>>
>>> C:\>dig urlhaus.abuse.ch.
>>>
>>> ; <<>> DiG 9.16.6 <<>> urlhaus.abuse.ch.
>>>
>>> ;; global options: +cmd
>>>
>>> ;; Got answer:
>>>
>>> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1870
>>>
>>> ;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 1
>>>
>>> ;; OPT PSEUDOSECTION:
>>>
>>> ; EDNS: version: 0, flags:; udp: 4096
>>>
>>> ;; QUESTION SECTION:
>>>
>>> ;urlhaus.abuse.ch.              IN      A
>>>
>>> ;; ANSWER SECTION:
>>>
>>> urlhaus.abuse.ch.       3037    IN      CNAME
>> p2.shared.global.fastly.net.
>>>
>>> p2.shared.global.fastly.net. 29 IN      A       151.101.130.49
>>>
>>> p2.shared.global.fastly.net. 29 IN      A       151.101.194.49
>>>
>>> p2.shared.global.fastly.net. 29 IN      A       151.101.2.49
>>>
>>> p2.shared.global.fastly.net. 29 IN      A       151.101.66.49
>>>
>>> The URL Returns data like this:
>>>
>>> $TTL 30
>>>
>>> @ SOA rpz.urlhaus.abuse.ch. hostmaster.urlhaus.abuse.ch. 2010141440
>>> 300
>>> 1800 604800 30
>>>
>>> NS localhost.
>>>
>>> ;
>>>
>>> ; abuse.ch URLhaus Response Policy Zones (RPZ)
>>>
>>> ; Last updated: 2020-10-14 14:40:12 (UTC)
>>>
>>> ;
>>>
>>> ; Terms Of Use: https://urlhaus.abuse.ch/api/
>>> <https://urlhaus.abuse.ch/api/>
>>>
>>> ; For questions please contact urlhaus [at] abuse.ch
>>>
>>> ;
>>>
>>> testentry.rpz.urlhaus.abuse.ch CNAME . ; Test entry for testing
>>> URLhaus RPZ
>>>
>>> 1am.co.nz CNAME . ; Malware download (2020-08-17), see
>>> https://urlhaus.abuse.ch/host/1am.co.nz/
>>> <https://urlhaus.abuse.ch/host/1am.co.nz/>
>>>
>>> 1ca.co.za CNAME . ; Malware download (2020-08-28), see
>>> https://urlhaus.abuse.ch/host/1ca.co.za/
>>> <https://urlhaus.abuse.ch/host/1ca.co.za/>
>>>
>>> 1med.kiev.ua CNAME . ; Malware download (2020-10-14), see
>>> https://urlhaus.abuse.ch/host/1med.kiev.ua/
>>> <https://urlhaus.abuse.ch/host/1med.kiev.ua/>
>>>
>>> 21robo.com CNAME . ; Malware download (2019-02-20), see
>>> https://urlhaus.abuse.ch/host/21robo.com/
>>> <https://urlhaus.abuse.ch/host/21robo.com/>
>>>
>>
>>
> 
> 