[Unbound-users] Validation failure of DNSSEC signed domain names

Wed May 5 11:51:58 UTC 2010

Hi Wouter,

On 4.5.2010 10:48, W.C.A. Wijngaards wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi Zbynek,
>
> Can you try with the svn trunk r2107?  I have made a bugfix which may
> help, it stops unbound from disabling its dnssec expectation after it
> has seen a response fail without rrsigs.  This could turn into a long
> causation chain: dnssec expectation lost, therefore EDNS backoff
> possible (with timeouts happening), therefore EDNS backoff stored in
> cache, stopping it from sending DO bits (and a bug fixed in r2106 where
> this fact got 'stuck' into the cache).

Still the same :( The only difference is when "strange" state occurs then 
replies for DNSSEC queries which I never tried before are OK, but those I have 
already been tried are still SERVFAILed.

> After discussion with my colleague I think this fix may help... The
> evidence you kindly provided supports the hypothesis and this fix should
> stop it from failing continuously (but a single query still fails).
>
> Also, I believe there to be some trouble with your setup, which is
> causing unbound to have slower turnaround; can you email me (offlist if
> you want) your configuration (unbound.conf and what is your ulimit(open
> files) for unbound, did you compile with libevent) ?  I think this
> slower turnaround exists because you have that failing query.

Yes, Unbound is compiled with libevent.

# ulimit -n
32768

This is my config for testing purposes:

--- CONFIG ---
#
# Example configuration file.
#
# See unbound.conf(5) man page.
#
# this is a comment.

#Use this to include other text into the file.
#include: "otherfile.conf"

# The server clause sets the main parameters.
server:
	# whitespace is not necessary, but looks cleaner.

	# verbosity number, 0 is least verbose. 1 is default.
	#verbosity: 5
	verbosity: 1

	# print statistics to the log (for every thread) every N seconds.
	# Set to "" or 0 to disable. Default is disabled.
	# statistics-interval: 0

	# enable cumulative statistics, without clearing them after printing.
	# statistics-cumulative: no

	# enable extended statistics (query types, answer codes, status)
	# printed from unbound-control. default off, because of speed.
	# extended-statistics: no

	# number of threads to create. 1 disables threading.
	num-threads: 5

	# specify the interfaces to answer queries from by ip-address.
	# The default is to listen to localhost (127.0.0.1 and ::1).
	# specify 0.0.0.0 and ::0 to bind to all available interfaces.
	# specify every interface[@port] on a new 'interface:' labelled line.
	# The listen interfaces are not changed on reload, only on restart.
	# interface: 192.0.2.153
	# interface: 192.0.2.154
	# interface: 192.0.2.154 at 5003
	# interface: 2001:DB8::5

	# enable this feature to copy the source address of queries to reply.
	# Socket options are not supported on all platforms. experimental.
	# interface-automatic: no

	# port to answer queries from
	# port: 53
	port: 54

	# specify the interfaces to send outgoing queries to authoritative
	# server from by ip-address. If none, the default (all) interface
	# is used. Specify every interface on a 'outgoing-interface:' line.
	# outgoing-interface: 192.0.2.153
	# outgoing-interface: 2001:DB8::5
	# outgoing-interface: 2001:DB8::6

	# number of ports to allocate per thread, determines the size of the
	# port range that can be open simultaneously.
	# outgoing-range: 256

	# permit unbound to use this port number or port range for
	# making outgoing queries, using an outgoing interface.
	# outgoing-port-permit: 32768

	# deny unbound the use this of port number or port range for
	# making outgoing queries, using an outgoing interface.
	# Use this to make sure unbound does not grab a UDP port that some
	# other server on this computer needs. The default is to avoid
	# IANA-assigned port numbers.
	# outgoing-port-avoid: "3200-3208"

	# number of outgoing simultaneous tcp buffers to hold per thread.
	# outgoing-num-tcp: 10

	# number of incoming simultaneous tcp buffers to hold per thread.
	# incoming-num-tcp: 10

	# buffer size for UDP port 53 incoming (SO_RCVBUF socket option).
	# 0 is system default.  Use 4m to catch query spikes for busy servers.
	# so-rcvbuf: 0

	# EDNS reassembly buffer to advertise to UDP peers (the actual buffer
	# is set with msg-buffer-size). 1480 can solve fragmentation (timeouts).
	# edns-buffer-size: 4096

	# buffer size for handling DNS data. No messages larger than this
	# size can be sent or received, by UDP or TCP. In bytes.
	# msg-buffer-size: 65552

	# the amount of memory to use for the message cache.
	# plain value in bytes or you can append k, m or G. default is "4Mb".
	# msg-cache-size: 4m

	# the number of slabs to use for the message cache.
	# the number of slabs must be a power of 2.
	# more slabs reduce lock contention, but fragment memory usage.
	# msg-cache-slabs: 4

	# the number of queries that a thread gets to service.
	# num-queries-per-thread: 1024
	num-queries-per-thread: 4096

	# if very busy, 50% queries run to completion, 50% get timeout in msec
	# jostle-timeout: 200

	# the amount of memory to use for the RRset cache.
	# plain value in bytes or you can append k, m or G. default is "4Mb".
	# rrset-cache-size: 4m

	# the number of slabs to use for the RRset cache.
	# the number of slabs must be a power of 2.
	# more slabs reduce lock contention, but fragment memory usage.
	# rrset-cache-slabs: 4

	# the time to live (TTL) value lower bound, in seconds. Default 0.
	# If more than an hour could easily give trouble due to stale data.
	# cache-min-ttl: 0

	# the time to live (TTL) value cap for RRsets and messages in the
	# cache. Items are not cached for longer. In seconds.
	# cache-max-ttl: 86400

	# the time to live (TTL) value for cached roundtrip times and
	# EDNS version information for hosts. In seconds.
	# infra-host-ttl: 900

	# the time to live (TTL) value for cached lame delegations. In sec.
	#infra-lame-ttl: 900

	# the number of slabs to use for the Infrastructure cache.
	# the number of slabs must be a power of 2.
	# more slabs reduce lock contention, but fragment memory usage.
	# infra-cache-slabs: 4

	# the maximum number of hosts that are cached (roundtrip times, EDNS).
	# infra-cache-numhosts: 10000

	# the maximum size of the lame zones cached per host. in bytes.
	# infra-cache-lame-size: 10k

	# Enable IPv4, "yes" or "no".
	# do-ip4: yes

	# Enable IPv6, "yes" or "no".
	do-ip6: no

	# Enable UDP, "yes" or "no".
	# do-udp: yes

	# Enable TCP, "yes" or "no".
	# do-tcp: yes

	# Detach from the terminal, run in background, "yes" or "no".
	# do-daemonize: yes

	# control which clients are allowed to make (recursive) queries
	# to this server. Specify classless netblocks with /size and action.
	# By default everything is refused, except for localhost.
	# Choose deny (drop message), refuse (polite error reply),
	# allow (recursive ok), allow_snoop (recursive and nonrecursive ok)
	# access-control: 0.0.0.0/0 refuse
	# access-control: 127.0.0.0/8 allow
	# access-control: ::0/0 refuse
	# access-control: ::1 allow
	# access-control: ::ffff:127.0.0.1 allow

	# if given, a chroot(2) is done to the given directory.
	# i.e. you can chroot to the working directory, for example,
	# for extra security, but make sure all files are in that directory.
	#
	# If chroot is enabled, you should pass the configfile (from the
	# commandline) as a full path from the original root. After the
	# chroot has been performed the now defunct portion of the config
	# file path is removed to be able to reread the config after a reload.
	#
	# All other file paths (working dir, logfile, roothints, and
	# key files) can be specified in several ways:
	# 	o as an absolute path relative to the new root.
	# 	o as a relative path to the working directory.
	# 	o as an absolute path relative to the original root.
	# In the last case the path is adjusted to remove the unused portion.
	#
	# The pid file can be absolute and outside of the chroot, it is
	# written just prior to performing the chroot and dropping permissions.
	#
	# Additionally, unbound may need to access /dev/random (for entropy).
	# How to do this is specific to your OS.
	#
	# If you give "" no chroot is performed. The path must not end in a /.
	# chroot: "/var/lib/unbound"
	chroot: ""

	# if given, user privileges are dropped (after binding port),
	# and the given username is assumed. Default is user "unbound".
	# If you give "" no privileges are dropped.
	# username: "unbound"
	username: ""

	# the working directory. The relative files in this config are
	# relative to this directory. If you give "" the working directory
	# is not changed.
	# directory: "/etc/unbound"

	# the log file, "" means log to stderr.
	# Use of this option sets use-syslog to "no".
	logfile: "/var/log/unbound"

	# Log to syslog(3) if yes. The log facility LOG_DAEMON is used to
	# log to, with identity "unbound". If yes, it overrides the logfile.
	# use-syslog: yes

	# print UTC timestamp in ascii to logfile, default is epoch in seconds.
	# log-time-ascii: no

	# the pid file. Can be an absolute path outside of chroot/work dir.
	# pidfile: "/var/run/unbound.pid"

	# file to read root hints from.
	# get one from ftp://FTP.INTERNIC.NET/domain/named.cache
	# root-hints: ""

	# enable to not answer id.server and hostname.bind queries.
	# hide-identity: no

	# enable to not answer version.server and version.bind queries.
	# hide-version: no

	# the identity to report. Leave "" or default to return hostname.
	# identity: ""

	# the version to report. Leave "" or default to return package version.
	# version: ""

	# the target fetch policy.
	# series of integers describing the policy per dependency depth.
	# The number of values in the list determines the maximum dependency
	# depth the recursor will pursue before giving up. Each integer means:
	# 	-1 : fetch all targets opportunistically,
	# 	0: fetch on demand,
	#	positive value: fetch that many targets opportunistically.
	# Enclose the list of numbers between quotes ("").
	# target-fetch-policy: "3 2 1 0 0"

	# Harden against very small EDNS buffer sizes.
	# harden-short-bufsize: no

	# Harden against unseemly large queries.
	# harden-large-queries: no

	# Harden against out of zone rrsets, to avoid spoofing attempts.
	# harden-glue: yes

	# Harden against receiving dnssec-stripped data. If you turn it
	# off, failing to validate dnskey data for a trustanchor will
	# trigger insecure mode for that zone (like without a trustanchor).
	# Default on, which insists on dnssec data for trust-anchored zones.
	# harden-dnssec-stripped: yes

         # Harden the referral path by performing additional queries for
	# infrastructure data.  Validates the replies (if possible).
	# Default off, because the lookups burden the server.  Experimental
	# implementation of draft-wijngaards-dnsext-resolver-side-mitigation.
	# harden-referral-path: no

	# Use 0x20-encoded random bits in the query to foil spoof attempts.
	# This feature is an experimental implementation of draft dns-0x20.
	# use-caps-for-id: no

	# Enforce privacy of these addresses. Strips them away from answers.
	# It may cause DNSSEC validation to additionally mark it as bogus.
	# Protects against 'DNS Rebinding' (uses browser as network proxy).
	# Only 'private-domain' and 'local-data' names are allowed to have
	# these private addresses. No default.
	# private-address: 10.0.0.0/8
	# private-address: 172.16.0.0/12
	# private-address: 192.168.0.0/16
	# private-address: 192.254.0.0/16
	# private-address: fd00::/8
	# private-address: fe80::/10

	# Allow the domain (and its subdomains) to contain private addresses.
	# local-data statements are allowed to contain private addresses too.
	# private-domain: "example.com"

	# If nonzero, unwanted replies are not only reported in statistics,
	# but also a running total is kept per thread. If it reaches the
	# threshold, a warning is printed and a defensive action is taken,
	# the cache is cleared to flush potential poison out of it.
	# A suggested value is 10000000, the default is 0 (turned off).
	# unwanted-reply-threshold: 0

	# Do not query the following addresses. No DNS queries are sent there.
	# List one address per entry. List classless netblocks with /size,
	# do-not-query-address: 127.0.0.1/8
	# do-not-query-address: ::1

	# if yes, the above default do-not-query-address entries are present.
	# if no, localhost can be queried (for testing and debugging).
	# do-not-query-localhost: yes

	# if yes, perform prefetching of almost expired message cache entries.
	# prefetch: no

	# if yes, perform key lookups adjacent to normal lookups.
	# prefetch-key: no

	# module configuration of the server. A string with identifiers
	# separated by spaces. "iterator" or "validator iterator"
	# module-config: "validator iterator"

	# File with DLV trusted keys. Same format as trust-anchor-file.
	# There can be only one DLV configured, it is trusted from root down.
	# Download http://ftp.isc.org/www/dlv/dlv.isc.org.key
	# dlv-anchor-file: "dlv.isc.org.key"

	# File with trusted keys for validation. Specify more than one file
	# with several entries, one file per entry.
	# Zone file format, with DS and DNSKEY entries.
	# trust-anchor-file: ""

	# File with trusted keys, kept uptodate using RFC5011 probes,
	# initial file like trust-anchor-file, then it stores metadata.
	# Use several entries, one per domain name, to track multiple zones.
	# auto-trust-anchor-file: ""

	# Trusted key for validation. DS or DNSKEY. specify the RR on a
	# single line, surrounded by "". TTL is ignored. class is IN default.
	# (These examples are from August 2007 and may not be valid anymore).
	# trust-anchor: "nlnetlabs.nl. DNSKEY 257 3 5 
AQPzzTWMz8qSWIQlfRnPckx2BiVmkVN6LPupO3mbz7FhLSnm26n6iG9N 
Lby97Ji453aWZY3M5/xJBSOS2vWtco2t8C0+xeO1bc/d6ZTy32DHchpW 
6rDH1vp86Ll+ha0tmwyy9QP7y2bVw5zSbFCrefk8qCUBgfHm9bHzMG1U BYtEIQ=="
	# trust-anchor: "jelte.nlnetlabs.nl. DS 42860 5 1 
14D739EB566D2B1A5E216A0BA4D17FA9B038BE4A"

	# File with trusted keys for validation. Specify more than one file
	# with several entries, one file per entry. Like trust-anchor-file
	# but has a different file format. Format is BIND-9 style format,
	# the trusted-keys { name flag proto algo "key"; }; clauses are read.
	trusted-keys-file: "/etc/unbound/keys.conf"

	# Ignore chain of trust. Domain is treated as insecure.
	# domain-insecure: "example.com"

	# Override the date for validation with a specific fixed date.
	# Do not set this unless you are debugging signature inception
	# and expiration. "" or "0" turns the feature off.
	# val-override-date: ""

	# The time to live for bogus data, rrsets and messages. This avoids
	# some of the revalidation, until the time interval expires. in secs.
	# val-bogus-ttl: 60

	# The signature inception and expiration dates are allowed to be off
	# by 10% of the signature lifetime (expir-incep) from our local clock.
	# This leeway is capped with a minimum and a maximum.  In seconds.
	# val-sig-skew-min: 3600
	# val-sig-skew-max: 86400

	# Should additional section of secure message also be kept clean of
	# unsecure data. Useful to shield the users of this validator from
	# potential bogus data in the additional section. All unsigned data
	# in the additional section is removed from secure messages.
	# val-clean-additional: yes

	# Turn permissive mode on to permit bogus messages. Thus, messages
	# for which security checks failed will be returned to clients,
	# instead of SERVFAIL. It still performs the security checks, which
	# result in interesting log files and possibly the AD bit in
	# replies if the message is found secure. The default is off.
	# val-permissive-mode: no

	# Have the validator log failed validations for your diagnosis.
	# 0: off. 1: A line per failed user query. 2: With reason and bad IP.
	val-log-level: 2

	# It is possible to configure NSEC3 maximum iteration counts per
	# keysize. Keep this table very short, as linear search is done.
	# A message with an NSEC3 with larger count is marked insecure.
	# List in ascending order the keysize and count values.
	# val-nsec3-keysize-iterations: "1024 150 2048 500 4096 2500"

	# instruct the auto-trust-anchor-file probing to add anchors after ttl.
	# add-holddown: 2592000 # 30 days

	# instruct the auto-trust-anchor-file probing to del anchors after ttl.
	# del-holddown: 2592000 # 30 days

	# auto-trust-anchor-file probing removes missing anchors after ttl.
	# If the value 0 is given, missing anchors are not removed.
	# keep-missing: 31622400 # 366 days

	# the amount of memory to use for the key cache.
	# plain value in bytes or you can append k, m or G. default is "4Mb".
	# key-cache-size: 4m

	# the number of slabs to use for the key cache.
	# the number of slabs must be a power of 2.
	# more slabs reduce lock contention, but fragment memory usage.
	# key-cache-slabs: 4

	# the amount of memory to use for the negative cache (used for DLV).
	# plain value in bytes or you can append k, m or G. default is "1Mb".
	# neg-cache-size: 1m

	# a number of locally served zones can be configured.
	# 	local-zone: <zone> <type>
	# 	local-data: "<resource record string>"
	# o deny serves local data (if any), else, drops queries.
	# o refuse serves local data (if any), else, replies with error.
	# o static serves local data, else, nxdomain or nodata answer.
	# o transparent gives local data, but resolves normally for other names
	# o redirect serves the zone data for any subdomain in the zone.
	# o nodefault can be used to normally resolve AS112 zones.
	#
	# defaults are localhost address, reverse for 127.0.0.1 and ::1
	# and nxdomain for AS112 zones. If you configure one of these zones
	# the default content is omitted, or you can omit it with 'nodefault'.
	#
	# If you configure local-data without specifying local-zone, by
	# default a transparent local-zone is created for the data.
	#
	# You can add locally served data with
	# local-zone: "local." static
	# local-data: "mycomputer.local. IN A 192.0.2.51"
	# local-data: 'mytext.local TXT "content of text record"'
	#
	# You can override certain queries with
	# local-data: "adserver.example.com A 127.0.0.1"
	#
	# You can redirect a domain to a fixed address with
	# (this makes example.com, www.example.com, etc, all go to 192.0.2.3)
	# local-zone: "example.com" redirect
	# local-data: "example.com A 192.0.2.3"
	#
	# Shorthand to make PTR records, "IPv4 name" or "IPv6 name".
	# You can also add PTR records using local-data directly, but then
	# you need to do the reverse notation yourself.
	# local-data-ptr: "192.0.2.3 www.example.com"

# Python config section. To enable:
# o use --with-pythonmodule to configure before compiling.
# o list python in the module-config string (above) to enable.
# o and give a python-script to run.
python:
	# Script file to load
	# python-script: "/etc/unbound/ubmodule-tst.py"

# Remote control config section.
remote-control:
	# Enable remote control with unbound-control(8) here.
	# set up the keys and certificates with unbound-control-setup.
	control-enable: yes

	# what interfaces are listened to for remote control.
	# give 0.0.0.0 and ::0 to listen to all interfaces.
	# control-interface: 127.0.0.1
	# control-interface: ::1

	# port number for remote control operations.
	control-port: 954

	# unbound server key file.
	# server-key-file: "/etc/unbound/unbound_server.key"

	# unbound server certificate file.
	# server-cert-file: "/etc/unbound/unbound_server.pem"

	# unbound-control key file.
	# control-key-file: "/etc/unbound/unbound_control.key"

	# unbound-control certificate file.
	# control-cert-file: "/etc/unbound/unbound_control.pem"

# Stub zones.
# Create entries like below, to make all queries for 'example.com' and
# 'example.org' go to the given list of nameservers. list zero or more
# nameservers by hostname or by ipaddress. If you set stub-prime to yes,
# the list is treated as priming hints (default is no).
# stub-zone:
#	name: "example.com"
#	stub-addr: 192.0.2.68
#	stub-prime: no
# stub-zone:
#	name: "example.org"
#	stub-host: ns.example.com.

# Forward zones
# Create entries like below, to make all queries for 'example.com' and
# 'example.org' go to the given list of servers. These servers have to handle
# recursion to other nameservers. List zero or more nameservers by hostname
# or by ipaddress. Use an entry with name "." to forward all queries.
# forward-zone:
# 	name: "example.com"
# 	forward-addr: 192.0.2.68
# 	forward-addr: 192.0.2.73 at 5355  # forward to port 5355.
# forward-zone:
# 	name: "example.org"
# 	forward-host: fwd.example.com
--- /CONFIG ---

Regards,
Zbynek

>
> Best regards,
>     Wouter
>
> On 04/30/2010 06:32 PM, Zbynek Michl wrote:
>> So in the "strange" state the resolver does not send queries with DO
>> bit, thus does not receive RRSIG for query type and therefore it can not
>> validate result. Then the remote authoritative server (who sent answer
>> without RRSIG) is added to the blacklist.
>>
>> In the log (complete version has been sent in previous mail):
>>
>> Request for DS nic.cz is sent to 194.0.12.1 (without DO bit), replied DS
>> is not validated due to missing its RRSIG and 194.0.12.1 is blacklisted.
>
>>>> Hmm, r2106 experiences the same issue :(
>>>>
>>>> It seems that there is no exact change between correct/incorrect
>>>> validation in the one time point. On the start there are all answers
>>>> correct, and when I am trying more and more (different in a few cycles)
>>>> requests, then there are more and more incorrect answers. And in some
>>>> time point all answers are incorrect from the resolver until cache is
>>>> flushed (probably).
>>>>
>>>> Regards,
>>>> Zbynek
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (GNU/Linux)
> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAkvf33cACgkQkDLqNwOhpPgwcwCeLSmiLxzpRZbPDfzRZwMMDqmI
> EjsAoKioC7qmqZBePZ86Wn1i7Emb1ie5
> =x4SU
> -----END PGP SIGNATURE-----