[Unbound-users] Strange Behavior Seen When Using Python Module and Queries Result in 100% Cache Misses

[Mountcastle, Sean]

I¹m seeing some strange behavior under what I think is relatively light
load. I have a tool which queries at a configurable rate for non-existent
domains under a particular zone (in this case a random 10 character
subdomain of verisign.com). When I run this tool at 50 queries per second to
my local Unbound instance for 10 seconds, Unbound drops the queries and is
then unresponsive for ~6 minutes following (manual digs to the server
timeout).

I¹m using Unbound 1.4.7, compiled with the following options:
--with-pthreads --with-ssl --with-ldns-builtin
--with-libevent=$(LIBEVENT_PATH) --with-libexpat=$(LIBEXPAT_PATH)
--with-pythonmodule CFLAGS="-std=c99" --enable-static-exe --disable-gost
--enable-debug. The versions of expat and libevent are expat-2.0.1 and
libevent-1.4.14b-stable. My configuration uses 2 threads each with a
num-queries-per-thread of 4096 and an outgoing-range of 32768. The so-rcvbuf
is 8m and there are 16 cache slabs of 4GB each.

During the ~6 minutes that Unbound is unresponsive, I see it spawning child
processes and using quite a bit of CPU. The children all behave the same
way, they spend the majority of their time closing all the descriptors
inherited from the parent process and then stat/read several files before
writing to an activity.log and exiting (the writing to the log is taking
place in a python module).  I wasn¹t expecting any children to be forked
since Unbound was configured to use threads instead but perhaps this is a
side-effect of SWIG invoking Python.

If I compile-in, but do not use, the Python module, Unbound is able to
handle 2,000 queries per second (again where each query results in a cache
miss) and never gets into a state where it¹s unresponsive (I also don¹t see
child processes being spawned in this case).

Has anyone else encountered this issue when using Python modules? Is there a
workaround to improve performance?

Regards,
Sean
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nlnetlabs.nl/pipermail/unbound-users/attachments/20110228/832407d7/attachment.htm>