[nsd-users] memory leak in 4.0.x

[W.C.A. Wijngaards]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Will, Andreas,

Searched for it and found two leaks; those are in the zone name when a
file is read and in the radixtree when reading (larger) zones from
file and from zone transfer.  The zonename is because the fileread
used to be in a separate executable, and the radixtree had a leak
because it allocated for type* and not for type, leaking the
difference (through our special purpose memory allocator).

Fixes are in svn trunk

Thanks for the report and the tests!

Best regards,
   Wouter

On 02/17/2014 06:30 PM, Will Pressly wrote:
> I have been tracing what appears to be the same issue on my end for
> a month or two (NSD 4.0.1 on a 64bit Ubuntu Linux 12.04 machine). I
> have built a test harness that reloads a single one-record-zone as
> fast as it can, capturing the memory utility, memory maps, etc of
> the parent and xfrd processes coupled with system slab allocator
> statistics.I have seen large multipliers in RAM utilization of the
> parent process between where it started and where it was at the
> latest sample. Another problem is that the time a reload takes
> skyrockets. I can observe the reload time taking 40x or more time
> than when the daemon freshly started. I am also restarting servers
> periodically -- which fixes both the memory consumption and the
> reload time issues.
> 
> In my testing, I see that the memory growth does eventually start
> into a upper-bounded saw-tooth function. Given that the memory
> growth does seem bounded (although it seems to use far too much
> ram) -- and I profiled NSD repeatedly with valgrind and saw no
> memory leak -- I have been pursuing this as a potential memory
> fragmentation issue. However, looking at slabinfo while running my
> test harness, I am seeing multiple hundred times multipliers
> (400-500x from where it started) on the shared_policy_node and the
> anon_vma node after thousands of reloads.
> 
> I have data from this test harness (and the harness available if
> needed) for review. At this point, does anyone have any input on
> this?
> 
> Thanks,
> 
> Will Pressly
> 
> 
> On Feb 17, 2014 5:07 AM, "Andreas Schulze" <sca at andreasschulze.de 
> <mailto:sca at andreasschulze.de>> wrote:
> 
> Hello,
> 
> I found my two of my name servers are consuming more and more
> memory over time. NSD is supervised and started by "/usr/sbin/nsd
> -c /var/lib/nsd/etc/nsd/nsd.conf -d" I have remote-control and
> chroot enabled. The server handle only one zone. Zone data updates
> are "pushed" by a bind master more then 1000 times per day (it's a
> dnsbl zone) There is also a rbldnsd on the same host but I assume
> it's unrelated to the problem.
> 
> * nsd-4.0.0.png shows the problem on server #1 * nsd-4.0.1.png
> shows my tests on server #2:
> 
> In week 3,4 and 5 it was a nsd-4.0.0. A daily cronjob called 
> "killall nsd". (NSD is restarted by the supervisor every time) In
> week 6 I updated to nsd-4.0.1 and stopped the daily "killall nsd"
> 
> I think there is a little amount of memory not released somewhere
> in NSD. But I have no idea how to find the memory leak.
> 
> Andreas
> 
> 
> 
> 
> 
> 
> 
> 
> 
> _______________________________________________ nsd-users mailing
> list nsd-users at NLnetLabs.nl 
> http://open.nlnetlabs.nl/mailman/listinfo/nsd-users
> 
> 
> 
> _______________________________________________ nsd-users mailing
> list nsd-users at NLnetLabs.nl 
> http://open.nlnetlabs.nl/mailman/listinfo/nsd-users
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJTA4BhAAoJEJ9vHC1+BF+Nc88P/AqEqm3Rq5wTZRt07ZVl5yGu
+bi4Tzu1+pIqhWhsWBwrLzusqIhcjd8f2l5GnJrZi/6og3l7u9K2JLuGhQS11EIF
TO+BiB+uBJqpd7keS1GyU/qJwlmtQ1dwB/I6/21U/44JiuxNABV20/PmCJTWVIdy
bvaXRgeR9hTavjPHXttc+Or571SHkZn2HCyLPegLay9zrSJkCsESmfTAEh7+z6ss
GrNZSCnlOUYOJk0w8tZJPOd4+fGgO1NMqylbuFA4793EEwmTETSKqPN9+bQs4VDV
8AN6/E2iJ7ii5a5zo9q+O7VYP7CJkVwXSNbjJ3Asqk6ObXjU5FpXPYwCF2ly8jw4
6WJTLjYwlTJT0bI7b0sL5k+6IK7Ha7wBT1128oFxkFvkovn5VaZ/+3g0Zby+U8tL
SKvW3NGVtWLA6OFEsXTgtRp7MfJI10SFw/A7D3N5PM2iPiLqbSKuBTtaMGBt+tz+
NPfMFx6Nwwg+QFdECRzGVY2krjj/fY5z5BA48j71hiaJfbtHe0Vxuk5z0YyOO/gR
CsVcOVGmfbTQGeSbaxRMUJT8UYQJmjttSQm63j1/9hDeijDMXbKGj1V1KXl55oqd
jKBdNGEHpIlEMjaxc5nCMG5v+ftIy1iuEW1jgVaZZKkb9ZzVoP7hsv/6y7EA2jzg
/fvOiquqse4oo75K3obe
=Ff1E
-----END PGP SIGNATURE-----