[nsd-users] NSD4 memory footprint

[W.C.A. Wijngaards]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Will,

On 07/14/2012 10:29 PM, Will Pressly wrote:
> Hello All,
> 
> Should we expect an increase in the memory footprint for NSD4, and
> if so, how much?  Also, what about the storage size of the db
> file?

Yes, NSD4 uses more memory to make it handle zone changes faster.

If you compile --without-nsec3, the memory size is the same as NSD3.
But --with-nsec3 (the default), the memory size is increased.  There
is extra information to enable IXFR for NSEC3 zones to be handled very
quickly (rehashing and precompilation).  For a 64bit machine this is
about 104 bytes more per domain name for a non-nsec3-zone, and about
360 bytes more per domain name for a zone that uses nsec3 (this is an
estimate).

For 1 million domain names, that works out to 99 Mb memory used more
than NSD3, or 343 Mb more when the zone is NSEC3-signed.  (this
compares to NSD3 with full prehash enabled (the default)).  For 32bit
compilation(gcc -m32), 78Mb or 200Mb, it is mostly pointers.

The storage size of the db file is significantly larger and adds to
the memory usage, because it is random-access.  This enables IXFRs to
be serviced faster and removes nsd-patch.  The file is memory mapped
(mmap(2) system call), and thus adds to the virtual memory size.  It
is not accessed during network service, but the reload process of NSD
accesses it when processing zone transfers.  The database size is
several Gb for a TLD-sized zone.  The database size is not constantly
growing (apart from your zone growth, of course), it is compacted when
space becomes freed.

Although the nsd.db counts with it mmap into the memory you see in the
process table, it is not necessary to have sufficient memory for that
(unless you want zone transfers to be handled quickly).  With not
enough memory it'll simply read and write to disk (this is normal mmap
behaviour) when processing a zone transfer.  The processes that
service port 53 do not access the mmap-area, so do not block on disk
access.

You can simply subtract the nsd.db size from the VSZ to get the size
you need for network service.  nsd4 also reports memory and disk usage
with nsd-control stats.

Best regards,
   Wouter
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJQBrgBAAoJEJ9vHC1+BF+NpIAP/3C71x7y8JXIOJxKU80+YRGg
+emoF/+jmFRWB1cOOhxDsITR6A98CokJ1WnmMPq1/lO38PwJqk1frUi7Tg/1FSq6
3hqacWx0MBX1DsORRmypAkrwaAAzxSIRu6KsNsbIYZf6ew1OhmqSfgbpqGGXozl7
fxM05ZDx5ljotF/JJnVaYNu7iiX1i4LCbvL/QRjlgU+porguZExkjQrPE4aqkA6V
DhTo8zXPVTNNBji8UerUiIH9UwDp+P/vPTmpYIqIkeFqeXKMx6yvHOZOGopVcTRq
emVIYCnci209UVcAH3HABY7/B/GJhz7UQ+YGbfamPGt+BM5YyEkD+yQZTMZS5/2u
GtONm3NQUuJiLLQ27hsIDq6vQi0i5VcfSCMeb+6dDbflQ2JfEkcEMGO4B3AiepQ/
Hs4G8JZHzZU1mO/rw1oC0fKY6XOOdF3tUKoSmOVcSbKH7S5OQ4O/OT3OqIGpnxpl
anhXHrIIF7a1IFP5klLOGBsyJvTGm9+WVWV4JuLY1NcWBI6/QXlWBNX2aY81LBFv
+QJIKSEcK+0VQciwp5ZE5Lphct+3o+yoNC+hZRjhuB5HTAV63gbi72dFsfAlQgtE
WB5BEjGfNPb7Gu/HhQSDB6paLChWWupfVNE76k5SyBj7dOTgqGop2pqcHGvph0EI
hnb3d/XB13IN6ljtiIAE
=vket
-----END PGP SIGNATURE-----