[nsd-users] NSD 4.3.9 stops updating zones

[Jeroen Koekkoek]

Hi Andand, Andreas,

Andand's observeration is correct in that at some point NSD may use
double the memory on reloads. In this case, one of it's strengths is
also one of it's weaknesses. The copy-on-write memory characteristics
of the fork call are used so there's no need for locking on read-access
to the in-memory database. Unfortunately, it also means that worst-case
twice the amount of memory is required if there are lots of changes.
Basically, there really are two versions of the data in memory at some
point.

There's two things that can be done to reduce the amount of memory
required. 1) Disable the on-disk database. i.e. 'database: ""' in the
configuration file. 2) (Re)compile NSD with --disable-radix-tree, as
the radix tree requires more memory to operate.

A tip from a colleague was to change the kernel overcommit value. I
have no experience with this, so you/we'd have to test this (happy to
look into this further). Allowing the kernel to always overcommit
allows the fork to succeed even though double the memory is not
available. The rationale here being that when a zone is updated, it's
not the entire size of the RAM, only some pages get overwritten and
therefore could do the trick.

All of that being said, the aforementioned semantics are fundamental to
the the design. That is not to say these will never be changed, but
there's no "quick-fix" in terms of a configuration option or oneline
code change.

I'm thinking the other, perhaps bigger, problem here is the stray
transfers that are left behind(?) There's some work being done on
slightly altering the behavior for zone transfers in porting zone
verification (CreDNS) to NSD. That should already improve the situation
quite a bit as only a certain window is being retried instead of all
current transfers before proceeding. Maybe, after that's in, we could
look into adding an option to throttle updates (not discussed with the
team).

I hope this provides some insight. Let me know if the suggestions work.
Also, it would be good to know how often this problem occurs.

Best regards,
Jeroen



On Sun, 2022-04-03 at 22:43 +0200, Anand Buddhdev via nsd-users wrote:
> On 03/04/2022 20:43, A. Schulze via nsd-users wrote:
> 
> Hi Andreas,
> 
> > do you have the chance to try version 4.4.0?
> > The announcement [1] promise "Lower memory usage of the XFRD
> > process by default."
> 
> Yes, I have also tried 4.4.0, but it has made no difference. I am not
> surprised, because the xfrd process isn't the one using most of the 
> memory. Most of the memory is used by the master nsd process that
> keeps 
> the zone database in memory.
> 
> Anand
> _______________________________________________
> nsd-users mailing list
> nsd-users at lists.nlnetlabs.nl
> https://lists.nlnetlabs.nl/mailman/listinfo/nsd-users