[Unbound-users] unbound should probably manage other RLIMITs as well...

[W.C.A. Wijngaards]

Hi Greg,

On 10/27/2009 11:35 PM, Greg A. Woods wrote:
> it was claiming that it ran out of memory.
> I have the following non-default memory-usage related settings:
> 	rrset-cache-size: 256m

Note that I expect due to malloc-overhead that unbound grows to
double the size of what it actually uses.  So, this grows to
512m + some for your other caches.  (this is because of the
libc memory allocator algorithm).

Note it is useful to also increase the message cache size -
you have been using 256m to store rrsets which helped recursion
but not answering clients from cache.  msg-cache-size: 128m.

Since you do not seem to need that much cache, perhaps:
rrset-cache-size: 64m
msg-cache-size: 32m
total 96, add some more for other stuff and double for malloc
overhead and this fits in 256m much more easily.

Or change rlimit of course

> However note the soft datasize RLIMIT is just exactly 256m, through some
> accident in how my system and kernel are configured, and this is of
> course slightly too small for Unbound's current configuration:

> So, this crash seems to have happening because the cache does finally
> fill close to the specified limit, but since the process can't grow that
> big, malloc() reports being out of memory and eventually Unbound gives
> up and dies.

Yes, I have looked through the kindly attached logs and fixed the
offending error to be more soft-fail.  This would make unbound continue
operation a little longer under such conditions.

> I think Unbound should probably try to manage other of its RLIMIT values
> besides just the number of open files.

I am hesitant, based on 'code bloat' reasons to do this.  I see this is
something for BSD and Solaris mostly as Linux does not impose limits by
default it seems.

> it could even try using getrusage(2) where available to discover initial
> memory requirements at startup (though on many systems getrusage(2)
> doesn't really provide enough useful information).
>
> Oct 27 02:02:48 once unbound: [12262:0] fatal error: prealloc: out of memory

getrlimit(RLIMIT_DATA) is more useful I guess.  The code bloat is in
trying to estimate its own memory usage.

Best regards,
    Wouter