[Unbound-users] Slowly memory leak?

[W.C.A. Wijngaards]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Tomoki,

On 05/21/2012 05:13 PM, fwkh7691 at nifty.com wrote:
> Hi, Wouter
> 
> Thank you for your reply.
> 
> I had thought the "msg cache memory" usage might be stop at 1000MB
> if I set "rrset-cache-size: 500m".
> 
> Actually, the "msg cache memory" will NOT be increased more than 
> about 500MB. "Real" memory usage is slowly increased by memory 
> fragmentation. Is this correct?

Yes, the counter that unbound reports is not more than 500Mb (what you
configure in unbound.conf).  The real memory usage reported by ps,
top, slowly increases because of memory fragmentation.  Up to double.

> So, if I set "rrset-cache-size: 500m" and "msg-cache-size: 250m", 
> real memory usage will be about 1500MB?

Theoretically, but the malloc code in FreeBSD is very good and may
keep it lower most of the time.

Best regards,
   Wouter

> On Fri, 18 May 2012 14:12:05 +0200 "W.C.A. Wijngaards"
> <wouter at nlnetlabs.nl> wrote:
> 
> Hi Tomoki,
> 
> On 05/17/2012 04:25 PM, fwkh7691 at nifty.com wrote:
>>>> Hi all.
>>>> 
>>>> I'm using unbound 1.4.16 with following server. 
>>>> ==================== FreeBSD9.0(32bit) RAM 4GB 1 CPU 
>>>> ====================
>>>> 
>>>> unbound.conf: ==================== msg-cache-size: 250m 
>>>> rrset-cache-size: 500m module-config: "iterator" 
>>>> ====================
>>>> 
>>>> I checked cache memories using "unbound-control
>>>> stats_noreset". Increment of msg cache memory is stop at
>>>> 266MB. Increment of RRset cache memory is stop at 532MB.
> 
> But the memory allocator has fragmentation overhead, this can 
> increase to double to allocation in total, 500m and 1000m memory. 
> That is the maximum really.
> 
>>>> But, increment of memory usage is not stop. I checked it
>>>> using ps command. The result is following.
>>>> ============================== # date ; ps aux | head -3 Thu
>>>> May 17 22:52:40 JST 2012 USER      PID %CPU %MEM    VSZ
>>>> RSS  TT  STAT STARTED        TIME COMMAND root 11  99.0  0.0
>>>> 0      8  ??  RL   13Apr12 48831:35.16 [idle] unbound 14313
>>>> 5.0 68.6 1445512 1426924  ??  Ss   17Apr12 175:47.60 /usr/lo
>>>> cal/sbin/unbound
>>>> 
>>>> # date ; ps aux | head -3 Thu May 17 23:11:18 JST 2012 USER 
>>>> PID  %CPU %MEM    VSZ    RSS  TT  STAT STARTED        TIME 
>>>> COMMAND root       11  94.0  0.0      0      8  ??  RL
>>>> 13Apr12 48848:17.65 [idle] unbound 14313   5.0 68.6 1445512
>>>> 1426944  ?? Ss 17Apr12   177:11.51 /usr/lo cal/sbin/unbound
>>>> 
>>>> Thu May 17 23:22:47 JST 2012 USER      PID  %CPU %MEM    VSZ 
>>>> RSS TT  STAT STARTED        TIME COMMAND root       11  90.0
>>>> 0.0 0      8  ??  RL   13Apr12 48858:33.42 [idle] unbound
>>>> 14313   6.9 68.6 1445512 1427020  ??  Ss   17Apr12
>>>> 178:04.08 /usr/lo cal/sbin/unbound
>>>> ============================== - Continuously incremented.
>>>> But, not always. Sometimes this increment is stop. - This
>>>> increment is not synchronized with the number of queries.
> 
> I think this is memory fragmentation (the space between allocated 
> memory blocks, wasted space), that increases slowly.
> 
> FreeBSD malloc has debug modes that can give you extra
> information, and if that is not sufficient, unbound has an internal
> alloc-check mode that tests memory as well (unbound's extra memory
> checks are slow) configure with --enable-alloc-lite or with 
> --enable-alloc-checks.  You can also enable 
> --enable-alloc-nonregional to see malloc statistics for things
> that are normally handled by a special purpose memory allocator.
> 
> Best regards, Wouter
>> _______________________________________________ Unbound-users
>> mailing list Unbound-users at unbound.net 
>> http://unbound.nlnetlabs.nl/mailman/listinfo/unbound-users
> 
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJPuz25AAoJEJ9vHC1+BF+NrGEQAJIB+0rPRnHjCCa7Mkv05gO0
yr0pIYKnZ9jadxvQ+m2jCcw8a4Mu8YDBvwPnD2YTHX0CJrxWON7VQqBuCBGFtSZA
0QcZv6GiKRvHAeY/dNBcn8PrT0Y4TmTWE8vdPIo4QZD23f1CesyTc/3mm5gFCWxa
dN3Bwq/KbNALoi3ubv88ZRh6Ngbgcfjrsanxe6qfUtlVvhu63pNcETEjAnPzQ5vM
BmGsflhpf1DCxrbQBE8v+mEFX5viOtjanwSJCKkhUGHSnu8xdgAT4Gej6yL+5hDz
urFoLYfnghg2cLSLC++nRFZfir30WHKJk4LdPYkF4IXPSxgQcNu/cqkDxhi7jSDz
BeZ0S6Nj0sXrFz1dadcQK5XT6T/xRBa/6fLMjRqZVT0nGhbsdi1x1GqKf7IV67Gu
paAAcBWVp7uVZaN55hMO6JNBUD9mjDCB5x6k53Q/72rvW8Sb2csswFtfgb0AoM4E
rmxDBegcTarK+JJROBThxgzZOEc3qwKSCEvJXhNTs2XgISGedv7vKibwQKxc9hFI
l8ebZNxvKxn/vAcSNy8Jn0NDsrIyKcJpTtJDp7X5jrjYC3YfgSNpX+XIiOC179Qb
fVcnnY1dmO8hFq9yYX5wz7yk8nGqi+/fbStJcSCLQC7JjHJJ0GR9o76BMsyDz0Jv
0P/BLi56HpJcOg6GVemO
=hGxS
-----END PGP SIGNATURE-----