[nsd-users] NSD database memory usage

[W.C.A. Wijngaards]

Hi Antti,

It is a memory leak, thank you for the report!  Fixed it, code is copied
below and in the code repository.  It happens when unknown RR formatted
RRs are read from zonefile.

Best regards, Wouter


Index: zparser.y
===================================================================
--- zparser.y	(revision 4813)
+++ zparser.y	(working copy)
@@ -1078,16 +1078,16 @@
 rdata_unknown:	URR sp STR sp str_sp_seq trail
     {
 	    /* $2 is the number of octets, currently ignored */
-	    $$ = zparser_conv_hex(parser->region, $5.str, $5.len);
+	    $$ = zparser_conv_hex(parser->rr_region, $5.str, $5.len);

     }
     |	URR sp STR trail
     {
-	    $$ = zparser_conv_hex(parser->region, "", 0);
+	    $$ = zparser_conv_hex(parser->rr_region, "", 0);
     }
     |	URR error NL
     {
-	    $$ = zparser_conv_hex(parser->region, "", 0);
+	    $$ = zparser_conv_hex(parser->rr_region, "", 0);
     }
     ;
 %%
Index: zonec.c
===================================================================
--- zonec.c	(revision 4813)
+++ zonec.c	(working copy)
@@ -1260,6 +1260,8 @@
 			zadd_rdata_wireformat(rdatas[i].data);
 		}
 	}
+	region_recycle(parser->region, rdatas,
+		rdata_count*sizeof(rdata_atom_type));
 }


@@ -1626,6 +1628,7 @@
 			name, domain_to_string(
 			parser->current_zone->soa_rrset->rrs[0].owner));
 	}
+	region_free_all(parser->rr_region);

 	parser_flush();
 	fclose(yyin);
@@ -1719,6 +1722,7 @@
 	/* remove origin if it was not used during the parse */
 	if(parser->origin != error_domain)
 		domain_table_deldomain(parser->db, parser->origin);
+	region_free_all(parser->rr_region);
 	zonec_desetup_string_parser();
 	parser_flush();
 	return errors;


On 06/02/18 10:54, Antti Ristimäki wrote:
> Hi Anand & list,
> 
> Actually I forgot to mention it in my first message, but we do have set the database to empty value in configuration.
> 
> For us restarting NSD every now and then is not a very big problem, as this instance is only a hidden master, but naturally a more elegant solution would be very welcome.
> 
> Antti
> 
> 
> ----- On 6 Feb, 2018, at 11:47, Anand Buddhdev anandb at ripe.net wrote:
> 
>> Hi Antti,
>>
>> This is certainly a problem, and I'm sure the developers will be happy
>> to investigate it with you.
>>
>> However, I'd like to suggest that you don't use the database mode. If
>> you set:
>>
>> database: ""
>>
>> in your nsd.conf, then nsd will load the zone from the zonefile into
>> RAM, and won't bother compiling the nsd.db file. You don't really gain
>> anything with the database file, and I've been advocating for the
>> database mode to be dropped completely in future versions of nsd.
>>
>> By the way, if you or the developers find the problem, please do let us
>> know here, because I'm also curious about it.
>>
>> Regards,
>> Anand
>>
>> On 06/02/2018 09:08, Antti Ristimäki wrote:
>>> Hello,
>>>
>>> We have an installation, where NSD (version 4.1.19) acts as a hidden master for
>>> the public DNS servers. NSD has only one large zone configured and the zone is
>>> periodically signed every 20 minutes and after each re-signing, "nsd-control
>>> reload <zone>" is given so that the NSD process reloads the new zone from the
>>> zonefile and notifies the slaves. However, we noticed that the database memory
>>> usage increases after every reload, finally resulting in memory allocation
>>> failure. We stat the memory usage by running "nsd-control stats_noreset" every
>>> minute and in the graph [1] one can see the increase in size.db.mem after each
>>> reload. We don't see similar behaviour with for example xfrd process memory
>>> usage.
>>>
>>> We have worked around the issue by restarting the NSD process periodically, but
>>> do you have any ideas about the possible root cause and a more long term
>>> solution?
>>>
>>> [1] http://nxdomain.fi/NSD_db_mem.png
>> _______________________________________________
>> nsd-users mailing list
>> nsd-users at NLnetLabs.nl
>> https://open.nlnetlabs.nl/mailman/listinfo/nsd-users
>>
>>
>> --
> _______________________________________________
> nsd-users mailing list
> nsd-users at NLnetLabs.nl
> https://open.nlnetlabs.nl/mailman/listinfo/nsd-users
> 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.nlnetlabs.nl/pipermail/nsd-users/attachments/20180206/64de2de6/attachment.bin>