[ldns-users] Possible memory leak in ldns_key_new_frm_algorithm

[Michael Sheldon]

So I've been chasing down a small memory leak when generating dnssec
signing keys.

valgrind backtrace was this:

==20296== 688 (136 direct, 552 indirect) bytes in 1 blocks are
definitely lost in loss record 43 of 44
==20296==    at 0x4A05FDE: malloc (vg_replace_malloc.c:236)
==20296==    by 0x39E905D95D: CRYPTO_malloc (in
/usr/lib64/libcrypto.so.1.0.0)
==20296==    by 0x39E90A2A4E: DSA_new_method (in
/usr/lib64/libcrypto.so.1.0.0)
==20296==    by 0x39E90A3AB9: DSA_generate_parameters (in
/usr/lib64/libcrypto.so.1.0.0)
==20296==    by 0x39E2429AAA: ldns_key_new_frm_algorithm (keys.c:851)

And similar for RSA.

Through searching, I found that DSA_generate_parameters calls:

01002 ldns_key_set_dsa_key(ldns_key *k, DSA *d)
01003 {
01004         EVP_PKEY *key = EVP_PKEY_new();
01005         EVP_PKEY_set1_DSA(key, d);
01006         k->_key.key  = key;
01007 }

The documentation for EVP_PKEY_set1_DSA(EVP_PKEY *pkey,RSA *key), which
is described as:  set the key referenced by pkey to key.

But also mentioned is EVP_PKEY_assign_RSA(EVP_PKEY *pkey,RSA *key),
which has the description: also set the referenced key to key however
these use the supplied key internally and so key will be freed when the
parent pkey is freed.

This implies that d is not freed when k->_key.key is freed, and d is not
itself explicitly freed by either ldns_key_set_dsa_key or
ldns_key_new_frm_algorithm which would account for the memory leak.

The same applies for r when passed to  ldns_key_set_rsa_key



Or am I smoking something?

Michael Sheldon
Dev-DNS Services
GoDaddy.com