From msheldon at godaddy.com Fri May 10 19:31:30 2013 From: msheldon at godaddy.com (Michael Sheldon) Date: Fri, 10 May 2013 12:31:30 -0700 Subject: [ldns-users] =?utf-8?q?Broken_code_in_ldns=5Fpkt=5Fquery=5Fnew=5F?= =?utf-8?q?frm=5Fstr?= Message-ID: <20130510123130.205a61dff9fc1684c258b274662bb912.68d30cffbc.wbe@email00.secureserver.net> Sometime fairly recently, code was added to the ldns_pkt_query_new_frm_str function to add an SOA record to the authority section if the query type is IXFR. Problem is, it's completely broken. This doesn't actually work unless the SOA serial value is correct for the relevant query. It also broke my systems, since I was adding a proper SOA to the authority myself, thus resulting in two SOA in the authority, and a resulting FORMERR from remote systems. Michael Sheldon Dev-DNS Services GoDaddy.com From erik.ostlyngen at uninett.no Mon May 13 11:59:38 2013 From: erik.ostlyngen at uninett.no (Erik P. Ostlyngen) Date: Mon, 13 May 2013 13:59:38 +0200 Subject: [ldns-users] ldns perl wrapper Message-ID: <5190D5AA.4020702@uninett.no> Hi, I have created a perl xs wrapper for the ldns library which might be useful for other programmers. My motivation for creating another DNS library for perl was that I needed a tool for doing zonefile analysis and dnssec with better performance than Net::DNS (which is otherwise the preferred library). https://github.com/erikoest/Net-LDNS Comments are welcome. Please be aware that it is still beta. The object freeing and reference counting is a bit involved. I wonder if it could have been done in a simpler way, e.g. using the built-in perl gc. If you find it useful, you may include it in the ldns source package. Regards, Erik ?stlyngen From matthijs at nlnetlabs.nl Mon May 13 14:28:27 2013 From: matthijs at nlnetlabs.nl (Matthijs Mekking) Date: Mon, 13 May 2013 16:28:27 +0200 Subject: [ldns-users] Broken code in ldns_pkt_query_new_frm_str In-Reply-To: <20130510123130.205a61dff9fc1684c258b274662bb912.68d30cffbc.wbe@email00.secureserver.net> References: <20130510123130.205a61dff9fc1684c258b274662bb912.68d30cffbc.wbe@email00.secureserver.net> Message-ID: <5190F88B.30804@nlnetlabs.nl> Hi Michael, That code was added to make drill perform a more useful IXFR query. When doing 'drill -t IXFR ...', it did not add a SOA RR in the authority section, and made bind9 return a FORMERR. Adding a default SOA RR will at least make bind9 to figure out that the serial does not match and will return an AXFR. So it is a drill bugfix. I agree that it is unfortunate that this changes the behavior for ldns_pkt_query_new_frm_str. It might be better to restore the function and fix the bug different. This would require a new function call that is an adaptation of ldns_resolver_query, adding a parameter to define an authority RR. Best regards, Matthijs On 05/10/2013 09:31 PM, Michael Sheldon wrote: > Sometime fairly recently, code was added to the > ldns_pkt_query_new_frm_str function to add an SOA record to the > authority section if the query type is IXFR. > > Problem is, it's completely broken. This doesn't actually work unless > the SOA serial value is correct for the relevant query. It also broke my > systems, since I was adding a proper SOA to the authority myself, thus > resulting in two SOA in the authority, and a resulting FORMERR from > remote systems. > > > Michael Sheldon > Dev-DNS Services > GoDaddy.com > > > _______________________________________________ > ldns-users mailing list > ldns-users at open.nlnetlabs.nl > http://open.nlnetlabs.nl/mailman/listinfo/ldns-users > From msheldon at godaddy.com Mon May 13 16:36:07 2013 From: msheldon at godaddy.com (Michael J. Sheldon) Date: Mon, 13 May 2013 16:36:07 +0000 Subject: [ldns-users] Broken code in ldns_pkt_query_new_frm_str In-Reply-To: <5190F88B.30804@nlnetlabs.nl> References: <20130510123130.205a61dff9fc1684c258b274662bb912.68d30cffbc.wbe@email00.secureserver.net>, <5190F88B.30804@nlnetlabs.nl> Message-ID: <996709eee74b40c0a5dbce91cf6bf388@CO1PR02MB095.namprd02.prod.outlook.com> I would have less problem with this *if* it actually worked. But without a relevant serial # in the SOA record, this won't work anyway Michael Sheldon Dev-DNS Services GoDaddy.com ________________________________________ From: Matthijs Mekking Sent: Monday, May 13, 2013 7:28 AM To: Michael J. Sheldon Cc: ldns-users at open.nlnetlabs.nl Subject: Re: [ldns-users] Broken code in ldns_pkt_query_new_frm_str Hi Michael, That code was added to make drill perform a more useful IXFR query. When doing 'drill -t IXFR ...', it did not add a SOA RR in the authority section, and made bind9 return a FORMERR. Adding a default SOA RR will at least make bind9 to figure out that the serial does not match and will return an AXFR. So it is a drill bugfix. I agree that it is unfortunate that this changes the behavior for ldns_pkt_query_new_frm_str. It might be better to restore the function and fix the bug different. This would require a new function call that is an adaptation of ldns_resolver_query, adding a parameter to define an authority RR. Best regards, Matthijs On 05/10/2013 09:31 PM, Michael Sheldon wrote: > Sometime fairly recently, code was added to the > ldns_pkt_query_new_frm_str function to add an SOA record to the > authority section if the query type is IXFR. > > Problem is, it's completely broken. This doesn't actually work unless > the SOA serial value is correct for the relevant query. It also broke my > systems, since I was adding a proper SOA to the authority myself, thus > resulting in two SOA in the authority, and a resulting FORMERR from > remote systems. > > > Michael Sheldon > Dev-DNS Services > GoDaddy.com > > > _______________________________________________ > ldns-users mailing list > ldns-users at open.nlnetlabs.nl > http://open.nlnetlabs.nl/mailman/listinfo/ldns-users > From willem at nlnetlabs.nl Tue May 14 08:26:54 2013 From: willem at nlnetlabs.nl (Willem Toorop) Date: Tue, 14 May 2013 10:26:54 +0200 Subject: [ldns-users] ldns perl wrapper In-Reply-To: <5190D5AA.4020702@uninett.no> References: <5190D5AA.4020702@uninett.no> Message-ID: <5191F54E.60103@nlnetlabs.nl> Erik! This is exciting! Can't wait to have a good look. I'll come back to it soon. -- Willem Op 13-05-13 13:59, Erik P. Ostlyngen schreef: > Hi, > > I have created a perl xs wrapper for the ldns library which might be > useful for other programmers. My motivation for creating another DNS > library for perl was that I needed a tool for doing zonefile analysis > and dnssec with better performance than Net::DNS (which is otherwise > the preferred library). > > https://github.com/erikoest/Net-LDNS > > Comments are welcome. Please be aware that it is still beta. The > object freeing and reference counting is a bit involved. I wonder if > it could have been done in a simpler way, e.g. using the built-in perl gc. > > If you find it useful, you may include it in the ldns source package. > > Regards, > Erik ?stlyngen > _______________________________________________ > ldns-users mailing list > ldns-users at open.nlnetlabs.nl > http://open.nlnetlabs.nl/mailman/listinfo/ldns-users > From matthijs at nlnetlabs.nl Tue May 14 09:53:38 2013 From: matthijs at nlnetlabs.nl (Matthijs Mekking) Date: Tue, 14 May 2013 11:53:38 +0200 Subject: [ldns-users] Broken code in ldns_pkt_query_new_frm_str In-Reply-To: <996709eee74b40c0a5dbce91cf6bf388@CO1PR02MB095.namprd02.prod.outlook.com> References: <20130510123130.205a61dff9fc1684c258b274662bb912.68d30cffbc.wbe@email00.secureserver.net>, <5190F88B.30804@nlnetlabs.nl> <996709eee74b40c0a5dbce91cf6bf388@CO1PR02MB095.namprd02.prod.outlook.com> Message-ID: <519209A2.2070208@nlnetlabs.nl> Well it works, at least for me (using it in one of our unit test). Could you say more about the behavior you see? One way or the other, I am tempted to implement this in the way I proposed earlier. Best regards, Matthijs On 05/13/2013 06:36 PM, Michael J. Sheldon wrote: > I would have less problem with this *if* it actually worked. But without a relevant serial # in the SOA record, this won't work anyway > > Michael Sheldon > Dev-DNS Services > GoDaddy.com > ________________________________________ > From: Matthijs Mekking > Sent: Monday, May 13, 2013 7:28 AM > To: Michael J. Sheldon > Cc: ldns-users at open.nlnetlabs.nl > Subject: Re: [ldns-users] Broken code in ldns_pkt_query_new_frm_str > > Hi Michael, > > That code was added to make drill perform a more useful IXFR query. When > doing 'drill -t IXFR ...', it did not add a SOA RR in the authority > section, and made bind9 return a FORMERR. Adding a default SOA RR will > at least make bind9 to figure out that the serial does not match and > will return an AXFR. > > So it is a drill bugfix. I agree that it is unfortunate that this > changes the behavior for ldns_pkt_query_new_frm_str. It might be better > to restore the function and fix the bug different. This would require a > new function call that is an adaptation of ldns_resolver_query, adding a > parameter to define an authority RR. > > Best regards, > Matthijs > > > > On 05/10/2013 09:31 PM, Michael Sheldon wrote: >> Sometime fairly recently, code was added to the >> ldns_pkt_query_new_frm_str function to add an SOA record to the >> authority section if the query type is IXFR. >> >> Problem is, it's completely broken. This doesn't actually work unless >> the SOA serial value is correct for the relevant query. It also broke my >> systems, since I was adding a proper SOA to the authority myself, thus >> resulting in two SOA in the authority, and a resulting FORMERR from >> remote systems. >> >> >> Michael Sheldon >> Dev-DNS Services >> GoDaddy.com >> >> >> _______________________________________________ >> ldns-users mailing list >> ldns-users at open.nlnetlabs.nl >> http://open.nlnetlabs.nl/mailman/listinfo/ldns-users >> > > From msheldon at godaddy.com Tue May 14 16:55:41 2013 From: msheldon at godaddy.com (Michael J. Sheldon) Date: Tue, 14 May 2013 16:55:41 +0000 Subject: [ldns-users] Broken code in ldns_pkt_query_new_frm_str In-Reply-To: <519209A2.2070208@nlnetlabs.nl> References: <20130510123130.205a61dff9fc1684c258b274662bb912.68d30cffbc.wbe@email00.secureserver.net>, <5190F88B.30804@nlnetlabs.nl> <996709eee74b40c0a5dbce91cf6bf388@CO1PR02MB095.namprd02.prod.outlook.com>, <519209A2.2070208@nlnetlabs.nl> Message-ID: Unless the serial of the SOA passed in the authority is a recognized previous version of the zone, you either get a full AXFR, or just the current SOA, depending on whether the serial passed is higher or lower than the current serial. So the only way for the current implementation of ldns_pkt_query_new_frm_str to work with IXFR is to then edit the SOA in the packet to properly set the serial value. IXFR is a unique query that doesn't match the simplicity of the others. It would be reasonable to me to have a dedicated function, something like ldns_pkt_query_new_ixfr with origin and serial as parameters. Michael Sheldon Dev-DNS Services GoDaddy.com ________________________________________ From: Matthijs Mekking Sent: Tuesday, May 14, 2013 2:53 AM To: Michael J. Sheldon Cc: ldns-users at open.nlnetlabs.nl Subject: Re: [ldns-users] Broken code in ldns_pkt_query_new_frm_str Well it works, at least for me (using it in one of our unit test). Could you say more about the behavior you see? One way or the other, I am tempted to implement this in the way I proposed earlier. Best regards, Matthijs On 05/13/2013 06:36 PM, Michael J. Sheldon wrote: > I would have less problem with this *if* it actually worked. But without a relevant serial # in the SOA record, this won't work anyway > > Michael Sheldon > Dev-DNS Services > GoDaddy.com > ________________________________________ > From: Matthijs Mekking > Sent: Monday, May 13, 2013 7:28 AM > To: Michael J. Sheldon > Cc: ldns-users at open.nlnetlabs.nl > Subject: Re: [ldns-users] Broken code in ldns_pkt_query_new_frm_str > > Hi Michael, > > That code was added to make drill perform a more useful IXFR query. When > doing 'drill -t IXFR ...', it did not add a SOA RR in the authority > section, and made bind9 return a FORMERR. Adding a default SOA RR will > at least make bind9 to figure out that the serial does not match and > will return an AXFR. > > So it is a drill bugfix. I agree that it is unfortunate that this > changes the behavior for ldns_pkt_query_new_frm_str. It might be better > to restore the function and fix the bug different. This would require a > new function call that is an adaptation of ldns_resolver_query, adding a > parameter to define an authority RR. > > Best regards, > Matthijs > > > > On 05/10/2013 09:31 PM, Michael Sheldon wrote: >> Sometime fairly recently, code was added to the >> ldns_pkt_query_new_frm_str function to add an SOA record to the >> authority section if the query type is IXFR. >> >> Problem is, it's completely broken. This doesn't actually work unless >> the SOA serial value is correct for the relevant query. It also broke my >> systems, since I was adding a proper SOA to the authority myself, thus >> resulting in two SOA in the authority, and a resulting FORMERR from >> remote systems. >> >> >> Michael Sheldon >> Dev-DNS Services >> GoDaddy.com >> >> >> _______________________________________________ >> ldns-users mailing list >> ldns-users at open.nlnetlabs.nl >> http://open.nlnetlabs.nl/mailman/listinfo/ldns-users >> > > From henri at asseily.com Fri May 17 07:27:32 2013 From: henri at asseily.com (Henri Asseily) Date: Fri, 17 May 2013 10:27:32 +0300 Subject: [ldns-users] ldns perl wrapper In-Reply-To: <5190D5AA.4020702@uninett.no> References: <5190D5AA.4020702@uninett.no> Message-ID: <5F6126BA-3EDD-4E48-8F1D-50B6DF05F666@asseily.com> Hello Erik, this is looking good. Clearly it's a pure low level API (i.e. a "thin" wrapper to the C API), but from there it becomes easy to create a more perl-ish API. For example, the creation of an RR shouldn't be as involved as: my $rr1 = new Net::LDNS::RR( type => LDNS_RR_TYPE_SOA, class => LDNS_RR_CLASS_CH, ttl => 1234, owner => new Net::LDNS::RData(LDNS_RDF_TYPE_DNAME, 'myzone.org'), rdata => [ new Net::LDNS::RData(LDNS_RDF_TYPE_DNAME, 'hostmaster.myzone.org'), new Net::LDNS::RData(LDNS_RDF_TYPE_DNAME, 'master.myzone.org'), new Net::LDNS::RData(LDNS_RDF_TYPE_INT32, '2012113030'), new Net::LDNS::RData(LDNS_RDF_TYPE_PERIOD, '12345'), new Net::LDNS::RData(LDNS_RDF_TYPE_PERIOD, '1827'), new Net::LDNS::RData(LDNS_RDF_TYPE_PERIOD, '2345678'), new Net::LDNS::RData(LDNS_RDF_TYPE_PERIOD, '87654') ], ); For starters, I would create a map for standard RR types that would know the content (and types) of the rdata array, or create new RR type classes. So one would be able to do: my $rr1 = new Net::LDNS::RR::SOA( ttl => 1234, owner => 'myzone.org', serial => 2012113030, refresh => 12345, update => 1827, expiry => 2345678, minimum => 87654, ); with of course default values where it makes sense. Again, this can (and should) all be done in pure perl, now that the low level interface is there. Regarding the garbage collection, you should indeed be leveraging the Perl GC. There shouldn't be a concept of an explicit owner in Perl if one can avoid it. You shouldn't have to worry about higher-ups "owning" via the C API lower level objects (i.e. RR owning RDATAs) since the RDATAs will be linked to the RR through the Perl layer anyway (RDATAs being array members of RRs). All you should worry about is that when destroy'ing the objects that you keep doing what you do: sub DESTROY { Net::LDNS::GC::free($_[0]); } (but see below) All the deep_free stuff is useless here, since the Perl GC will call DESTROY on every object that is refcounted to zero anyway, so all RDATAs for example will be DESTROYed in turn when an RR is destroyed, assuming the RDATA object only exists in one RR. I think you should be able to simply get totally rid of all your GC since XS will add xx_newmortal() in the generated code for your object constructors. That ensures that Perl "owns" the generated object and manages the refcounts. Now there's another thing you can do as well, which should make your code cleaner: you can get rid of your "sub DESTROY" in each of the Perl packages by putting the DESTROY code in the XS file since it all just calls the C destruction code. Say you have: MODULE = Net::LDNS PACKAGE = Net::LDNS::RR Replace it with: MODULE = Net::LDNS PACKAGE = Net::LDNS::RR PREFIX = ldnsrr_ and then you add: void ldnsrr_DESTROY(rr) ldns_rr *rr CODE: ldns_rr_free(rr) I hope that what I wrote above doesn't confuse more than it helps. --- Henri Asseily henri.tel "For me, it is far better to grasp the Universe as it really is than to persist in delusion, however satisfying and reassuring." ?Carl Sagan On May 13, 2013, at 2:59 PM, "Erik P. Ostlyngen" wrote: > Hi, > > I have created a perl xs wrapper for the ldns library which might be > useful for other programmers. My motivation for creating another DNS > library for perl was that I needed a tool for doing zonefile analysis > and dnssec with better performance than Net::DNS (which is otherwise > the preferred library). > > https://github.com/erikoest/Net-LDNS > > Comments are welcome. Please be aware that it is still beta. The > object freeing and reference counting is a bit involved. I wonder if > it could have been done in a simpler way, e.g. using the built-in perl gc. > > If you find it useful, you may include it in the ldns source package. > > Regards, > Erik ?stlyngen > _______________________________________________ > ldns-users mailing list > ldns-users at open.nlnetlabs.nl > http://open.nlnetlabs.nl/mailman/listinfo/ldns-users From erik.ostlyngen at uninett.no Sat May 18 13:39:59 2013 From: erik.ostlyngen at uninett.no (=?windows-1252?Q?Erik_=D8stlyngen?=) Date: Sat, 18 May 2013 15:39:59 +0200 Subject: [ldns-users] ldns perl wrapper In-Reply-To: <5F6126BA-3EDD-4E48-8F1D-50B6DF05F666@asseily.com> References: <5190D5AA.4020702@uninett.no> <5F6126BA-3EDD-4E48-8F1D-50B6DF05F666@asseily.com> Message-ID: <519784AF.5030203@uninett.no> Hi Henry, Thanks a lot for for your comments. I agree that creating specialized constructors for common RRs is a good idea. So far I've focused on only mapping the C functions and make them available for perl. There are probably many other higher level methods which could be useful also, which could be added in pure perl. I'm not sure that I understand your comments about the gc. Using the built-in perl gc in the 'naive' way is imo problematic because freeing each single struct instead of doing a deep free is very cpu and memory consuming for large zonefiles because it is then necessary to map the whole structure of rrlists, rrs and rdatas to their own perl object at them moment the parent object is created (for example when a zone is read from file). Some of the reason for making a library wrapper rather than just doing pure perl then falls away. After all the perl wrapper will have to mimic the whole data structure that the c-library makes for you. My approach has been to create a perl object of a sub structure only if and when it is accessed. Another design choise I made is that if a substruct is accessed, the returned perl-wrapper object references the original struct rather than making a clone. But if a structure is _inserted_ as a substructure into an object, a clone is made. I've made a small comment on this in the LDNS.pm perldoc. In your example code, you indicate subclassing the RR into RR::SOA, etc. This is imo a good idea which I have been thinking about without finding a good way to do it. The problem is that each time an rr is returned (and a perl object is wrapped around it), I then have to determine which kind of rr is returned and re-bless the perl object into the right kind of subclass. I'm not sure that can be done directly with the xs typemap functionality. But I'm a novice when it comes to perl xs. If I find a good way to do this, other classes might be subclassed too, e.g. rrlist -> rrset and rdata -> dname. Btw. There are some known bugs which are not commented. One of them is the default values in the constructors which do not work. Parts of the ldns library has not yet been mapped, for example the *_buffer and *_wire functions and some of the functions for handling specific key types. Regards, Erik ?stlyngen On 05/17/2013 09:27 AM, Henri Asseily wrote: > Hello Erik, this is looking good. Clearly it's a pure low level API > (i.e. a "thin" wrapper to the C API), but from there it becomes easy > to create a more perl-ish API. For example, the creation of an RR > shouldn't be as involved as: > > my $rr1 = new Net::LDNS::RR( type => LDNS_RR_TYPE_SOA, class => > LDNS_RR_CLASS_CH, ttl => 1234, owner => new > Net::LDNS::RData(LDNS_RDF_TYPE_DNAME, 'myzone.org'), rdata => [ new > Net::LDNS::RData(LDNS_RDF_TYPE_DNAME, 'hostmaster.myzone.org'), new > Net::LDNS::RData(LDNS_RDF_TYPE_DNAME, 'master.myzone.org'), new > Net::LDNS::RData(LDNS_RDF_TYPE_INT32, '2012113030'), new > Net::LDNS::RData(LDNS_RDF_TYPE_PERIOD, '12345'), new > Net::LDNS::RData(LDNS_RDF_TYPE_PERIOD, '1827'), new > Net::LDNS::RData(LDNS_RDF_TYPE_PERIOD, '2345678'), new > Net::LDNS::RData(LDNS_RDF_TYPE_PERIOD, '87654') ], ); > > For starters, I would create a map for standard RR types that would > know the content (and types) of the rdata array, or create new RR > type classes. So one would be able to do: > > my $rr1 = new Net::LDNS::RR::SOA( ttl => 1234, owner => > 'myzone.org', serial => 2012113030, refresh => 12345, update => > 1827, expiry => 2345678, minimum => 87654, ); > > with of course default values where it makes sense. > > Again, this can (and should) all be done in pure perl, now that the > low level interface is there. > > Regarding the garbage collection, you should indeed be leveraging the > Perl GC. There shouldn't be a concept of an explicit owner in Perl if > one can avoid it. You shouldn't have to worry about higher-ups > "owning" via the C API lower level objects (i.e. RR owning RDATAs) > since the RDATAs will be linked to the RR through the Perl layer > anyway (RDATAs being array members of RRs). All you should worry > about is that when destroy'ing the objects that you keep doing what > you do: > > sub DESTROY { Net::LDNS::GC::free($_[0]); } > > (but see below) > > All the deep_free stuff is useless here, since the Perl GC will call > DESTROY on every object that is refcounted to zero anyway, so all > RDATAs for example will be DESTROYed in turn when an RR is destroyed, > assuming the RDATA object only exists in one RR. > > I think you should be able to simply get totally rid of all your GC > since XS will add xx_newmortal() in the generated code for your > object constructors. That ensures that Perl "owns" the generated > object and manages the refcounts. > > Now there's another thing you can do as well, which should make your > code cleaner: you can get rid of your "sub DESTROY" in each of the > Perl packages by putting the DESTROY code in the XS file since it all > just calls the C destruction code. > > Say you have: > > MODULE = Net::LDNS PACKAGE = Net::LDNS::RR > > Replace it with: > > MODULE = Net::LDNS PACKAGE = Net::LDNS::RR PREFIX = ldnsrr_ > > and then you add: > > void ldnsrr_DESTROY(rr) ldns_rr *rr CODE: ldns_rr_free(rr) > > > I hope that what I wrote above doesn't confuse more than it helps. > > --- Henri Asseily henri.tel > > "For me, it is far better to grasp the Universe as it really is than > to persist in delusion, however satisfying and reassuring." ?Carl > Sagan > > On May 13, 2013, at 2:59 PM, "Erik P. Ostlyngen" > wrote: > >> Hi, >> >> I have created a perl xs wrapper for the ldns library which might >> be useful for other programmers. My motivation for creating another >> DNS library for perl was that I needed a tool for doing zonefile >> analysis and dnssec with better performance than Net::DNS (which is >> otherwise the preferred library). >> >> https://github.com/erikoest/Net-LDNS >> >> Comments are welcome. Please be aware that it is still beta. The >> object freeing and reference counting is a bit involved. I wonder >> if it could have been done in a simpler way, e.g. using the >> built-in perl gc. >> >> If you find it useful, you may include it in the ldns source >> package. >> >> Regards, Erik ?stlyngen >> _______________________________________________ ldns-users mailing >> list ldns-users at open.nlnetlabs.nl >> http://open.nlnetlabs.nl/mailman/listinfo/ldns-users > From willem at nlnetlabs.nl Thu May 23 13:33:23 2013 From: willem at nlnetlabs.nl (Willem Toorop) Date: Thu, 23 May 2013 15:33:23 +0200 Subject: [ldns-users] 'example' tool: ldns-3597 In-Reply-To: <5176A4C1.7050602@nlnetlabs.nl> References: <5171544E.8040000@sidn.nl> <5176A4C1.7050602@nlnetlabs.nl> Message-ID: <519E1AA3.6040304@nlnetlabs.nl> Op 23-04-13 17:12, Willem Toorop schreef: > We could add a flag for struct ldns_output_format. FYI: I've taken the ldns_output_format approach (with ability to set and unset RR types). ldns-read-zone has two new options illustrating its usage: -u Mark for printing in unknown type format -U Mark for not printing in unknown type format When only -U options are given, all types are printed in unknown type format except the given RR types -- Willem From jelte.jansen at sidn.nl Mon May 27 10:05:14 2013 From: jelte.jansen at sidn.nl (Jelte Jansen) Date: Mon, 27 May 2013 12:05:14 +0200 Subject: [ldns-users] suggestion for a parse-rrs callback method Message-ID: <51A32FDA.2000802@sidn.nl> I just posted a blog article about a possible extension i'd like to see in ldns (in dutch): http://www.sidnlabs.nl/laatste-berichten/nieuwsdetail/article/dns-resource-record-handling/ lemme quickly summarize it in english here: I often find (like in the recent case of the 3597 translation), that I simply need to read a bunch of resource records from either dig/drill output or a zone file, and do something with them. Since I'd like to focus on that 'something' and not on the zone parsing itself this usually only works with a pretty specific 'flattened' input format (generally the output of ldns-read-zone), and not with the full flexibility a zone parser gives you. So what would be very useful is a general construct to define a function that is called for every RR in a given file stream, e.g. a callback that gets an RR and probably a callback-specific void* 'state' object. Said construct would keep track of the state involved in 'real' parsing so that I don't have to repeat the same code over and over again. Ideally, the calling program would then just have to define its callback, open the file, and call the ldns parser code, and be done with it, e.g. in the case of ldns I'd want something like: // Callback to print the RR, no state. int printer(ldns_rr* rr, void* state) { ldns_rr_print(stdout, rr); return 0; } in = fopen("my.zone", "r"); if (in != NULL) { ldns_parse_rr_stream(in, printer, NULL); fclose(in); } So no more keeping ldns_rr's around (unless you need them later), no more tracking of origin, ttl, etc. Some details need to be hashed out (e.g. is ownership of the rr transfered, should there be an initial origin/default ttl, etc.) But in general, I think this would be a reasonably simple but highly useful addition. This is an initial stab at an implementation, it's really not that much right now (but I expect it to grow a little bit): ldns_status ldns_parse_rr_stream(FILE* in, int (*callback)(ldns_rr*, void*), void* callback_state) { ldns_rdf* origin = NULL; uint32_t ttl = 3600; ldns_rdf* prev = NULL; ldns_rr* cur_rr = NULL; ldns_status status; while (!feof(in)) { status = ldns_rr_new_frm_fp(&cur_rr, in, &ttl, &origin, &prev); if (!callback(cur_rr, callback_state)) { return LDNS_STATUS_ERR; } ldns_rr_free(cur_rr); cur_rr = NULL; switch (status) { case LDNS_STATUS_OK: case LDNS_STATUS_SYNTAX_EMPTY: /* empty line was seen */ case LDNS_STATUS_SYNTAX_TTL: /* the function set the ttl */ case LDNS_STATUS_SYNTAX_ORIGIN: /* the function set the origin */ break; default: if (prev != NULL) { ldns_rdf_deep_free(prev); } return status; } } if (prev != NULL) { ldns_rdf_deep_free(prev); } return LDNS_STATUS_OK; } From edmonds at debian.org Mon May 27 15:17:31 2013 From: edmonds at debian.org (Robert Edmonds) Date: Mon, 27 May 2013 11:17:31 -0400 Subject: [ldns-users] suggestion for a parse-rrs callback method In-Reply-To: <51A32FDA.2000802@sidn.nl> References: <51A32FDA.2000802@sidn.nl> Message-ID: <20130527151731.GA6761@mycre.ws> Jelte Jansen wrote: > So what would be very useful is a general construct to define a > function that is called for every RR in a given file stream, e.g. a > callback that gets an RR and probably a callback-specific void* > 'state' object. Said construct would keep track of the state > involved in 'real' parsing so that I don't have to repeat the same > code over and over again. > > Ideally, the calling program would then just have to define its > callback, open the file, and call the ldns parser code, and be done > with it, e.g. in the case of ldns I'd want something like: > > // Callback to print the RR, no state. > int printer(ldns_rr* rr, void* state) { > ldns_rr_print(stdout, rr); > return 0; > } > > in = fopen("my.zone", "r"); > if (in != NULL) { > ldns_parse_rr_stream(in, printer, NULL); > fclose(in); > } this seems like a very simple iterator pattern. why complicate it with a callback? e.g., in = fopen("my.zone", "r"); if (in != NULL) { ldns_rr *rr; ldns_rr_iter *iter = ldns_parse_rr_iter_frm_fp(in); while ((rr = ldns_rr_iter_next(iter)) != NULL) ldns_rr_print(stdout, rr); ldns_rr_iter_free(iter); } this makes the interface much easier to call in languages that are not C or C++. -- Robert Edmonds edmonds at debian.org From Willem at NLnetLabs.nl Mon May 27 17:28:52 2013 From: Willem at NLnetLabs.nl (Willem Toorop) Date: Mon, 27 May 2013 19:28:52 +0200 Subject: [ldns-users] suggestion for a parse-rrs callback method In-Reply-To: <20130527151731.GA6761@mycre.ws> References: <51A32FDA.2000802@sidn.nl> <20130527151731.GA6761@mycre.ws> Message-ID: <51A397D4.5040204@NLnetLabs.nl> +1 I am actually thinking along these lines (iterators and other more interface oriented (and less data centric) approaches) for some time now. I guess the parser is a good place to start out with as some of the example tools would benefit right away! For example ldns-verify-zone wouldn't have to read in the complete zone just to check the apex!. -- Willem Op 27-05-13 17:17, Robert Edmonds schreef: > Jelte Jansen wrote: >> So what would be very useful is a general construct to define a >> function that is called for every RR in a given file stream, e.g. a >> callback that gets an RR and probably a callback-specific void* >> 'state' object. Said construct would keep track of the state >> involved in 'real' parsing so that I don't have to repeat the same >> code over and over again. >> >> Ideally, the calling program would then just have to define its >> callback, open the file, and call the ldns parser code, and be done >> with it, e.g. in the case of ldns I'd want something like: >> >> // Callback to print the RR, no state. >> int printer(ldns_rr* rr, void* state) { >> ldns_rr_print(stdout, rr); >> return 0; >> } >> >> in = fopen("my.zone", "r"); >> if (in != NULL) { >> ldns_parse_rr_stream(in, printer, NULL); >> fclose(in); >> } > > this seems like a very simple iterator pattern. why complicate it with > a callback? e.g., > > in = fopen("my.zone", "r"); > if (in != NULL) { > ldns_rr *rr; > ldns_rr_iter *iter = ldns_parse_rr_iter_frm_fp(in); > while ((rr = ldns_rr_iter_next(iter)) != NULL) > ldns_rr_print(stdout, rr); > ldns_rr_iter_free(iter); > } > > this makes the interface much easier to call in languages that are not C > or C++. > From miek at miek.nl Mon May 27 18:58:08 2013 From: miek at miek.nl (Miek Gieben) Date: Mon, 27 May 2013 19:58:08 +0100 Subject: [ldns-users] suggestion for a parse-rrs callback method In-Reply-To: <51A397D4.5040204@NLnetLabs.nl> References: <51A32FDA.2000802@sidn.nl> <20130527151731.GA6761@mycre.ws> <51A397D4.5040204@NLnetLabs.nl> Message-ID: <20130527185808.GB10544@miek.nl> [ Quoting in "Re: [ldns-users] suggestion for a p..." ] > +1 > > I am actually thinking along these lines (iterators and other more > interface oriented (and less data centric) approaches) for some time > now. I guess the parser is a good place to start out with as some of the > example tools would benefit right away! For example ldns-verify-zone > wouldn't have to read in the complete zone just to check the apex!. SGTM Maybe this is also an opportunity to cut down on the API size of ldns? grtz Miek -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From lmxhappy at gmail.com Tue May 28 21:17:25 2013 From: lmxhappy at gmail.com (=?GB2312?B?wfXD99DH?=) Date: Wed, 29 May 2013 05:17:25 +0800 Subject: [ldns-users] sha1_block_data_order_ssse3 segment error Message-ID: I use ldns to program a multithread programs to query soa RRs for many domain names. gdb the core file and execute bt command and found that there is a function sha1_block_data_order_ssse3() in the result. -------------- next part -------------- An HTML attachment was scrubbed... URL: From willem at nlnetlabs.nl Wed May 29 07:36:02 2013 From: willem at nlnetlabs.nl (Willem Toorop) Date: Wed, 29 May 2013 09:36:02 +0200 Subject: [ldns-users] sha1_block_data_order_ssse3 segment error In-Reply-To: References: Message-ID: <51A5AFE2.60704@nlnetlabs.nl> Hi ???, Which version of ldns do you use? Could you copy/paste the complete backtrace? i.e. the openssl sha1_block_data_order_ssse3 function is not called from ldns directly. Thanks! -- Willem Op 28-05-13 23:17, ??? schreef: > I use ldns to program a multithread programs to query soa RRs for many > domain names. > > gdb the core file and execute bt command and found that there is a > function sha1_block_data_order_ssse3() in the result. > > > > > _______________________________________________ > ldns-users mailing list > ldns-users at open.nlnetlabs.nl > http://open.nlnetlabs.nl/mailman/listinfo/ldns-users >