testbound tests & convergence with Deckard

[W.C.A. Wijngaards]

Hi Petr,

On 22/03/17 19:16, Petr Špaček via Unbound-users wrote:
> Hello,
> 
> and thank you for testbound test suite!
> 
> Please accept my apology if this is not the right list but I was not
> able to find unbound-devel list.
> 
> I've spent last weeks working on Deckard DNS test tool [1], which is
> heavily inspired by testbound. It can be seen as generalization of
> testbound for other DNS software. In fact it can run one set of tests on
> Knot resolver, Unbound, and Power DNS Recursor. For some reason BIND
> does not work under Deckard but it might have been a problem between
> chair and keyboard.
> 
> My hope is that Deckard test tool and associated test suite [2] will
> eventually match testbound capabilities and after that keep growing and
> working on multiple resolvers at the same time. That would allow us to
> share the same (large!) set of tests and test tooling, which could
> hopefully lead to more reliable tool and bigger test coverage.

Yes, that would be very nice.

You should note that not all resolvers work in the same way.  So it is
likely that tests may not work; i.e. they need a particular
implementation choice.

So, maybe, in the file include an option that enables one or more flags,
and if the current software product has this flag the test result is
ignored.

Anyway, this is not necessary today.

> 
> 
> Sometimes I modify imported tests either to add some comments to them
> (to ease my life later on) or improving the tests to e.g. work with
> query name minimization and so on. For now I plan to send these small
> changes to you so both test suites can stay syncronized until:
> a) both test suites merge
> or
> b) the plan envisioned above turns out to be infeasible
> 
> What is the right approach to submit such test changes?
> Would you accept pull requests on Github [3]?
> It would be easier for me than sending e-mails around.

You can send me some sort of diff or patch or link.  And then I'll apply
it after inspection.

Best regards, Wouter

> 
> Let me know if you are interested in further cooperation on DNS software
> testing.
> 
> Have a nice day!
> 
> [1] https://gitlab.labs.nic.cz/knot/deckard/blob/master/README.rst
> [2] https://gitlab.labs.nic.cz/knot/deckard/tree/master/sets/resolver
> [3] https://github.com/NLnetLabs/unbound
> 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.nlnetlabs.nl/pipermail/unbound-users/attachments/20170323/a07a74c2/attachment.bin>