[ldns-users] adding FILE* support for Python3

Fri Aug 26 11:36:44 UTC 2011

Hi Willem,

I've modified the configure.ac to perform an additional test for the
presence of >=SWIG-2.0.4 when compiling with Python >=3.2 support. The
usage of SWIG-2.0.4 is, in this case, necessary.

However according to the autoconf-archive the so far used SWIG m4 macros
have some version checking and Ubuntu specific issues.
http://git.savannah.gnu.org/gitweb/?p=autoconf-archive.git;a=history;f=m4/ax_pkg_swig.m4
Therefore I've also added the current version of autoconf SWIG macros
which is making the old SWIG m4 file obsolete.

After applying the patch please run:
svn rm --force trunk/ac_pkg_swig.m4
svn add trunk/ax_pkg_swig.m4

Best regards,
K.

Dne 25.8.2011 18:43, Karel Slany napsal(a):
> Hello Willem,
> 
> I've installed python3.2 on my Gentoo to reproduce your problem.
> However, the compilation succeeded.
> 
> But I've run into an additional incompatibility problem. The issue is
> described in the SWIG bug tracker
> http://sourceforge.net/tracker/index.php?func=detail&aid=3057804&group_id=1645&atid=101645
> and appears to be solved in SWIG-2.0.4. I'm going to test it and
> consequently post another patch.
> 
> Best regards,
> K.
> 
> 
> Dne 25.8.2011 16:15, Willem Toorop napsal(a):
>> Hi Karel,
>>
>> The patch is applied to trunk. Hopefully someone would like (and
>> succeed) to test it out now.
>>
>> I did not get it compiled on Ubuntu 11.04 however. In config.log I see:
>>
>> /usr/bin/ld: cannot find -lpython3.2
>>
>> I do have /usr/lib/libpython3.2mu.a, this I tried to create a symbolic
>> link from /usr/lib/libpython3.2mu.a to /usr/lib/libpython3.2.a just to
>> see if it would link, but then it produced lots and lots of undefined
>> references errors.
>> Apparently some libraries linked to in libpython3.2.a have to be linked
>> with the test as well.
>>
>> I didn't have those problems with python2. I suppose configure.ac still
>> needs some work :) Good luck with that. In the mean time I will give it
>> a try on FreeBSD.
>>
>> Cheers, Willem
>>
>> PS. In the mean time I did found a bug in ldns_rdf.i. On line 170 the
>> variable status is used, but not available. Bedrich?
>>
>> On 22-08-11 17:33, Karel Slany wrote:
>>> Hello again,
>>
>>> I've modified the patch. The source now contains only those statements
>>> which can be found in the SWIG documentation. All undocumented code
>>> originating from SWIG source/templates has been removed. The file is now
>>> BSD licensed.
>>
>>> The patch makes the previous posted patch obsolete. The application
>>> procedure remains the same. After patching please run:
>>
>>> chmod +x trunk/contrib/python/examples/python3/ldns-*.py
>>> svn add trunk/contrib/python/file_py3.i
>>> svn add trunk/contrib/python/examples/python3
>>
>>> Best regards,
>>> K.
>>
>>> Dne 22.8.2011 14:44, W.C.A. Wijngaards napsal(a):
>>>> Hi Karel,
>>>>
>>>> Your effort is commendable, but there looks to be license trouble with
>>>> it.  This is where the trouble seems to be from:
>>>>
>>>> +++ trunk/contrib/python/file_py3.i	(revision 0)
>>>> @@ -0,0 +1,122 @@
>>>> +/*
>>>> + * file_py3.i: Typemaps for FILE* for Python 3. Based on SWIG's file.i (See
>>>> + * the COPYRIGHT file for a list of contributors to SWIG and their
>>>> copyright
>>>> + * notices.)
>>>> + *
>>>> + * Copyright (c) 2011, Karel Slany (karel.slany AT nic.cz)
>>>> + *
>>>> + * The redistribution and/or modification terms follow the SWIG licensing
>>>> + * and distribution terms.
>>>> + */
>>>>
>>>> That COPYRIGHT file is from swig, lists their 100s of authors.  So it
>>>> looks like that file.i was part of the swig sourcecode.  The swig
>>>> sourcecode has the GPLv3 license (they have a legal stuff webpage).
>>>> Code generated by swig has very liberal licensing, that is what we have
>>>> now, we use the output of swig to generate code for us.  But it seems
>>>> that file.i was part of the swig sourcecode ...
>>>>
>>>> So, that means, assuming it is created by editing the GPLv3 work, that
>>>> the patch could be hosted in the contrib/ dir, and we explain the
>>>> license on that patch in a README.  And it does not get applied to the
>>>> main code.  Not exactly sure if we should put GPL code in the contrib of
>>>> a BSD tarball, perhaps a web url advertising the patch would be neater.
>>>>
>>>> Not sure if it may be easier to simply have the FILE I/O with python 3
>>>> io lib and feed that into the string parser routines of ldns.
>>>>
>>>> The ldns project uses the BSD license.
>>>>
>>>> Best regards,
>>>>    Wouter
>>>>
>>>> On 08/22/2011 01:42 PM, Karel Slany wrote:
>>>>> Hi Wouter,
>>>>
>>>>> many thanks for applying the previous patches. Here is another one. As
>>>>> promised it contains a workaround to enable SWIG to generate FILE*
>>>>> wrapper code for Python3. The Python2 wrapper code remains unaffected.
>>>>
>>>>> Here is a brief explanation why this patch does what it does:
>>>>>  The I/O in Python2 relies on C standard library -- it uses FILE* and
>>>>> the Python2/C API contains a straightforward solution how to access
>>>>> these structures from C.
>>>>>  Python3 uses the io module containing layers of buffers over the low
>>>>> level I/O which the OS provides. The Python3/C file API was reduced --
>>>>> no more functions for direct FILE* access. Also the io module C API does
>>>>> not contain support for direct FILE* access.
>>>>>  The Python3/C file API provides a way how to obtain a file descriptor.
>>>>> But since the Python3 has their own buffers the documentation warns
>>>>> against mixing access between Python3 and OS-level file descriptors.
>>>>
>>>>> What this patch does:
>>>>>  In order to get a valid FILE* pointer from a Python object it wraps a
>>>>> ldns C function which takes a FILE* argument with C code that:
>>>>> * ensures itself whether the object is a file object
>>>>> * flushes the python buffers using a python method call
>>>>> * gets the file descriptor and flags and creates a copy of the file
>>>>> descriptor
>>>>> * opens a FILE* for the descriptor copy
>>>>> * passes the opened FILE* to the wrapped function
>>>>> * after the execution of the function flushes the FILE* and closes it
>>>>
>>>>> The buffer flushes in Python and C ensure the correct order of written
>>>>> data but also may slow-down the code. I have not tested the performance
>>>>> impacts very thoroughly. The goal was to make FILE* work in Python3. I
>>>>> also do not know what the code will be behaving like when executed in a
>>>>> multi-threaded application.
>>>>
>>>>> In order to get rid of the sometimes unnecessary buffer flushes the
>>>>> Python3 wrapper can be extended with function opening a file for ldns
>>>>> only. Such an approach, however, introduces an inconvenience of
>>>>> closing/reopening/closing the file when accessing the files from Python
>>>>> (but giving the user control over unnecessary buffer flushing).
>>>>
>>>>> The patch contains new files to the SVN repository so please do not
>>>>> forget to make the Python3 example scripts executable and to 'svn add'
>>>>> all the new files:
>>>>
>>>>> chmod +x trunk/contrib/python/examples/python3/ldns-*.py
>>>>> svn add trunk/contrib/python/file_py3.i
>>>>> svn add trunk/contrib/python/examples/python3
>>>>
>>>>> I have tested the new code under the GNU/Linux userland using Python
>>>>> 3.1. I intent to test it also under FreeBSD -- don't know whether it
>>>>> would compile under Windows.
>>>>
>>>>> Best regards,
>>>>> K.
>>>>
>>>>
>>>>
>>>>> _______________________________________________
>>>>> ldns-users mailing list
>>>>> ldns-users at open.nlnetlabs.nl
>>>>> http://open.nlnetlabs.nl/mailman/listinfo/ldns-users
>>>>
>>> _______________________________________________
>>> ldns-users mailing list
>>> ldns-users at open.nlnetlabs.nl
>>> http://open.nlnetlabs.nl/mailman/listinfo/ldns-users
>>
>>
>>
>>
>>> _______________________________________________
>>> ldns-users mailing list
>>> ldns-users at open.nlnetlabs.nl
>>> http://open.nlnetlabs.nl/mailman/listinfo/ldns-users
>>
> _______________________________________________
> ldns-users mailing list
> ldns-users at open.nlnetlabs.nl
> http://open.nlnetlabs.nl/mailman/listinfo/ldns-users
> 
> _______________________________________________
> ldns-users mailing list
> ldns-users at open.nlnetlabs.nl
> http://open.nlnetlabs.nl/mailman/listinfo/ldns-users

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: py3.2_swig_dependency.diff
URL: <http://lists.nlnetlabs.nl/pipermail/ldns-users/attachments/20110826/39fe7592/attachment.ksh>