[ldns-users] adding FILE* support for Python3

Karel Slany karel.slany at nic.cz
Thu Aug 25 16:43:11 UTC 2011


Hello Willem,

I've installed python3.2 on my Gentoo to reproduce your problem.
However, the compilation succeeded.

But I've run into an additional incompatibility problem. The issue is
described in the SWIG bug tracker
http://sourceforge.net/tracker/index.php?func=detail&aid=3057804&group_id=1645&atid=101645
and appears to be solved in SWIG-2.0.4. I'm going to test it and
consequently post another patch.

Best regards,
K.


Dne 25.8.2011 16:15, Willem Toorop napsal(a):
> Hi Karel,
> 
> The patch is applied to trunk. Hopefully someone would like (and
> succeed) to test it out now.
> 
> I did not get it compiled on Ubuntu 11.04 however. In config.log I see:
> 
> /usr/bin/ld: cannot find -lpython3.2
> 
> I do have /usr/lib/libpython3.2mu.a, this I tried to create a symbolic
> link from /usr/lib/libpython3.2mu.a to /usr/lib/libpython3.2.a just to
> see if it would link, but then it produced lots and lots of undefined
> references errors.
> Apparently some libraries linked to in libpython3.2.a have to be linked
> with the test as well.
> 
> I didn't have those problems with python2. I suppose configure.ac still
> needs some work :) Good luck with that. In the mean time I will give it
> a try on FreeBSD.
> 
> Cheers, Willem
> 
> PS. In the mean time I did found a bug in ldns_rdf.i. On line 170 the
> variable status is used, but not available. Bedrich?
> 
> On 22-08-11 17:33, Karel Slany wrote:
>> Hello again,
> 
>> I've modified the patch. The source now contains only those statements
>> which can be found in the SWIG documentation. All undocumented code
>> originating from SWIG source/templates has been removed. The file is now
>> BSD licensed.
> 
>> The patch makes the previous posted patch obsolete. The application
>> procedure remains the same. After patching please run:
> 
>> chmod +x trunk/contrib/python/examples/python3/ldns-*.py
>> svn add trunk/contrib/python/file_py3.i
>> svn add trunk/contrib/python/examples/python3
> 
>> Best regards,
>> K.
> 
>> Dne 22.8.2011 14:44, W.C.A. Wijngaards napsal(a):
>>> Hi Karel,
>>>
>>> Your effort is commendable, but there looks to be license trouble with
>>> it.  This is where the trouble seems to be from:
>>>
>>> +++ trunk/contrib/python/file_py3.i	(revision 0)
>>> @@ -0,0 +1,122 @@
>>> +/*
>>> + * file_py3.i: Typemaps for FILE* for Python 3. Based on SWIG's file.i (See
>>> + * the COPYRIGHT file for a list of contributors to SWIG and their
>>> copyright
>>> + * notices.)
>>> + *
>>> + * Copyright (c) 2011, Karel Slany (karel.slany AT nic.cz)
>>> + *
>>> + * The redistribution and/or modification terms follow the SWIG licensing
>>> + * and distribution terms.
>>> + */
>>>
>>> That COPYRIGHT file is from swig, lists their 100s of authors.  So it
>>> looks like that file.i was part of the swig sourcecode.  The swig
>>> sourcecode has the GPLv3 license (they have a legal stuff webpage).
>>> Code generated by swig has very liberal licensing, that is what we have
>>> now, we use the output of swig to generate code for us.  But it seems
>>> that file.i was part of the swig sourcecode ...
>>>
>>> So, that means, assuming it is created by editing the GPLv3 work, that
>>> the patch could be hosted in the contrib/ dir, and we explain the
>>> license on that patch in a README.  And it does not get applied to the
>>> main code.  Not exactly sure if we should put GPL code in the contrib of
>>> a BSD tarball, perhaps a web url advertising the patch would be neater.
>>>
>>> Not sure if it may be easier to simply have the FILE I/O with python 3
>>> io lib and feed that into the string parser routines of ldns.
>>>
>>> The ldns project uses the BSD license.
>>>
>>> Best regards,
>>>    Wouter
>>>
>>> On 08/22/2011 01:42 PM, Karel Slany wrote:
>>>> Hi Wouter,
>>>
>>>> many thanks for applying the previous patches. Here is another one. As
>>>> promised it contains a workaround to enable SWIG to generate FILE*
>>>> wrapper code for Python3. The Python2 wrapper code remains unaffected.
>>>
>>>> Here is a brief explanation why this patch does what it does:
>>>>  The I/O in Python2 relies on C standard library -- it uses FILE* and
>>>> the Python2/C API contains a straightforward solution how to access
>>>> these structures from C.
>>>>  Python3 uses the io module containing layers of buffers over the low
>>>> level I/O which the OS provides. The Python3/C file API was reduced --
>>>> no more functions for direct FILE* access. Also the io module C API does
>>>> not contain support for direct FILE* access.
>>>>  The Python3/C file API provides a way how to obtain a file descriptor.
>>>> But since the Python3 has their own buffers the documentation warns
>>>> against mixing access between Python3 and OS-level file descriptors.
>>>
>>>> What this patch does:
>>>>  In order to get a valid FILE* pointer from a Python object it wraps a
>>>> ldns C function which takes a FILE* argument with C code that:
>>>> * ensures itself whether the object is a file object
>>>> * flushes the python buffers using a python method call
>>>> * gets the file descriptor and flags and creates a copy of the file
>>>> descriptor
>>>> * opens a FILE* for the descriptor copy
>>>> * passes the opened FILE* to the wrapped function
>>>> * after the execution of the function flushes the FILE* and closes it
>>>
>>>> The buffer flushes in Python and C ensure the correct order of written
>>>> data but also may slow-down the code. I have not tested the performance
>>>> impacts very thoroughly. The goal was to make FILE* work in Python3. I
>>>> also do not know what the code will be behaving like when executed in a
>>>> multi-threaded application.
>>>
>>>> In order to get rid of the sometimes unnecessary buffer flushes the
>>>> Python3 wrapper can be extended with function opening a file for ldns
>>>> only. Such an approach, however, introduces an inconvenience of
>>>> closing/reopening/closing the file when accessing the files from Python
>>>> (but giving the user control over unnecessary buffer flushing).
>>>
>>>> The patch contains new files to the SVN repository so please do not
>>>> forget to make the Python3 example scripts executable and to 'svn add'
>>>> all the new files:
>>>
>>>> chmod +x trunk/contrib/python/examples/python3/ldns-*.py
>>>> svn add trunk/contrib/python/file_py3.i
>>>> svn add trunk/contrib/python/examples/python3
>>>
>>>> I have tested the new code under the GNU/Linux userland using Python
>>>> 3.1. I intent to test it also under FreeBSD -- don't know whether it
>>>> would compile under Windows.
>>>
>>>> Best regards,
>>>> K.
>>>
>>>
>>>
>>>> _______________________________________________
>>>> ldns-users mailing list
>>>> ldns-users at open.nlnetlabs.nl
>>>> http://open.nlnetlabs.nl/mailman/listinfo/ldns-users
>>>
>> _______________________________________________
>> ldns-users mailing list
>> ldns-users at open.nlnetlabs.nl
>> http://open.nlnetlabs.nl/mailman/listinfo/ldns-users
> 
> 
> 
> 
>> _______________________________________________
>> ldns-users mailing list
>> ldns-users at open.nlnetlabs.nl
>> http://open.nlnetlabs.nl/mailman/listinfo/ldns-users
> 
_______________________________________________
ldns-users mailing list
ldns-users at open.nlnetlabs.nl
http://open.nlnetlabs.nl/mailman/listinfo/ldns-users




More information about the ldns-users mailing list