[Unbound-users] Threading in Solaris with libevent

[Mike Emigh]

Hey Wouter,

Here's the error I get (with every version of gcc I've tried).  I
previously compiled openssl0.9.8m into /opt/unbound (thus the include
flags).

./libtool --quiet --tag=CC --mode=compile /usr/sfw/bin/gcc -I.
-Ildns-src/include -I/opt/unbound/include -I/opt/install/lib
event-1.4.13-stable -I/opt/install/libevent-1.4.13-stable/include
-lnsl -lsocket -I/opt/unbound/include -g -O2 -D_REENTRANT -pthreads -o
build/util/rtt.lo -c util/rtt.c
util/netevent.c: In function `comm_point_send_udp_msg_if':
util/netevent.c:373: error: structure has no member named `msg_control'
util/netevent.c:375: error: structure has no member named `msg_controllen'
util/netevent.c:377: error: structure has no member named `msg_flags'
util/netevent.c:380: warning: assignment makes pointer from integer
without a cast
util/netevent.c:386: warning: passing arg 1 of `memmove' makes pointer
from integer without a cast
util/netevent.c:399: warning: passing arg 1 of `memmove' makes pointer
from integer without a cast
util/netevent.c:405: warning: passing arg 1 of `memset' makes pointer
from integer without a cast
util/netevent.c:408: error: structure has no member named `msg_controllen'
util/netevent.c: In function `comm_point_udp_ancil_callback':
util/netevent.c:468: error: structure has no member named `msg_control'
util/netevent.c:470: error: structure has no member named `msg_controllen'
util/netevent.c:472: error: structure has no member named `msg_flags'
util/netevent.c:485: warning: assignment makes pointer from integer
without a cast
util/netevent.c:486: warning: assignment makes pointer from integer
without a cast
util/netevent.c:491: warning: passing arg 2 of `memmove' makes pointer
from integer without a cast
util/netevent.c:498: warning: passing arg 2 of `memmove' makes pointer
from integer without a cast
gmake: *** [build/util/netevent.lo] Error 1


I was mistaken--the crashes occur when using event ports.  Libevent
seems to work fine with /dev/poll (even with lots of threads), but I
would still like it to work with evports :)  /dev/poll works when
using --without-pthreads --with-solaris-threads as well as with
--with-pthreads.  Event ports crashes on both.

Thanks,
Mike


On Wed, Mar 10, 2010 at 4:39 AM, W.C.A. Wijngaards <wouter at nlnetlabs.nl> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi Mike,
>
> Couple of data points:
> * compile with gcc on solaris works fine for me.  What is the gcc
> compile error?
> * /dev/poll support has been flakey in libevent and libev in the past.
> There is also evports on solaris, can you try to use that?
> If you compile with solaris-threads, but not with libevent (or get
> libevent to use select() instead) does that work?  I.e. is there some
> interference between unbounds use of kernel-mutexes and dev/poll ?
> * The warning: increased limit(open files): is harmless here, it signals
> that the default ulimit for you is 256 open files and obviously with
> several threads it needs more open (network-)sockets.  It notifies you
> that it is overriding the ulimit.
>
> Best regards,
>   Wouter
>
> On 03/10/2010 09:37 AM, Mike Emigh wrote:
>> Hello,
>>
>> I'm having some trouble getting Unbound 1.4.2 (and every other
>> version) to run with libevent with multiple threads on Solaris 10
>> 10/09 x86.  I am using the Sun Studio 12 compiler to building from
>> source (I have tried with many different versions of gcc but the
>> compile always errors out).
>>
>> I have tried multiple scenarios:
>> 1. Compiling with the options --without-pthreads
>> --without-solaris-threads works fine.  The server starts and works as
>> expected.
>>
>> 2. Compiling with no options works fine.  I can run with 1 thread or
>> many and the server works as expected.
>>
>> 3. Compiling with --with-libevent and using libevent-1.4.13. This
>> works fine when the configuration is set to start only 1 thread.  If
>> more than 1 thread is started, then the logs fill up with messages
>> "unbound: [ID 993594 daemon.error] [10258:1] error: could not
>> event_del on close".  When I stop the test, a few seconds later when
>> the threads stop working, unbound crashes. I could only find one
>> previous thread related to this while using select in Solaris, but
>> this does not appear to be the same.  libevent should be using
>> /dev/poll and I have kept the file descriptors low and still get the
>> messages and crash: "[1268198366] unbound[10262:0] warning: increased
>> limit(open files) from 256 to 734".
>>
>> I've seen better performance from 2.0.4-alpha, but it also crashes the
>> server and it does not pass the regression tests after compiling.  I
>> plan to post on the libevent mailing list, but looks like there is a
>> routing loop preventing me from accessing it right now.
>>
>> 4. Compiling with --with-libevent and using libev-3.9.  The server
>> starts fine and responds to queries.  If I run performance tests where
>> I hit with multiple queries (slow or fast) each thread will bring a
>> CPU to 100% and stay that way until the thread is done working.  When
>> I stop the test, a few seconds later when the threads stop working,
>> unbound crashes.  No errors messages are every produced.
>>
>> Does anyone have any ideas on how I could get this working?  I would
>> prefer to have a multithreaded Unbound over the multi-process Unbound
>> (I am able to get this working).
>>
>> Thanks,
>> Mike
>> _______________________________________________
>> Unbound-users mailing list
>> Unbound-users at unbound.net
>> http://unbound.nlnetlabs.nl/mailman/listinfo/unbound-users
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (GNU/Linux)
> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAkuXaO4ACgkQkDLqNwOhpPgrBACdGc0K+qg5e4sNkTxVWFxBjcs6
> Ty0AnRHqDHJd/ixzCNqZDGizxfMJMyij
> =fr2Z
> -----END PGP SIGNATURE-----
> _______________________________________________
> Unbound-users mailing list
> Unbound-users at unbound.net
> http://unbound.nlnetlabs.nl/mailman/listinfo/unbound-users
>