[Unbound-users] [PATCH] performance increase with SO_REUSEPORT on Linux 3.9+

[W.C.A. Wijngaards]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Robert,

On 01/07/2014 04:05 AM, Robert Edmonds wrote:
> Hi,
> 
> There is a SO_REUSEPORT socket option available on Linux 3.9+,
> described in this LWN article:

Thank you very much for this patch.  I have applied it.
(small changed in header comments, and documentation entry).

The only thing I wonder if it wouldn't be better to make this the
default setting.  It could be disabled if you then mention it in the
config file?  This would still only apply to Linux systems (unless we
had some way for a similar effect on other systems, and then we have
to rename this linux-reuseport?)

Best regards,
   Wouter

> 
> https://lwn.net/Articles/542629/
> 
> It is of particular interest to DNS servers, since it allows more
> evenly distributing incoming queries among multiple UDP server
> sockets bound to the same port:
> 
> As with TCP, SO_REUSEPORT allows multiple UDP sockets to be bound
> to the same port. This facility could, for example, be useful in a
> DNS server operating over UDP. With SO_REUSEPORT, each thread could
> use recv() on its own socket to accept datagrams arriving on the
> port. The traditional approach is that all threads would compete
> to perform recv() calls on a single shared socket. As with the
> second of the traditional TCP scenarios described above, this can
> lead to unbalanced loads across the threads. By contrast,
> SO_REUSEPORT distributes datagrams evenly across all of the
> receiving threads.
> 
> Tom noted that the traditional SO_REUSEADDR socket option already 
> allows multiple UDP sockets to be bound to, and accept datagrams
> on, the same UDP port. However, by contrast with SO_REUSEPORT, 
> SO_REUSEADDR does not prevent port hijacking and does not
> distribute datagrams evenly across the receiving threads.
> 
> There is also a slide deck available here:
> 
> http://domsch.com/linux/lpc2010/Scaling_techniques_for_servers_with_high_connection%20rates.pdf
>
>  I've written a patch for Unbound that enables per-thread server
> sockets with SO_REUSEPORT on Linux.  It must be explicitly
> requested with "so-reuseport: yes" because Linux < 3.9 doesn't
> support the socket option (it will fail with ENOPROTOOPT) and
> because it has different semantics on Linux than on other platforms
> with a similarly named socket option it has no effect on non-Linux
> platforms.
> 
> Under my simplistic 100% cache hit microbenchmark, Unbound with 
> SO_REUSEPORT support running on Linux 3.12 achieves somewhat lower
> CPU utilization at all query rates except the very highest (I
> suspect this is because the patched Unbound is delivering a
> slightly higher response rate).  E.g., at 810Kq/s from the traffic
> generator, unbound trunk (r3036) consumes 75% of the system's 4
> CPUs, while unbound with SO_REUSEPORT consumes 66%.  I've attached
> a plot showing the benchmark data graphically.
> 
> 
> 
> _______________________________________________ Unbound-users
> mailing list Unbound-users at unbound.net 
> http://unbound.nlnetlabs.nl/mailman/listinfo/unbound-users
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJS4lFLAAoJEJ9vHC1+BF+N0igP/2wia/EHJYQGTCo5rOOtSheL
Pc67573qbQKzuzQHQoU+eH7NFCpphdrEfYCDGal0ODZhjARqGj4o7Z+nTfvw8Ane
TeRoDaiHytXLPFGbTRnznuqYbAMNxR03mz8oiAxUWoCbIJm5BqjJxfaDARwEtsXv
DcPOrv0YGXqDWZpU0xys1whguYK6/+029JX+PXAqmka84o0TqSsdlKhhZdvDa9r9
oDoRNwv9b+Kv3RGEmcut+jRDLYbCR2wgg1pz3bdu2dNq0OC7h2do01LK0xBASVnU
rkTRjsBshXCEHpOJRvfHbqxEBuyT6M7zy6KYYi6itE1YHzsQwyOT6Ot61qLGs7lz
vSz3mUMCBkUm0d0Mq1pWXX0KE2koS6O/vAqljSjeWod1YB8Le4tmY+70Vb90MujF
VqRMY+QuVepPEE3g8pSKF4D3hHqzWd3xkE0niOms3+jMYd2vtOSU8M8OpaL/LIGR
tXbnU6UOT84IWTlvDUZnmOqKGidxWmsYmC1CX/d9cnOySEvFmah2APdtehOOqMuz
pGtBPfDbaZo1yAU55piHzggSXUBw1sC56oaMBtWj3I0NmbLA3PzZkssuNhInvGly
yHd7QECcEnNYiZreNympaIouaOa6LZabrfINLCI3PgUytu8relxfp5VO700UDqzh
q1JVH5t9Qvb8iaf/EKPm
=MEvu
-----END PGP SIGNATURE-----