socketserver.Unix*Server may attempt to set SO_REUSEPORT on Unix sockets, causing EOPNOTSUPP, e.g. in test_asyncio

Bug report

Bug description:

When socketserver.Unix*Server.server_bind() is called with allow_reuse_port attribute set to True, it attempts to call self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1) on a Unix socket, and it fails with: OSError: [Errno 95] Operation not supported. In particular, this happens in test_asyncio tests.

This seems to be a recent change on my system (probably in Linux kernel), as the relevant tests used to pass when I tested CPython 3.14.0a3, and now the same version failed. Now I'm at 6.12.9, back then it was 6.12.5 (but I don't think there's a point in actually verifying that).

Output from `test_asyncio.test_sock_lowlevel`

$ ./python -E  -m test -v test_asyncio.test_sock_lowlevel
== CPython 3.14.0a4+ (heads/main:3193cb5ef87, Jan 16 2025, 13:54:51) [GCC 14.2.1 20241221]
== Linux-6.12.9-gentoo-dist-x86_64-AMD_Ryzen_5_3600_6-Core_Processor-with-glibc2.40 little-endian
== Python build: release
== cwd: /home/mgorny/git/cpython/build/test_python_worker_532605æ
== CPU count: 12
== encodings: locale=UTF-8 FS=utf-8
== resources: all test resources are disabled, use -u option to unskip tests

Using random seed: 3831794787
0:00:00 load avg: 0.54 Run 1 test sequentially in a single process
0:00:00 load avg: 0.54 [1/1] test_asyncio.test_sock_lowlevel
test_cancel_sock_accept (test.test_asyncio.test_sock_lowlevel.EPollEventLoopTests.test_cancel_sock_accept) ... ok
test_create_connection_sock (test.test_asyncio.test_sock_lowlevel.EPollEventLoopTests.test_create_connection_sock) ... ok
test_huge_content (test.test_asyncio.test_sock_lowlevel.EPollEventLoopTests.test_huge_content) ... ok
test_huge_content_recvinto (test.test_asyncio.test_sock_lowlevel.EPollEventLoopTests.test_huge_content_recvinto) ... ok
test_recvfrom (test.test_asyncio.test_sock_lowlevel.EPollEventLoopTests.test_recvfrom) ... ok
test_recvfrom_into (test.test_asyncio.test_sock_lowlevel.EPollEventLoopTests.test_recvfrom_into) ... ok
test_sendto_blocking (test.test_asyncio.test_sock_lowlevel.EPollEventLoopTests.test_sendto_blocking) ... ok
test_sock_accept (test.test_asyncio.test_sock_lowlevel.EPollEventLoopTests.test_sock_accept) ... ok
test_sock_client_connect_racing (test.test_asyncio.test_sock_lowlevel.EPollEventLoopTests.test_sock_client_connect_racing) ... ok
test_sock_client_fail (test.test_asyncio.test_sock_lowlevel.EPollEventLoopTests.test_sock_client_fail) ... ok
test_sock_client_ops (test.test_asyncio.test_sock_lowlevel.EPollEventLoopTests.test_sock_client_ops) ... ok
test_sock_client_racing (test.test_asyncio.test_sock_lowlevel.EPollEventLoopTests.test_sock_client_racing) ... ok
test_unix_sock_client_ops (test.test_asyncio.test_sock_lowlevel.EPollEventLoopTests.test_unix_sock_client_ops) ... ERROR
test_cancel_sock_accept (test.test_asyncio.test_sock_lowlevel.PollEventLoopTests.test_cancel_sock_accept) ... ok
test_create_connection_sock (test.test_asyncio.test_sock_lowlevel.PollEventLoopTests.test_create_connection_sock) ... ok
test_huge_content (test.test_asyncio.test_sock_lowlevel.PollEventLoopTests.test_huge_content) ... ok
test_huge_content_recvinto (test.test_asyncio.test_sock_lowlevel.PollEventLoopTests.test_huge_content_recvinto) ... ok
test_recvfrom (test.test_asyncio.test_sock_lowlevel.PollEventLoopTests.test_recvfrom) ... ok
test_recvfrom_into (test.test_asyncio.test_sock_lowlevel.PollEventLoopTests.test_recvfrom_into) ... ok
test_sendto_blocking (test.test_asyncio.test_sock_lowlevel.PollEventLoopTests.test_sendto_blocking) ... ok
test_sock_accept (test.test_asyncio.test_sock_lowlevel.PollEventLoopTests.test_sock_accept) ... ok
test_sock_client_connect_racing (test.test_asyncio.test_sock_lowlevel.PollEventLoopTests.test_sock_client_connect_racing) ... ok
test_sock_client_fail (test.test_asyncio.test_sock_lowlevel.PollEventLoopTests.test_sock_client_fail) ... ok
test_sock_client_ops (test.test_asyncio.test_sock_lowlevel.PollEventLoopTests.test_sock_client_ops) ... ok
test_sock_client_racing (test.test_asyncio.test_sock_lowlevel.PollEventLoopTests.test_sock_client_racing) ... ok
test_unix_sock_client_ops (test.test_asyncio.test_sock_lowlevel.PollEventLoopTests.test_unix_sock_client_ops) ... ERROR
test_cancel_sock_accept (test.test_asyncio.test_sock_lowlevel.SelectEventLoopTests.test_cancel_sock_accept) ... ok
test_create_connection_sock (test.test_asyncio.test_sock_lowlevel.SelectEventLoopTests.test_create_connection_sock) ... ok
test_huge_content (test.test_asyncio.test_sock_lowlevel.SelectEventLoopTests.test_huge_content) ... ok
test_huge_content_recvinto (test.test_asyncio.test_sock_lowlevel.SelectEventLoopTests.test_huge_content_recvinto) ... ok
test_recvfrom (test.test_asyncio.test_sock_lowlevel.SelectEventLoopTests.test_recvfrom) ... ok
test_recvfrom_into (test.test_asyncio.test_sock_lowlevel.SelectEventLoopTests.test_recvfrom_into) ... ok
test_sendto_blocking (test.test_asyncio.test_sock_lowlevel.SelectEventLoopTests.test_sendto_blocking) ... ok
test_sock_accept (test.test_asyncio.test_sock_lowlevel.SelectEventLoopTests.test_sock_accept) ... ok
test_sock_client_connect_racing (test.test_asyncio.test_sock_lowlevel.SelectEventLoopTests.test_sock_client_connect_racing) ... ok
test_sock_client_fail (test.test_asyncio.test_sock_lowlevel.SelectEventLoopTests.test_sock_client_fail) ... ok
test_sock_client_ops (test.test_asyncio.test_sock_lowlevel.SelectEventLoopTests.test_sock_client_ops) ... ok
test_sock_client_racing (test.test_asyncio.test_sock_lowlevel.SelectEventLoopTests.test_sock_client_racing) ... ok
test_unix_sock_client_ops (test.test_asyncio.test_sock_lowlevel.SelectEventLoopTests.test_unix_sock_client_ops) ... ERROR

======================================================================
ERROR: test_unix_sock_client_ops (test.test_asyncio.test_sock_lowlevel.EPollEventLoopTests.test_unix_sock_client_ops)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mgorny/git/cpython/Lib/test/test_asyncio/test_sock_lowlevel.py", line 462, in test_unix_sock_client_ops
    with test_utils.run_test_unix_server() as httpd:
         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/mgorny/git/cpython/Lib/contextlib.py", line 141, in __enter__
    return next(self.gen)
  File "/home/mgorny/git/cpython/Lib/test/test_asyncio/utils.py", line 261, in run_test_unix_server
    yield from _run_test_server(address=path, use_ssl=use_ssl,
                                server_cls=SilentUnixWSGIServer,
                                server_ssl_cls=UnixSSLWSGIServer)
  File "/home/mgorny/git/cpython/Lib/test/test_asyncio/utils.py", line 188, in _run_test_server
    httpd = server_class(address, SilentWSGIRequestHandler)
  File "/home/mgorny/git/cpython/Lib/socketserver.py", line 457, in __init__
    self.server_bind()
    ~~~~~~~~~~~~~~~~^^
  File "/home/mgorny/git/cpython/Lib/test/test_asyncio/utils.py", line 217, in server_bind
    UnixHTTPServer.server_bind(self)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/mgorny/git/cpython/Lib/test/test_asyncio/utils.py", line 207, in server_bind
    socketserver.UnixStreamServer.server_bind(self)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/mgorny/git/cpython/Lib/socketserver.py", line 472, in server_bind
    self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 95] Operation not supported

======================================================================
ERROR: test_unix_sock_client_ops (test.test_asyncio.test_sock_lowlevel.PollEventLoopTests.test_unix_sock_client_ops)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mgorny/git/cpython/Lib/test/test_asyncio/test_sock_lowlevel.py", line 462, in test_unix_sock_client_ops
    with test_utils.run_test_unix_server() as httpd:
         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/mgorny/git/cpython/Lib/contextlib.py", line 141, in __enter__
    return next(self.gen)
  File "/home/mgorny/git/cpython/Lib/test/test_asyncio/utils.py", line 261, in run_test_unix_server
    yield from _run_test_server(address=path, use_ssl=use_ssl,
                                server_cls=SilentUnixWSGIServer,
                                server_ssl_cls=UnixSSLWSGIServer)
  File "/home/mgorny/git/cpython/Lib/test/test_asyncio/utils.py", line 188, in _run_test_server
    httpd = server_class(address, SilentWSGIRequestHandler)
  File "/home/mgorny/git/cpython/Lib/socketserver.py", line 457, in __init__
    self.server_bind()
    ~~~~~~~~~~~~~~~~^^
  File "/home/mgorny/git/cpython/Lib/test/test_asyncio/utils.py", line 217, in server_bind
    UnixHTTPServer.server_bind(self)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/mgorny/git/cpython/Lib/test/test_asyncio/utils.py", line 207, in server_bind
    socketserver.UnixStreamServer.server_bind(self)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/mgorny/git/cpython/Lib/socketserver.py", line 472, in server_bind
    self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 95] Operation not supported

======================================================================
ERROR: test_unix_sock_client_ops (test.test_asyncio.test_sock_lowlevel.SelectEventLoopTests.test_unix_sock_client_ops)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mgorny/git/cpython/Lib/test/test_asyncio/test_sock_lowlevel.py", line 462, in test_unix_sock_client_ops
    with test_utils.run_test_unix_server() as httpd:
         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/mgorny/git/cpython/Lib/contextlib.py", line 141, in __enter__
    return next(self.gen)
  File "/home/mgorny/git/cpython/Lib/test/test_asyncio/utils.py", line 261, in run_test_unix_server
    yield from _run_test_server(address=path, use_ssl=use_ssl,
                                server_cls=SilentUnixWSGIServer,
                                server_ssl_cls=UnixSSLWSGIServer)
  File "/home/mgorny/git/cpython/Lib/test/test_asyncio/utils.py", line 188, in _run_test_server
    httpd = server_class(address, SilentWSGIRequestHandler)
  File "/home/mgorny/git/cpython/Lib/socketserver.py", line 457, in __init__
    self.server_bind()
    ~~~~~~~~~~~~~~~~^^
  File "/home/mgorny/git/cpython/Lib/test/test_asyncio/utils.py", line 217, in server_bind
    UnixHTTPServer.server_bind(self)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/mgorny/git/cpython/Lib/test/test_asyncio/utils.py", line 207, in server_bind
    socketserver.UnixStreamServer.server_bind(self)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/mgorny/git/cpython/Lib/socketserver.py", line 472, in server_bind
    self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 95] Operation not supported

----------------------------------------------------------------------
Ran 39 tests in 1.296s

FAILED (errors=3)
test test_asyncio.test_sock_lowlevel failed
test_asyncio.test_sock_lowlevel failed (3 errors)

== Tests result: FAILURE ==

1 test failed:
    test_asyncio.test_sock_lowlevel

Total duration: 1.4 sec
Total tests: run=39
Total test files: run=1/1 failed=1
Result: FAILURE

Bisect points to 192d17c:

commit 192d17c3fd9945104bc0303cf248bb0d074d260e
Author:     Idan Kapustian <71190257+idankap@users.noreply.github.com>
AuthorDate: 2024-06-16 14:15:03 +0200
Commit:     GitHub <noreply@github.com>
CommitDate: 2024-06-16 14:15:03 +0200

    gh-120485: Add an override of `allow_reuse_port` on classes subclassing `socketserver.TCPServer` (GH-120488)
    
    Co-authored-by: Vinay Sajip <vinay_sajip@yahoo.co.uk>

CPython versions tested on:

3.14, CPython main branch

Operating systems tested on:

Linux

Linked PRs

Indeed, here's the kernel commit: torvalds/linux@5b0af62

@idankap

CC @idankap @vsajip

Not sure if we should just ignore allow_reuse_port on Unix sockets, or assume it's invalid to set it to True, and modify the tests not to do that. Also, unclear on allow_reuse_address — it doesn't fail (right now), but no clue if it actually does anything.

Yes, I suppose neither allow_reuse_XXX really makes sense for domain sockets. I guess if the change is done in socketserver.Unix*Server to ignore those settings when a domain socket is in use, nothing else needs to change?

Yes, the following seems to fix the issue:

diff --git a/Lib/socketserver.py b/Lib/socketserver.py
index cd028ef1c63..d1d292c623f 100644
--- a/Lib/socketserver.py
+++ b/Lib/socketserver.py
@@ -722,9 +722,15 @@ class ThreadingTCPServer(ThreadingMixIn, TCPServer): pass
     class UnixStreamServer(TCPServer):
         address_family = socket.AF_UNIX
 
+        allow_reuse_address = False
+        allow_reuse_port = False
+
     class UnixDatagramServer(UDPServer):
         address_family = socket.AF_UNIX
 
+        allow_reuse_address = False
+        allow_reuse_port = False
+
     class ThreadingUnixStreamServer(ThreadingMixIn, UnixStreamServer): pass
 
     class ThreadingUnixDatagramServer(ThreadingMixIn, UnixDatagramServer): pass

Should I make it into a pull request, or do you want to handle it from here? I suppose a comment noting that this overrides the value from http.server.HTTPServer may help too, given that the directly inherited TCPServer and UDPServer both default to False.

I'm confused. I just looked at the main branch of the repository:

$ rg allow_reuse_port Lib | more
Lib/socketserver.py:    - allow_reuse_port
Lib/socketserver.py:    - allow_reuse_port
Lib/socketserver.py:    allow_reuse_port = False
Lib/socketserver.py:        if self.allow_reuse_port and hasattr(socket, "SO_REUSEPORT"):
Lib/socketserver.py:    allow_reuse_port = False
Lib/test/test_logging.py:    allow_reuse_port = True
Lib/xmlrpc/server.py:    allow_reuse_port = True
Lib/logging/config.py:        allow_reuse_port = True
Lib/http/server.py:    allow_reuse_port = True

Since TCPServer and UDPServer already set the allow_reuse_port class attribute to False, which value should be inherited by UnixStreamServer and UnixDatagramServer respectively, there should be no need to override those values as in your patch, right? I can't immediately see where the code is that sets that attribute to True which would explain the failure of test_asyncio.test_sock_lowlevel.

That's because UnixHTTPServer inherits HTTPServer, and the True comes from there:

cpython/Lib/test/test_asyncio/utils.py

Lines 204 to 209 in ea6cc26

    
           class UnixHTTPServer(socketserver.UnixStreamServer, HTTPServer): 
        
               def server_bind(self): 
        
                   socketserver.UnixStreamServer.server_bind(self) 
        
                   self.server_name = '127.0.0.1' 
        
                   self.server_port = 80

I suppose that alternatively we could override these attributes there directly (and on the other servers in that file). Do you want me to test and prepare a PR doing that instead?

Another way of looking at it is this: the problem is caused by calling setsockopt() with SO_REUSEPORT on the wrong kind of socket. So perhaps the better fix is to check the socket type using getsockopt() and not make the setsockopt() call if it's a domain socket?

I suppose that makes sense too. After all, the point of "allowing reuse" is well, not to block, and with unix sockets this is never a problem.

FWICS SO_REUSEADDR is valid for at least a few different address families, so I'll focus on SO_REUSEPORT only. I'll have a PR shortly.

Filed #128933

Excellent, thank you very much.

gh-128916: Do not set `SO_REUSEPORT` on non-`AF_INET*` sockets (GH-12…

pythongh-128916: Do not set `SO_REUSEPORT` on non-`AF_INET*` sockets (p…

thanks! merged and set to merge.

[3.12] gh-128916: Do not set `SO_REUSEPORT` on non-`AF_INET*` sockets (…

[3.13] gh-128916: Do not set `SO_REUSEPORT` on non-`AF_INET*` sockets (…

pythongh-128916: Do not set `SO_REUSEPORT` on non-`AF_INET*` sockets (p…

00448: pythongh-128916: Do not set `SO_REUSEPORT` on non-`AF_INET*` s…

Use SO_REUSEPORT only for AF_INET sockets

Use SO_REUSEPORT only for AF_INET sockets (#303)

Use SO_REUSEPORT only for AF_INET sockets (Thriftpy#303)

opt: avoid setting SO_REUSEPORT on AF_UNIX sockets

opt: avoid setting SO_REUSEPORT on AF_UNIX sockets (#695)

Navigation Menu

Repository navigation

`socketserver.Unix*Server` may attempt to set `SO_REUSEPORT` on Unix sockets, causing `EOPNOTSUPP`, e.g. in `test_asyncio` #128916

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	class UnixHTTPServer(socketserver.UnixStreamServer, HTTPServer):

	def server_bind(self):
	socketserver.UnixStreamServer.server_bind(self)
	self.server_name = '127.0.0.1'
	self.server_port = 80

socketserver.Unix*Server may attempt to set SO_REUSEPORT on Unix sockets, causing EOPNOTSUPP, e.g. in test_asyncio #128916

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

mgorny commented on Jan 16, 2025

mgorny commented on Jan 16, 2025

vsajip commented on Jan 16, 2025

mgorny commented on Jan 16, 2025

vsajip commented on Jan 16, 2025

mgorny commented on Jan 17, 2025

vsajip commented on Jan 17, 2025

mgorny commented on Jan 17, 2025

mgorny commented on Jan 17, 2025

mgorny commented on Jan 17, 2025

vsajip commented on Jan 17, 2025

gpshead commented on Jan 18, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

`socketserver.Unix*Server` may attempt to set `SO_REUSEPORT` on Unix sockets, causing `EOPNOTSUPP`, e.g. in `test_asyncio` #128916