Skip to content

loop.create_server does not detect if the interface is IPv6 enabled #75128

Closed
Closed
loop.create_server does not detect if the interface is IPv6 enabled#75128
Labels
topic-asynciotype-bugAn unexpected behavior, bug, or error

Description

@cecton
cecton
mannequin
opened on Jul 17, 2017
BPO 30945
Nosy @pitrou, @1st1, @asottile, @jnwatson, @cecton
PRs
  • gh-75128: Fix create_server to handle the case when iface isn't IPv6 enabled. #7207
    		•
    Files
  • test_ipv6.py: Example file to reproduce the issue
  • eaddrnotavail_asyncio.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2017-07-17.10:04:34.577>
labels = ['type-bug', '3.7', 'expert-asyncio']
title = 'loop.create_server does not detect if the interface is IPv6 enabled'
updated_at = <Date 2018-10-19.21:14:34.454>
user = 'https://github.com/cecton'

    bugs.python.org fields:

    activity = <Date 2018-10-19.21:14:34.454>
actor = 'jnwatson'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['asyncio']
creation = <Date 2017-07-17.10:04:34.577>
creator = 'cecton'
dependencies = []
files = ['47019', '47026']
hgrepos = []
issue_num = 30945
keywords = ['patch']
message_count = 12.0
messages = ['298474', '298683', '298684', '298714', '298716', '298718', '311915', '311916', '311917', '311942', '317895', '318045']
nosy_count = 5.0
nosy_names = ['pitrou', 'yselivanov', 'Anthony Sottile', 'jnwatson', 'cecton']
pr_nums = ['7207']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue30945'
versions = ['Python 3.5', 'Python 3.6', 'Python 3.7']

    Linked PRs

    Activity

    1. cecton commented on Jul 17, 2017

      @cecton
      cectonmannequin
      on Jul 17, 2017
      MannequinAuthor

      The IPv6 detection in asyncio.base_events.create_server only detect if IPv6 is available instead of checking if the interface can actually support it.

      I noticed that by using Python in a Docker container (example code to reproduce in attachment):

      docker run -it --rm -v /tmp/test_ipv6.py:/src/test_ipv6.py python:3.6 python /src/test_ipv6.py

      Will result in:

      Traceback (most recent call last):
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 1043, in create_server
    sock.bind(sa)
OSError: [Errno 99] Cannot assign requested address

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/src/test_ipv6.py", line 11, in <module>
    server = loop.run_until_complete(server_creation)
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 466, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 1047, in create_server
    % (sa, err.strerror.lower()))
OSError: [Errno 99] error while attempting to bind on address ('::1', 27015, 0, 0): cannot assign requested address

      By default Docker containers have only IPv4 enabled:

      1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
      inet 127.0.0.1/8 scope host lo
      valid_lft forever preferred_lft forever
      38: eth0@if39: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
      link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff
      inet 172.17.0.2/16 scope global eth0
      valid_lft forever preferred_lft forever

      I believe this detection mechanism should rely on the interface requested. I found this on the web for Python 2 that manage to get the info per interface: https://pastebin.com/VEnhF1Ht but it's using an external library.

      However if you change the hostname to 127.0.0.1 it works normally.

    2. cecton
      added
      topic-asyncio
      type-bugAn unexpected behavior, bug, or error
      on Jul 17, 2017

    3. pitrou commented on Jul 19, 2017

      @pitrou
      pitrou
      on Jul 19, 2017
      Member

      Better than trying to detect IPv6 compatibility beforehand would probably to recognize the error and simply ignore it.

      Note: errno 99 is EADDRNOTAVAIL.

      Something like this could work (untested):

      diff --git a/Lib/asyncio/base_events.py b/Lib/asyncio/base_events.py
index 33b8f48..413161a 100644
--- a/Lib/asyncio/base_events.py
+++ b/Lib/asyncio/base_events.py
@@ -1038,6 +1038,11 @@ class BaseEventLoop(events.AbstractEventLoop):
                     try:
                         sock.bind(sa)
                     except OSError as err:
+                        if err.errno == errno.EADDRNOTAVAIL:
+                            # See bpo-30945
+                            sockets.pop()
+                            sock.close()
+                            continue
                         raise OSError(err.errno, 'error while attempting '
                                       'to bind on address %r: %s'
                                       % (sa, err.strerror.lower())) from None
    4. pitrou
      added
      3.7 (EOL)end of life
      on Jul 19, 2017

    5. 1st1 commented on Jul 19, 2017

      @1st1
      1st1
      on Jul 19, 2017
      Member

      Better than trying to detect IPv6 compatibility beforehand would probably to recognize the error and simply ignore it.

      +1

    6. pitrou commented on Jul 20, 2017

      @pitrou
      pitrou
      on Jul 20, 2017
      Member

      Cécile, could you try the following patch? I have no easy way to test here.

    7. cecton commented on Jul 20, 2017

      @cecton
      cectonmannequin
      on Jul 20, 2017
      MannequinAuthor

      Sure! It seems to work, the process returns an exit code of 0 and I see no traceback but the message is still displayed in the terminal.

      (Also I did something weird because your patch applies on branch master and I ran it with Python 3.6... I suppose it shouldn't be a problem)

      [0] [11:54:13] /tmp > d run -it --rm -v /tmp:/tmp:ro -v ~/repos/cpython/Lib/asyncio:/usr/local/lib/python3.6/asyncio:ro python:3.6 python /tmp/test_ipv6.py
error while attempting to bind on address ('::1', 27015, 0, 0): cannot assign requested address
[0] [11:54:19] /tmp > d run -it --rm -v /tmp:/tmp:ro python:3.6 python /tmp/test_ipv6.py
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 1043, in create_server
    sock.bind(sa)
OSError: [Errno 99] Cannot assign requested address

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/test_ipv6.py", line 11, in <module>
    server = loop.run_until_complete(server_creation)
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 466, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.6/asyncio/base_events.py", line 1047, in create_server
    % (sa, err.strerror.lower()))
OSError: [Errno 99] error while attempting to bind on address ('::1', 27015, 0, 0): cannot assign requested address
[1] [11:54:52] /tmp >

    8. pitrou commented on Jul 20, 2017

      @pitrou
      pitrou
      on Jul 20, 2017
      Member

      Cécile, thank you. The reason the message is still displayed is that I turned the error into a warning (in other words, it is expected).

    9. asottile commented on Feb 9, 2018

      @asottile
      asottilemannequin
      on Feb 9, 2018
      Mannequin

      Seeing this as well when running the cpython test suite in docker:

      $ ./python -m test.test_asyncio

...

[18 similar traces omitted]
======================================================================
ERROR: test_sock_sendfile_zero_size (test.test_asyncio.test_unix_events.SelectorEventLoopUnixSockSendfileTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/code/Lib/test/test_asyncio/test_unix_events.py", line 559, in test_sock_sendfile_zero_size
    sock, proto = self.prepare()
  File "/code/Lib/test/test_asyncio/test_unix_events.py", line 483, in prepare
    lambda: proto, support.HOST, port))
  File "/code/Lib/test/test_asyncio/test_unix_events.py", line 476, in run_loop
    return self.loop.run_until_complete(coro)
  File "/code/Lib/asyncio/base_events.py", line 566, in run_until_complete
    return future.result()
  File "/code/Lib/asyncio/base_events.py", line 1346, in create_server
    % (sa, err.strerror.lower())) from None
OSError: [Errno 99] error while attempting to bind on address ('::1', 39527, 0, 0): cannot assign requested address

----------------------------------------------------------------------

      I'm going to try and write a patch to skip these tests (there's already a helper)

    10. asottile commented on Feb 9, 2018

      @asottile
      asottilemannequin
      on Feb 9, 2018
      Mannequin

      Actually, my issue seems to be something more strange.

      The host being passed in is localhost which resolves to:

      >>> pprint.pprint(socket.getaddrinfo('localhost', 80))
[(<AddressFamily.AF_INET: 2>,
  <SocketKind.SOCK_STREAM: 1>,
  6,
  '',
  ('127.0.0.1', 80)),
 (<AddressFamily.AF_INET: 2>,
  <SocketKind.SOCK_DGRAM: 2>,
  17,
  '',
  ('127.0.0.1', 80)),
 (<AddressFamily.AF_INET: 2>,
  <SocketKind.SOCK_RAW: 3>,
  0,
  '',
  ('127.0.0.1', 80)),
 (<AddressFamily.AF_INET6: 10>,
  <SocketKind.SOCK_STREAM: 1>,
  6,
  '',
  ('::1', 80, 0, 0)),
 (<AddressFamily.AF_INET6: 10>,
  <SocketKind.SOCK_DGRAM: 2>,
  17,
  '',
  ('::1', 80, 0, 0)),
 (<AddressFamily.AF_INET6: 10>,
  <SocketKind.SOCK_RAW: 3>,
  0,
  '',
  ('::1', 80, 0, 0))]

      asyncio is picking ipv6 because of this code:

      cpython/Lib/asyncio/base_events.py

      Lines 1334 to 1340 in a445feb

      # Disable IPv4/IPv6 dual stack support (enabled by
      # default on Linux) which makes a single socket
      # listen on both address families.
      if af == AF_INET6 and hasattr(socket, 'IPPROTO_IPV6'):
      sock.setsockopt(socket.IPPROTO_IPV6,
      socket.IPV6_V6ONLY,
      True)

      despite my host not actually having an ipv6 network hooked up.

    11. asottile commented on Feb 9, 2018

      @asottile
      asottilemannequin
      on Feb 9, 2018
      Mannequin

      Applying this patch makes the tests pass for me, but I don't think the patch is appropriate (just hides the bug):

      $ git diff
diff --git a/Lib/test/test_asyncio/test_unix_events.py b/Lib/test/test_asyncio/test_unix_events.py
index 5bd76d3..ff6c4e1 100644
--- a/Lib/test/test_asyncio/test_unix_events.py
+++ b/Lib/test/test_asyncio/test_unix_events.py
@@ -480,7 +480,7 @@ class SelectorEventLoopUnixSockSendfileTests(test_utils.TestCase):
         proto = self.MyProto(self.loop)
         port = support.find_unused_port()
         server = self.run_loop(self.loop.create_server(
-            lambda: proto, support.HOST, port))
+            lambda: proto, support.HOSTv4, port))
         self.run_loop(self.loop.sock_connect(sock, (support.HOST, port)))
 
         def cleanup():

    12. cecton commented on Feb 10, 2018

      @cecton
      cectonmannequin
      on Feb 10, 2018
      MannequinAuthor

      I see that the patch hasn't been applied to master on GitHub. Is there anything else expected from me on this ticket?

    13. 1st1 commented on May 28, 2018

      @1st1
      1st1
      on May 28, 2018
      Member

      Does anybody wants to make a PR to fix this?

    14. 1st1 commented on May 29, 2018

      @1st1
      1st1
      on May 29, 2018
      Member

      asyncio is picking ipv6 because of this code:

      cpython/Lib/asyncio/base_events.py

      Lines 1334 to 1340 in a445feb

      # Disable IPv4/IPv6 dual stack support (enabled by
      # default on Linux) which makes a single socket
      # listen on both address families.
      if af == AF_INET6 and hasattr(socket, 'IPPROTO_IPV6'):
      sock.setsockopt(socket.IPPROTO_IPV6,
      socket.IPV6_V6ONLY,
      True)

      despite my host not actually having an ipv6 network hooked up.

      Which should be fine; create_server enumerates all addresses and tries to connect to each one. IPV6_V6ONLY is only applied to one socket for one IPv6 address. I think Antoine's patch is OK.

    15. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    16. ezio-melotti
      added this to asyncioon Jul 17, 2022
    17. ezio-melotti
      moved this to Todo in asyncioon Jul 17, 2022
    18. kumaraditya303
      removed
      3.7 (EOL)end of life
      on Oct 18, 2022
    19. serhiy-storchaka
      added 2 commits that reference this issue on Jan 22, 2024

      pythongh-75128: Ignore EADDRNOTAVAIL error in asyncio.BaseEventLoop.c…

      0c836ae

      gh-75128: Ignore EADDRNOTAVAIL error in asyncio.BaseEventLoop.create_…

      a53e56e
    20. miss-islington
      added 2 commits that reference this issue on Jan 22, 2024

      pythongh-75128: Ignore EADDRNOTAVAIL error in asyncio.BaseEventLoop.c…

      0d12dee

      pythongh-75128: Ignore EADDRNOTAVAIL error in asyncio.BaseEventLoop.c…

      6833cf4

    21. serhiy-storchaka commented on Jan 22, 2024

      @serhiy-storchaka
      serhiy-storchaka
      on Jan 22, 2024
      Member

      Thank you for your patch Antoine. I am surprised that it was not merged at its time. It fixes a constant failure on some buildbots. Buildbot failure could perhaps be fixed by changing the configuration (I think that I encountered such errors on different buildbots in the past), but it is better to fix it in the core.

    22. serhiy-storchaka
      closed this as completedon Jan 22, 2024
    23. github-project-automation
      moved this from Todo to Done in asyncioon Jan 22, 2024
    24. serhiy-storchaka
      added 2 commits that reference this issue on Jan 22, 2024

      [3.12] gh-75128: Ignore EADDRNOTAVAIL error in asyncio.BaseEventLoop.…

      4bbb9f6

      [3.11] gh-75128: Ignore EADDRNOTAVAIL error in asyncio.BaseEventLoop.…

      acea9d8
    25. aisk
      added a commit that references this issue on Feb 11, 2024

      pythongh-75128: Ignore EADDRNOTAVAIL error in asyncio.BaseEventLoop.c…

      5c3f5de
    26. Glyphack
      added a commit that references this issue on Sep 2, 2024

      pythongh-75128: Ignore EADDRNOTAVAIL error in asyncio.BaseEventLoop.c…

      e00d614
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      topic-asynciotype-bugAn unexpected behavior, bug, or error

      Projects

      • asyncio
        Status
        Done

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions