shielded task exception never retrieved when outer task cancelled

@asvetlov

BPO	42636
Nosy	@asvetlov, @1st1, @natano
Files	trigger-warning.py: Short script for showing the issue regression-test.patch: Regression test

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-12-14.10:36:21.291>
labels = ['3.7', '3.8', '3.9', '3.10', 'expert-asyncio']
title = 'shielded task exception never retrieved when outer task cancelled'
updated_at = <Date 2020-12-15.09:44:41.561>
user = 'https://github.com/natano'

bugs.python.org fields:

activity = <Date 2020-12-15.09:44:41.561>
actor = 'natano'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['asyncio']
creation = <Date 2020-12-14.10:36:21.291>
creator = 'natano'
dependencies = []
files = ['49679', '49680']
hgrepos = []
issue_num = 42636
keywords = ['patch']
message_count = 4.0
messages = ['382975', '382976', '382979', '383039']
nosy_count = 3.0
nosy_names = ['asvetlov', 'yselivanov', 'natano']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue42636'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9', 'Python 3.10']

Linked PRs

A task created with asyncio.shield() never retrieves the task exception, which results in a log message being generated. See the attached script for a minimal example. Output looks something like this:

<snip>
Task exception was never retrieved
future: <Task finished name='Task-2' coro=<some_coroutine() done, defined at /Users/natano/python-bug/trigger-warning.py:4> exception=Exception('foo')>
Traceback (most recent call last):
  File "/Users/natano/python-bug/trigger-warning.py", line 6, in some_coroutine
    raise Exception('foo')
Exception: foo
</snip>

I believe this behaviour is not intended and is an unintended side-effect of commit b35acc5 (b35acc5). The _inner_done_callback has has code to retrieve the exception when the outer task was cancelled, but since that commit the _inner_done_callback is not called anymore when the outer task was cancelled.

(Tested with 3.8, 3.9 and 3.10. I don't have a python 3.7 version available for testing, but I think this issue also affects 3.7.)

I'm not sure how to fix this, but here is a regression test for the issue. (failing on the current master branch)

Thanks for the report.

I didn't test but looks like outer.exception() call can suppress the message about not retrieved exception.

How would calling outer.exception() suppress the message? This happens when the outer task is cancelled while the inner task is still running. In that case the exception is never set on outer.

pythongh-86802: Fix asyncio memory leak; shielded tasks where cancell…

gh-86802: Fix asyncio memory leak; shielded task exceptions log once …

pythongh-86802: Fix asyncio memory leak; shielded task exceptions log…

[3.14] gh-86802: Fix asyncio memory leak; shielded task exceptions lo…

Revert "pythongh-86802: Fix asyncio memory leak; shielded task except…

Merge branch 'pythongh-86802' into pythongh-90871

Revert "pythongh-86802: Fix asyncio memory leak; shielded task except…

pythongh-86802: Fix asyncio memory leak; shielded task exceptions log…

Navigation Menu

Repository navigation

shielded task exception never retrieved when outer task cancelled #86802

Linked PRs

5 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

shielded task exception never retrieved when outer task cancelled #86802

Description

Linked PRs

Activity

natano commented on Dec 14, 2020

natano commented on Dec 14, 2020

asvetlov commented on Dec 14, 2020

natano commented on Dec 15, 2020

5 remaining items

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions