pathlib.Path objects can be used as context managers

@brettcannon

BPO	39682
Nosy	@brettcannon, @pitrou, @barneygale
PRs	bpo-39682: make `pathlib.Path` immutable by removing (undocumented) support for "closing" a path by using it as a context manager #18846 bpo-46556: emit `DeprecationWarning` from `pathlib.Path.__enter__()` #30971

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-04-01.14:11:09.381>
created_at = <Date 2020-02-19.02:10:35.842>
labels = ['type-bug', 'library', '3.9']
title = 'pathlib.Path objects can be used as context managers'
updated_at = <Date 2022-01-27.23:22:28.820>
user = 'https://github.com/barneygale'

bugs.python.org fields:

activity = <Date 2022-01-27.23:22:28.820>
actor = 'barneygale'
assignee = 'none'
closed = True
closed_date = <Date 2020-04-01.14:11:09.381>
closer = 'pitrou'
components = ['Library (Lib)']
creation = <Date 2020-02-19.02:10:35.842>
creator = 'barneygale'
dependencies = []
files = []
hgrepos = []
issue_num = 39682
keywords = ['patch']
message_count = 14.0
messages = ['362244', '362840', '362934', '363206', '363209', '363218', '363260', '363261', '363285', '363298', '363383', '363468', '363469', '365473']
nosy_count = 4.0
nosy_names = ['brett.cannon', 'pitrou', 'Isaac Muse', 'barneygale']
pr_nums = ['18846', '30971']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue39682'
versions = ['Python 3.9']

Linked PRs

GH-83863: Drop support for using pathlib.Path objects as context managers #104807

pathlib.Path objects can be used as context managers, but this functionality is undocumented and makes little sense. Example:

>>> import pathlib
>>> root = pathlib.Path("/")
>>> with root:
...     print(1)
... 
1
>>> with root:
...     print(2)
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/barney/.pyenv/versions/3.7.3/lib/python3.7/pathlib.py", line 1028, in __enter__
    self._raise_closed()
  File "/home/barney/.pyenv/versions/3.7.3/lib/python3.7/pathlib.py", line 1035, in _raise_closed
    raise ValueError("I/O operation on closed path")
ValueError: I/O operation on closed path

Path objects don't acquire any resources on new/init/enter, nor do they release any resources on exit. The whole notion of the path being _closed seems to exist purely to make impure Path methods unusable after exiting from the context manager. I can't personally think of a compelling use case for this, and suggest that it be removed.

A use-case of "closing" a path is to cheaply flag that a path object is no longer valid, e.g. you deleted the underlying file and so you don't want others using the object anymore without paying the penalty of having to do the I/O to trigger the failure due to the path no longer existing.

Now whether anyone is using this functionality I don't know, but removing it will inevitably break code for someone, so someone will need to make at least some form of an attempt to see if there's any use in the wild to justify simplifying the code by removing the functionality.

A problem of having this bit of state in paths is that it violates assumptions based on immutability/hashability, e.g. with

    from pathlib import Path

    p = Path("foo")
    q = Path("foo")

    with p: pass

    unique_paths = {p, q}

    print(len(unique_paths))  # == 1, as p == q
    for path in unique_paths:
        print(path.resolve())  # Fails if the path is closed.

The last line fails with unique_paths = {p, q} as p has been closed (and apparently sets keep the first element when multiple "equal" elements are passed in), but not with unique_paths = {q, p}.

It would also prevent optimizations based on immutability as proposed in https://bugs.python.org/issue39783.

I guess a question is whether we want immutability guarantees (I for one didn't even know you could hash Path objects until now).

I'm personally fine with that idea as it mentally makes sense to not need paths to be mutable. But as I said, someone needs to take at least some stab at seeing if any preexisting code out there will break if the _closed flag is removed before we can continue this discussion.

Immutability and hashability are listed first among "general properties" of paths (https://docs.python.org/3/library/pathlib.html#general-properties), and in the PEP proposing pathlib (https://www.python.org/dev/peps/pep-0428/#immutability). Looking at it again I *guess* the docs version could be read as only guaranteeing this for PurePaths (because that's in the PurePath section) rather than Path, but the PEP version clearly implies that this is true for both PurePaths and concrete Paths.

It may be tricky to check usage in third-party packages, given that one would need to look for with <path-object>: ... rather than, say, a method name which can be grepped. Do you have any suggestions here?

As with the Accessor abstraction, the original idea was to support Path objects backed by a directory file descriptor (for use with openat() and friends). That idea was abandoned but it looks like the context manager stayed. It's certainly not useful currently.

Can I ask what sort of backwards-compatibility guarantees Python provides for these cases? In the case of using a Path object as a context manager, I think we can say:

It's easy to do - there's no need to call any underscore-prefixed methods for example
It's undocumented
It's pretty hard to determine existing usage statically - grepping for `with p:` is years of work.

Also, thank you Antoine for your explanation :-)

Can I ask what sort of backwards-compatibility guarantees Python provides for these cases?

A deprecation warning for two releases (i.e. two years). Then it can be removed. So if people want to move forward with removing this then a DeprecationWarning would need to be raised in all places where _closed is set with a message stating the functionality is deprecated in 3.9 and slated for removal in 3.11 (along with appropriate doc updates, both for the module and What's New).

Note you could simply remove the "closed" flag and the context manager live. That way, you don't have to deprecate anything.

@antoine

@antoine I just quite follow what you mean. Are you saying ditch _closed and just leave the context manager to be a no-op?

remove pathlib.Path use as a contextmanager

pythonGH-83863: Drop support for using `pathlib.Path` objects as cont…

GH-83863: Drop support for using `pathlib.Path` objects as context ma…

Oh well. I was actually using this functionality.
It was super convenient to do something like:

with Path('debian') / 'fill.copyright.blanks.yml' as debian_copyright_overrides:
    if debian_copyright_overrides.is_file():
        ...
    else:
        ...

Basically it was a very visual way to structure code that worked with different subpaths. So no matter that it didn’t "close" path, it allowed to temporarily assign a shortcut to that path, do some work, and then forget about it.

Perhaps contextlib.nullcontext() would help?

But context managers are all about acquiring and releasing resources. It could be considered misleading to use a context manager only for code formating purposes. I would write:

debian_copyright_overrides = Path('debian') / 'fill.copyright.blanks.yml'
if debian_copyright_overrides.is_file():
    ...
else:
    ...

repo-setup: Drop context manager for posix path

Import Debian version 0.9.38

Navigation Menu

Repository navigation

pathlib.Path objects can be used as context managers #83863

Linked PRs

6 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

pathlib.Path objects can be used as context managers #83863

Description

Linked PRs

Activity

barneygale commented on Feb 19, 2020

brettcannon commented on Feb 27, 2020

anntzer commented on Feb 28, 2020

brettcannon commented on Mar 2, 2020

anntzer commented on Mar 2, 2020

pitrou commented on Mar 2, 2020

barneygale commented on Mar 3, 2020

barneygale commented on Mar 3, 2020

brettcannon commented on Mar 3, 2020

pitrou commented on Mar 3, 2020

brettcannon commented on Mar 4, 2020

6 remaining items

andrewshadura commented on Oct 12, 2023

barneygale commented on Oct 13, 2023

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions