Skip to content

Pickled files with pathlib.Path cannot be loaded in py >= 3.13 #133789

Closed

Description

@kushaangupta

In python <=3.12, the resolved path of Path was pathlib.Path, however upon conversion of pathlib into a package now pickle stores the pickled objects as pathlib._local.Path. This leads to backwards incompatibility of files generated with Python >= 3.13. I suppose one can still load pickle files saved using py <= 3.12 into 3.13, but I was wondering if this is as intended.

_local.__all__)

Linked PRs

Activity

  1. added
    type-bugAn unexpected behavior, bug, or error
    stdlibStandard Library Python modules in the Lib/ directory
    3.13only security fixes
    3.14bugs and security fixes
    3.15bugs and security fixes
    on May 9, 2025
  2. picnixz commented on May 9, 2025

    @picnixz
    Member
  3. added a commit that references this issue on May 10, 2025
  4. barneygale commented on May 10, 2025

    @barneygale
    Contributor

    In retrospect, moving the class definitions to pathlib._local in 3.13 was a mistake. As you point out, it means that path objects pickled in 3.13 can't be unpickled in 3.12. @picnixz do you think we ought to guarantee that works? It's more like forward-compatibility than backward-compatibility I think...

    The PR I've logged against this issue doesn't fix the bug reported, instead it fixes a related issue where paths pickled in 3.13 can't be unpickled in 3.14. I misread your report initially, sorry 😅

  5. picnixz commented on May 10, 2025

    @picnixz
    Member

    It's more like forward-compatibility than backward-compatibility I think...

    Yes, it's about forward compatibility in this case. So maybe not a real issue (I think we should only guarantee that anything pickled in previous versions should be unpickable in later versions but not the other way around, namely if you pickle something in X.Y, you need at least version X.Y to unpickle it).

  6. 2sn commented on Oct 17, 2025

    @2sn

    This issue is still present when I try to load a pickle file in Python 3.14 that was written in Python 3.13.

    Python 3.13.7 (main, Sep 18 2025, 12:37:09) [GCC 15.2.1 20250808 (Red Hat 15.2.1-1)]
    Type 'copyright', 'credits' or 'license' for more information
    IPython 9.6.0 -- An enhanced Interactive Python. Type '?' for help.
    
    import pathlib
    x = pathlib.Path('x')
    import pickle
    f = open('test', 'wb')
    pickle.dump(x,f)

    and then

    Python 3.14.0 (main, Oct 11 2025, 09:23:46) [GCC 15.2.1 20250808 (Red Hat 15.2.1-1)]
    Type 'copyright', 'credits' or 'license' for more information
    IPython 9.6.0 -- An enhanced Interactive Python. Type '?' for help.
    
    import pickle
    f = open('test', 'rb')
    pickle.load(f)

    result

    ---------------------------------------------------------------------------
    ModuleNotFoundError                       Traceback (most recent call last)
    Cell In[3], line 1
    ----> 1 pickle.load(f)
    
    ModuleNotFoundError: No module named 'pathlib._local'

    I would like to propose a bug fix in Python 3.14.1 to automatically replace the reference. I would like to argue that this is a critical issue with backward compatibility with the standard librar

  7. 2sn commented on Oct 17, 2025

    @2sn

    Obviously, once can work around, but this is not something users should usually have to do ...

    class Unpickle314(pickle.Unpickler):
        def find_class(self, module, name):
            if module == "pathlib._local":
                module = "pathlib"
                print(f' [{self.__class__.__name__}] Loading Python < 3.14 data.')
            return super().find_class(module, name)
        try:
            item = pickle.load(f)
        except ModuleNotFoundError:
            if sys.version_info.minor >= 14:
                assert sys.version_info.major >= 3
                item = Unpickle314(f).load()
  8. sergey-miryanov commented on Oct 17, 2025

    @sergey-miryanov
    Contributor

    This issue is still present when I try to load a pickle file in Python 3.14 that was written in Python 3.13.

    Yes, it is because the proposed PR is not merged yet. Can you please try you example on the liked PR's branch?

  9. 2sn commented on Oct 17, 2025

    @2sn

    @sergey-miryanov How would I do this?

    I read some text above suggesting to fix Python 3.13 the written files, but this would not fix old files that already exist.
    Could you please clarify?
    I would only be interested in fix for Python 3.14.x, nothing else makes sense.

    What does the patch fix, and how do I check out a version with the patch. Happy to test.

  10. sergey-miryanov commented on Oct 17, 2025

    @sergey-miryanov
    Contributor

    IIUC the patch, it should fix issue when you have files from 3.13 version and want to load them in 3.14.
    IIUC, to test the linked patch you have to copy _local.py to Lib/pathlib of your local installation.

    Or you have to wait when PR will be merged.

  11. 2sn commented on Oct 17, 2025

    @2sn

    @sergey-miryanov I just don't know how to get the patch. How this here works. I clicked on all hyper content I could find on this page and don't get anywhere. In circles at best. How can I obtain a patch to try out?

  12. sergey-miryanov commented on Oct 17, 2025

    @sergey-miryanov
    Contributor

    @2sn There is a file Lib/pathlib/_local.py. Please download it and copy to your local installation, as I wrote above.

  13. 2sn commented on Oct 17, 2025

    @2sn

    @sergey-miryanov I just copied the contents into the installed ~/Python/lib/python3.14/pathlib directory as _local.py and with that path, loading works fine. Thanks.

    I confirm working of the patch.

    My apologies I could not quite follow what you wrote earlier at first. Thank you for your patience. The patch is very much appreciated!

  14. sergey-miryanov commented on Oct 17, 2025

    @sergey-miryanov
    Contributor

    Great, thanks!

  15. added 2 commits that reference this issue on Oct 17, 2025
  16. added a commit that references this issue on Oct 17, 2025
  17. added a commit that references this issue on Oct 17, 2025
  18. barneygale commented on Oct 17, 2025

    @barneygale
    Contributor

    Sorry for the delay. Fix has landed in main and will be available from 3.14.1.

  19. added a commit that references this issue on Dec 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.13only security fixes3.14bugs and security fixes3.15bugs and security fixesstdlibStandard Library Python modules in the Lib/ directorytopic-pathlibtype-bugAn unexpected behavior, bug, or error

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions