Skip to content

runpy.run_path mishandles Path objects #99437

Closed
#107644
Closed
runpy.run_path mishandles Path objects#99437
#107644
Labels
stdlibStandard Library Python modules in the Lib/ directorytopic-importlibtype-bugAn unexpected behavior, bug, or error

Description

@nedbat
nedbat
opened on Nov 13, 2022

Bug report

runpy.run_path will accept a Path object, and run it, but various code inspection tools will be unhappy with it because it is not a string. Should run_path convert its argument to a true string before execution?

Coverage.py is one of those tools; this report is based on coveragepy/coveragepy#1417.

Create debugme.py:

from pathlib import Path
import runpy, sys

runme = Path(__file__).resolve().parent / sys.argv[1]
print(f"Running {runme!r}")
runpy.run_path(runme)

Create inspectme.py:

import inspect

print(inspect.stack())
print('here')

Run it:

% python3.12 debugme.py inspectme.py
Running PosixPath('/System/Volumes/Data/root/src/bugs/bug1417/inspectme.py')
Traceback (most recent call last):
  File "/System/Volumes/Data/root/src/bugs/bug1417/debugme.py", line 6, in <module>
    runpy.run_path(runme)
  File "<frozen runpy>", line 291, in run_path
  File "<frozen runpy>", line 98, in _run_module_code
  File "<frozen runpy>", line 88, in _run_code
  File "/System/Volumes/Data/root/src/bugs/bug1417/inspectme.py", line 3, in <module>
    print(inspect.stack())
          ^^^^^^^^^^^^^^^
  File "/usr/local/pyenv/pyenv/versions/3.12.0a1/lib/python3.12/inspect.py", line 1748, in stack
    return getouterframes(sys._getframe(1), context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/pyenv/pyenv/versions/3.12.0a1/lib/python3.12/inspect.py", line 1723, in getouterframes
    traceback_info = getframeinfo(frame, context)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/pyenv/pyenv/versions/3.12.0a1/lib/python3.12/inspect.py", line 1685, in getframeinfo
    lines, lnum = findsource(frame)
                  ^^^^^^^^^^^^^^^^^
  File "/usr/local/pyenv/pyenv/versions/3.12.0a1/lib/python3.12/inspect.py", line 1060, in findsource
    module = getmodule(object, file)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/pyenv/pyenv/versions/3.12.0a1/lib/python3.12/inspect.py", line 983, in getmodule
    f = getabsfile(module)
        ^^^^^^^^^^^^^^^^^^
  File "/usr/local/pyenv/pyenv/versions/3.12.0a1/lib/python3.12/inspect.py", line 952, in getabsfile
    _filename = getsourcefile(object) or getfile(object)
                ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/pyenv/pyenv/versions/3.12.0a1/lib/python3.12/inspect.py", line 928, in getsourcefile
    if any(filename.endswith(s) for s in all_bytecode_suffixes):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/pyenv/pyenv/versions/3.12.0a1/lib/python3.12/inspect.py", line 928, in <genexpr>
    if any(filename.endswith(s) for s in all_bytecode_suffixes):
           ^^^^^^^^^^^^^^^^^
AttributeError: 'PosixPath' object has no attribute 'endswith'

Coverage.py fails earlier, but for a similar reason:

% coverage run debugme.py inspectme.py
Running PosixPath('/System/Volumes/Data/root/src/bugs/bug1417/inspectme.py')
Traceback (most recent call last):
  File "/System/Volumes/Data/root/src/bugs/bug1417/debugme.py", line 6, in <module>
    runpy.run_path(runme)
  File "/usr/local/pyenv/pyenv/versions/3.9.15/lib/python3.9/runpy.py", line 288, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/usr/local/pyenv/pyenv/versions/3.9.15/lib/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/local/pyenv/pyenv/versions/3.9.15/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/local/virtualenvs/tmp-6c4153bbeaa11b2/lib/python3.9/site-packages/coverage/control.py", line 344, in _should_trace
    disp = self._inorout.should_trace(filename, frame)
  File "/usr/local/virtualenvs/tmp-6c4153bbeaa11b2/lib/python3.9/site-packages/coverage/inorout.py", line 335, in should_trace
    filename = source_for_file(dunder_file)
  File "/usr/local/virtualenvs/tmp-6c4153bbeaa11b2/lib/python3.9/site-packages/coverage/python.py", line 105, in source_for_file
    if filename.endswith(".py"):
AttributeError: 'PosixPath' object has no attribute 'endswith'

Using coverage.py's --timid flag uses a Python trace function and also shows the error:

% coverage run --timid debugme.py inspectme.py
Running PosixPath('/System/Volumes/Data/root/src/bugs/bug1417/inspectme.py')
Traceback (most recent call last):
  File "/System/Volumes/Data/root/src/bugs/bug1417/debugme.py", line 6, in <module>
    runpy.run_path(runme)
  File "/usr/local/pyenv/pyenv/versions/3.9.15/lib/python3.9/runpy.py", line 288, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/usr/local/pyenv/pyenv/versions/3.9.15/lib/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/local/pyenv/pyenv/versions/3.9.15/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/System/Volumes/Data/root/src/bugs/bug1417/inspectme.py", line 1, in <module>
    import inspect
  File "/usr/local/virtualenvs/tmp-6c4153bbeaa11b2/lib/python3.9/site-packages/coverage/pytracer.py", line 166, in _trace
    disp = self.should_trace(filename, frame)
  File "/usr/local/virtualenvs/tmp-6c4153bbeaa11b2/lib/python3.9/site-packages/coverage/control.py", line 344, in _should_trace
    disp = self._inorout.should_trace(filename, frame)
  File "/usr/local/virtualenvs/tmp-6c4153bbeaa11b2/lib/python3.9/site-packages/coverage/inorout.py", line 335, in should_trace
    filename = source_for_file(dunder_file)
  File "/usr/local/virtualenvs/tmp-6c4153bbeaa11b2/lib/python3.9/site-packages/coverage/python.py", line 105, in source_for_file
    if filename.endswith(".py"):
AttributeError: 'PosixPath' object has no attribute 'endswith'

I don't understand why a simpler trace example gets a string for co_filename? Create traceme.py:

import sys

def foo():
    print("FOO")

def trace(frame, event, arg):
    if "traceme" in str(frame.f_code.co_filename):
        print(type(frame.f_code.co_filename))
        print(f"{frame.f_code.co_filename=}")

sys.settrace(trace)

foo()

print('here')

and run it:

% python debugme.py traceme.py
Running PosixPath('/System/Volumes/Data/root/src/bugs/bug1417/traceme.py')
<class 'str'>
frame.f_code.co_filename='/System/Volumes/Data/root/src/bugs/bug1417/traceme.py'
FOO
here

Your environment

I'm on a Mac, and this behavior seems consistent on versions 3.7 through 3.12.0a1.

Linked PRs

Activity

  1. nedbat
    added
    type-bugAn unexpected behavior, bug, or error
    on Nov 13, 2022
  2. AlexWaygood
    added
    topic-pathlib
    on Nov 13, 2022

  3. nedbat commented on Nov 13, 2022

    @nedbat
    nedbat
    on Nov 13, 2022
    MemberAuthor

    A bit more detail... Create modname.py:

    import sys

def func(): pass

print(f"{func.__module__=}")
print(f"{sys.modules[func.__module__].__file__=}")

print('here')

    and run it:

    % python3.12 debugme.py modname.py
Running PosixPath('/System/Volumes/Data/root/src/bugs/bug1417/modname.py')
func.__module__='<run_path>'
sys.modules[func.__module__].__file__=PosixPath('/System/Volumes/Data/root/src/bugs/bug1417/modname.py')
here

    The executed file has a module with __file__ as a Path object instead of a string.

  4. ronaldoussoren
    added
    topic-importlib
    on Nov 14, 2022

  5. Jacob-Stevens-Haas commented on Nov 16, 2022

    @Jacob-Stevens-Haas
    Jacob-Stevens-Haas
    on Nov 16, 2022

    Some of this is a bit above my level, but FWIW verified the same errors and output on Ubuntu 22.04 and python 3.10.6

  6. brettcannon commented on Nov 18, 2022

    @brettcannon
    brettcannon
    on Nov 18, 2022
    Member

    There's probably an implicit os.fsdecode() call missing to convert whatever gets passed in as a path to a string.

  7. iritkatriel
    added
    stdlibStandard Library Python modules in the Lib/ directory
    on Nov 23, 2023
  8. barneygale
    removed
    topic-pathlib
    on Jan 13, 2024

  9. FFY00 commented on Jan 15, 2024

    @FFY00
    FFY00
    on Jan 15, 2024
    Member

    The runpy.run_path documentation does not mention support for passing path-like objects, but since it somewhat works, I think it's better to not break it and instead fix this issue by properly supporting this use-case.

  10. FFY00
    closed this as completedin #107644on Jan 15, 2024
  11. FFY00
    added a commit that references this issue on Jan 15, 2024

    gh-99437: runpy: decode path-like objects before setting globals

    d457345
  12. kulikjak
    added a commit that references this issue on Jan 22, 2024

    pythongh-99437: runpy: decode path-like objects before setting globals

    50b3dcc
  13. miss-islington
    added a commit that references this issue on Feb 1, 2024

    pythongh-99437: runpy: decode path-like objects before setting globals

    4a81d43
  14. aisk
    added a commit that references this issue on Feb 11, 2024

    pythongh-99437: runpy: decode path-like objects before setting globals

    0830d68
  15. hugovk
    added a commit that references this issue on Aug 12, 2024

    [3.12] gh-99437: runpy: decode path-like objects before setting globa…

    0585a3f
  16. Glyphack
    added a commit that references this issue on Sep 2, 2024

    pythongh-99437: runpy: decode path-like objects before setting globals

    3a9df48
  17. fhchl
    added a commit that references this issue on Oct 19, 2024

    convert paths to strings to avoid bug in runpy

    a823a51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    stdlibStandard Library Python modules in the Lib/ directorytopic-importlibtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions