Module names containing null bytes bypass the sys.modules cache

Bug report

Bug description:

find_frozen in Python/import.c converts the module name to a C string using PyUnicode_AsUTF8, which returns a \0-terminated pointer into the Unicode object's internal buffer. When the name contains an embedded null byte -- for example "codecs\x00junk" -- the C string passed to look_up_frozen is silently truncated to "codecs", so the frozen table lookup succeeds.

The module is then created and registered in sys.modules under the full key "codecs\x00junk" rather than "codecs", producing a second, independent frozen codecs module that is not the cached one. Every subsequent call with such a name bypasses the cache and allocates a fresh copy of all the module's functions/etc.

The following reproducer shows the inconsistency:

import sys

before = set(sys.modules.keys())
m = __import__('codecs\x00junk')
new_keys = set(sys.modules.keys()) - before
print("Added to sys.modules.keys:", new_keys)
print("Name of the newly added module:", m.__name__)
print("Modules are the same?", m is sys.modules.get('codecs'))

... which shows:

Added to sys.modules.keys: {'codecs\x00junk'}
Name of the newly added module: codecsjunk
Modules are the same? False

This bug was discovered by fuzzing pickle.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

@KowalskiThomas

Thanks for the report @KowalskiThomas.

My take on this is that __import__() and importlib.import_module() (which your report doesn't mention, but is subject to similar issues), should explicitly reject null bytes in their input. __import__('codecs\x00junk') and importlib.import_module('codecs\x00junk') should both raise ValueError, just as open() will do in this case:

>>> open('codecs\x00junk')
Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    open('codecs\x00junk')
    ~~~~^^^^^^^^^^^^^^^^^^
ValueError: embedded null byte

The reasoning is this: while not explicitly described in the language reference docs (a gap that should also be closed), the language grammar defines a module name as a dotted sequence of identifiers, and identifiers cannot contain null bytes. Further, AFAIK no file system on any platform that Python is compatible with allows null bytes in file names.

I don't think there's a case to be made to explicitly disallow other non-ASCII characters (e.g. 'codecs\x01junk') given that PEP 3131 allows for non-ASCII identifiers, even though there are restrictions in which non-ASCII characters can appear in identifiers.

A stronger argument could be made that these APIs should reject anything that is not an identifier.

>>> 'codecs\x00junk'.isidentifier()
False
>>> 'codecs\x01junk'.isidentifier()
False
>>> 'café'.isidentifier()
True

and in fact, you'll get SyntaxErrors if you try to use these non-identifiers in import statements:

>>> import codecs\x01junk
  File "<python-input-13>", line 1
    import codecs\x01junk
                  ^
SyntaxError: unexpected character after line continuation character
>>> import codecs\x00junk
  File "<python-input-14>", line 1
    import codecs\x00junk
                  ^
SyntaxError: unexpected character after line continuation character
>>> import café
Traceback (most recent call last):
  File "<python-input-15>", line 1, in <module>
    import café
ModuleNotFoundError: No module named 'café'

That said, I don't know whether we want to pay the performance penalty for checking for identifier-ness given that it's likely you'll get a ModuleNotFoundError in those cases. Plus, it's not uncommon to stash weird strings in sys.modules as marker objects:

>>> from types import ModuleType
>>> import sys
>>> sys.modules['codecs\x01junk'] = ModuleType('codecs\x01junk')
>>> __import__('codecs\x01junk')
<module 'codecs\x01junk'>
>>>

Technically, disallowing embedded null bytes is a backward incompatible change, which means it could only go in Python 3.16, and there are deprecation concerns to think about. I personally think we should not worry ourselves about that as it's a pretty niche corner case, but I'm open to other opinions.

Thus my proposal is:

Explicitly raise ValueError if an embedded null is found in the arguments to __import__() and importlib.import_module()
In the Language Reference manual, document that import statements require dotted-identifiers and that these two functional APIs to the import system have looser requirements (i.e. only embedded null is explicitly disallowed)
Add this breaking change to Python 3.16 with no deprecation cycle

@brettcannon @ncoghlan @gvanrossum (and anyone else) for thoughts.

y take on this is that __import__() and importlib.import_module() (which your report doesn't mention, but is subject to similar issues), should explicitly reject null bytes in their input. __import__('codecs\x00junk') and importlib.import_module('codecs\x00junk') should both raise ValueError

I agree; just reject it outright.

/ in the module name also can cause issues. Or \ and : on Windows.

In general, we should handle null bytes close to the place where they cause issues. For example, if we convert a Python string to null-terminated C string, we should check that the Python string does not have embedded null characters. So we guarantee that we handle it independently of the way we came here. In some cases we may add an additional check at the higher level if it provides important context or saves us from doing redundant work in common case.

The main problem with nul is that it truncates the string, so 'codecs\x00junk' tries to import codecs. Other characters can definitely cause problems, but if they don't spoof the module name and end up raising legit ModuleNotFound exceptions, that should be good enough.

There's still the issue of poking non-importable names into sys.modules but that's also a legit use case. I don't think we have to prevent adding embedded-nul names into sys.modules though.

I agree, I only say that there is no need to specially reject null characters at the top level, in __import__() and importlib.import_module() if they are rejected at the lower levels, where they can cause truncation. And they should always be rejected at the same level where they can cause truncation.

#150634 does this right, I only have some questions to the NEWS entry and tests.

As long as you don't look up the unsanitized key in a cache.

gh-150633: Properly handle null characters in the name when importing…

[3.13] gh-150633: Properly handle null characters in the name when im…

[3.14] gh-150633: Properly handle null characters in the name when im…

[3.15] gh-150633: Properly handle null characters in the name when im…

gh-150633: Minor improvement of a newly added test (#151103)

[3.15] gh-150633: Minor improvement of a newly added test (GH-151103) (…

pythongh-150633: Properly handle null characters in the name when imp…

pythongh-150633: Minor improvement of a newly added test (python#151103)

Navigation Menu

Repository navigation

Module names containing null bytes bypass the `sys.modules` cache #150633

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Module names containing null bytes bypass the sys.modules cache #150633

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

warsaw commented on Jun 3, 2026

brettcannon commented on Jun 5, 2026

serhiy-storchaka commented on Jun 6, 2026

warsaw commented on Jun 6, 2026

serhiy-storchaka commented on Jun 7, 2026

gvanrossum commented on Jun 7, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

Module names containing null bytes bypass the `sys.modules` cache #150633