Repository navigation
Add -X importtime=2 for additional logging when an imported module is already loaded #102567
Description
Feature or enhancement
Add special handling for -X importtime=2 that provides additional output when already-loaded modules are imported. This will allow users to get a complete picture of runtime imports.
Pitch
While -X importtime is incredibly useful for analyzing module import times, by design, it doesn't log anything if an imported module has already been loaded. -X importtime=2 would provide additional output for every module that's already been loaded:
>>> import uuid
import time: cached | cached | _io
import time: cached | cached | _io
import time: cached | cached | os
import time: cached | cached | sys
import time: cached | cached | enum
import time: cached | cached | _io
import time: cached | cached | _io
import time: cached | cached | collections
import time: cached | cached | os
import time: cached | cached | re
import time: cached | cached | sys
import time: cached | cached | functools
import time: cached | cached | itertools
import time: 151 | 151 | _wmi
import time: 18290 | 18440 | platform
import time: 372 | 372 | _uuid
import time: 10955 | 29766 | uuid
In codebases with convoluted/poorly managed import graphs (and consequently, workloads that suffer from long import times), the ability to record all paths to an expensive dependency–not just the first-imported–can help expedite refactoring (and help scale identification of this type of issue). More generally, this flag would provide a more efficient path to tracking runtime dependencies.
The changes required are largely unintrusive: here's my reference implementation: [will relink, need to do some paperwork].
Previous discussion
Discussion: https://discuss.python.org/t/x-importtrace-to-supplement-x-importtime-for-loaded-modules/23882/5
Prior email chain: https://mail.python.org/archives/list/python-ideas@python.org/thread/GEISYQ5BXWGKT33RWF77EOSOMMMFUBUS/
Linked PRs
- gh-102567: Add -X importtime=2 for logging an importtime message for already-loaded modules #118655
- gh-102567: Add -X importtime=2 for logging an importtime message for already-loaded modules #133443
- gh-102567: Add missing newline to
--help-all#136391 - [3.14] gh-102567: Add missing newline to
--help-all(GH-136391) #136403
Activity
noahbkim commentedon Nov 15, 2023
@willingc Hey Carol, tagging you based on our recent conversation. We need some clarification on how to fill out the CLA before we can start contributing code to CPython; I'll quote the questions my colleague had directly (cc @tjstum):
We tried reaching out to the PSF directly around the time this issue was created but didn't get a response. Thanks!
vstinner commentedon May 29, 2024
Can you paste an example of output before and after for comparison? You propose to add "cached" lines?
gh-102567: Add -X importtime=2 for logging an importtime message for …
gh-102567: Add missing newline to `--help-all` (GH-136391)
pythongh-102567: Add missing newline to `--help-all` (pythonGH-136391)
[3.14] gh-102567: Add missing newline to `--help-all` (GH-136391) (GH…
pythongh-102567: Add missing newline to `--help-all` (pythonGH-136391)
pythongh-102567: Add -X importtime=2 for logging an importtime messag…
pythongh-102567: Add missing newline to `--help-all` (pythonGH-136391)
pythongh-102567: Add missing newline to `--help-all` (pythonGH-136391)
pythongh-102567: Add missing newline to `--help-all` (pythonGH-136391)
pythongh-102567: Add missing newline to `--help-all` (pythonGH-136391)