Skip to content

Optimize repeated Map lookups with lazy index - #16

Draft
mlin wants to merge 1 commit into
mainfrom
codex/optimize-map-access-efficiency
Draft

Optimize repeated Map lookups with lazy index#16
mlin wants to merge 1 commit into
mainfrom
codex/optimize-map-access-efficiency

Conversation

@mlin

@mlin mlin commented Apr 29, 2026

Copy link
Copy Markdown
Owner

Motivation

  • Repeated WDL Map key lookups were doing a linear scan of Value.Map.value on every access, which is inefficient for common cases with many lookups.
  • Make a small, low-risk optimization that reuses a Python dict index after the first lookup to improve performance without large refactors.

Description

  • Add cached index fields to WDL.Value.Map: _index and _index_failed, and initialize them in __init__.
  • Implement WDL.Value.Map.get(key) which lazily builds _index on first lookup, uses _index for subsequent lookups, and falls back to the original linear scan when keys are unhashable in Python.
  • Provide _index_key() helper that derives the Python dict key from a Value.Base (currently returns key.value).
  • Switch stdlib map access in _At._call_eager to use Value.Map.get() so the optimization applies to map[key] evaluations.

Testing

  • Ran formatting and static checks: ruff format WDL/Value.py WDL/StdLib.py and ruff check WDL/Value.py WDL/StdLib.py, which passed.
  • Ran type checks with mypy WDL/Value.py WDL/StdLib.py, which passed.
  • Ran targeted unit tests with pytest -q tests/test_0eval.py -k map, which passed (1 passed, 22 deselected).

Codex Task

@coveralls

Copy link
Copy Markdown

Coverage Report for CI Build 25129537403

Coverage decreased (-0.09%) to 95.143%

Details

  • Coverage decreased (-0.09%) from the base build.
  • Patch coverage: 7 uncovered changes across 1 file (14 of 21 lines covered, 66.67%).
  • 1 coverage regression across 1 file.

Uncovered Changes

File Changed Covered %
WDL/Value.py 20 13 65.0%

Coverage Regressions

1 previously-covered line in 1 file lost coverage.

File Lines Losing Coverage Coverage
WDL/runtime/backend/docker_swarm.py 1 91.62%

Coverage Stats

Coverage Status
Relevant Lines: 7926
Covered Lines: 7541
Line Coverage: 95.14%
Coverage Strength: 0.95 hits per line

💛 - Coveralls

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants