Skip to content

[ty] Use more parallelism when running corpus tests - #18711

Merged
AlexWaygood merged 2 commits into
mainfrom
alex/split-corpus-tests-more
Jun 16, 2025
Merged

[ty] Use more parallelism when running corpus tests#18711
AlexWaygood merged 2 commits into
mainfrom
alex/split-corpus-tests-more

Conversation

@AlexWaygood

@AlexWaygood AlexWaygood commented Jun 16, 2025

Copy link
Copy Markdown
Member

Summary

Now that we've moved the corpus tests to the ty_python_semantic crate, running cargo test -p ty_python_semantic takes an annoying amount of time locally. (This also hasn't been helped by the fact that we run a lot more corpus tests after #18531.) We can speed things up quite a bit by splitting the corpus tests up more, so that they run in parallel.

Test Plan

On main locally:

~/dev/ruff (main)⚡ % cargo test -p ty_python_semantic --test corpus              
   Compiling ty_python_semantic v0.0.0 (/Users/alexw/dev/ruff/crates/ty_python_semantic)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.81s
     Running tests/corpus.rs (target/debug/deps/corpus-1d987a322669ccb0)

running 6 tests
test corpus_no_panic ... ok
test linter_stubs_no_panic ... ok
test parser_no_panic ... ok
test linter_af_no_panic ... ok
test linter_gz_no_panic ... ok
test typeshed_no_panic ... ok

test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 8.63s

On this PR branch locally:

~/dev/ruff (alex/split-corpus-tests-more)⚡ % cargo test -p ty_python_semantic --test corpus
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.08s
     Running tests/corpus.rs (target/debug/deps/corpus-1d987a322669ccb0)

running 12 tests
test corpus_no_panic ... ok
test linter_no_panic::_a_e_expects ... ok
test linter_no_panic::_g_o_expects ... ok
test linter_stubs_no_panic ... ok
test parser_no_panic ... ok
test typeshed_no_panic::_f_k_expects ... ok
test typeshed_no_panic::_l_p_expects ... ok
test typeshed_no_panic::_a_e_expects ... ok
test linter_no_panic::_q_z_expects ... ok
test typeshed_no_panic::_q_z_expects ... ok
test linter_no_panic::_p_expects ... ok
test linter_no_panic::_f_expects ... ok

test result: ok. 12 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.77s

I.e., a speedup of around 5s

@AlexWaygood AlexWaygood added testing Related to testing Ruff or ty ty The ty type checker labels Jun 16, 2025
@github-actions

github-actions Bot commented Jun 16, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

mypy_primer results

No ecosystem changes detected ✅

@MichaReiser

MichaReiser commented Jun 16, 2025

Copy link
Copy Markdown
Member

This could make the tests on ci (or systems with fewer cores in general) somewhat slower because we now have to infer typeshed more often but I think this change still makes sense.

MichaReiser
MichaReiser approved these changes Jun 16, 2025
View reviewed changes
Comment thread crates/ty_python_semantic/tests/corpus.rs
@AlexWaygood
AlexWaygood enabled auto-merge (squash) June 16, 2025 17:36
@AlexWaygood
AlexWaygood merged commit 5e57e46 into main Jun 16, 2025
@AlexWaygood
AlexWaygood deleted the alex/split-corpus-tests-more branch June 16, 2025 17:38
dcreager added a commit that referenced this pull request Jun 16, 2025
@dcreager
Merge branch 'main' into dcreager/tuple-spec
604b484 
* main: (38 commits)
  [`pyupgrade`] Suppress `UP008` diagnostic if `super` symbol is not builtin (#18688)
  [pylint] Fix `PLW0128` to check assignment targets in square brackets and after asterisks (#18665)
  [`refurb`] Make the fix for `FURB163` unsafe for `log2`, `log10`, `*args`, and deleted comments (#18645)
  [ty] allow `T: Never` as subtype of `Never` (#18687)
  [ty] Use more parallelism when running corpus tests (#18711)
  [ty] Support `dataclasses.KW_ONLY` (#18677)
  [`ruff`] Check for non-context-manager use of `pytest.raises`, `pytest.warns`, and `pytest.deprecated_call` (`RUF061`) (#17368)
  Add syntax error when conversion flag does not immediately follow exclamation mark (#18706)
  [`flake8-pyi`] Fix `custom-typevar-for-self` with string annotations (`PYI019`) (#18311)
  Drop confusing second `*` from glob pattern example (#18709)
  [ty] Stabilize completions (#18650)
  [ty] Correctly label typeshed-sync PRs (#18702)
  Update Rust crate memchr to v2.7.5 (#18696)
  Update dependency react-resizable-panels to v3.0.3 (#18691)
  Update Rust crate clap to v4.5.40 (#18692)
  Update Rust crate libcst to v1.8.2 (#18695)
  Update Rust crate jiff to v0.2.15 (#18693)
  Update Rust crate libc to v0.2.173 (#18694)
  Update Rust crate syn to v2.0.103 (#18698)
  Update Rust crate toml to v0.8.23 (#18699)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@MichaReiser MichaReiser MichaReiser approved these changes
@carljm carljm Awaiting requested review from carljm
@sharkdp sharkdp Awaiting requested review from sharkdp
@dcreager dcreager Awaiting requested review from dcreager

Assignees

No one assigned

Labels

testing Related to testing Ruff or ty ty The ty type checker

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@AlexWaygood @MichaReiser