perf: optimize version script from rustc by karolzwolak · Pull Request #1355 · wild-linker/wild

karolzwolak · 2025-12-03T21:16:42Z

karolzwolak · 2025-12-03T21:18:57Z

Actually this doesn't speed the things up that much.
Benching libbevy_dylib (560k line version script):

Benchmark 1: ./run-with /home/karolz/dev/contrib/wild/target/release/wild --no-fork
  Time (mean ± σ):      1.068 s ±  0.017 s    [User: 7.228 s, System: 1.607 s]
  Range (min … max):    1.042 s …  1.092 s    10 runs
 
Benchmark 2: ./run-with /home/karolz/dev/contrib/wild/target/release/wild-before --no-fork
  Time (mean ± σ):      1.088 s ±  0.032 s    [User: 7.129 s, System: 1.636 s]
  Range (min … max):    1.054 s …  1.148 s    10 runs
 
Summary
  ./run-with /home/karolz/dev/contrib/wild/target/release/wild --no-fork ran
    1.02 ± 0.03 times faster than ./run-with /home/karolz/dev/contrib/wild/target/release/wild-before --no-fork

Did I miss something performance wise or just this doesn't provide any substantial speedup?

EDIT: this actually is slower for compiling smaller libs like librustc_driver (although 300ms is probably too low to be a representative test)

Benchmark 1: ./run-with /home/karolz/dev/contrib/wild/target/release/wild --no-fork
  Time (mean ± σ):     292.5 ms ±   7.9 ms    [User: 1992.1 ms, System: 536.3 ms]
  Range (min … max):   280.4 ms … 306.7 ms    10 runs
 
Benchmark 2: ./run-with /home/karolz/dev/contrib/wild/target/release/wild-before --no-fork
  Time (mean ± σ):     286.2 ms ±   8.9 ms    [User: 1968.3 ms, System: 536.9 ms]
  Range (min … max):   277.6 ms … 305.7 ms    10 runs
 
Summary
  ./run-with /home/karolz/dev/contrib/wild/target/release/wild-before --no-fork ran
    1.02 ± 0.04 times faster than ./run-with /home/karolz/dev/contrib/wild/target/release/wild --no-fork

davidlattimore

That's unfortunate that there wasn't a speedup for librustc-driver. Although, I see a 9.9% speedup for the bevy dylib, which is still definitely something! I just looked at some profiles and it seems that analyze_glob_pattern is taking quite a bit of time.

I wonder if the changes you've made here would be a step towards moving the call to analyze_glob_pattern to run in parallel. e.g. we parse the file from a single thread, but don't identify the kinds of patterns, then from multiple threads, we classify matchers. I wouldn't suggest doing that in this PR, but we could explore doing it afterwards. If you don't want to, then I might.

mati865 · 2025-12-04T03:32:19Z

If you don't want to regress performance in the short term, we could merge it into a new branch, do the optimizations, and then rebase main onto that branch (without squashing).
Although I'm not sure if it's worth risking conflicts.

davidlattimore · 2025-12-04T03:35:40Z

The performance regression for librustc-driver was pretty small, about 1% and went away completely (at least for me) once the extra phase was run in parallel with dropping a bunch of Vecs.

karolzwolak · 2025-12-04T20:45:15Z

I wonder if the changes you've made here would be a step towards moving the call to analyze_glob_pattern to run in parallel. e.g. we parse the file from a single thread, but don't identify the kinds of patterns, then from multiple threads, we classify matchers. I wouldn't suggest doing that in this PR, but we could explore doing it afterwards. If you don't want to, then I might.

Sure, I will look into that when this is merged.

karolzwolak · 2025-12-04T21:27:52Z

+        let mut per_symbol_flags = PerSymbolFlags::new(per_symbol_flags);

-        verbose_timing_phase!("Apply linker scripts");
+        rayon::join(


We should probably benchmark that this doesn't slow things down when the version script isn't coming from rustc. I can do that tomorrow.

karolzwolak · 2025-12-04T21:40:34Z

-        for script in linker_scripts {
-            index.apply_linker_script(script);
-        }
+                {


Maybe we should move this scope out of the rayon task and only do the join if the version script is coming from rustc

It's not clear to me which is better, this way we are always dropping in parallel but not sure that's actually worth it when the script isn't from rustc.

rayon::join is pretty lightweight. It doesn't even heap-allocate the closures and we're not in a loop here, so this is one extra join during the entire linker execution.

…ript

karolzwolak · 2025-12-05T20:45:51Z

That's unfortunate that there wasn't a speedup for librustc-driver. Although, I see a 9.9% speedup for the bevy dylib, which is still definitely something! I just looked at some profiles and it seems that analyze_glob_pattern is taking quite a bit of time.

This is great! The speedup is negligible on my system.

./run-with /home/karolz/dev/contrib/wild/target/release/wild --no-fork ran
    1.01 ± 0.02 times faster than ./run-with /home/karolz/dev/contrib/wild/target/release/wild-before --no-fork

Could you do the benchmark again on the latest revision?

davidlattimore · 2025-12-05T21:49:41Z

Sure! Still looks good to me:

B=bevy F="--threads=32 --strip-debug" R=/home/david/save/$B/run-with OUT=/run/user/1000/ttt bench poop -d 30000  "$R $HOME/wild-builds/2025-12-06 $F" "$R target/release/wild $F"
  Temperature: 54.8 C
Benchmark 1 (87 runs): /home/david/save/bevy/run-with /home/d/wild-builds/2025-12-06 --threads=32 --strip-debug
  measurement          mean ± σ            min … max           outliers         delta
  wall_time           345ms ± 7.04ms     332ms …  366ms          2 ( 2%)        0%
  peak_rss           7.29MB ± 24.2KB    7.24MB … 7.30MB         16 (18%)        0%
  cpu_cycles         10.5G  ±  136M     10.1G  … 10.7G           0 ( 0%)        0%
  instructions       10.2G  ± 17.2M     10.2G  … 10.2G           0 ( 0%)        0%
  cache_references    237M  ± 1.56M      233M  …  243M           2 ( 2%)        0%
  cache_misses       61.2M  ±  350K     60.3M  … 62.5M           3 ( 3%)        0%
  branch_misses      17.2M  ± 61.0K     17.1M  … 17.4M           0 ( 0%)        0%
Benchmark 2 (99 runs): /home/david/save/bevy/run-with target/release/wild --threads=32 --strip-debug
  measurement          mean ± σ            min … max           outliers         delta
  wall_time           305ms ± 6.35ms     296ms …  326ms          5 ( 5%)        ⚡- 11.4% ±  0.6%
  peak_rss           7.29MB ± 21.4KB    7.20MB … 7.34MB         19 (19%)          +  0.1% ±  0.1%
  cpu_cycles         10.6G  ±  111M     10.2G  … 10.7G           4 ( 4%)          +  0.5% ±  0.3%
  instructions       10.1G  ± 16.8M     10.1G  … 10.2G           1 ( 1%)          -  0.4% ±  0.0%
  cache_references    239M  ± 1.78M      236M  …  246M           4 ( 4%)          +  0.9% ±  0.2%
  cache_misses       61.8M  ±  342K     61.3M  … 63.2M           1 ( 1%)          +  1.0% ±  0.2%
  branch_misses      17.5M  ± 58.8K     17.4M  … 17.7M           5 ( 5%)        💩+  1.6% ±  0.1%
  Temperature: 61.4 C

No significant change on librustc-driver.

karolzwolak added 10 commits December 3, 2025 21:52

prepare

77c9227

add rustc optimized version script

4d32ccd

naive solution

be18da3

properly detect rust version script

c90e422

add test for parsing rust version script

36b6c57

remove the DOWNGRADE_TO_LOCAL flag in parallel

5922c60

hash just in time

2aa84ba

also apply rust-like opt when there are multiple local matchers

2fcd0dc

rename the struct

69eef0d

better docs, rename field

4d1816d

karolzwolak marked this pull request as draft December 3, 2025 21:23

davidlattimore reviewed Dec 4, 2025

View reviewed changes

karolzwolak added 3 commits December 4, 2025 21:30

implement AtomicValueFlags::remove() using fetch_and

c6783d1

remove (inline) VersionScript::version_iter()

0985e0f

drop per group outputs in a separate thread & timing phase

446aa75

rayon::join instead of rayon::scope

0c4353a

karolzwolak commented Dec 4, 2025

View reviewed changes

karolzwolak added 2 commits December 5, 2025 21:40

fix typo

939ad49

don't set all symbols to non-interposable when having rust version sc…

159f4b2

…ript

karolzwolak marked this pull request as ready for review December 5, 2025 20:41

davidlattimore reviewed Dec 5, 2025

View reviewed changes

Comment thread libwild/src/symbol_db.rs Outdated

create a separete function for handling rust version script

d4a9b79

davidlattimore approved these changes Dec 6, 2025

View reviewed changes

davidlattimore merged commit 653fdde into wild-linker:main Dec 6, 2025
20 checks passed

Uh oh!

Conversation

karolzwolak commented Dec 3, 2025

Uh oh!

karolzwolak commented Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

davidlattimore left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mati865 commented Dec 4, 2025

Uh oh!

davidlattimore commented Dec 4, 2025

Uh oh!

karolzwolak commented Dec 4, 2025

Uh oh!

karolzwolak Dec 4, 2025

Choose a reason for hiding this comment

Uh oh!

karolzwolak Dec 4, 2025

Choose a reason for hiding this comment

Uh oh!

karolzwolak Dec 4, 2025

Choose a reason for hiding this comment

Uh oh!

davidlattimore Dec 4, 2025

Choose a reason for hiding this comment

Uh oh!

karolzwolak commented Dec 5, 2025

Uh oh!

davidlattimore commented Dec 5, 2025

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

karolzwolak commented Dec 3, 2025 •

edited

Loading