Plumb

Plumb the depth of your IR.

A static cost-model analysis pass for LLVM that knows the difference between an add and a chase through a function pointer.

./build.sh && ./run.sh

./run.sh analyses the testcases, then starts the live dashboard at http://localhost:8420/dashboard.html (so recommendation Compare works). Use ./run.sh --no-open in CI, or ./run.sh --file for a plain file:// open.

A naive instruction count ranks two functions equal if they have the same number of instructions, even when one is a tight integer loop and the other is a chain of memory-bound calls. Plumb assigns a cost to every IR instruction, multiplies it by loop nesting depth, surcharges indirect calls, then traces the worst-case path through every CFG.

Every cost number traces back to (group, count, weight, depth) — no black-box scores.

Real example. On the matmul() from testcases/test_floatmm.c, raw instruction count says 78. Plumb at default weights says 335 — and pinpoints bb.6 (the depth-3 inner accumulator) as the hot block carrying ~45% of total cost. After -O2, that drops to 81 (−76%), with the hotspot moving cleanly out of the inner body. See §3.4 in EVALUATION.md

Validated at scale. Across 83 programs from LLVM's official test-suite (1,165 functions, 166 runs), Plumb's median -O2 cost reduction is 55%, and its top hot function lines up with the canonical kernel name (Jacobi, FFT, kernel_*) on every numerical workload. Full sweep in benchmarks/SUMMARY.md

• At a glance
• Architecture
• Cost model in 30 seconds
• Quick start
• Testcase suite
• Plumb on real code
• The dashboard
• Output formats
• Pass options
• Repository layout
• Documents
• Tech stack
• License

The pipeline is three stages: produce IR, run the analysis pass, render the report. Every stage is decoupled — Plumb.cpp doesn't know about the dashboard, and the dashboard doesn't know about LLVM.

Inside the pass itself, each function is processed through a fixed sequence of stages:

¹ pJ figures inspired by Horowitz, ISSCC 2014 — order-of-magnitude only, not absolute device wattage.

Why static depth and not block frequency? Because every cost number must trace back to (count, weight, depth). A BlockFrequencyInfo multiplier is a heuristic guess; a depth integer is auditable. Full rationale in DESIGN.md §3.

Plumb deliberately targets the legacy pass-manager API (FunctionPass / RegisterPass) which was removed in LLVM 17. Versions 14, 15, and 16 are supported. The build script auto-detects all three.

./build.sh        # -> build/libPlumb.{so,dylib}
./run.sh          # -> ir/, results/, opens dashboard

./run.sh --no-open   # skip browser launch (CI-friendly)

If LLVM auto-detection ever fails, pin a specific install:

LLVM_DIR=$(brew --prefix llvm@14)/lib/cmake/llvm ./build.sh   # macOS
LLVM_DIR=/usr/lib/llvm-14/lib/cmake/llvm        ./build.sh   # Ubuntu

pass built: build/libPlumb.dylib
-- test_floatmm @ -O0 -----------------------------------
[Plumb] Loaded weights from: config/weights.cfg
+==========================================================+
  Plumb  >>  Function: matmul
+==========================================================+
  Total weighted cost     : 335
  Loop count / max depth  : 1 / 3
  Most expensive group    : memory (cost=246)
  Critical Path (worst-case):  cost = 264
    bb.0 -> bb.1 -> bb.2 -> bb.3 -> bb.4 -> bb.5 -> bb.6 -> bb.7
  Recommendations         : HOTSPOT
  *** HOTSPOT WARNING: cost 335 exceeds threshold 30 ***

...then the dashboard opens with twelve JSON reports ready to load.

Six C programs, each engineered to stress a different cost class. Each is run at both -O0 and -O2, producing 12 reports.

Per-testcase findings, per-function deltas, and the failure cases live in EVALUATION.md.

The depth-3 inner loop (bb.5 → bb.6 → bb.7) carries the cost. bb.6 alone is 150 / 335 = 45% of the function — exactly because every load gets a ×3 depth multiplier on top of its weight 3.

The six testcases in testcases/ are designed to stress specific cost classes. To validate Plumb against actual workloads, benchmarks/ sweeps the official LLVM test-suite SingleSource/Benchmarks subset (Stanford, Misc, Polybench, Shootout).

   ┌────────────────────────────────────────────────────────────┐
   │   83 programs · 1,165 functions · 166 runs (O0 + O2)       │
   │                                                            │
   │   Median cost reduction at -O2     ─→   55 %               │
   │   Programs whose hottest fn          memory-dominated 99 % │
   │   Inliner-inflation failure mode    ─→   4 % of programs   │
   │                                                            │
   │   Top-3 hottest at -O0:                                    │
   │     1.  Misc/himenobmtxpa  →  jacobi          (cost 10669) │
   │     2.  Misc/oourafft       →  cftmdl          (cost  3181) │
   │     3.  Polybench/deriche  →  kernel_deriche  (cost  1587) │
   └────────────────────────────────────────────────────────────┘

Plumb's hot-function detection lines up with what an HPC engineer would call out as the kernel: Jacobi solvers, FFT inner loops, Polybench's kernel_* functions all surface at the top of the cost ranking. The full report — top-15 hottest programs, per-suite breakdown, and the 3 programs where -O2 actually increased reported cost (the failure mode from EVALUATION.md §5.1 confirmed in the wild) — lives in benchmarks/SUMMARY.md.

To reproduce:

./build.sh
./benchmarks/fetch.sh            # one-time sparse-clone of test-suite
./benchmarks/run_bench.sh        # ~20 s end-to-end on Apple Silicon
python3 benchmarks/analyze.py    # regenerates SUMMARY.md

Open dashboard/dashboard.html. Single self-contained HTML, no build step, CDN-loaded libs (Chart.js, Cytoscape.js, html2canvas, jsPDF).

./scripts/validation_server.sh   # opens http://localhost:8420/dashboard.html

Pick any already-analyzed program from the header, or upload a brand-new .c file the project has never seen — the dashboard compiles it, runs Plumb, and every recommendation tag already shown in "Optimisation Recommendations" grows a ▶ Validate button right on the chip. Click one and the dashboard shells out to the real opt (-always-inline, -loop-vectorize, -tailcallelim) against that exact function, live, and the result lands in "Recommendation Validation" right below. This is not replayed data: below, main from test_floatmm.c gets vectorized live and its cost actually drops 112 → 78 (−30%) — a different, better outcome than the fixed case study's matrix_add example, because it's a different function, measured for real.

Opened without the server (just double-clicking the HTML file), the same section still works, but falls back to a clearly-labeled fixed case study — EVALUATION.md §6 rendered, covering Plumb's own 3 testcases only. It never pretends to reflect a file it hasn't actually analyzed.

The rest of the dashboard is the per-run analysis tooling, populated after loading a results.json (or automatically, once you pick/upload a program above):

Demo screenshots live in docs/screenshots/.

+==========================================================+
  Plumb  >>  Function: matmul
+==========================================================+

  Instruction-Type Analysis:
  +----------+-------+--------+--------+--------------+
  | Group    | Count | Weight |  Cost  | Contribution |
  +----------+-------+--------+--------+--------------+
  | memory   |  38   |   3    | 246    |    73%       |
  | branch   |  12   |   1    |  22    |     6%       |
  | mul      |   3   |   2    |  16    |     4%       |
  | ...                                              |
  +----------+-------+--------+--------+--------------+

  BasicBlock cost bar chart:
  bb.6  |##############################| 150  [CRIT]
  bb.8  |#########                     |  46
  bb.0  |####                          |  24
  ...

  Critical Path (worst-case):  cost = 264
  bb.0 -> bb.1 -> bb.2 -> bb.3 -> bb.4 -> bb.5 -> bb.6 -> bb.7

function,group,count,weight,cost,pct
matmul,memory,38,3,246,73.4
matmul,branch,12,1,22,6.6
matmul,mul,3,2,16,4.8
matmul,call,1,5,15,4.5
matmul,add,6,1,14,4.2
...

{
  "metadata": {
    "tool": "Plumb", "runLabel": "O0",
    "weights":      { "add":1, "mul":2, "memory":3, "call":5 },
    "energyModelPj": { "add":0.4, "mul":3.4, "memory":50 },
    "totals":       { "totalCost": 514, "functionCount": 3 }
  },
  "functions": [
    {
      "name": "matmul",
      "totalCost": 335,
      "totalInstructions": 78,
      "cyclomaticComplexity": 4,
      "maxLoopDepth": 3,
      "loopCount": 1,
      "isRecursive": false,
      "energyPj": 4487.9,
      "mostExpensiveGroup": "memory",
      "recommendations": ["HOTSPOT"],
      "criticalPath":     ["bb.0","bb.1","bb.2","bb.3","bb.4","bb.5","bb.6","bb.7"],
      "criticalPathCost": 264,
      "groups":      [ /* per-group {count,weight,cost,pct,indirect} */ ],
      "basicBlocks": [ /* per-BB {label,cost,instructions,loopDepth,isCritical,successors} */ ]
    }
  ]
}

All flags use the plumb- prefix to avoid clashing with LLVM's own command-line options (notably the inliner's built-in -inline-threshold, which would trip CommandLineParser::addOption at dlopen time without the prefix). Full story in IMPLEMENTATION.md §8

opt -enable-new-pm=0 \
    -load build/libPlumb.dylib \
    -plumb \
    -plumb-weight-file=config/weights.cfg \
    -plumb-hot-threshold=30 \
    -plumb-inline-threshold=20 \
    -plumb-run-label=O0 \
    -plumb-out-file=results.csv \
    -plumb-json-file=results.json \
    -disable-output input.ll

plumb/
├── README.md              <- this file
├── LICENSE                <- MIT
├── build.sh               <- compiles the pass
├── run.sh                 <- runs analysis, opens dashboard
├── src/
│   ├── Plumb.cpp          <- the LLVM pass (~890 LoC)
│   └── CMakeLists.txt     <- LLVM-14/15/16 build glue
├── config/
│   └── weights.cfg        <- editable cost table
├── testcases/             <- 6 C programs, each stressing a different class
│   ├── test_arith.c
│   ├── test_branchy.c
│   ├── test_callchain.c
│   ├── test_floatmm.c
│   ├── test_memheavy.c
│   └── test_recursive.c
├── benchmarks/            <- Plumb sweep across 83 LLVM-test-suite programs
│   ├── README.md
│   ├── fetch.sh           <- sparse-clone llvm-test-suite
│   ├── run_bench.sh       <- compile + analyse every program
│   ├── analyze.py         <- aggregate JSONs into SUMMARY.md
│   └── SUMMARY.md         <- the headline report (auto-gen, committed)
├── dashboard/
│   └── dashboard.html     <- interactive UI (single file, CDN libs)
├── scripts/
│   ├── _llvm_env.sh                  <- shared LLVM toolchain detection
│   ├── _plumb_lib.py                 <- shared IR-transform helpers (both scripts below import this)
│   ├── validate_recommendations.sh   <- fixed case study: applies + measures all 5 tags on Plumb's own testcases
│   ├── validate_recommendations.py   <- IR surgery + JSON analysis behind the above
│   ├── validation_server.sh          <- live validation: local HTTP server for the dashboard
│   └── validation_server.py          <- compiles/analyzes uploads, runs transforms on-demand
└── docs/
    ├── DESIGN.md          <- approach, alternatives, tradeoffs
    ├── IMPLEMENTATION.md  <- LLVM-API specifics, build details
    ├── EVALUATION.md      <- measured results across 6 testcases
    ├── CHANGELOG.md       <- bugs found and fixed, each with a reproducible before/after
    ├── diagrams/          <- README diagrams (matplotlib-rendered PNGs)
    └── screenshots/       <- demo captures

After running ./run.sh, a few more directories appear (all .gitignored):

build/                          compiled pass library
ir/                             LLVM IR for every testcase x {O0,O2}
results/                        one CSV + one JSON per (testcase, opt-level) pair
validation/                     IR/JSON from ./scripts/validate_recommendations.sh
uploads/                        programs analyzed live via ./scripts/validation_server.sh
benchmarks/llvm-test-suite/     sparse clone of llvm/llvm-test-suite (~30 MB)
benchmarks/ir/                  IR for every benchmark program
benchmarks/results/             166 JSONs (one per program × opt-level)

Verified end-to-end on:

• macOS — Apple Silicon, Homebrew LLVM 14, bash 3.2
• Linux — Ubuntu 22.04, clang-14 / opt-14, bash 5

MIT. See LICENSE.