Skip to content

bench: heaptrack integration & baseline - #7

Merged
mikucionisaau merged 12 commits into
mainfrom
bench/heaptrack-integration
Jul 1, 2026
Merged

bench: heaptrack integration & baseline#7
mikucionisaau merged 12 commits into
mainfrom
bench/heaptrack-integration

Conversation

@mikucionisaau

@mikucionisaau mikucionisaau commented Jun 8, 2026

Copy link
Copy Markdown
Member

Summary

This PR adds a heaptrack-based benchmarking harness and CI workflow to capture performance and memory artifacts and compare them against a committed baseline.

What changed

  • tools/bench/run_bench.sh: builds non-sanitized preset, runs test binaries under heaptrack and /usr/bin/time, produces artifacts/ (heap-.zst, heap-.txt, time-*.txt) and artifacts/current_results.json.
  • tools/bench/compare_results.py: compares artifacts/current_results.json against benchmarks/baseline.json and exits non-zero when regressions exceed thresholds (defaults: 5% perf, 5% mem).
  • .github/workflows/benchmarks.yml: CI job that installs heaptrack, runs the bench script, uploads artifacts, and publishes current_results.json.
  • benchmarks/baseline.json: initial baseline committed from a local run (relative paths).
  • .github/copilot-instructions.md: repository guidance for Copilot sessions (added alongside this work).

Notes & usage

  • To run locally: chmod +x tools/bench/run_bench.sh && ./tools/bench/run_bench.sh (requires heaptrack and heaptrack_print).
  • Compare locally: python3 tools/bench/compare_results.py [perf_pct_threshold] [mem_pct_threshold] (defaults: 5 5).
  • Baseline is stored in benchmarks/baseline.json. Update baseline by running the bench script on a known-good machine and committing the new file.

Caveats

  • CI runs the benchmark job on ubuntu-latest; absolute performance numbers vary between runners. Prefer comparing percentage changes and set thresholds conservatively.
  • Artifacts/ contains raw heaptrack outputs (zst) and summaries; they are committed in this branch to provide an initial baseline and speed review. Consider moving large artifacts to CI-only storage or LFS if undesired in main.

Checklist

  • Run bench locally and verify outputs
  • Review thresholds in tools/bench/compare_results.py
  • Decide baseline storage policy (in-repo vs artifact store)
  • Consider gating PRs on perf/memory regression once stable

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

mikucionisaau and others added 4 commits June 8, 2026 11:34
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…line

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ffer leaks

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mikucionisaau

Copy link
Copy Markdown
Member Author

Follow-up: fixed heaptrack-reported FILE buffer leaks by closing stdout/stderr at program exit in test/utils.h. Rebuilt and re-ran bench: heaptrack now reports 0 leaked allocations for all test binaries. Committed fix in this branch.

mikucionisaau and others added 8 commits June 8, 2026 13:01
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…espace, non-copyable, cast returns, inline const)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…lts.json

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…and time files)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…l commits

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mikucionisaau
mikucionisaau merged commit 1d35bb9 into main Jul 1, 2026
7 checks passed
@mikucionisaau
mikucionisaau deleted the bench/heaptrack-integration branch July 1, 2026 06:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant