ci: add e2e benchmarks for lint rules by ematipico · Pull Request #11048 · biomejs/biome

ematipico · 2026-07-23T10:57:45Z

Summary

This PR adds three new benchmarks to test some specific lint rules in an e2e setting. I chose the following rules:

noUnusedVariables: semantic model heavy, and used a lot of times. It regressed some time ago.
noUnresolvedImports a project rule that is stable
noMiusedPromises and noFloatingPromises: two type-aware lint rules that are very important

I created the benchmarks using a coding agent

Test Plan

Green CI

Docs

N/A

changeset-bot · 2026-07-23T10:57:52Z

⚠️ No Changeset found

Latest commit: 8237753

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

github-actions · 2026-07-23T10:57:57Z

✅ Organic activity

No automation signals detected in the analyzed events.

View full analysis →

_{This is an automated analysis by AgentScan}

codspeed · 2026-07-23T11:04:05Z

Merging this PR will degrade performance by 3.17%

❌ 2 regressed benchmarks
✅ 264 untouched benchmarks
🆕 4 new benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Benchmark	`BASE`	`HEAD`	Efficiency
❌	`db_17847247775464589309.json[uncached]`	13.6 ms	14.1 ms	-3.34%
❌	`react.production.min_3378072959512366797.js[cached]`	1.8 ms	1.8 ms	-3%
🆕	`e2e_no_floating_promises`	N/A	874.4 µs	N/A
🆕	`e2e_no_misused_promises`	N/A	970.6 µs	N/A
🆕	`e2e_no_unresolved_imports`	N/A	1 ms	N/A
🆕	`e2e_no_unused_variables`	N/A	864.2 µs	N/A

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

_{Comparing bench/e2e-rules (8237753) with main (2a708ce)}

coderabbitai · 2026-07-23T11:05:11Z

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 01180595-7217-4cc7-82fc-e1f55b06327c

📥 Commits

Reviewing files that changed from the base of the PR and between 22ae5e3 and 8237753.

📒 Files selected for processing (2)

.github/workflows/benchmark.yml
crates/biome_service/Cargo.toml

Walkthrough

Adds an end-to-end Divan benchmark for workspace.pull_diagnostics using TypeScript fixtures covering four lint rules and multiple scan modes. Updates the service crate manifest with benchmark configuration, dependencies, and platform-specific allocators. Extends the benchmark workflow to detect service-related changes and run the new biome_service benchmark job.

Possibly related PRs

biomejs/biome#9963: Restructures benchmark workflow change detection and conditional jobs, which this change extends.
biomejs/biome#10943: Also modifies crates/biome_service/Cargo.toml.

Suggested labels: A-Linter, A-Tooling

Suggested reviewers: denbezrukov, siketyan

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly captures the main change: adding e2e benchmarks for lint rules.
Description check	✅ Passed	The description matches the change set and explains the new e2e lint-rule benchmarks.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch bench/e2e-rules

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 Checkov (3.3.8)

.github/workflows/benchmark.yml

Traceback (most recent call last):
File "/usr/local/bin/checkov", line 2, in
from checkov.main import Checkov
ModuleNotFoundError: No module named 'checkov'

_{Comment @coderabbitai help to get the list of available commands.}

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/benchmark.yml:
- Around line 150-163: Update the service trigger path filters to include the
biome_diagnostics crate source, and include its manifest if manifest changes are
intended to trigger the benchmark job. Preserve the existing service paths and
placement in the workflow.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: f1fbfcbd-03b0-4c16-b7e5-318f697305d7

📥 Commits

Reviewing files that changed from the base of the PR and between 50a9bd8 and 22ae5e3.

⛔ Files ignored due to path filters (1)

Cargo.lock is excluded by !**/*.lock and included by **

📒 Files selected for processing (3)

.github/workflows/benchmark.yml
crates/biome_service/Cargo.toml
crates/biome_service/benches/pull_diagnostics.rs

bench: e2e lint rules

337f2a4

ematipico requested review from a team July 23, 2026 10:57

github-actions Bot added the A-Project Area: project label Jul 23, 2026

ematipico changed the title ~~bench: e2e lint rules~~ ci: add e2e benchmarks for lint rules Jul 23, 2026

create step

22ae5e3

coderabbitai Bot reviewed Jul 23, 2026

View reviewed changes

Comment thread .github/workflows/benchmark.yml Outdated

fix

6756dcf

ematipico commented Jul 23, 2026

View reviewed changes

Comment thread .github/workflows/benchmark.yml Outdated

Apply suggestion from @ematipico

8237753

ematipico merged commit 623dbbb into main Jul 23, 2026
14 of 15 checks passed

ematipico deleted the bench/e2e-rules branch July 23, 2026 12:16

Navigation Menu

Uh oh!

ci: add e2e benchmarks for lint rules - #11048

ci: add e2e benchmarks for lint rules#11048
ematipico merged 4 commits into
mainfrom
bench/e2e-rules

ematipico commented Jul 23, 2026

Uh oh!

changeset-bot Bot commented Jul 23, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 23, 2026

Uh oh!

codspeed Bot commented Jul 23, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Jul 23, 2026 •

edited

Loading

Review failed

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

ematipico commented Jul 23, 2026

Summary

Test Plan

Docs

Uh oh!

changeset-bot Bot commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

github-actions Bot commented Jul 23, 2026

✅ Organic activity

Uh oh!

codspeed Bot commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will degrade performance by 3.17%

Performance Changes

Uh oh!

coderabbitai Bot commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review failed

Walkthrough

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

changeset-bot Bot commented Jul 23, 2026 •

edited

Loading

codspeed Bot commented Jul 23, 2026 •

edited

Loading

coderabbitai Bot commented Jul 23, 2026 •

edited

Loading