Repository navigation
ci(e2e): speed up e2e job via Ginkgo proc oversubscription - #2085
Merged
Merged
Conversation
The e2e specs are dominated by container startup and health-check waits rather than CPU, so running more Ginkgo processes than the runner has cores improves wall-clock time. Locally this cut a full run from 244s (-p, 8 procs/8 cores) to 185s (--procs=16). - Add GINKGO_E2E_PROCS (default -p, unchanged locally); CI sets --procs=8. - Add DOCKER_E2E_CACHE_FLAGS so CI can reuse e2e build layers via the GitHub Actions cache (type=gha), mirroring development-docker.yml. Local default behaviour is unchanged; both knobs are CI-only overrides.
No-op commit to measure e2e build time with the GitHub Actions cache warm (populated by the previous run).
The type=gha cache flags silently no-op with a raw 'docker buildx build' in a plain run: step (no ACTIONS_RUNTIME_TOKEN/ACTIONS_CACHE_URL exposed), so the build recompiled from scratch on warm runs and saved nothing. Removing the flags to avoid GitHub Actions cache-storage pressure. The win is the Ginkgo proc oversubscription (--procs=8 on the 4-core runner): e2e-test job 6m45s -> ~4m40s (-30%), 137/137 specs, 0 skips.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR speeds up the CI e2e-test job by making Ginkgo’s parallelism configurable for e2e runs and oversubscribing processes on GitHub Actions runners where e2e time is dominated by container startup/health-check waits rather than CPU.
Changes:
- Add
GINKGO_E2E_PROCSMakefile knob (default-p) to control e2e Ginkgo parallelism without changing local defaults. - Update e2e Makefile targets to use
GINKGO_E2E_PROCSinstead of hardcoding-p. - Set
GINKGO_E2E_PROCS=--procs=8in the GitHub Actions workflow env to speed up CI.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| Makefile | Introduces GINKGO_E2E_PROCS and wires it into e2e-test and e2e-test-coverage Ginkgo invocations. |
| .github/workflows/makefile.yml | Sets GINKGO_E2E_PROCS in CI to oversubscribe Ginkgo processes for faster e2e runs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2085 +/- ##
=======================================
Coverage 86.21% 86.21%
=======================================
Files 112 112
Lines 8367 8367
=======================================
Hits 7214 7214
Misses 932 932
Partials 221 221 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
dadezzz
pushed a commit
to dadezzz/infra_docker-compose
that referenced
this pull request
Jun 20, 2026
This PR contains the following updates: | Package | Update | Change | |---|---|---| | [ghcr.io/0xerr0r/blocky](https://github.com/0xERR0R/blocky) | minor | `v0.31.0` → `v0.32.0` | --- ### Release Notes <details> <summary>0xERR0R/blocky (ghcr.io/0xerr0r/blocky)</summary> ### [`v0.32.0`](https://github.com/0xERR0R/blocky/releases/tag/v0.32.0) [Compare Source](0xERR0R/blocky@v0.31.0...v0.32.0) #### Changelog ##### Features - [`c851293`](0xERR0R/blocky@c851293): feat(blocking): log the matched rule in the block reason ([#​2091](0xERR0R/blocky#2091)) ([@​0xERR0R](https://github.com/0xERR0R)) - [`87be127`](0xERR0R/blocky@87be127): feat(cache): Shard the result cache to remove the single-lock read ceiling ([#​2097](0xERR0R/blocky#2097)) ([@​0xERR0R](https://github.com/0xERR0R)) - [`7c6da15`](0xERR0R/blocky@7c6da15): feat(config): structurally merge config folder files ([#​1827](0xERR0R/blocky#1827)) ([#​2112](0xERR0R/blocky#2112)) ([@​0xERR0R](https://github.com/0xERR0R)) - [`e7958e0`](0xERR0R/blocky@e7958e0): feat(lists): opt-in on-disk download cache with conditional revalidation ([#​2087](0xERR0R/blocky#2087)) ([@​0xERR0R](https://github.com/0xERR0R)) - [`b82199b`](0xERR0R/blocky@b82199b): feat(querylog): ignore domains (exact/wildcard/regex) in query log ([#​2084](0xERR0R/blocky#2084)) ([@​0xERR0R](https://github.com/0xERR0R)) - [`bee2d8b`](0xERR0R/blocky@bee2d8b): feat(resolver): UDP-first plain-DNS upstream and EDNS0 buffer floor ([#​2100](0xERR0R/blocky#2100)) ([@​0xERR0R](https://github.com/0xERR0R)) - [`0b70e5c`](0xERR0R/blocky@0b70e5c): feat(resolver): add DNS rebinding protection ([#​2111](0xERR0R/blocky#2111)) ([@​0xERR0R](https://github.com/0xERR0R)) - [`1b8e08a`](0xERR0R/blocky@1b8e08a): feat(resolver): pool DoT connections and enable TLS session resumption ([#​2098](0xERR0R/blocky#2098)) ([@​0xERR0R](https://github.com/0xERR0R)) - [`4018d9e`](0xERR0R/blocky@4018d9e): feat: in-memory statistics subsystem with /api/stats REST endpoint ([#​2093](0xERR0R/blocky#2093)) ([@​0xERR0R](https://github.com/0xERR0R)) - [`7abca44`](0xERR0R/blocky@7abca44): feat: support PROXY protocol on proxied DoT/DoH listeners ([#​2094](0xERR0R/blocky#2094)) ([@​kastakhov](https://github.com/kastakhov)) ##### Bug fixes - [`2496d12`](0xERR0R/blocky@2496d12): fix(dnssec): close DNSSEC validation bypass & cache-scope pollution (GHSA-x845-2f78-7v36) ([#​2119](0xERR0R/blocky#2119)) ([@​0xERR0R](https://github.com/0xERR0R)) - [`06555e0`](0xERR0R/blocky@06555e0): fix(metrics): bound reason label cardinality for blocked responses ([#​2114](0xERR0R/blocky#2114)) ([@​0xERR0R](https://github.com/0xERR0R)) - [`e0ea9b3`](0xERR0R/blocky@e0ea9b3): fix(resolver): eliminate recursive RLock deadlock in blocking group resolution ([#​2106](0xERR0R/blocky#2106)) ([@​0xERR0R](https://github.com/0xERR0R)) - [`80f742e`](0xERR0R/blocky@80f742e): fix(server): answer browser CORS preflights for custom headers and Private Network Access ([#​2109](0xERR0R/blocky#2109)) ([@​0xERR0R](https://github.com/0xERR0R)) - [`9702c8f`](0xERR0R/blocky@9702c8f): fix(stats): populate allow/denylist counts in /api/stats at startup ([#​2113](0xERR0R/blocky#2113)) ([@​0xERR0R](https://github.com/0xERR0R)) ##### Build and dependencies - [`821e378`](0xERR0R/blocky@821e378): build(deps): bump codecov/codecov-action from 6 to 7 ([#​2089](0xERR0R/blocky#2089)) ([@​dependabot](https://github.com/dependabot)\[bot]) - [`d63f84b`](0xERR0R/blocky@d63f84b): build(deps): bump github.com/0xERR0R/expiration-cache from 0.1.0 to 0.2.0 ([#​2095](0xERR0R/blocky#2095)) ([@​dependabot](https://github.com/dependabot)\[bot]) - [`81f2b2a`](0xERR0R/blocky@81f2b2a): build(deps): bump github.com/onsi/ginkgo/v2 from 2.29.0 to 2.30.0 ([#​2108](0xERR0R/blocky#2108)) ([@​dependabot](https://github.com/dependabot)\[bot]) - [`9c5abdf`](0xERR0R/blocky@9c5abdf): build(deps): bump github.com/onsi/ginkgo/v2 from 2.30.0 to 2.31.0 ([#​2116](0xERR0R/blocky#2116)) ([@​dependabot](https://github.com/dependabot)\[bot]) - [`6c0ec5f`](0xERR0R/blocky@6c0ec5f): build(deps): bump github.com/onsi/gomega from 1.41.0 to 1.42.0 ([#​2115](0xERR0R/blocky#2115)) ([@​dependabot](https://github.com/dependabot)\[bot]) - [`4beb9fe`](0xERR0R/blocky@4beb9fe): build(deps): bump github.com/quic-go/quic-go from 0.59.1 to 0.60.0 ([#​2090](0xERR0R/blocky#2090)) ([@​dependabot](https://github.com/dependabot)\[bot]) - [`8fe2202`](0xERR0R/blocky@8fe2202): build(deps): bump golang.org/x/net from 0.55.0 to 0.56.0 ([#​2101](0xERR0R/blocky#2101)) ([@​dependabot](https://github.com/dependabot)\[bot]) - [`52fa6a4`](0xERR0R/blocky@52fa6a4): build(deps): bump golang.org/x/sys from 0.45.0 to 0.46.0 ([#​2096](0xERR0R/blocky#2096)) ([@​dependabot](https://github.com/dependabot)\[bot]) ##### Misc - [`1c9f717`](0xERR0R/blocky@1c9f717): ci(e2e): speed up e2e job via Ginkgo proc oversubscription ([#​2085](0xERR0R/blocky#2085)) ([@​0xERR0R](https://github.com/0xERR0R)) - [`8d49922`](0xERR0R/blocky@8d49922): ci: cross-compile multi-arch docker image instead of QEMU emulation ([#​2081](0xERR0R/blocky#2081)) ([@​0xERR0R](https://github.com/0xERR0R)) - [`355a9a1`](0xERR0R/blocky@355a9a1): docs(grafana): redesign Grafana dashboard and update integration guide ([#​2110](0xERR0R/blocky#2110)) ([@​0xERR0R](https://github.com/0xERR0R)) - [`c791e80`](0xERR0R/blocky@c791e80): perf(e2e): probe blocky healthcheck at 250ms during start period ([#​2086](0xERR0R/blocky#2086)) ([@​0xERR0R](https://github.com/0xERR0R)) - [`a0fb375`](0xERR0R/blocky@a0fb375): perf(lists): speed up blocklist loading (cache build + parsing) ([#​2083](0xERR0R/blocky#2083)) ([@​0xERR0R](https://github.com/0xERR0R)) - [`316b073`](0xERR0R/blocky@316b073): perf(resolver): pre-classify client groups to cut per-query allocations ([#​2103](0xERR0R/blocky#2103)) ([@​0xERR0R](https://github.com/0xERR0R)) - [`d502806`](0xERR0R/blocky@d502806): perf(server): let Truncate decide response compression ([#​2102](0xERR0R/blocky#2102)) ([@​0xERR0R](https://github.com/0xERR0R)) - [`8b32512`](0xERR0R/blocky@8b32512): test(e2e): de-flake PROXY protocol and upstream init.strategy specs ([#​2099](0xERR0R/blocky#2099)) ([@​0xERR0R](https://github.com/0xERR0R)) *** #### ❤️ Support Blocky Blocky is free and open source, developed in my spare time — no telemetry, no ads, no hidden filtering. If it's useful to you, please consider supporting its continued development: [GitHub Sponsors](https://github.com/sponsors/0xERR0R) · [thanks.dev](https://thanks.dev/u/gh/0xERR0R) · [Liberapay](https://liberapay.com/spx01) · [Ko-fi](https://ko-fi.com/0xerr0r) · [PayPal](https://paypal.me/spx01) Thank you to everyone supporting Blocky! 🙏 </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - At any time (no schedule defined) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4yMjIuMSIsInVwZGF0ZWRJblZlciI6IjQzLjIyMi4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The
e2e-testtarget runs Ginkgo with-p(one process per core). e2e specsare dominated by container startup and health-check waits, not CPU — across
the suite, ~1663s of summed spec runtime compresses to ~244s wall-clock, and
even trivial specs cost 15–30s because each one spins up a fresh Docker network
BeforeEach. That means we can safely runmore Ginkgo processes than the runner has cores.
This adds a
GINKGO_E2E_PROCSknob (default-p, so local runs are unchanged)and sets
--procs=8in CI to oversubscribe the 4-coreubuntu-latestrunner.Measured impact (CI
make (e2e-test)job)-p)--procs=8Verified across 3 CI runs: 137/137 specs pass, 0 skips, 0 flakes — the higher
parallelism does not trigger any IPv6 fixed-subnet / network-collision skips.
Local reference (8-core): 244s (
-p) → 185s (--procs=16).Notes
GINKGO_E2E_PROCSdefaults to-p); only CIoverrides it.
GINKGO_PROCS: --procs=1in CI only ever applied to thetest/racetargets, never to e2e — so this is not a regression of any prior serialization.
type=ghaDocker build cache but dropped it: it silentlyno-ops with a raw
docker buildx buildin a plainrun:step (the backendneeds
ACTIONS_RUNTIME_TOKEN, which onlydocker/build-push-actioninjects),and the e2e image build is mostly per-commit recompile + image load anyway.
Please squash-merge (the branch has an extra throwaway commit from
measuring warm-cache build times).