Skip to content

transport: avoid allocating a closure for writeQuota.replenish on every stream - #9493

Open
arjan-bal wants to merge 1 commit into
grpc:masterfrom
arjan-bal:heap-alloc
Open

arjan-bal wants to merge 1 commit into
grpc:masterfrom
arjan-bal:heap-alloc

Conversation

@arjan-bal

Copy link
Copy Markdown
Contributor

writeQuota.init stored the bound method value w.realReplenish in the replenish func field. Taking a method value that captures its receiver allocates a closure, so this cost one heap allocation per stream on both the client and the server, i.e. 2 allocations per unary RPC. The field only existed so that a test could observe replenishment; production code calls it in a single place (loopyWriter.processData).

This PR makes replenish a regular method and replaces the overridable field with an optional, test-only observer, onReplenishForTesting, which is nil in production and is called in addition to (not instead of) the real flow-control update. TestLargeMessageWithDelayRead, the only user of the old override, is updated to use the hook.

Benchmarks

benchmain with -bufconn -networkMode=Local -compression=off -benchtime=60s. Each round runs master and this PR at the same time.

Unary, 100 concurrent calls, 0-byte payloads

Round QPS Avg latency p99 latency Bytes/op Allocs/op
1 +0.48% -0.49% +0.17% 9324 → 9293 135.19 → 133.19
2 +0.34% -0.32% -1.35% 9325 → 9292 135.19 → 133.19
3 +0.97% -0.95% -1.42% 9325 → 9289 135.19 → 133.19
Mean +0.60% -0.59% -0.87% -33 B (-0.36%) -2.00 (-1.48%)

Streaming, 200 concurrent calls, 100-byte payloads

Round QPS Avg latency p99 latency Bytes/op Allocs/op
1 -0.40% +0.39% +1.09% 0.00% 0.00%
2 -0.20% +0.13% +1.62% 0.00% 0.00%
3 +1.17% -1.32% -0.66% 0.00% 0.00%
Mean +0.19% -0.27% +0.68% 0.00% 0.00%

Streaming is unaffected as expected: streams are long-lived, so the per-stream allocation was already amortized away. A heap profile of the unary benchmark confirms that writeQuota.init now only allocates its notification channel.

RELEASE NOTES: None

@arjan-bal arjan-bal added this to the 1.86 Release milestone Oct 9, 2026
@arjan-bal arjan-bal added Type: Performance Performance improvements (CPU, network, memory, etc) Area: Transport Includes HTTP/2 client/server and HTTP server handler transports and advanced transport features. labels Oct 9, 2026
@codecov

codecov Bot commented Oct 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.72%. Comparing base (39ebc82) to head (59dafb8).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #9493      +/-   ##
==========================================
+ Coverage   87.71%   87.72%   +0.01%     
==========================================
  Files         436      436              
  Lines       30822    30823       +1     
==========================================
+ Hits        27036    27040       +4     
+ Misses       3785     3783       -2     
+ Partials        1        0       -1     
Files with missing lines Coverage Δ
internal/transport/flowcontrol.go 98.63% <100.00%> (+0.01%) ⬆️

... and 24 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@easwars

easwars commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

I understand that the part of invoking a callback defined by tests from non-test code is existing art. But it makes me a bit uncomfortable though. I think we can change this piece of code to not have this kind of coupling with the test code.

The only thing that the test needs it the actual amount of quota that we being replenished. Based on the tests, this value is completely computable in the test. If the test does that, then it can atomically read the quota value and compare it against the expected value. And if we do this, we can completely also get rid of the replenish field in writeQuota.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: Transport Includes HTTP/2 client/server and HTTP server handler transports and advanced transport features. Type: Performance Performance improvements (CPU, network, memory, etc)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants