Skip to content

Have JDS ping local mempool less frequently#1299

Merged
GitGab19 merged 1 commit into
stratum-mining:mainfrom
Sjors:2024/12/chill
Dec 16, 2024
Merged

Have JDS ping local mempool less frequently#1299
GitGab19 merged 1 commit into
stratum-mining:mainfrom
Sjors:2024/12/chill

Conversation

@Sjors

@Sjors Sjors commented Dec 16, 2024

Copy link
Copy Markdown
Collaborator

Otherwise Bitcoin Core -debug=rpc logging becomes too noisy.

It's probably also too high for production on mainnet, "distracting" the node from its other work, but that's something that benchmarking might find.

@codecov

codecov Bot commented Dec 16, 2024

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 19.30%. Comparing base (cf4c625) to head (3a1da94).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1299   +/-   ##
=======================================
  Coverage   19.30%   19.30%           
=======================================
  Files         164      164           
  Lines       10849    10849           
=======================================
  Hits         2094     2094           
  Misses       8755     8755           
Flag Coverage Δ
binary_codec_sv2-coverage 0.00% <ø> (ø)
binary_serde_sv2-coverage 3.65% <ø> (ø)
binary_sv2-coverage 5.48% <ø> (ø)
bip32_derivation-coverage 0.00% <ø> (ø)
buffer_sv2-coverage 25.02% <ø> (ø)
codec_sv2-coverage 0.01% <ø> (ø)
common_messages_sv2-coverage 0.13% <ø> (ø)
const_sv2-coverage 0.00% <ø> (ø)
error_handling-coverage ?
framing_sv2-coverage 0.29% <ø> (ø)
jd_client-coverage 0.00% <ø> (ø)
jd_server-coverage 7.79% <ø> (ø)
job_declaration_sv2-coverage 0.00% <ø> (ø)
key-utils-coverage 2.39% <ø> (ø)
mining-coverage 2.49% <ø> (-0.02%) ⬇️
mining_device-coverage 0.00% <ø> (ø)
mining_proxy_sv2-coverage 0.70% <ø> (ø)
noise_sv2-coverage 4.35% <ø> (ø)
pool_sv2-coverage 1.38% <ø> (ø)
protocols 24.72% <ø> (ø)
roles 6.55% <ø> (ø)
roles_logic_sv2-coverage 8.08% <ø> (ø)
sv2_ffi-coverage 0.00% <ø> (ø)
template_distribution_sv2-coverage 0.00% <ø> (ø)
translator_sv2-coverage 9.60% <ø> (ø)
utils 25.13% <ø> (ø)
v1-coverage 2.47% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@github-actions

github-actions Bot commented Dec 16, 2024

Copy link
Copy Markdown
Contributor

🐰 Bencher Report

Branch2024/12/chill
Testbedsv1
Click to view all benchmark results
BenchmarkEstimated CyclesBenchmark Result
estimated cycles
(Result Δ%)
Upper Boundary
estimated cycles
(Limit %)
InstructionsBenchmark Result
instructions
(Result Δ%)
Upper Boundary
instructions
(Limit %)
L1 AccessesBenchmark Result
accesses
(Result Δ%)
Upper Boundary
accesses
(Limit %)
L2 AccessesBenchmark Result
accesses
(Result Δ%)
Upper Boundary
accesses
(Limit %)
RAM AccessesBenchmark Result
accesses
(Result Δ%)
Upper Boundary
accesses
(Limit %)
get_authorize📈 view plot
🚷 view threshold
8,482.00
(+0.24%)
8,683.75
(97.68%)
📈 view plot
🚷 view threshold
3,663.00
(-2.06%)
3,861.71
(94.85%)
📈 view plot
🚷 view threshold
5,107.00
(-2.54%)
5,446.56
(93.77%)
📈 view plot
🚷 view threshold
10.00
(+30.88%)
14.85
(67.36%)
📈 view plot
🚷 view threshold
95.00
(+4.44%)
95.68
(99.28%)
get_submit📈 view plot
🚷 view threshold
95,366.00
(-0.04%)
95,794.73
(99.55%)
📈 view plot
🚷 view threshold
59,263.00
(-0.29%)
59,713.04
(99.25%)
📈 view plot
🚷 view threshold
85,071.00
(-0.35%)
85,823.80
(99.12%)
📈 view plot
🚷 view threshold
50.00
(+13.15%)
59.28
(84.34%)
📈 view plot
🚷 view threshold
287.00
(+2.34%)
290.08
(98.94%)
get_subscribe📈 view plot
🚷 view threshold
7,961.00
(-0.66%)
8,233.64
(96.69%)
📈 view plot
🚷 view threshold
2,758.00
(-2.41%)
2,939.88
(93.81%)
📈 view plot
🚷 view threshold
3,831.00
(-2.93%)
4,137.63
(92.59%)
📈 view plot
🚷 view threshold
14.00
(+18.91%)
19.41
(72.11%)
📈 view plot
🚷 view threshold
116.00
(+1.28%)
117.76
(98.51%)
serialize_authorize📈 view plot
🚷 view threshold
12,271.00
(-0.05%)
12,518.09
(98.03%)
📈 view plot
🚷 view threshold
5,240.00
(-1.36%)
5,426.70
(96.56%)
📈 view plot
🚷 view threshold
7,276.00
(-1.72%)
7,601.99
(95.71%)
📈 view plot
🚷 view threshold
12.00
(+26.52%)
17.80
(67.43%)
📈 view plot
🚷 view threshold
141.00
(+2.26%)
142.09
(99.23%)
serialize_deserialize_authorize📈 view plot
🚷 view threshold
24,640.00
(-0.28%)
25,182.01
(97.85%)
📈 view plot
🚷 view threshold
9,786.00
(-0.93%)
10,033.57
(97.53%)
📈 view plot
🚷 view threshold
13,785.00
(-1.12%)
14,199.86
(97.08%)
📈 view plot
🚷 view threshold
36.00
(+3.00%)
44.99
(80.03%)
📈 view plot
🚷 view threshold
305.00
(+0.77%)
313.66
(97.24%)
serialize_deserialize_handle_authorize📈 view plot
🚷 view threshold
30,265.00
(-0.22%)
30,706.91
(98.56%)
📈 view plot
🚷 view threshold
11,989.00
(-0.66%)
12,202.27
(98.25%)
📈 view plot
🚷 view threshold
16,950.00
(-0.81%)
17,308.97
(97.93%)
📈 view plot
🚷 view threshold
59.00
(+6.31%)
66.96
(88.11%)
📈 view plot
🚷 view threshold
372.00
(+0.42%)
379.79
(97.95%)
serialize_deserialize_handle_submit📈 view plot
🚷 view threshold
126,460.00
(-0.01%)
126,786.40
(99.74%)
📈 view plot
🚷 view threshold
73,117.00
(-0.21%)
73,518.09
(99.45%)
📈 view plot
🚷 view threshold
104,760.00
(-0.27%)
105,475.87
(99.32%)
📈 view plot
🚷 view threshold
112.00
(+5.94%)
124.25
(90.14%)
📈 view plot
🚷 view threshold
604.00
(+1.12%)
606.97
(99.51%)
serialize_deserialize_handle_subscribe📈 view plot
🚷 view threshold
27,871.00
(-0.06%)
28,475.23
(97.88%)
📈 view plot
🚷 view threshold
9,577.00
(-0.68%)
9,752.11
(98.20%)
📈 view plot
🚷 view threshold
13,516.00
(-0.87%)
13,823.69
(97.77%)
📈 view plot
🚷 view threshold
71.00
(+11.66%)
76.05
(93.36%)
📈 view plot
🚷 view threshold
400.00
(+0.48%)
413.06
(96.84%)
serialize_deserialize_submit📈 view plot
🚷 view threshold
115,189.00
(-0.06%)
115,735.50
(99.53%)
📈 view plot
🚷 view threshold
67,894.00
(-0.28%)
68,416.93
(99.24%)
📈 view plot
🚷 view threshold
97,354.00
(-0.36%)
98,277.51
(99.06%)
📈 view plot
🚷 view threshold
74.00
(+16.92%)
82.28
(89.94%)
📈 view plot
🚷 view threshold
499.00
(+1.33%)
500.25
(99.75%)
serialize_deserialize_subscribe📈 view plot
🚷 view threshold
23,244.00
(-0.26%)
23,871.94
(97.37%)
📈 view plot
🚷 view threshold
8,129.00
(-0.84%)
8,308.92
(97.83%)
📈 view plot
🚷 view threshold
11,424.00
(-1.03%)
11,731.89
(97.38%)
📈 view plot
🚷 view threshold
40.00
(+5.98%)
47.77
(83.73%)
📈 view plot
🚷 view threshold
332.00
(+0.42%)
344.48
(96.38%)
serialize_submit📈 view plot
🚷 view threshold
99,831.00
(+0.00%)
100,228.04
(99.60%)
📈 view plot
🚷 view threshold
61,325.00
(-0.26%)
61,740.37
(99.33%)
📈 view plot
🚷 view threshold
87,926.00
(-0.32%)
88,652.98
(99.18%)
📈 view plot
🚷 view threshold
57.00
(+22.13%)
63.82
(89.32%)
📈 view plot
🚷 view threshold
332.00
(+2.09%)
335.31
(99.01%)
serialize_subscribe📈 view plot
🚷 view threshold
11,444.00
(+0.30%)
11,620.35
(98.48%)
📈 view plot
🚷 view threshold
4,111.00
(-1.52%)
4,280.43
(96.04%)
📈 view plot
🚷 view threshold
5,689.00
(-1.98%)
5,990.03
(94.97%)
📈 view plot
🚷 view threshold
17.00
(+30.93%)
21.54
(78.94%)
📈 view plot
🚷 view threshold
162.00
(+2.33%)
163.38
(99.16%)
🐰 View full continuous benchmarking report in Bencher

@github-actions

github-actions Bot commented Dec 16, 2024

Copy link
Copy Markdown
Contributor

🐰 Bencher Report

Branch2024/12/chill
Testbedsv1
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)
Upper Boundary
nanoseconds (ns)
(Limit %)
client-submit-serialize📈 view plot
🚷 view threshold
6,529.90
(-1.58%)
7,080.45
(92.22%)
client-submit-serialize-deserialize📈 view plot
🚷 view threshold
7,387.00
(-1.53%)
8,088.04
(91.33%)
client-submit-serialize-deserialize-handle/client-submit-serialize-deserialize-handle📈 view plot
🚷 view threshold
7,999.80
(-1.34%)
8,574.63
(93.30%)
client-sv1-authorize-serialize-deserialize-handle/client-sv1-authorize-serialize-deserialize-handle📈 view plot
🚷 view threshold
860.94
(-0.40%)
923.37
(93.24%)
client-sv1-authorize-serialize-deserialize/client-sv1-authorize-serialize-deserialize📈 view plot
🚷 view threshold
670.12
(-0.13%)
703.07
(95.31%)
client-sv1-authorize-serialize/client-sv1-authorize-serialize📈 view plot
🚷 view threshold
249.11
(+0.36%)
268.48
(92.79%)
client-sv1-get-authorize/client-sv1-get-authorize📈 view plot
🚷 view threshold
155.91
(-0.68%)
164.00
(95.07%)
client-sv1-get-submit📈 view plot
🚷 view threshold
6,341.30
(-1.48%)
7,050.96
(89.94%)
client-sv1-get-subscribe/client-sv1-get-subscribe📈 view plot
🚷 view threshold
275.66
(-1.42%)
304.21
(90.61%)
client-sv1-subscribe-serialize-deserialize-handle/client-sv1-subscribe-serialize-deserialize-handle📈 view plot
🚷 view threshold
742.23
(+2.01%)
780.83
(95.06%)
client-sv1-subscribe-serialize-deserialize/client-sv1-subscribe-serialize-deserialize📈 view plot
🚷 view threshold
582.69
(-1.18%)
631.88
(92.21%)
client-sv1-subscribe-serialize/client-sv1-subscribe-serialize📈 view plot
🚷 view threshold
205.72
(-0.08%)
216.64
(94.96%)
🐰 View full continuous benchmarking report in Bencher

@github-actions

github-actions Bot commented Dec 16, 2024

Copy link
Copy Markdown
Contributor

🐰 Bencher Report

Branch2024/12/chill
Testbedsv2
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)
Upper Boundary
nanoseconds (ns)
(Limit %)
client_sv2_handle_message_common📈 view plot
🚷 view threshold
44.85
(+0.63%)
46.69
(96.05%)
client_sv2_handle_message_mining📈 view plot
🚷 view threshold
74.03
(-3.72%)
99.78
(74.20%)
client_sv2_mining_message_submit_standard📈 view plot
🚷 view threshold
14.67
(+0.05%)
14.76
(99.38%)
client_sv2_mining_message_submit_standard_serialize📈 view plot
🚷 view threshold
283.80
(+6.12%)
302.80
(93.72%)
client_sv2_mining_message_submit_standard_serialize_deserialize📈 view plot
🚷 view threshold
619.96
(+0.49%)
667.22
(92.92%)
client_sv2_open_channel📈 view plot
🚷 view threshold
169.98
(+4.57%)
188.88
(89.99%)
client_sv2_open_channel_serialize📈 view plot
🚷 view threshold
297.60
(+4.44%)
313.43
(94.95%)
client_sv2_open_channel_serialize_deserialize📈 view plot
🚷 view threshold
378.35
(-1.41%)
412.04
(91.82%)
client_sv2_setup_connection📈 view plot
🚷 view threshold
183.53
(+13.55%)
185.01
(99.20%)
client_sv2_setup_connection_serialize📈 view plot
🚷 view threshold
472.55
(+0.42%)
540.08
(87.50%)
client_sv2_setup_connection_serialize_deserialize📈 view plot
🚷 view threshold
985.14
(-1.07%)
1,073.73
(91.75%)
🐰 View full continuous benchmarking report in Bencher

@github-actions

github-actions Bot commented Dec 16, 2024

Copy link
Copy Markdown
Contributor

🐰 Bencher Report

Branch2024/12/chill
Testbedsv2
Click to view all benchmark results
BenchmarkEstimated CyclesBenchmark Result
estimated cycles
(Result Δ%)
Upper Boundary
estimated cycles
(Limit %)
InstructionsBenchmark Result
instructions
(Result Δ%)
Upper Boundary
instructions
(Limit %)
L1 AccessesBenchmark Result
accesses
(Result Δ%)
Upper Boundary
accesses
(Limit %)
L2 AccessesBenchmark Result
accesses
(Result Δ%)
Upper Boundary
accesses
(Limit %)
RAM AccessesBenchmark Result
accesses
(Result Δ%)
Upper Boundary
accesses
(Limit %)
client_sv2_handle_message_common📈 view plot
🚷 view threshold
2,205.00
(+4.75%)
2,222.07
(99.23%)
📈 view plot
🚷 view threshold
473.00
(-0.07%)
488.90
(96.75%)
📈 view plot
🚷 view threshold
730.00
(-0.78%)
757.76
(96.34%)
📈 view plot
🚷 view threshold
8.00
(+54.45%)
13.22
(60.52%)
📈 view plot
🚷 view threshold
41.00
(+6.82%)
41.52
(98.76%)
client_sv2_handle_message_mining📈 view plot
🚷 view threshold
8,284.00
(+0.84%)
8,363.01
(99.06%)
📈 view plot
🚷 view threshold
2,137.00
(-0.03%)
2,143.08
(99.72%)
📈 view plot
🚷 view threshold
3,154.00
(-0.19%)
3,171.27
(99.46%)
📈 view plot
🚷 view threshold
39.00
(+10.08%)
41.50
(93.98%)
📈 view plot
🚷 view threshold
141.00
(+1.17%)
143.46
(98.29%)
client_sv2_mining_message_submit_standard📈 view plot
🚷 view threshold
6,365.00
(+1.11%)
6,437.57
(98.87%)
📈 view plot
🚷 view threshold
1,750.00
(-0.05%)
1,767.24
(99.02%)
📈 view plot
🚷 view threshold
2,545.00
(-0.30%)
2,576.07
(98.79%)
📈 view plot
🚷 view threshold
22.00
(+28.94%)
24.48
(89.86%)
📈 view plot
🚷 view threshold
106.00
(+1.45%)
108.45
(97.74%)
client_sv2_mining_message_submit_standard_serialize📈 view plot
🚷 view threshold
14,824.00
(+0.66%)
14,948.28
(99.17%)
📈 view plot
🚷 view threshold
4,694.00
(-0.02%)
4,711.24
(99.63%)
📈 view plot
🚷 view threshold
6,744.00
(-0.20%)
6,785.37
(99.39%)
📈 view plot
🚷 view threshold
55.00
(+24.27%)
58.05
(94.75%)
📈 view plot
🚷 view threshold
223.00
(+0.74%)
226.95
(98.26%)
client_sv2_mining_message_submit_standard_serialize_deserialize📈 view plot
🚷 view threshold
27,792.00
(+0.82%)
27,948.21
(99.44%)
📈 view plot
🚷 view threshold
10,645.00
(+0.46%)
10,675.29
(99.72%)
📈 view plot
🚷 view threshold
15,502.00
(+0.51%)
15,554.70
(99.66%)
📈 view plot
🚷 view threshold
92.00
(+11.86%)
94.31
(97.55%)
📈 view plot
🚷 view threshold
338.00
(+0.83%)
342.17
(98.78%)
client_sv2_open_channel📈 view plot
🚷 view threshold
4,523.00
(+3.09%)
4,556.50
(99.26%)
📈 view plot
🚷 view threshold
1,461.00
(-0.02%)
1,476.90
(98.92%)
📈 view plot
🚷 view threshold
2,153.00
(-0.35%)
2,183.60
(98.60%)
📈 view plot
🚷 view threshold
12.00
(+43.55%)
15.04
(79.79%)
📈 view plot
🚷 view threshold
66.00
(+5.72%)
67.19
(98.23%)
client_sv2_open_channel_serialize📈 view plot
🚷 view threshold
14,116.00
(+0.67%)
14,234.76
(99.17%)
📈 view plot
🚷 view threshold
5,064.00
(-0.01%)
5,079.90
(99.69%)
📈 view plot
🚷 view threshold
7,316.00
(-0.14%)
7,351.09
(99.52%)
📈 view plot
🚷 view threshold
44.00
(+24.05%)
45.38
(96.95%)
📈 view plot
🚷 view threshold
188.00
(+0.94%)
192.39
(97.72%)
client_sv2_open_channel_serialize_deserialize📈 view plot
🚷 view threshold
22,888.00
(+0.94%)
22,962.28
(99.68%)
📈 view plot
🚷 view threshold
8,040.00
(+0.14%)
8,058.67
(99.77%)
📈 view plot
🚷 view threshold
11,688.00
(+0.06%)
11,722.89
(99.70%)
📈 view plot
🚷 view threshold
84.00
(+12.28%)
86.28
(97.35%)
📈 view plot
🚷 view threshold
308.00
(+1.51%)
310.67
(99.14%)
client_sv2_setup_connection📈 view plot
🚷 view threshold
4,749.00
(+1.33%)
4,786.30
(99.22%)
📈 view plot
🚷 view threshold
1,502.00
(-0.02%)
1,517.90
(98.95%)
📈 view plot
🚷 view threshold
2,274.00
(-0.18%)
2,299.41
(98.89%)
📈 view plot
🚷 view threshold
12.00
(+23.87%)
16.52
(72.65%)
📈 view plot
🚷 view threshold
69.00
(+2.33%)
70.10
(98.44%)
client_sv2_setup_connection_serialize📈 view plot
🚷 view threshold
16,264.00
(+0.70%)
16,341.40
(99.53%)
📈 view plot
🚷 view threshold
5,963.00
(-0.01%)
5,978.90
(99.73%)
📈 view plot
🚷 view threshold
8,649.00
(-0.17%)
8,691.26
(99.51%)
📈 view plot
🚷 view threshold
53.00
(+32.63%)
53.91
(98.30%)
📈 view plot
🚷 view threshold
210.00
(+0.87%)
213.00
(98.59%)
client_sv2_setup_connection_serialize_deserialize📈 view plot
🚷 view threshold
35,786.00
(+0.61%)
35,855.74
(99.81%)
📈 view plot
🚷 view threshold
14,888.00
(+0.18%)
14,909.35
(99.86%)
📈 view plot
🚷 view threshold
21,866.00
(+0.15%)
21,912.10
(99.79%)
📈 view plot
🚷 view threshold
110.00
(+19.10%)
112.55
(97.74%)
📈 view plot
🚷 view threshold
382.00
(+0.72%)
384.94
(99.24%)
🐰 View full continuous benchmarking report in Bencher

@GitGab19

Copy link
Copy Markdown
Member

nACK on setting it to 5s, since it's too much. In that way, we are going to have an outdated mempool, leading to a lot ProvideMissingTransactions/ProvideMissingTransactions.Success just because of that.

I'm running some tests to find the most accurate value, I'll keep you posted

@Sjors

Sjors commented Dec 16, 2024

Copy link
Copy Markdown
Collaborator Author

Do you expect anyone to run the local config example on mainnet? Because on testnet4 and the custom signet the transaction frequency is pretty low.

@GitGab19

Copy link
Copy Markdown
Member

Do you expect anyone to run the local config example on mainnet? Because on testnet4 and the custom signet the transaction frequency is pretty low.

We are doing it in the benchmarking-tool, which leads to an increased bandwidth usage because of ProvideMissingTransactions/ProvideMissingTransactions.Success messages.

@Sjors

Sjors commented Dec 16, 2024

Copy link
Copy Markdown
Collaborator Author

Maybe the benchmarking-tool should have its own config file though?

@GitGab19

GitGab19 commented Dec 16, 2024

Copy link
Copy Markdown
Member

Yes, it already has its own config file, my point was that we shouldn't put an high value because otherwise we are going to have too many PMT messages. I'm ok with putting the value back to 1s at most, but not 5s.

Otherwise Bitcoin Core -debug=rpc logging becomes too noisy.
@Sjors

Sjors commented Dec 16, 2024

Copy link
Copy Markdown
Collaborator Author

Ok, changed to 1s.

[mempool_update_interval]
unit = "secs"
value = 0.1
value = 5

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think @GitGab19 did some experiments and found an optimal value for this

#1292 (comment)

Suggested change
value = 5
value = 0.5

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A typical terminal on macOS is just 25 lines, so you can't see any other log messages, like SubmitSolution after just 12 seconds with that default.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1s is perfect to me

@Shourya742 Shourya742 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK

@Shourya742 Shourya742 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tack

@GitGab19 GitGab19 merged commit e750e1d into stratum-mining:main Dec 16, 2024
@Sjors Sjors deleted the 2024/12/chill branch December 16, 2024 12:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants