Skip to content

Sniffer Improvements#1258

Merged
plebhash merged 2 commits into
stratum-mining:mainfrom
jbesraa:2024-11-18-sniffer-improvements
Nov 20, 2024
Merged

Sniffer Improvements#1258
plebhash merged 2 commits into
stratum-mining:mainfrom
jbesraa:2024-11-18-sniffer-improvements

Conversation

@jbesraa

@jbesraa jbesraa commented Nov 18, 2024

Copy link
Copy Markdown
Contributor

Partially resolves #1253

@jbesraa jbesraa marked this pull request as ready for review November 18, 2024 09:21
@codecov

codecov Bot commented Nov 18, 2024 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 19.30%. Comparing base (0520c58) to head (27eec57).
Report is 3 commits behind head on main.

Additional details and impacted files 
@@           Coverage Diff           @@
##             main    #1258   +/-   ##
=======================================
  Coverage   19.30%   19.30%           
=======================================
  Files         164      164           
  Lines       10849    10849           
=======================================
  Hits         2094     2094           
  Misses       8755     8755
Flag Coverage Δ
binary_codec_sv2-coverage 0.00% <ø> (ø)
binary_serde_sv2-coverage 3.65% <ø> (ø)
binary_sv2-coverage 5.46% <ø> (ø)
bip32_derivation-coverage 0.00% <ø> (ø)
buffer_sv2-coverage 25.02% <ø> (ø)
codec_sv2-coverage 0.01% <ø> (ø)
common_messages_sv2-coverage 0.13% <ø> (ø)
const_sv2-coverage 0.00% <ø> (ø)
error_handling-coverage 0.00% <ø> (ø)
framing_sv2-coverage 0.29% <ø> (ø)
jd_client-coverage 0.00% <ø> (ø)
jd_server-coverage 7.79% <ø> (ø)
job_declaration_sv2-coverage 0.00% <ø> (ø)
key-utils-coverage 2.39% <ø> (ø)
mining-coverage 2.51% <ø> (+0.01%) ⬆️
mining_device-coverage 0.00% <ø> (ø)
mining_proxy_sv2-coverage 0.70% <ø> (ø)
noise_sv2-coverage 4.35% <ø> (ø)
pool_sv2-coverage 1.38% <ø> (ø)
protocols 24.72% <ø> (ø)
roles 6.55% <ø> (ø)
roles_logic_sv2-coverage 8.06% <ø> (ø)
sv1-mining-device-coverage 0.00% <ø> (ø)
sv2_ffi-coverage 0.00% <ø> (ø)
template_distribution_sv2-coverage 0.00% <ø> (ø)
translator_sv2-coverage 9.60% <ø> (ø)
utils 25.13% <ø> (ø)
v1-coverage 2.47% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚨 Try these New Features:

@github-actions

github-actions Bot commented Nov 18, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor

🐰 Bencher Report

Branch2024-11-18-sniffer-improvements
Testbedsv1
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)		Upper Boundary
nanoseconds (ns)
(Limit %)
client-submit-serialize📈 view plot
🚷 view threshold		6,949.60
(+3.22%)		7,247.78
(95.89%)
client-submit-serialize-deserialize📈 view plot
🚷 view threshold		7,826.20
(+2.22%)		8,059.21
(97.11%)
client-submit-serialize-deserialize-handle/client-submit-serialize-deserialize-handle📈 view plot
🚷 view threshold		8,426.00
(+2.54%)		8,711.49
(96.72%)
client-sv1-authorize-serialize-deserialize-handle/client-sv1-authorize-serialize-deserialize-handle📈 view plot
🚷 view threshold		879.03
(+0.44%)		939.73
(93.54%)
client-sv1-authorize-serialize-deserialize/client-sv1-authorize-serialize-deserialize📈 view plot
🚷 view threshold		678.27
(+0.73%)		716.19
(94.71%)
client-sv1-authorize-serialize/client-sv1-authorize-serialize📈 view plot
🚷 view threshold		246.88
(-0.56%)		261.23
(94.51%)
client-sv1-get-authorize/client-sv1-get-authorize📈 view plot
🚷 view threshold		158.15
(+0.34%)		163.21
(96.90%)
client-sv1-get-submit📈 view plot
🚷 view threshold		6,730.60
(+3.27%)		7,242.95
(92.93%)
client-sv1-get-subscribe/client-sv1-get-subscribe📈 view plot
🚷 view threshold		288.51
(+3.40%)		305.45
(94.45%)
client-sv1-subscribe-serialize-deserialize-handle/client-sv1-subscribe-serialize-deserialize-handle📈 view plot
🚷 view threshold		751.24
(+1.78%)		815.21
(92.15%)
client-sv1-subscribe-serialize-deserialize/client-sv1-subscribe-serialize-deserialize📈 view plot
🚷 view threshold		590.85
(-1.34%)		644.25
(91.71%)
client-sv1-subscribe-serialize/client-sv1-subscribe-serialize📈 view plot
🚷 view threshold		201.78
(-2.16%)		225.26
(89.58%)
🐰 View full continuous benchmarking report in Bencher

@github-actions

github-actions Bot commented Nov 18, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor

🐰 Bencher Report

Branch2024-11-18-sniffer-improvements
Testbedsv2
Click to view all benchmark results
BenchmarkEstimated CyclesBenchmark Result
estimated cycles
(Result Δ%)		Upper Boundary
estimated cycles
(Limit %)		InstructionsBenchmark Result
instructions
(Result Δ%)		Upper Boundary
instructions
(Limit %)		L1 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		L2 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		RAM AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)
client_sv2_handle_message_common📈 view plot
🚷 view threshold		2,069.00
(-1.18%)		2,206.26
(93.78%)		📈 view plot
🚷 view threshold		473.00
(-0.07%)		488.51
(96.82%)		📈 view plot
🚷 view threshold		734.00
(-0.26%)		757.27
(96.93%)		📈 view plot
🚷 view threshold		8.00
(+47.34%)		14.07
(56.86%)		📈 view plot
🚷 view threshold		37.00
(-2.67%)		41.17
(89.87%)
client_sv2_handle_message_mining📈 view plot
🚷 view threshold		8,208.00
(+0.05%)		8,285.19
(99.07%)		📈 view plot
🚷 view threshold		2,137.00
(-0.03%)		2,143.08
(99.72%)		📈 view plot
🚷 view threshold		3,158.00
(-0.09%)		3,170.61
(99.60%)		📈 view plot
🚷 view threshold		37.00
(+4.09%)		42.15
(87.79%)		📈 view plot
🚷 view threshold		139.00
(-0.01%)		140.87
(98.68%)
client_sv2_mining_message_submit_standard📈 view plot
🚷 view threshold		6,297.00
(+0.28%)		6,391.62
(98.52%)		📈 view plot
🚷 view threshold		1,750.00
(-0.05%)		1,766.87
(99.05%)		📈 view plot
🚷 view threshold		2,547.00
(-0.26%)		2,575.63
(98.89%)		📈 view plot
🚷 view threshold		22.00
(+29.77%)		24.57
(89.54%)		📈 view plot
🚷 view threshold		104.00
(-0.02%)		107.12
(97.09%)
client_sv2_mining_message_submit_standard_serialize📈 view plot
🚷 view threshold		14,728.00
(-0.01%)		14,959.79
(98.45%)		📈 view plot
🚷 view threshold		4,694.00
(-0.02%)		4,710.87
(99.64%)		📈 view plot
🚷 view threshold		6,753.00
(-0.08%)		6,781.90
(99.57%)		📈 view plot
🚷 view threshold		48.00
(+10.34%)		53.34
(89.99%)		📈 view plot
🚷 view threshold		221.00
(-0.24%)		227.57
(97.11%)
client_sv2_mining_message_submit_standard_serialize_deserialize📈 view plot
🚷 view threshold		27,772.00
(+0.87%)		27,794.16
(99.92%)		📈 view plot
🚷 view threshold		10,645.00
(+0.60%)		10,650.04
(99.95%)		📈 view plot
🚷 view threshold		15,507.00
(+0.70%)		15,507.55
(100.00%)		📈 view plot
🚷 view threshold		87.00
(+6.91%)		89.92
(96.76%)		📈 view plot
🚷 view threshold		338.00
(+0.87%)		341.64
(98.93%)
client_sv2_open_channel📈 view plot
🚷 view threshold		4,379.00
(-0.36%)		4,601.79
(95.16%)		📈 view plot
🚷 view threshold		1,461.00
(-0.02%)		1,476.51
(98.95%)		📈 view plot
🚷 view threshold		2,159.00
(-0.04%)		2,184.30
(98.84%)		📈 view plot
🚷 view threshold		10.00
(+12.58%)		17.33
(57.70%)		📈 view plot
🚷 view threshold		62.00
(-0.94%)		68.12
(91.02%)
client_sv2_open_channel_serialize📈 view plot
🚷 view threshold		14,020.00
(-0.24%)		14,421.41
(97.22%)		📈 view plot
🚷 view threshold		5,064.00
(-0.01%)		5,079.51
(99.69%)		📈 view plot
🚷 view threshold		7,325.00
(-0.01%)		7,351.87
(99.63%)		📈 view plot
🚷 view threshold		37.00
(+5.46%)		43.25
(85.56%)		📈 view plot
🚷 view threshold		186.00
(-0.66%)		197.78
(94.05%)
client_sv2_open_channel_serialize_deserialize📈 view plot
🚷 view threshold		22,838.00
(+0.77%)		22,903.67
(99.71%)		📈 view plot
🚷 view threshold		8,040.00
(+0.24%)		8,073.69
(99.58%)		📈 view plot
🚷 view threshold		11,693.00
(+0.21%)		11,752.43
(99.49%)		📈 view plot
🚷 view threshold		80.00
(+7.53%)		85.24
(93.85%)		📈 view plot
🚷 view threshold		307.00
(+1.16%)		311.30
(98.62%)
client_sv2_setup_connection📈 view plot
🚷 view threshold		4,655.00
(-0.62%)		4,772.90
(97.53%)		📈 view plot
🚷 view threshold		1,502.00
(-0.02%)		1,517.51
(98.98%)		📈 view plot
🚷 view threshold		2,275.00
(-0.16%)		2,299.14
(98.95%)		📈 view plot
🚷 view threshold		14.00
(+47.98%)		16.23
(86.23%)		📈 view plot
🚷 view threshold		66.00
(-2.04%)		69.81
(94.55%)
client_sv2_setup_connection_serialize📈 view plot
🚷 view threshold		16,126.00
(-0.27%)		16,453.23
(98.01%)		📈 view plot
🚷 view threshold		5,963.00
(-0.01%)		5,978.51
(99.74%)		📈 view plot
🚷 view threshold		8,661.00
(-0.03%)		8,691.57
(99.65%)		📈 view plot
🚷 view threshold		44.00
(+10.26%)		53.22
(82.68%)		📈 view plot
🚷 view threshold		207.00
(-0.84%)		216.14
(95.77%)
client_sv2_setup_connection_serialize_deserialize📈 view plot
🚷 view threshold		35,670.00
(+0.33%)		35,754.48
(99.76%)		📈 view plot
🚷 view threshold		14,888.00
(+0.26%)		14,908.34
(99.86%)		📈 view plot
🚷 view threshold		21,880.00
(+0.30%)		21,921.26
(99.81%)		📈 view plot
🚷 view threshold		98.00
(+5.96%)		112.77
(86.90%)		📈 view plot
🚷 view threshold		380.00
(+0.20%)		384.63
(98.80%)
🐰 View full continuous benchmarking report in Bencher

@github-actions

github-actions Bot commented Nov 18, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor

🐰 Bencher Report

Branch2024-11-18-sniffer-improvements
Testbedsv2
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)		Upper Boundary
nanoseconds (ns)
(Limit %)
client_sv2_handle_message_common📈 view plot
🚷 view threshold		44.81
(+0.28%)		46.99
(95.36%)
client_sv2_handle_message_mining📈 view plot
🚷 view threshold		73.02
(-2.62%)		84.95
(85.95%)
client_sv2_mining_message_submit_standard📈 view plot
🚷 view threshold		14.76
(+0.56%)		14.89
(99.11%)
client_sv2_mining_message_submit_standard_serialize📈 view plot
🚷 view threshold		261.97
(-1.71%)		317.15
(82.60%)
client_sv2_mining_message_submit_standard_serialize_deserialize📈 view plot
🚷 view threshold		626.23
(+3.52%)		741.10
(84.50%)
client_sv2_open_channel📈 view plot
🚷 view threshold		162.88
(+9.35%)		175.46
(92.83%)
client_sv2_open_channel_serialize📈 view plot
🚷 view threshold		293.09
(+2.47%)		315.94
(92.77%)
client_sv2_open_channel_serialize_deserialize📈 view plot
🚷 view threshold		388.03
(+2.55%)		411.55
(94.29%)
client_sv2_setup_connection📈 view plot
🚷 view threshold		159.42
(-0.95%)		174.10
(91.57%)
client_sv2_setup_connection_serialize📈 view plot
🚷 view threshold		482.32
(+3.34%)		558.32
(86.39%)
client_sv2_setup_connection_serialize_deserialize📈 view plot
🚷 view threshold		1,076.60
(+9.57%)		1,170.84
(91.95%)
🐰 View full continuous benchmarking report in Bencher

@github-actions

github-actions Bot commented Nov 18, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor

🐰 Bencher Report

Branch2024-11-18-sniffer-improvements
Testbedsv1
Click to view all benchmark results
BenchmarkEstimated CyclesBenchmark Result
estimated cycles
(Result Δ%)		Upper Boundary
estimated cycles
(Limit %)		InstructionsBenchmark Result
instructions
(Result Δ%)		Upper Boundary
instructions
(Limit %)		L1 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		L2 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		RAM AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)
get_authorize📈 view plot
🚷 view threshold		8,447.00
(-0.19%)		8,767.33
(96.35%)		📈 view plot
🚷 view threshold		3,689.00
(-1.61%)		3,868.04
(95.37%)		📈 view plot
🚷 view threshold		5,157.00
(-1.91%)		5,434.70
(94.89%)		📈 view plot
🚷 view threshold		7.00
(-2.82%)		13.03
(53.71%)		📈 view plot
🚷 view threshold		93.00
(+2.68%)		95.57
(97.31%)
get_submit📈 view plot
🚷 view threshold		95,479.00
(+0.05%)		95,924.51
(99.54%)		📈 view plot
🚷 view threshold		59,346.00
(-0.19%)		59,685.41
(99.43%)		📈 view plot
🚷 view threshold		85,219.00
(-0.22%)		85,754.21
(99.38%)		📈 view plot
🚷 view threshold		43.00
(-4.31%)		63.96
(67.22%)		📈 view plot
🚷 view threshold		287.00
(+2.47%)		289.41
(99.17%)
get_subscribe📈 view plot
🚷 view threshold		8,000.00
(-0.25%)		8,317.77
(96.18%)		📈 view plot
🚷 view threshold		2,765.00
(-2.47%)		2,949.25
(93.75%)		📈 view plot
🚷 view threshold		3,845.00
(-2.96%)		4,129.88
(93.10%)		📈 view plot
🚷 view threshold		12.00
(+0.33%)		20.67
(58.06%)		📈 view plot
🚷 view threshold		117.00
(+2.42%)		119.20
(98.15%)
serialize_authorize📈 view plot
🚷 view threshold		12,172.00
(-0.88%)		12,595.47
(96.64%)		📈 view plot
🚷 view threshold		5,266.00
(-1.03%)		5,436.74
(96.86%)		📈 view plot
🚷 view threshold		7,327.00
(-1.25%)		7,594.59
(96.48%)		📈 view plot
🚷 view threshold		10.00
(+9.40%)		16.27
(61.46%)		📈 view plot
🚷 view threshold		137.00
(-0.41%)		143.02
(95.79%)
serialize_deserialize_authorize📈 view plot
🚷 view threshold		24,404.00
(-1.14%)		25,253.33
(96.64%)		📈 view plot
🚷 view threshold		9,838.00
(-0.54%)		10,038.64
(98.00%)		📈 view plot
🚷 view threshold		13,889.00
(-0.55%)		14,188.85
(97.89%)		📈 view plot
🚷 view threshold		31.00
(-10.55%)		43.85
(70.69%)		📈 view plot
🚷 view threshold		296.00
(-1.78%)		315.49
(93.82%)
serialize_deserialize_handle_authorize📈 view plot
🚷 view threshold		29,976.00
(-1.13%)		30,772.05
(97.41%)		📈 view plot
🚷 view threshold		12,015.00
(-0.56%)		12,209.40
(98.41%)		📈 view plot
🚷 view threshold		17,011.00
(-0.58%)		17,297.47
(98.34%)		📈 view plot
🚷 view threshold		52.00
(-6.40%)		67.61
(76.91%)		📈 view plot
🚷 view threshold		363.00
(-1.76%)		381.76
(95.09%)
serialize_deserialize_handle_submit📈 view plot
🚷 view threshold		126,220.00
(-0.20%)		126,967.05
(99.41%)		📈 view plot
🚷 view threshold		73,199.00
(-0.12%)		73,538.67
(99.54%)		📈 view plot
🚷 view threshold		104,920.00
(-0.14%)		105,467.54
(99.48%)		📈 view plot
🚷 view threshold		102.00
(-4.70%)		133.32
(76.51%)		📈 view plot
🚷 view threshold		594.00
(-0.39%)		605.25
(98.14%)
serialize_deserialize_handle_subscribe📈 view plot
🚷 view threshold		27,626.00
(-0.74%)		28,625.49
(96.51%)		📈 view plot
🚷 view threshold		9,584.00
(-0.67%)		9,765.34
(98.14%)		📈 view plot
🚷 view threshold		13,541.00
(-0.78%)		13,820.11
(97.98%)		📈 view plot
🚷 view threshold		66.00
(+4.03%)		75.89
(86.97%)		📈 view plot
🚷 view threshold		393.00
(-0.81%)		417.05
(94.23%)
serialize_deserialize_submit📈 view plot
🚷 view threshold		115,172.00
(-0.06%)		115,844.10
(99.42%)		📈 view plot
🚷 view threshold		68,059.00
(-0.07%)		68,403.91
(99.50%)		📈 view plot
🚷 view threshold		97,657.00
(-0.08%)		98,226.64
(99.42%)		📈 view plot
🚷 view threshold		59.00
(-5.88%)		78.24
(75.41%)		📈 view plot
🚷 view threshold		492.00
(+0.17%)		498.89
(98.62%)
serialize_deserialize_subscribe📈 view plot
🚷 view threshold		23,080.00
(-0.74%)		24,027.43
(96.06%)		📈 view plot
🚷 view threshold		8,143.00
(-0.75%)		8,321.07
(97.86%)		📈 view plot
🚷 view threshold		11,460.00
(-0.84%)		11,725.94
(97.73%)		📈 view plot
🚷 view threshold		35.00
(-5.90%)		44.22
(79.15%)		📈 view plot
🚷 view threshold		327.00
(-0.56%)		348.40
(93.86%)
serialize_submit📈 view plot
🚷 view threshold		99,833.00
(-0.01%)		100,323.86
(99.51%)		📈 view plot
🚷 view threshold		61,407.00
(-0.16%)		61,721.60
(99.49%)		📈 view plot
🚷 view threshold		88,078.00
(-0.19%)		88,591.02
(99.42%)		📈 view plot
🚷 view threshold		48.00
(+2.49%)		64.68
(74.21%)		📈 view plot
🚷 view threshold		329.00
(+1.33%)		333.90
(98.53%)
serialize_subscribe📈 view plot
🚷 view threshold		11,347.00
(-0.48%)		11,710.12
(96.90%)		📈 view plot
🚷 view threshold		4,118.00
(-1.54%)		4,293.70
(95.91%)		📈 view plot
🚷 view threshold		5,707.00
(-1.93%)		5,984.07
(95.37%)		📈 view plot
🚷 view threshold		15.00
(+16.65%)		20.71
(72.44%)		📈 view plot
🚷 view threshold		159.00
(+0.85%)		163.59
(97.19%)
🐰 View full continuous benchmarking report in Bencher

@plebhash

plebhash commented Nov 19, 2024

Copy link
Copy Markdown
Member

from description of #1253:

  1. Introduce a function to check if a message was exchanged regardless of the order(this is good because sometimes in a SubmitShare scenario, upstream next message can be SubmitShare[STATUS] or it can be something totally different, while it is handling the received share

I don't see this being implemented here.

Is that the reason why this PR says it only partially solves #1253?

will that be done in a follow up PR?

other than that, LGTM

@jbesraa

jbesraa commented Nov 20, 2024

Copy link
Copy Markdown
Contributor Author

from description of #1253:

  1. Introduce a function to check if a message was exchanged regardless of the order(this is good because sometimes in a SubmitShare scenario, upstream next message can be SubmitShare[STATUS] or it can be something totally different, while it is handling the received share

I don't see this being implemented here.

Is that the reason why this PR says it only partially solves #1253?

will that be done in a follow up PR?

other than that, LGTM

Indeed, will be pushed in a different pr

@plebhash plebhash added the ready-to-be-merged triggers auto rebase bot label Nov 20, 2024
@jbesraa
Add identifier property to Sniffer
9c94856 
..This should make it easier to debug multiple sniffers
by printing the sniffer identifier if a problem occurs during the `drop`
function.
@jbesraa
Add check_on_drop property to Sniffer
27eec57 
..If this flag is set to true, the `Sniffer` **will**
validate that all the downstrea and upstream messages were handled
before dropping, otherwise they are ignored.
@plebhash plebhash merged commit 40c93c8 into stratum-mining:main Nov 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@plebhash plebhash plebhash approved these changes

Assignees

No one assigned

Labels

ready-to-be-merged triggers auto rebase bot

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Sniffer improvements

2 participants

@jbesraa @plebhash