Add tproxy initializer for Integration Tests#1254
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #1254 +/- ##
=======================================
Coverage 19.30% 19.30%
=======================================
Files 164 164
Lines 10849 10849
=======================================
Hits 2094 2094
Misses 8755 8755
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚨 Try these New Features:
|
|
| Branch | 2024-11-14-tproxy-test-initializer |
| Testbed | sv1 |
Click to view all benchmark results
| Benchmark | Estimated Cycles | Benchmark Result estimated cycles (Result Δ%) | Upper Boundary estimated cycles (Limit %) | Instructions | Benchmark Result instructions (Result Δ%) | Upper Boundary instructions (Limit %) | L1 Accesses | Benchmark Result accesses (Result Δ%) | Upper Boundary accesses (Limit %) | L2 Accesses | Benchmark Result accesses (Result Δ%) | Upper Boundary accesses (Limit %) | RAM Accesses | Benchmark Result accesses (Result Δ%) | Upper Boundary accesses (Limit %) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| get_authorize | 📈 view plot 🚷 view threshold | 8,379.00 (-0.94%) | 8,801.86 (95.20%) | 📈 view plot 🚷 view threshold | 3,689.00 (-1.57%) | 3,881.85 (95.03%) | 📈 view plot 🚷 view threshold | 5,159.00 (-1.84%) | 5,450.80 (94.65%) | 📈 view plot 🚷 view threshold | 7.00 (-2.71%) | 13.01 (53.82%) | 📈 view plot 🚷 view threshold | 91.00 (+0.56%) | 95.87 (94.92%) |
| get_submit | 📈 view plot 🚷 view threshold | 95,407.00 (-0.02%) | 95,958.18 (99.43%) | 📈 view plot 🚷 view threshold | 59,346.00 (-0.19%) | 59,699.54 (99.41%) | 📈 view plot 🚷 view threshold | 85,222.00 (-0.21%) | 85,770.34 (99.36%) | 📈 view plot 🚷 view threshold | 42.00 (-6.59%) | 63.98 (65.64%) | 📈 view plot 🚷 view threshold | 285.00 (+1.80%) | 289.36 (98.49%) |
| get_subscribe | 📈 view plot 🚷 view threshold | 7,872.00 (-1.79%) | 8,357.07 (94.20%) | 📈 view plot 🚷 view threshold | 2,765.00 (-2.43%) | 2,962.24 (93.34%) | 📈 view plot 🚷 view threshold | 3,847.00 (-2.88%) | 4,144.74 (92.82%) | 📈 view plot 🚷 view threshold | 14.00 (+16.74%) | 20.75 (67.45%) | 📈 view plot 🚷 view threshold | 113.00 (-1.00%) | 119.67 (94.42%) |
| serialize_authorize | 📈 view plot 🚷 view threshold | 12,210.00 (-0.54%) | 12,627.44 (96.69%) | 📈 view plot 🚷 view threshold | 5,266.00 (-1.00%) | 5,451.07 (96.60%) | 📈 view plot 🚷 view threshold | 7,325.00 (-1.26%) | 7,611.40 (96.24%) | 📈 view plot 🚷 view threshold | 11.00 (+20.55%) | 16.13 (68.21%) | 📈 view plot 🚷 view threshold | 138.00 (+0.35%) | 143.35 (96.27%) |
| serialize_deserialize_authorize | 📈 view plot 🚷 view threshold | 24,340.00 (-1.39%) | 25,270.82 (96.32%) | 📈 view plot 🚷 view threshold | 9,838.00 (-0.53%) | 10,048.99 (97.90%) | 📈 view plot 🚷 view threshold | 13,890.00 (-0.53%) | 14,202.90 (97.80%) | 📈 view plot 🚷 view threshold | 32.00 (-7.83%) | 44.11 (72.54%) | 📈 view plot 🚷 view threshold | 294.00 (-2.43%) | 315.62 (93.15%) |
| serialize_deserialize_handle_authorize | 📈 view plot 🚷 view threshold | 30,068.00 (-0.83%) | 30,780.36 (97.69%) | 📈 view plot 🚷 view threshold | 12,015.00 (-0.55%) | 12,219.64 (98.33%) | 📈 view plot 🚷 view threshold | 17,003.00 (-0.61%) | 17,312.03 (98.21%) | 📈 view plot 🚷 view threshold | 58.00 (+4.18%) | 67.98 (85.32%) | 📈 view plot 🚷 view threshold | 365.00 (-1.22%) | 381.76 (95.61%) |
| serialize_deserialize_handle_submit | 📈 view plot 🚷 view threshold | 126,369.00 (-0.07%) | 127,028.60 (99.48%) | 📈 view plot 🚷 view threshold | 73,200.00 (-0.11%) | 73,570.56 (99.50%) | 📈 view plot 🚷 view threshold | 104,914.00 (-0.14%) | 105,510.30 (99.43%) | 📈 view plot 🚷 view threshold | 105.00 (-1.93%) | 133.33 (78.75%) | 📈 view plot 🚷 view threshold | 598.00 (+0.29%) | 605.42 (98.77%) |
| serialize_deserialize_handle_subscribe | 📈 view plot 🚷 view threshold | 27,468.00 (-1.30%) | 28,638.51 (95.91%) | 📈 view plot 🚷 view threshold | 9,584.00 (-0.66%) | 9,779.75 (98.00%) | 📈 view plot 🚷 view threshold | 13,543.00 (-0.76%) | 13,838.47 (97.86%) | 📈 view plot 🚷 view threshold | 69.00 (+8.69%) | 76.06 (90.72%) | 📈 view plot 🚷 view threshold | 388.00 (-2.06%) | 417.14 (93.01%) |
| serialize_deserialize_submit | 📈 view plot 🚷 view threshold | 115,117.00 (-0.10%) | 115,890.07 (99.33%) | 📈 view plot 🚷 view threshold | 68,060.00 (-0.06%) | 68,421.70 (99.47%) | 📈 view plot 🚷 view threshold | 97,657.00 (-0.08%) | 98,248.58 (99.40%) | 📈 view plot 🚷 view threshold | 62.00 (-1.15%) | 78.25 (79.24%) | 📈 view plot 🚷 view threshold | 490.00 (-0.22%) | 499.30 (98.14%) |
| serialize_deserialize_subscribe | 📈 view plot 🚷 view threshold | 22,842.00 (-1.74%) | 24,053.70 (94.96%) | 📈 view plot 🚷 view threshold | 8,143.00 (-0.73%) | 8,335.98 (97.68%) | 📈 view plot 🚷 view threshold | 11,467.00 (-0.76%) | 11,744.97 (97.63%) | 📈 view plot 🚷 view threshold | 35.00 (-6.04%) | 44.45 (78.74%) | 📈 view plot 🚷 view threshold | 320.00 (-2.66%) | 348.72 (91.76%) |
| serialize_submit | 📈 view plot 🚷 view threshold | 99,796.00 (-0.05%) | 100,357.68 (99.44%) | 📈 view plot 🚷 view threshold | 61,408.00 (-0.16%) | 61,736.95 (99.47%) | 📈 view plot 🚷 view threshold | 88,081.00 (-0.19%) | 88,608.21 (99.41%) | 📈 view plot 🚷 view threshold | 47.00 (+0.35%) | 64.69 (72.66%) | 📈 view plot 🚷 view threshold | 328.00 (+1.05%) | 333.97 (98.21%) |
| serialize_subscribe | 📈 view plot 🚷 view threshold | 11,317.00 (-0.71%) | 11,742.18 (96.38%) | 📈 view plot 🚷 view threshold | 4,118.00 (-1.51%) | 4,307.28 (95.61%) | 📈 view plot 🚷 view threshold | 5,707.00 (-1.91%) | 5,999.49 (95.12%) | 📈 view plot 🚷 view threshold | 16.00 (+24.42%) | 20.71 (77.27%) | 📈 view plot 🚷 view threshold | 158.00 (+0.26%) | 163.88 (96.41%) |
|
| Branch | 2024-11-14-tproxy-test-initializer |
| Testbed | sv2 |
Click to view all benchmark results
| Benchmark | Latency | Benchmark Result nanoseconds (ns) (Result Δ%) | Upper Boundary nanoseconds (ns) (Limit %) |
|---|---|---|---|
| client_sv2_handle_message_common | 📈 view plot 🚷 view threshold | 44.23 (-0.84%) | 47.22 (93.67%) |
| client_sv2_handle_message_mining | 📈 view plot 🚷 view threshold | 74.63 (-1.02%) | 84.29 (88.54%) |
| client_sv2_mining_message_submit_standard | 📈 view plot 🚷 view threshold | 14.65 (-0.07%) | 14.72 (99.56%) |
| client_sv2_mining_message_submit_standard_serialize | 📈 view plot 🚷 view threshold | 269.56 (+1.32%) | 316.40 (85.19%) |
| client_sv2_mining_message_submit_standard_serialize_deserialize | 📈 view plot 🚷 view threshold | 615.03 (+1.82%) | 750.34 (81.97%) |
| client_sv2_open_channel | 📈 view plot 🚷 view threshold | 147.10 (-0.36%) | 156.94 (93.73%) |
| client_sv2_open_channel_serialize | 📈 view plot 🚷 view threshold | 280.53 (-1.55%) | 316.13 (88.74%) |
| client_sv2_open_channel_serialize_deserialize | 📈 view plot 🚷 view threshold | 389.11 (+2.91%) | 415.15 (93.73%) |
| client_sv2_setup_connection | 📈 view plot 🚷 view threshold | 158.74 (-1.33%) | 175.75 (90.32%) |
| client_sv2_setup_connection_serialize | 📈 view plot 🚷 view threshold | 490.17 (+4.66%) | 573.42 (85.48%) |
| client_sv2_setup_connection_serialize_deserialize | 📈 view plot 🚷 view threshold | 991.81 (+1.84%) | 1,069.19 (92.76%) |
|
| Branch | 2024-11-14-tproxy-test-initializer |
| Testbed | sv2 |
🚨 1 Alert
| Benchmark | Measure Units | View | Benchmark Result (Result Δ%) | Upper Boundary (Limit %) |
|---|---|---|---|---|
| client_sv2_mining_message_submit_standard_serialize_deserialize | Estimated Cycles estimated cycles | 📈 plot 🚨 alert (🔔) 🚷 threshold | 27,882.00 (+1.27%) | 27,787.35 (100.34%) |
Click to view all benchmark results
| Benchmark | Estimated Cycles | Benchmark Result estimated cycles (Result Δ%) | Upper Boundary estimated cycles (Limit %) | Instructions | Benchmark Result instructions (Result Δ%) | Upper Boundary instructions (Limit %) | L1 Accesses | Benchmark Result accesses (Result Δ%) | Upper Boundary accesses (Limit %) | L2 Accesses | Benchmark Result accesses (Result Δ%) | Upper Boundary accesses (Limit %) | RAM Accesses | Benchmark Result accesses (Result Δ%) | Upper Boundary accesses (Limit %) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| client_sv2_handle_message_common | 📈 view plot 🚷 view threshold | 2,103.00 (+0.45%) | 2,206.38 (95.31%) | 📈 view plot 🚷 view threshold | 473.00 (-0.07%) | 488.52 (96.82%) | 📈 view plot 🚷 view threshold | 733.00 (-0.39%) | 757.28 (96.79%) | 📈 view plot 🚷 view threshold | 8.00 (+46.92%) | 14.11 (56.69%) | 📈 view plot 🚷 view threshold | 38.00 (-0.04%) | 41.17 (92.30%) |
| client_sv2_handle_message_mining | 📈 view plot 🚷 view threshold | 8,204.00 (-0.00%) | 8,286.46 (99.00%) | 📈 view plot 🚷 view threshold | 2,137.00 (-0.03%) | 2,143.08 (99.72%) | 📈 view plot 🚷 view threshold | 3,159.00 (-0.06%) | 3,170.60 (99.63%) | 📈 view plot 🚷 view threshold | 36.00 (+1.25%) | 42.18 (85.36%) | 📈 view plot 🚷 view threshold | 139.00 (-0.02%) | 140.89 (98.66%) |
| client_sv2_mining_message_submit_standard | 📈 view plot 🚷 view threshold | 6,323.00 (+0.69%) | 6,391.88 (98.92%) | 📈 view plot 🚷 view threshold | 1,750.00 (-0.05%) | 1,766.88 (99.04%) | 📈 view plot 🚷 view threshold | 2,548.00 (-0.23%) | 2,575.62 (98.93%) | 📈 view plot 🚷 view threshold | 20.00 (+17.92%) | 24.54 (81.49%) | 📈 view plot 🚷 view threshold | 105.00 (+0.94%) | 107.12 (98.02%) |
| client_sv2_mining_message_submit_standard_serialize | 📈 view plot 🚷 view threshold | 14,838.00 (+0.73%) | 14,962.93 (99.17%) | 📈 view plot 🚷 view threshold | 4,694.00 (-0.02%) | 4,710.88 (99.64%) | 📈 view plot 🚷 view threshold | 6,748.00 (-0.16%) | 6,782.04 (99.50%) | 📈 view plot 🚷 view threshold | 50.00 (+14.86%) | 53.51 (93.44%) | 📈 view plot 🚷 view threshold | 224.00 (+1.11%) | 227.63 (98.41%) |
| client_sv2_mining_message_submit_standard_serialize_deserialize | 📈 view plot 🚨 view alert (🔔) 🚷 view threshold | 27,882.00 (+1.27%) | 27,787.35 (100.34%) | 📈 view plot 🚷 view threshold | 10,645.00 (+0.60%) | 10,647.81 (99.97%) | 📈 view plot 🚷 view threshold | 15,502.00 (+0.68%) | 15,502.80 (99.99%) | 📈 view plot 🚷 view threshold | 89.00 (+9.32%) | 90.16 (98.71%) | 📈 view plot 🚷 view threshold | 341.00 (+1.77%) | 341.64 (99.81%) |
| client_sv2_open_channel | 📈 view plot 🚷 view threshold | 4,409.00 (+0.30%) | 4,605.96 (95.72%) | 📈 view plot 🚷 view threshold | 1,461.00 (-0.02%) | 1,476.52 (98.95%) | 📈 view plot 🚷 view threshold | 2,159.00 (-0.04%) | 2,184.41 (98.84%) | 📈 view plot 🚷 view threshold | 9.00 (+1.05%) | 17.43 (51.65%) | 📈 view plot 🚷 view threshold | 63.00 (+0.61%) | 68.23 (92.34%) |
| client_sv2_open_channel_serialize | 📈 view plot 🚷 view threshold | 14,046.00 (-0.08%) | 14,431.12 (97.33%) | 📈 view plot 🚷 view threshold | 5,064.00 (-0.01%) | 5,079.52 (99.69%) | 📈 view plot 🚷 view threshold | 7,326.00 (+0.00%) | 7,352.03 (99.65%) | 📈 view plot 🚷 view threshold | 35.00 (-0.22%) | 43.23 (80.95%) | 📈 view plot 🚷 view threshold | 187.00 (-0.16%) | 198.06 (94.42%) |
| client_sv2_open_channel_serialize_deserialize | 📈 view plot 🚷 view threshold | 22,854.00 (+0.84%) | 22,903.41 (99.78%) | 📈 view plot 🚷 view threshold | 8,040.00 (+0.24%) | 8,073.98 (99.58%) | 📈 view plot 🚷 view threshold | 11,689.00 (+0.18%) | 11,753.01 (99.46%) | 📈 view plot 🚷 view threshold | 84.00 (+12.87%) | 85.46 (98.30%) | 📈 view plot 🚷 view threshold | 307.00 (+1.15%) | 311.41 (98.58%) |
| client_sv2_setup_connection | 📈 view plot 🚷 view threshold | 4,681.00 (-0.07%) | 4,772.88 (98.07%) | 📈 view plot 🚷 view threshold | 1,502.00 (-0.02%) | 1,517.52 (98.98%) | 📈 view plot 🚷 view threshold | 2,276.00 (-0.11%) | 2,299.13 (98.99%) | 📈 view plot 🚷 view threshold | 12.00 (+27.57%) | 16.04 (74.83%) | 📈 view plot 🚷 view threshold | 67.00 (-0.58%) | 69.80 (95.99%) |
| client_sv2_setup_connection_serialize | 📈 view plot 🚷 view threshold | 16,190.00 (+0.12%) | 16,458.18 (98.37%) | 📈 view plot 🚷 view threshold | 5,963.00 (-0.01%) | 5,978.52 (99.74%) | 📈 view plot 🚷 view threshold | 8,660.00 (-0.04%) | 8,691.71 (99.64%) | 📈 view plot 🚷 view threshold | 43.00 (+7.69%) | 53.29 (80.69%) | 📈 view plot 🚷 view threshold | 209.00 (+0.10%) | 216.26 (96.64%) |
| client_sv2_setup_connection_serialize_deserialize | 📈 view plot 🚷 view threshold | 35,722.00 (+0.48%) | 35,753.45 (99.91%) | 📈 view plot 🚷 view threshold | 14,888.00 (+0.26%) | 14,907.75 (99.87%) | 📈 view plot 🚷 view threshold | 21,882.00 (+0.31%) | 21,920.23 (99.83%) | 📈 view plot 🚷 view threshold | 94.00 (+1.52%) | 113.21 (83.03%) | 📈 view plot 🚷 view threshold | 382.00 (+0.72%) | 384.65 (99.31%) |
| template_provider | ||
| } | ||
|
|
||
| pub async fn start_sv2_translator(upstream: SocketAddr) -> SocketAddr { |
There was a problem hiding this comment.
let's try to keep starter APIs consistent, as highlighted in #1234
I'm not opinionated towards any specific approach
I just feel it's important to be careful in fragmenting patterns because that will eventually result in a poor UX for SRI contributors writing tests while working on this like e.g.: #1229 (which will likely happen sooner, rather than later)
There was a problem hiding this comment.
I think the approach here is mostly what we want. No need for the end user to call get_available_address but the role itself does that and returns it
| listening_address | ||
| } | ||
|
|
||
| fn measure_hashrate(duration_secs: u64) -> f64 { |
There was a problem hiding this comment.
interesting approach of dynamically measuring local CPU hashrate
this strategy will save us A LOT of headache and it's quite a game changer in comparison to the static nature of MG tests
| .expect("failed"); | ||
| let listening_address = get_available_address(); | ||
| let listening_port = listening_address.port(); | ||
| let hashrate = measure_hashrate(3) as f32 / 100.0; |
There was a problem hiding this comment.
This is done just part of the testing, I think this will change also until we figure out the best numbers here and in measure_hashrate
| let mut hashes: u64 = 0; | ||
| let duration = std::time::Duration::from_secs(duration_secs); | ||
|
|
||
| let hash = |share: &mut [u8; 80]| -> Target { |
There was a problem hiding this comment.
why return Target, it seems to be used nowhere?
| let mut nonce = u64::from_le_bytes(nonce); | ||
| nonce += 1; | ||
| share[0..8].copy_from_slice(&nonce.to_le_bytes()); | ||
| let hash = Sha256::digest(&share).to_vec(); |
There was a problem hiding this comment.
Bitcoin protocol uses sha256d, which means we actually calculate sha256 2x
every ASIC hashrate will tell you how many sha256d hashes were calculated in a second, not sha256
There was a problem hiding this comment.
here's an easy way to calculate sha256d hashes with stratum_common:
use stratum_common::bitcoin::hashes::{sha256d, Hash, HashEngine};
let input = 0;
let mut engine = sha256d::Hash::engine();
engine.input(&input);
let hashed = sha256d::Hash::from_engine(engine).into_inner();| let mut share = { | ||
| let mut rng = thread_rng(); | ||
| let mut arr = [0u8; 80]; | ||
| rng.fill(&mut arr[..]); | ||
| arr | ||
| }; | ||
| let start_time = std::time::Instant::now(); | ||
| let mut hashes: u64 = 0; | ||
| let duration = std::time::Duration::from_secs(duration_secs); |
There was a problem hiding this comment.
I'm fine with this approach but it seems a bit convoluted and confusing to read/understand
I'm not convinced we really need to replicate the bitcoin protocol so closely if all we want is to know the capacity of the system to calculate sha256d (attention to the d, it's 2x sha256)
why not simply hash some simple input?
if you REALLY want to closely replicate what happens on bitcoin mining, I would suggest leveraging rust-bitcoin
you can initialize a Header struct and then call header.block_hash() to "mine" it.
that's how both mining-device and sv1-mining-device do it
|
@plebhash the |
I must point out that my comments about in it's current form, this I don't think we should introduce something with broken functionality to fix later, especially given that it's pretty straightforward to fix it now. please see #1254 (comment) |
|
after discussing with @GitGab19 (he did not write this code) we tracked down the root cause of the confusion around stratum/roles/translator/src/lib/downstream_sv1/diff_management.rs Lines 374 to 400 in 8c98647 we should not blindly copy-paste this code because it will result in a broken ITF I'm taking note of this on #1229 so it can be fixed as we modularize vardiff (which is what these unit tests are for) |
tproxy initializertproxy initializer for Integration Tests
resolves #1250