Skip to content

analisaperlengkapan/crypto_benchmark

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

10 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

๐Ÿ” Crypto Benchmark - Comprehensive Cryptographic Performance Analysis

Rust License Build Status

A high-performance benchmarking tool for comparing classical and post-quantum cryptographic algorithms. Features optimized implementations with statistical analysis and accurate performance measurements.


โœจ Key Features

๐ŸŽฏ Algorithm Support

Digital Signatures:

  • Classical: Ed25519, RSA-PSS (2048-bit), ECDSA (P-256)
  • Post-Quantum: ML-DSA-44 (Dilithium), Falcon-512

Key Exchange Mechanisms:

  • Classical: X25519 (Curve25519 DH), ECDH (P-256), RSA-KEM
  • Post-Quantum: ML-KEM-512 (Kyber)

โšก Performance Optimizations

  • Pre-generated Keys: Keys generated once and reused (70-80% speedup)
  • Statistical Analysis: 50-100 iterations per operation
  • Accurate Timing: High-resolution measurements with standard deviation
  • Dual Benchmark Modes:
    • Interactive mode with real-time output
    • Criterion benchmarks with HTML reports

๐Ÿ“Š Benchmark Results

Interactive Benchmark (cargo run --release)

System Info: Ubuntu Linux, Rust 1.70+
Total Time: 0.49s (0.28s key generation + 0.21s benchmarking)

Digital Signatures Performance

Algorithm Operation Mean Time Min Max Key/Sig Size
Ed25519 Sign 16 ฮผs 16 ฮผs 29 ฮผs 32 / 64 bytes
Verify 35 ฮผs 32 ฮผs 64 ฮผs
RSA-2048 Sign 1,388 ฮผs 1,207 ฮผs 4,276 ฮผs 2048 bits / 256 bytes
Verify 176 ฮผs 148 ฮผs 818 ฮผs
ECDSA P-256 Sign 181 ฮผs 154 ฮผs 891 ฮผs 32 / 64 bytes
Verify 283 ฮผs 261 ฮผs 574 ฮผs
ML-DSA-44 Sign 69 ฮผs 32 ฮผs 230 ฮผs 1312 / 2420 bytes
Verify 26 ฮผs 25 ฮผs 35 ฮผs
Falcon-512 Sign 366 ฮผs 192 ฮผs 1,254 ฮผs 897 / 752 bytes
Verify 37 ฮผs 36 ฮผs 70 ฮผs

Key Exchange Mechanisms Performance

Algorithm Operation Mean Time Min Max Key Sizes
X25519 Key Exchange 45 ฮผs 43 ฮผs 62 ฮผs 32 bytes
ECDH P-256 Key Exchange 136 ฮผs 131 ฮผs 329 ฮผs 32 bytes
ML-KEM-512 Encapsulate 8 ฮผs 7 ฮผs 19 ฮผs 800 / 768 bytes
Decapsulate 8 ฮผs 7 ฮผs 16 ฮผs

Criterion Benchmark (cargo bench)

More detailed statistical analysis with warmup, outlier detection, and HTML reports.

Signature Algorithms (Criterion)

Algorithm Operation Mean Time StdDev Outliers
Ed25519 Sign 36.6 ฮผs ยฑ0.9 ฮผs 11%
Verify 84.8 ฮผs ยฑ2.3 ฮผs 4%
RSA-2048 Sign 163.7 ms ยฑ14.9 ms 1%
Verify 157.5 ms ยฑ14.6 ms 2%
ECDSA P-256 Sign 340.8 ฮผs ยฑ4.9 ฮผs 12%
Verify 611.8 ฮผs ยฑ10.5 ฮผs 6%
ML-DSA-44 Sign 113.3 ฮผs ยฑ4.7 ฮผs 9%
Verify 138.4 ฮผs ยฑ2.8 ฮผs 2%
Falcon-512 Sign 7.09 ms ยฑ0.15 ms 0%
Verify 7.23 ms ยฑ0.21 ms 6%

KEM Algorithms (Criterion)

Algorithm Operation Mean Time StdDev Outliers
RSA-2048 KEM Encapsulate 166.6 ms ยฑ20.3 ms 5%
Decapsulate 176.6 ms ยฑ19.0 ms 4%
X25519 DH Encapsulate 65.4 ฮผs ยฑ0.7 ฮผs 7%
Decapsulate 65.3 ฮผs ยฑ0.4 ฮผs 6%
ECDH P-256 Encapsulate 289.8 ฮผs ยฑ1.7 ฮผs 4%
Decapsulate 271.7 ฮผs ยฑ4.1 ฮผs 7%
ML-KEM-512 Encapsulate 18.1 ฮผs ยฑ1.1 ฮผs 12%
Decapsulate 9.7 ฮผs ยฑ0.2 ฮผs 1%

๐Ÿš€ Quick Start

Prerequisites

# Rust 1.70 or later
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Installation

git clone <repository-url>
cd enc_test
cargo build --release

Running Benchmarks

1. Interactive Benchmark (Recommended)

Run the optimized benchmark with real-time output:

cargo run --release

Command Line Options:

# Show help
cargo run --release -- --help

# Show version
cargo run --release -- --version

# Run comparison mode (shows performance improvement)
cargo run --release -- comparison

Expected Output:

โ•”โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•—
โ•‘         Cryptographic Benchmarking Tool v0.2.0               โ•‘
โ•‘    Classical & Post-Quantum Cryptography Performance         โ•‘
โ•šโ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•

โณ Generating benchmark keys...
โœ“ All keys generated successfully (0.28s)

โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
                   SIGNATURE ALGORITHMS
โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•

--- Ed25519 ---
Sign:   16.000 ฮผs (min: 16 ฮผs, max: 29 ฮผs, ฯƒ: 1.23 ฮผs)
Verify: 35.000 ฮผs (min: 32 ฮผs, max: 64 ฮผs, ฯƒ: 3.45 ฮผs)
...

โ•”โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•—
โ•‘                 BENCHMARK SUMMARY                      โ•‘
โ•šโ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
  ๐Ÿ“Š Statistics:      100 iterations per operation
  โšก Key Generation:  0.28s (one-time cost)
  ๐Ÿ”ฌ Benchmark Time:  0.21s (all operations)
  โฑ๏ธ  Total Time:      0.49s

โœ… All benchmarks completed successfully!

2. Criterion Benchmark (Statistical Analysis)

cargo bench

Results will be saved to target/criterion/ with detailed HTML reports.

View Reports:

# Open in browser
firefox target/criterion/report/index.html
# Or
xdg-open target/criterion/report/index.html

๐Ÿ“ˆ Performance Analysis

Key Insights

๐Ÿ† Fastest Algorithms

Signatures:

  1. ML-KEM-512 Decapsulate: 8 ฮผs (Post-Quantum winner!)
  2. Ed25519 Sign: 16 ฮผs (Classical winner)
  3. ML-DSA-44 Verify: 26 ฮผs

Key Exchange:

  1. ML-KEM-512: 8-18 ฮผs (Revolutionary speed!)
  2. X25519 DH: 45-65 ฮผs (Battle-tested)
  3. ECDH P-256: 136-290 ฮผs

โš ๏ธ Slowest Algorithms

Signatures:

  1. Falcon-512 Sign: 7.09 ms (Criterion) / 366 ฮผs (Interactive)*
  2. Falcon-512 Verify: 7.23 ms (Criterion) / 37 ฮผs (Interactive)*
  3. RSA-2048 Sign: 163.7 ms (Needs key generation!)
  4. RSA-2048 Verify: 157.5 ms

Key Exchange:

  1. RSA-KEM: 166-177 ms (Very slow!)

*Note: Large discrepancy between Criterion and interactive benchmarks for Falcon suggests key generation overhead in Criterion helper functions.

๐Ÿ“Š Classical vs Post-Quantum

Metric Classical Best Post-Quantum Best Winner
Sign Speed Ed25519 (16 ฮผs) ML-DSA-44 (69 ฮผs) ๐Ÿ† Classical
Verify Speed Ed25519 (35 ฮผs) ML-DSA-44 (26 ฮผs) ๐Ÿ† Post-Quantum!
Sign Size Ed25519 (64 B) Falcon-512 (752 B) ๐Ÿ† Classical
KEM Speed X25519 (45 ฮผs) ML-KEM-512 (8 ฮผs) ๐Ÿ† Post-Quantum!
Key Size X25519 (32 B) ML-KEM-512 (800 B) ๐Ÿ† Classical

Surprising Result: Post-quantum ML-KEM-512 is 3x faster than classical X25519! ๐ŸŽ‰


๐Ÿ—๏ธ Architecture

Project Structure

enc_test/
โ”œโ”€โ”€ src/
โ”‚   โ”œโ”€โ”€ main.rs              # Interactive benchmark CLI
โ”‚   โ”œโ”€โ”€ lib.rs               # Library exports
โ”‚   โ”œโ”€โ”€ signatures.rs        # Signature algorithms + helpers
โ”‚   โ”œโ”€โ”€ kem.rs               # KEM algorithms + helpers
โ”‚   โ”œโ”€โ”€ keys.rs              # Pre-generated key management
โ”‚   โ”œโ”€โ”€ measurement.rs       # Statistical benchmarking
โ”‚   โ”œโ”€โ”€ error.rs             # Error handling
โ”‚   โ””โ”€โ”€ constants.rs         # Configuration constants
โ”œโ”€โ”€ benches/
โ”‚   โ””โ”€โ”€ crypto_bench.rs      # Criterion benchmarks
โ”œโ”€โ”€ Cargo.toml               # Dependencies
โ””โ”€โ”€ README.md                # This file

Key Components

1. Pre-generated Keys (keys.rs)

pub struct BenchmarkKeys {
    pub ed25519_signing: Ed25519SigningKey,
    pub rsa_private: RsaPrivateKey,
    pub dilithium_public: mldsa44::PublicKey,
    // ... all algorithm keys
}

Benefits:

  • RSA key generation (~200ms) done once
  • Post-quantum key generation amortized
  • 70-80% performance improvement

2. Statistical Measurement (measurement.rs)

pub struct BenchmarkResult {
    pub mean_duration: Duration,
    pub std_deviation: f64,
    pub min_duration: Duration,
    pub max_duration: Duration,
}

Features:

  • Warmup cycles (10 iterations)
  • Multiple measurements (50-100 iterations)
  • Outlier detection
  • Standard deviation calculation

๐Ÿ”ง Configuration

Customize Iterations

Edit src/constants.rs:

pub const DEFAULT_MEASUREMENT_ITERATIONS: usize = 100;
pub const DEFAULT_WARMUP_ITERATIONS: usize = 10;

Custom Message

pub const DEFAULT_MESSAGE: &[u8] = b"Your custom test message";

Adjust Criterion Settings

Edit benches/crypto_bench.rs:

criterion_group!(
    name = benches;
    config = Criterion::default()
        .sample_size(100)      // Number of samples
        .warm_up_time(Duration::from_secs(3));
    targets = benchmark_signatures, benchmark_kem
);

๐Ÿ“š API Documentation

Generate Documentation

cargo doc --open

Example Usage

use crypto_benchmark::{BenchmarkKeys, signatures, kem};

fn main() {
    // Generate keys once
    let keys = BenchmarkKeys::generate().unwrap();
    
    // Run benchmarks
    signatures::benchmark_signatures_optimized(&keys);
    kem::benchmark_kem_optimized(&keys);
}

๐Ÿงช Testing

# Run tests
cargo test

# Run tests with output
cargo test -- --nocapture

# Check code
cargo check --all-targets

# Lint
cargo clippy --all-targets

๐Ÿ“Š Interpretation Guide

Understanding Results

Mean Time: Average execution time (most important) Min/Max: Best and worst case performance StdDev: Consistency (lower = more consistent) Outliers: Anomalous measurements (higher % = less reliable)

Performance Tiers

Time Range Performance Examples
< 50 ฮผs ๐ŸŸข Excellent Ed25519, ML-KEM
50-200 ฮผs ๐ŸŸก Good ECDSA, Falcon verify
200-500 ฮผs ๐ŸŸ  Moderate ECDH, Falcon sign
> 1 ms ๐Ÿ”ด Slow RSA operations

๐Ÿ’ก Best Practices

For Production

  1. Use Ed25519 for speed: Fastest classical signature
  2. Use ML-KEM-512 for quantum resistance: Faster than classical!
  3. Avoid RSA: Too slow for modern applications
  4. Pre-generate keys: Never generate keys in hot path

For Research

  1. Run multiple times: Results vary by ~5-10%
  2. Use release mode: Debug is 10x slower
  3. Close other apps: Reduces CPU noise
  4. Check outliers: High outliers = unreliable results

For Benchmarking

# Best accuracy
cargo run --release

# Most detailed
cargo bench

# Both
cargo run --release && cargo bench

๐Ÿ”ฌ Technical Details

Algorithms

NIST Standards:

  • ML-DSA (FIPS 204 - Digital Signatures)
  • ML-KEM (FIPS 203 - Key Encapsulation)

Classical Standards:

  • Ed25519 (RFC 8032)
  • RSA-PSS (RFC 8017)
  • ECDSA (FIPS 186-4)

Dependencies

ed25519-dalek = "2.0"      # Ed25519 signatures
rsa = "0.9"                # RSA cryptography
p256 = "0.13"              # ECDSA & ECDH
pqcrypto-mldsa = "0.1.2"   # ML-DSA (Dilithium)
pqcrypto-mlkem = "0.1.1"   # ML-KEM (Kyber)
pqcrypto-falcon = "0.4.1"  # Falcon signatures
criterion = "0.7"          # Benchmarking framework

โš ๏ธ Important Notes on Falcon Benchmarks

Discrepancy Alert: Falcon shows significant performance differences between benchmark modes:

Mode Sign Verify Reason
Interactive 366 ฮผs 37 ฮผs Uses pre-generated keys โœ…
Criterion 7.09 ms 7.23 ms Helper includes key generation โš ๏ธ

Explanation: The Criterion helper functions (falcon_sign, falcon_verify) generate new keys on each iteration, while the interactive benchmark uses pre-generated keys from BenchmarkKeys. This demonstrates the importance of key reuse in production systems.

Recommendation: For production performance estimates, use the interactive benchmark results which reflect real-world usage with key caching.


๐ŸŽฏ Use Cases

Academic Research

  • Compare classical vs post-quantum performance
  • Study cryptographic algorithm characteristics
  • Generate performance data for papers

Security Audits

  • Assess algorithm performance impact
  • Plan migration to post-quantum crypto
  • Evaluate resource requirements

System Design

  • Choose appropriate algorithms
  • Performance budgeting
  • Capacity planning

๐Ÿค Contributing

We welcome contributions! Whether you're fixing bugs, adding new algorithms, improving documentation, or optimizing performance, your help is appreciated.

Quick Start for Contributors

  1. Read CONTRIBUTING.md for detailed guidelines
  2. Fork the repository
  3. Create a feature branch (git checkout -b feature/amazing-feature)
  4. Make your changes with tests
  5. Test thoroughly (cargo test && cargo clippy && cargo fmt)
  6. Commit with conventional commits (git commit -m 'feat: add amazing feature')
  7. Push to your fork (git push origin feature/amazing-feature)
  8. Open a Pull Request

What Can You Contribute?

  • ๐Ÿ› Bug Fixes: Fix issues or improve error handling
  • โœจ New Algorithms: Add SPHINCS+, BIKE, or other cryptographic algorithms
  • โšก Performance: Optimize existing implementations
  • ๐Ÿ“š Documentation: Improve README, add examples, fix typos
  • ๐Ÿงช Tests: Increase test coverage or add integration tests
  • ๐Ÿ”ง CI/CD: Improve build process or add automation

See CONTRIBUTING.md for complete guidelines including:

  • Coding standards and style guide
  • Testing requirements
  • Commit message format
  • Pull request process
  • How to add new algorithms

๐Ÿ“ License

This project is licensed under the MIT License - see the LICENSE file for details.

What This Means

โœ… You can:

  • Use this software for commercial purposes
  • Modify and distribute the code
  • Use it privately
  • Sublicense the code

โš ๏ธ You must:

  • Include the original copyright notice
  • Include a copy of the license

โŒ Limitation:

  • The software is provided "as is" without warranty

For the full license text, see LICENSE.


๏ฟฝ Contributors

This project exists thanks to all the people who contribute!

Want to contribute? See CONTRIBUTING.md to get started!

How to Contribute

We're always looking for help with:

  • Adding new cryptographic algorithms
  • Performance optimizations
  • Documentation improvements
  • Bug fixes and testing
  • Code reviews

๏ฟฝ๐Ÿ™ Acknowledgments

  • NIST PQC Project: Post-quantum algorithm standardization
  • Rust Crypto: Excellent cryptographic libraries
  • Criterion.rs: Professional benchmarking framework
  • All Contributors: Thank you for making this project better!

๐Ÿ“ž Support


๐Ÿ”ฎ Roadmap

  • Add more post-quantum algorithms (SPHINCS+, BIKE)
  • GPU acceleration support
  • WebAssembly benchmarks
  • CI/CD integration
  • Performance regression testing
  • JSON/CSV export
  • Comparison charts

โšก Fast. Secure. Future-proof.

Last updated: October 1, 2025
Version: 0.2.0

About

Comprehensive Cryptographic Performance Analysis

Topics

Resources

License

Contributing

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors