Skip to content

tomtomwombat/fast-atoi

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

32 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

fast-atoi

Github

fast-atoi is a SIMD-accelerated integer and digit parsing library for Rust. It matches the behavior of std parsing while achieving state-of-the-art performance on common inputs. fast-atoi also exposes low-level building blocks for constructing custom numeric parsers. fast-atoi is up to 3x faster than existing Rust parsers and particularly excels at unpredictable length inputs.

Usage

use fast_atoi::*;

let _ = atoi::<u64>(b"42").unwrap();
let _ = atoi::<i8>(b"-42").unwrap();
let _ = atoi::<u32>(b"00042").unwrap();
let _ = atoi::<u128>(b"+42").unwrap();

// Ignoring the pesky '+' is faster
let _ = atoi_no_plus::<u64>(b"42").unwrap();
assert!(atoi_no_plus::<u64>(b"+42").is_err());

let _: u64 = "42".parse_radix10().unwrap();
let _: u64 = b"42".parse_radix10().unwrap();
use fast_atoi::{parse_4, parse_2};

fn my_really_fast_6_digit_parser(mut src: &[u8]) -> Result<u64, ()> {
      assert_eq!(src.len(), 6);
      let mut is_err = 0;
      let left = parse_4(&mut src, &mut is_err);
      let right = parse_2(&mut src, &mut is_err);
      if is_err > 0 {
            return Err(());
      }
      Ok(left.wrapping_mul(100).wrapping_add(right))
}

assert_eq!(my_really_fast_6_digit_parser(b"123456"), Ok(123456));

SIMD Support

fast-atoi automatically uses SIMD on x86_64 (SSE2) if available. No configuration or feature flags are required. For maximum performance, users may compile with:

RUSTFLAGS="-C target-cpu=native"

This is optional and not required for correctness. If SIMD support is not detected, a fallback is automatically used instead.

Performance

Benchmark source and more results: https://github.com/tomtomwombat/atoi-benchmark.

  • AMD Ryzen 9 5900X 12-Core Processor (3.70 GHz)
  • 64-bit operating system, x64-based processor
u64parsen u64parse1n i64parsen i64parse1n

Should I Use This?

Yes. fast-atoi's behavior is of 1-1 parity with std. At worst, fast-atoi matches the performance of the next fastest parser. At best (for larger inputs) fast-atoi is 2-3x faster. fast-atoi is extensively tested:

  • exhaustive testing for correct inputs
  • exhaustive testing for all 4-byte combinations at different alignments
  • no unsafe code (except for SIMD)
  • miri for undefined behavior
  • extensive property testing
  • performance tested on a variety of input patterns

To Do

Below are some ideas for features. Create an issue if you have a use-case for any.

  • General radix
  • Unchecked parsing
  • Parsing aligned data
  • Parsing buffered data (i.e. input has trailing buffer)
  • AVX and NEON support

References

License

Licensed under either of

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

About

Extremely fast SIMD-accelerated integer parser.

Resources

License

Apache-2.0, MIT licenses found

Licenses found

Apache-2.0
LICENSE-APACHE
MIT
LICENSE-MIT

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages