Skip to content

perf: cell reader buffer reuse, optimise maps#611

Merged
jmcnamara merged 2 commits into
tafia:masterfrom
alexander-beedie:perf-cell-reader-bufs
Mar 6, 2026
Merged

perf: cell reader buffer reuse, optimise maps#611
jmcnamara merged 2 commits into
tafia:masterfrom
alexander-beedie:perf-cell-reader-bufs

Conversation

@alexander-beedie
Copy link
Copy Markdown
Contributor

@alexander-beedie alexander-beedie commented Feb 27, 2026

Three optimisations - two (very) small/minor, one medium, no public API changes:

Optimisations

  1. Replaced s.chunks(4) with s.chunks_exact(4) for the most micro of micro-optimisations.

  2. As per @tafia's review suggestion perf: cache path and attr lookups, reuse buffers, minimise allocs #606 (review), replaced use of the standard HashMap with hashbrown::HashMap (which uses ahash). This was already a transitive dependency via zip, so doesn't actually expand the current set of deps (though is now explicit in Cargo.toml). Also replaced BTreeMap with HashMap in two places where the map does not actually need to maintain ordering ("read_relationships" and "number_formats") to get O(n) instead of O(log n); marginal impact though as these maps are rarely (if ever?) going to be large.

Update: reverted explicit hashbrown integration (benefit was too marginal to reliably measure - maps are likely too small in relation to everything else), but kept the switch from BTreeMap to HashMap in places where order isn't needed..

  1. Extended the same buffer-reuse optimisation that was part of perf: cache path and attr lookups, reuse buffers, minimise allocs #606 to XlsxCellReader, but cleaned it up a bit by consolidating the buffers in a new ValueBufs struct. No more use of read_string, so dropped it (only the variant that takes shared buffers is used now).

Miscellaneous

clippy complained about one of the functions having one too many args, so I also consolidated workbook-related args into a WorkbookContext struct; seems tidy?

Results

~6-7% speedup1 when testing on a large (10 million cell) mixed-dtype Workbook.

Footnotes

  1. Benchmarked on: Apple Silicon M3 Max

@alexander-beedie alexander-beedie force-pushed the perf-cell-reader-bufs branch 4 times, most recently from 68cd545 to 5ffad42 Compare February 27, 2026 13:05
Copy link
Copy Markdown
Collaborator

@jmcnamara jmcnamara left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 on adding the context.

@jmcnamara
Copy link
Copy Markdown
Collaborator

Overall looks good. My only concern is with adding hashbrown as an explicit additional dependency (even if it is included by zip.rs anyway). What % performance does that give in your test case. Add @jqnatividad for a second opinion on the adding a dependency.

Also, I think the first commit could have been 3 functionally separate commits (even if they are small). You don't need to go back and change that though. Just a note for the future.

@alexander-beedie
Copy link
Copy Markdown
Contributor Author

alexander-beedie commented Feb 28, 2026

What % performance does that give in your test case. Add @jqnatividad for a second opinion on the adding a dependency.

Honestly? Couldn't even measure it; just kept it as the dependency was already implicit 😄
Totally happy to switch it back to a regular HashMap to keep things simple 👍

(Update: done)

Also, I think the first commit could have been 3 functionally separate commits (even if they are small). You don't need to go back and change that though. Just a note for the future.

Noted (and agreed ;)

@jqnatividad
Copy link
Copy Markdown
Collaborator

LGTM! Every little bit of performance is always appreciated!

@jmcnamara jmcnamara merged commit d10b642 into tafia:master Mar 6, 2026
6 checks passed
@jmcnamara
Copy link
Copy Markdown
Collaborator

Merged. Thanks.

@alexander-beedie alexander-beedie deleted the perf-cell-reader-bufs branch March 6, 2026 14:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants