Skip to content

Speed up loading: fewer allocations and a faster lexer - #450

Merged
unknwon merged 3 commits into
mainfrom
perf-parser
Nov 19, 2026
Merged

unknwon merged 3 commits into
mainfrom
perf-parser

Conversation

@unknwon

@unknwon unknwon commented Nov 19, 2026 •

Copy link
Copy Markdown
Member

Describe the pull request

Profiling Load showed about half the time in allocation and GC. The largest single cost was growing each section's map[string]*Key, then copying the input through a growing strings.Builder, then strings.IndexAny on every line, which runs IndexRune per byte on short strings.

  • Key index: sections search their key slice until they have more than 8 keys, then build byName once, sized to the key count. Most sections never get a map, and lookups get faster too.
  • Key allocation: keys come from per-file chunks that grow from 8 to 128. A deleted key's memory is freed with the rest of its chunk.
  • Input: data sources return the input as a string instead of an io.Reader that parse copied into a growing strings.Builder, so each source is copied once at its final size.
  • Lexer: comment symbols and delimiters are precomputed byte sets (non-ASCII sets fall back to strings.IndexAny). Whitespace trimming has an ASCII fast path, and key fills the token in place.
  • Inlining: Key.current keeps the rare missing-key path in a separate function so it still inlines. Without that, Key.Value got about 30% slower.

Benchmarks, v2 before and after (8 runs, GOMAXPROCS=4):

Benchmark Before After Allocations
Load 10,000 keys in one section 2.44 ms 2.00 ms (−18%) 10,115 → 192
Load 10,000 keys in 1,000 sections 2.59 ms 2.14 ms (−17%) 18,046 → 8,124
Load 10,000 keys, mixed syntax 2.79 ms 2.27 ms (−18%) 18,434 → 8,412
Write 10,000 keys 3.3–4.5 ms 3.2–4.0 ms (−4% to −12%) unchanged
Section.Key(...).Value(), Locking(false) 14.6 ns 8.8 ns (−40%) 0

Load allocates 3–5% more bytes, mostly unused slots in the last key chunk. MapTo, ReflectFrom and locked reads are unchanged.

This PR also adds Load benchmarks to bench_test.go and refreshes the README performance table against v1.67.3. The harness that produced the old table isn't in the repository, so I rebuilt it and reran every row, loading from files on disk. My mixed-syntax input has 100 keys per section, so v1 is only 2.9× slower on it rather than 26×. With this 13-field struct, MapTo and ReflectFrom measure slower than v1; that was already true before this change.

Verification:

  • go test -race ./... and golangci-lint run pass.
  • FuzzWriteRoundTrip ran for 45 s and FuzzStructMapping for 20 s, with no failures.
  • Old and new parsers give byte-identical results on testdata/inputs.txt plus the .ini files in testdata, under 6 option sets including non-ASCII comment symbols and delimiters. The comparison covers sections, keys, comments, shadows, nested values, raw bodies, errors and the written output.
  • The new byte-set and trim helpers match strings.IndexAny, startsWithAny and strings.TrimLeftFunc(s, unicode.IsSpace) on 300k random strings with Unicode and invalid UTF-8.

Link to the issue: n/a

Checklist

  • I agree to follow the Code of Conduct by submitting this pull request.
  • I have read and acknowledge the Contributing guide.
  • I have added test cases to cover the new code.

AI usage

Written with Amp: profiling, the code changes, benchmarks and the verification above.

Profiling Load showed half the time in allocation and GC, with per-section
key maps the largest single cost, then the copy of the input through a
growing strings.Builder and strings.IndexAny on every line.

- Sections search their keys until they have more than 8, and only then
  index them in a map, sized once.
- Keys are allocated in chunks of up to 128 per file.
- Data sources return the input as a string, without the extra copy for
  files and readers.
- The lexer looks up comment symbols and delimiters in precomputed byte
  sets and trims ASCII whitespace without the unicode package.
- Key.current keeps the missing-key path apart so it still inlines.

Load is 18-22% faster with 54-98% fewer allocations, and lookups through
Section.Key without locking 40% faster. Add Load benchmarks and refresh
the README performance table.
@unknwon
unknwon merged commit 68bedac into main Nov 19, 2026
7 checks passed
@unknwon
unknwon deleted the perf-parser branch November 19, 2026 03:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant