Repository navigation
Speed up loading: fewer allocations and a faster lexer - #450
Merged
Merged
Conversation
Profiling Load showed half the time in allocation and GC, with per-section key maps the largest single cost, then the copy of the input through a growing strings.Builder and strings.IndexAny on every line. - Sections search their keys until they have more than 8, and only then index them in a map, sized once. - Keys are allocated in chunks of up to 128 per file. - Data sources return the input as a string, without the extra copy for files and readers. - The lexer looks up comment symbols and delimiters in precomputed byte sets and trims ASCII whitespace without the unicode package. - Key.current keeps the missing-key path apart so it still inlines. Load is 18-22% faster with 54-98% fewer allocations, and lookups through Section.Key without locking 40% faster. Add Load benchmarks and refresh the README performance table.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Describe the pull request
Profiling
Loadshowed about half the time in allocation and GC. The largest single cost was growing each section'smap[string]*Key, then copying the input through a growingstrings.Builder, thenstrings.IndexAnyon every line, which runsIndexRuneper byte on short strings.byNameonce, sized to the key count. Most sections never get a map, and lookups get faster too.io.Readerthatparsecopied into a growingstrings.Builder, so each source is copied once at its final size.strings.IndexAny). Whitespace trimming has an ASCII fast path, andkeyfills the token in place.Key.currentkeeps the rare missing-key path in a separate function so it still inlines. Without that,Key.Valuegot about 30% slower.Benchmarks, v2 before and after (8 runs,
GOMAXPROCS=4):Section.Key(...).Value(),Locking(false)Loadallocates 3–5% more bytes, mostly unused slots in the last key chunk.MapTo,ReflectFromand locked reads are unchanged.This PR also adds
Loadbenchmarks tobench_test.goand refreshes the README performance table against v1.67.3. The harness that produced the old table isn't in the repository, so I rebuilt it and reran every row, loading from files on disk. My mixed-syntax input has 100 keys per section, so v1 is only 2.9× slower on it rather than 26×. With this 13-field struct,MapToandReflectFrommeasure slower than v1; that was already true before this change.Verification:
go test -race ./...andgolangci-lint runpass.FuzzWriteRoundTripran for 45 s andFuzzStructMappingfor 20 s, with no failures.testdata/inputs.txtplus the.inifiles intestdata, under 6 option sets including non-ASCII comment symbols and delimiters. The comparison covers sections, keys, comments, shadows, nested values, raw bodies, errors and the written output.strings.IndexAny,startsWithAnyandstrings.TrimLeftFunc(s, unicode.IsSpace)on 300k random strings with Unicode and invalid UTF-8.Link to the issue: n/a
Checklist
AI usage
Written with Amp: profiling, the code changes, benchmarks and the verification above.