fix: make text parsers safe for long lines#128
Conversation
📝 WalkthroughWalkthroughAdds explicit ChangesScanner buffer size fix
Estimated code review effort: 2 (Simple) | ~12 minutes Sequence Diagram(s)sequenceDiagram
participant Caller
participant Parser as instr.ParseAll / program.Parse
participant Scanner as bufio.Scanner
Caller->>Parser: parse input reader
Parser->>Scanner: Buffer(buf, maxParseLineBytes)
loop scan lines
Scanner-->>Parser: Scan() token
Parser->>Parser: increment line counter
end
Parser->>Scanner: Err()
alt token too long
Scanner-->>Parser: "token too long"
Parser-->>Caller: line N exceeds maximum allowed size
else other/no error
Scanner-->>Parser: raw error or nil
Parser-->>Caller: raw error or success
end
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #128 +/- ##
==========================================
+ Coverage 46.80% 46.83% +0.02%
==========================================
Files 77 77
Lines 17340 17352 +12
==========================================
+ Hits 8116 8126 +10
- Misses 8306 8309 +3
+ Partials 918 917 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@internal/textparse/scanner.go`:
- Around line 19-25: LineError currently drops line-number context for every
scanner error except bufio.ErrTooLong, regressing the messages produced by
instr.ParseAll and program.Parse. Update LineError to wrap all non-nil errors
with the line prefix while still special-casing ErrTooLong for the max-size
message, and use errors.Is instead of direct equality when checking
bufio.ErrTooLong.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: e022795b-333f-49e1-a0c5-97e1023ce3b2
📒 Files selected for processing (5)
instr/parse.goinstr/parse_test.gointernal/textparse/scanner.goprogram/parse.goprogram/parse_test.go
There was a problem hiding this comment.
🧹 Nitpick comments (2)
instr/parse.go (1)
151-154: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winPrefer
errors.Isover string matching for the scanner error.
strings.Contains(err.Error(), "token too long")is fragile compared to checking against the exportedbufio.ErrTooLongsentinel error witherrors.Is.♻️ Proposed fix
import ( "bufio" + "errors" "fmt" "io" "math" "strconv" "strings" ) ... if err := scanner.Err(); err != nil { - if strings.Contains(err.Error(), "token too long") { + if errors.Is(err, bufio.ErrTooLong) { return nil, fmt.Errorf("line %d exceeds maximum allowed size of %d bytes", line, maxParseLineBytes) } return nil, err }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@instr/parse.go` around lines 151 - 154, The scanner error check in parse logic relies on string matching, which should be replaced with the exported sentinel error check. Update the error handling around the line-size validation in the parsing flow to use errors.Is against bufio.ErrTooLong instead of strings.Contains(err.Error(), "token too long"), while keeping the existing wrapped message for the oversized line case and preserving the fallback return of the original error.program/parse.go (1)
39-42: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winSame fragile string match for scanner error, occurring twice in this file.
Both scan phases detect the oversized-line condition via
strings.Contains(err.Error(), "token too long"). Useerrors.Is(err, bufio.ErrTooLong)instead, which is more robust and idiomatic for this stdlib sentinel error.♻️ Proposed fix
import ( "bufio" + "errors" "fmt" "io" "strings" "github.com/siyul-park/minivm/instr" "github.com/siyul-park/minivm/types" ) ... if err := scanner.Err(); err != nil { - if strings.Contains(err.Error(), "token too long") { + if errors.Is(err, bufio.ErrTooLong) { return nil, fmt.Errorf("line %d exceeds maximum allowed size of %d bytes", lineNum+1, maxParseLineBytes) } return nil, err } ... if err := scanner.Err(); err != nil { - if strings.Contains(err.Error(), "token too long") { + if errors.Is(err, bufio.ErrTooLong) { return nil, fmt.Errorf("line %d exceeds maximum allowed size of %d bytes", lineNum+1, maxParseLineBytes) } return nil, err }Also applies to: 84-87
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@program/parse.go` around lines 39 - 42, The oversized-line handling in parse.go is using a fragile string check for the scanner error in both scan phases. Update the error handling in the relevant parsing logic around the scanner/line processing paths to use errors.Is with bufio.ErrTooLong instead of strings.Contains(err.Error(), "token too long"), and apply the same change wherever this check appears in the file so the behavior stays consistent.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@instr/parse.go`:
- Around line 151-154: The scanner error check in parse logic relies on string
matching, which should be replaced with the exported sentinel error check.
Update the error handling around the line-size validation in the parsing flow to
use errors.Is against bufio.ErrTooLong instead of strings.Contains(err.Error(),
"token too long"), while keeping the existing wrapped message for the oversized
line case and preserving the fallback return of the original error.
In `@program/parse.go`:
- Around line 39-42: The oversized-line handling in parse.go is using a fragile
string check for the scanner error in both scan phases. Update the error
handling in the relevant parsing logic around the scanner/line processing paths
to use errors.Is with bufio.ErrTooLong instead of strings.Contains(err.Error(),
"token too long"), and apply the same change wherever this check appears in the
file so the behavior stays consistent.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 7e319d49-f2be-4c44-84fb-4011dafd033e
📒 Files selected for processing (4)
instr/parse.goinstr/parse_test.goprogram/parse.goprogram/parse_test.go
🚧 Files skipped from review as they are similar to previous changes (1)
- instr/parse_test.go
Summary
bufio.Scannerbuffers directly ininstr.ParseAllandprogram.Parsewith a 1 MiB maximum line size.internal/textparsehelper.Fixes #109.
Validation
Summary by CodeRabbit