Tags: wwwcojp/ja_sentence_segmenter
Tags
Merge branch 'fix/concatenator-dos-guard': bound the concatenation, o… …pt-in Fixes the one finding that survived a Claude Security scan: the former matching rule was re-applied to an unboundedly accumulated buffer once per input line, and the buffer was rebuilt each time, making concatenate_matching quadratic in the input size — 30s of CPU for a 1.8MB input. Adds max_concatenate_length, defaulting to None. A workflow review showed a default-on bound silently changes emitted text, not just split positions, so the bound is opt-in and its three side effects are documented rather than hidden. Also accepts str input, raises TypeError instead of discarding unsupported iterables, and compiles the rules once per call. Releases 0.2.0.