The CLI does not recognize a multibyte delimiter when it straddles two read buffers. This undercounts records and can crash --update when two numeric records are passed together to literal_eval.
printf '1é2é3' | python -m tqdm --delim 'é' --buf-size 2 --bar-format '{n}it'
# reports 2it, expected 3it
printf '1é2é3' | python -m tqdm --delim 'é' --buf-size 2 --update --bar-format '{n}it'
# SyntaxError while parsing '1é2', expected 6it
posix_pipe searches each fresh chunk before combining it with the saved prefix. Search the combined bytes so split UTF-8 delimiters can match. The stdout payload should remain byte-for-byte unchanged.
Reproduced against master 9cf5a12, Python 3.12.13/macOS. Both public CLI regressions fail before the fix. Investigated with OpenAI Codex assistance.
The CLI does not recognize a multibyte delimiter when it straddles two read buffers. This undercounts records and can crash
--updatewhen two numeric records are passed together toliteral_eval.posix_pipesearches each fresh chunk before combining it with the saved prefix. Search the combined bytes so split UTF-8 delimiters can match. The stdout payload should remain byte-for-byte unchanged.Reproduced against master 9cf5a12, Python 3.12.13/macOS. Both public CLI regressions fail before the fix. Investigated with OpenAI Codex assistance.