What happens
POST /_bulk truncates an action-line _id (and _index, op_type) at the first backslash-escaped quote. Two distinct ids that share a prefix up to an escaped quote collapse to the same stored id, and the second document silently overwrites the first. The single-document API handles the same id correctly.
Reproduce (any node; this was a throwaway --insecure node built from 02aa319d, the code in question is unchanged from main)
printf '%s\n' '{"index":{"_index":"t-bulkid","_id":"a\"b"}}' '{"n":1}' \
'{"index":{"_index":"t-bulkid","_id":"a\"c"}}' '{"n":2}' > bulkid.ndjson
curl -s -XPOST -H 'content-type: application/x-ndjson' \
'http://127.0.0.1:9854/_bulk?refresh=true' --data-binary @bulkid.ndjson
curl -s 'http://127.0.0.1:9854/t-bulkid/_search'
Observed: the bulk response reports _id: "a\\" for both items (created, then updated), and the index holds ONE document, _id = a\ with {"n":2}. {"n":1} is gone, and the response says nothing went wrong.
Expected: two documents, ids a"b and a"c — which is what PUT /t-bulkid/_doc/x%22y does ("_id":"x\"y", created).
Root cause
engine/crates/xerj-engine/src/bulk.rs, the fast-path find_field closure (around line 567-616): after the opening quote it advances with while j < inner.len() && inner[j] != b'"' and returns the raw bytes. It does not skip \", and it does not JSON-unescape the value, so \\, \/ and \uXXXX in an id are also stored as their escape sequences rather than the characters they name.
Suggested fix
Either fall back to the serde_json slow path when the value contains a backslash, or skip escaped bytes in the scan and unescape the slice. A test with two ids differing only after an escaped quote pins the overwrite.
Found while loading hostile-id fixtures for the security review of #945; it is not caused by that PR.
What happens
POST /_bulktruncates an action-line_id(and_index,op_type) at the first backslash-escaped quote. Two distinct ids that share a prefix up to an escaped quote collapse to the same stored id, and the second document silently overwrites the first. The single-document API handles the same id correctly.Reproduce (any node; this was a throwaway
--insecurenode built from02aa319d, the code in question is unchanged frommain)Observed: the bulk response reports
_id: "a\\"for both items (created, thenupdated), and the index holds ONE document,_id=a\with{"n":2}.{"n":1}is gone, and the response says nothing went wrong.Expected: two documents, ids
a"banda"c— which is whatPUT /t-bulkid/_doc/x%22ydoes ("_id":"x\"y", created).Root cause
engine/crates/xerj-engine/src/bulk.rs, the fast-pathfind_fieldclosure (around line 567-616): after the opening quote it advances withwhile j < inner.len() && inner[j] != b'"'and returns the raw bytes. It does not skip\", and it does not JSON-unescape the value, so\\,\/and\uXXXXin an id are also stored as their escape sequences rather than the characters they name.Suggested fix
Either fall back to the
serde_jsonslow path when the value contains a backslash, or skip escaped bytes in the scan and unescape the slice. A test with two ids differing only after an escaped quote pins the overwrite.Found while loading hostile-id fixtures for the security review of #945; it is not caused by that PR.