Skip to content

Hritikd/mend

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

mend

Repair malformed JSON from LLMs into valid JSON.
Trailing commas, single quotes, unquoted keys, markdown fences, comments, Python literals, and output that just got cut off — mended into something json.loads accepts.

CI Python 3.10+ License: MIT runtime deps: 0

▶ Try the live playground — paste broken JSON, watch it get repaired in your browser.


The problem

You asked the model for JSON. You got:

Sure! Here's the JSON:
```json
{
  'name': 'Ada',        // single quotes + a comment
  "skills": ["py", "ml",],   // trailing comma
  "active": True,        // Python literal
  "bio": "cut off mid-str

json.loads throws on every line of that. When you call a hosted model you get a string, not logits — so you can't constrain decoding; you have to repair what came back. mend does exactly that.

Install

pip install mend-json

Zero runtime dependencies — pure standard library.

Usage

import mend

mend.loads("{'ok': True, 'items': [1, 2, 3")   # tolerant json.loads
# {'ok': True, 'items': [1, 2, 3]}

mend.repair('```json\n{name: "Ada", age: 36,}\n```')   # -> valid JSON string
# '{"name": "Ada", "age": 36}'

From the shell:

cat llm_output.txt | mend --indent 2

What it fixes

Defect Example in → out
Markdown fences ```json {...} ```{...}
Surrounding prose Here you go: {...} thanks!{...}
Trailing commas [1, 2, 3,][1, 2, 3]
Single quotes {'a': 'b'}{"a": "b"}
Unquoted keys {a: 1}{"a": 1}
Python literals {"x": True, "y": None}{"x": true, "y": null}
// and /* */ comments {"a": 1 /* c */}{"a": 1}
Truncated / streaming output {"a": {"b": [1, 2{"a": {"b": [1, 2]}}
Unterminated strings {"msg": "hello{"msg": "hello"}

How it works

mend is a forgiving recursive-descent scanner that never throws on input. It strips fences and comments, jumps to the first real value, and parses leniently — tolerating stray commas, either quote style, and bare keywords. It keeps a stack of open containers and, at end-of-input, auto-closes whatever is still open (that's the trick that recovers truncated/streaming output). The result is a Python object, which repair() re-serializes as guaranteed- valid JSON.

Does it actually work? (reproducible benchmark)

16 malformed samples covering every defect above:

approach recovered into valid JSON
json.loads (stdlib) 0/16 (0%)
mend 16/16 (100%)

~122,000 repairs/sec, pure Python. The test suite (49 cases) asserts that every repair is parseable by the real json module and that valid JSON is returned unchanged.

python benchmarks/bench.py        # reproduce

Scope & honest notes

  • mend is best-effort recovery, not a validator. It assumes the input is meant to be JSON and makes it parseable; it won't tell you the model got the schema wrong (pair it with a schema check, or prevent the problem upstream with constrained decoding).
  • Ambiguous cases resolve pragmatically: an unrecognized bare word becomes a string ({"status": active}{"status": "active"}); a value buried in pure prose with no {/[ raises JSONRepairError rather than guessing.
  • Numbers, strings, and nesting are preserved exactly; round-tripping valid JSON returns the same object.

Development

make install   # pip install -e ".[dev]"
make test      # 49 tests
make lint      # ruff
make bench     # regenerate benchmarks/results.md

License

MIT © Hritik Datta

About

Repair malformed JSON from LLMs into valid JSON — trailing commas, single quotes, unquoted keys, markdown fences, truncated output. Zero-dependency, with a live playground.

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages