Skip to content

fix(body): reject invalid UTF-8 in JSON requests - #142

Merged
hjianbo merged 1 commit into
emqx:masterfrom
hjianbo:fix/json-utf8-r14
Oct 10, 2026
Merged

hjianbo merged 1 commit into
emqx:masterfrom
hjianbo:fix/json-utf8-r14

Conversation

@hjianbo

@hjianbo hjianbo commented Sep 29, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Reject malformed UTF-8 and unpaired Unicode surrogate escapes in application/json requests before downstream schema validation.
  • Keep the fixed HTTP 400 BAD_REQUEST response without reflecting request data.
  • Cover valid Unicode round trips, malformed keys/comments/values, 1 MiB inputs, and characters split across HTTP chunks through real HTTP requests.

Valid emoji, paired surrogate escapes, and U+FFFD remain supported. Clients that relied on replacing malformed Unicode now receive HTTP 400; other JSX strictness options are unchanged.

Related issues: emqx/emqx#19168, emqx/emqx#19169.
Target: minirest master (1.5.x).

Rollout plan

  1. Merge into minirest master and publish a new 1.5.x patch. No backport to minirest 1.4.x or EMQX 6.0–6.2 is planned.
  2. Update EMQX dev-63, dev-70, and master (EMQX >=6.3), with the corresponding release branches following the release process.
  3. After each dependency update, verify the listener REST API scenarios: malformed input returns a bounded error and creates no listener.

Validation

  • Local OTP 27 on the master-based patch: make fmt-check, make xref, make eunit, make ct (22/22), and make cover passed. No EUnit cases are defined.
  • The invalid-input and chunked-request regression tests previously detected HTTP 200 instead of HTTP 400 against the original decoder.

This comment was marked as outdated.

@hjianbo
hjianbo marked this pull request as ready for review October 8, 2026 07:50
@hjianbo
hjianbo requested a review from a team October 8, 2026 07:56
@zmstone

zmstone commented Oct 8, 2026

Copy link
Copy Markdown
Member

recommend fix to master, no back port to minirest 1.4 (for emqx, it means >=6.3).

Reject malformed Unicode before schema validation while retaining the fixed BAD_REQUEST response. Cover valid Unicode round trips, malformed keys and values, bounded errors for large inputs, and UTF-8 split across HTTP chunks.
@hjianbo
hjianbo force-pushed the fix/json-utf8-r14 branch from 7d64017 to 9b1d2bd Compare October 9, 2026 00:59
@hjianbo
hjianbo changed the base branch from release-1.4 to master October 9, 2026 00:59
@hjianbo
hjianbo requested a balanced review from Copilot October 9, 2026 01:05

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The focused decoder change is well covered by end-to-end tests and preserves the established bounded error response.

0 open findings

🧠 Review effort: Balanced

@hjianbo
hjianbo merged commit 669b90e into emqx:master Oct 10, 2026
@hjianbo
hjianbo deleted the fix/json-utf8-r14 branch October 10, 2026 00:42
@hjianbo hjianbo mentioned this pull request Oct 10, 2026
3 tasks done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants