End lists before trailing blank lines in mdast - #269

Summary

In 1.0.0, to_mdast("* a\n\n", &Default::default()) ends the list and the list item at offset 5, after the blank line. mdast-util-from-markdown ends both at 3. GFM footnote definitions end late the same way.

markdown-rs events exit a container after its trailing line endings and blank lines. mdast-util-from-markdown moves those ends back before them, in prepareList for list items. This does the same in to_mdast, for lists, list items, and footnote definitions: on exit, it walks back over trailing line endings and whitespace. When the last child is a container, it reuses that child's moved end, so each line ending is walked once.

Moving the exit events instead, as micromark does, gives the same trees but moves HTML line endings: 5 of 30,000 generated inputs regressed. Block quotes stay as they are: mdast-util-from-markdown keeps trailing > lines inside the quote. HTML doesn't change.

Performance

Callgrind, with one codegen unit, against 1.0.0 built the same way, across 7 option sets. On small documents, instructions change by −0.6% to +0.3%, within the noise of this method.

Tests

• * a\n\n asserts the exact tree. One footnote definition in tests/gfm_footnote.rs now ends at offset 10, where mdast-util-from-markdown ends it.
• The differential against 1.0.0 runs the spec examples, 463 local files, and a million random strings, in 7 option sets. Of its 1,001,481 inputs, 57,718 comparisons change, all in positions. Compared with mdast-util-from-markdown, 20,277 trees now match, and none that matched before now differ.
• In trees that still differ elsewhere, 13,408 changed nodes moved toward mdast-util-from-markdown and 26 moved away. The 26 are footnote definitions followed by indented lines, where mdast-util-from-markdown's end already differs from 1.0.0's.

As of 2026-09-29, CI on main fails for two reasons unrelated to this change. One is a new clippy lint on Rust 1.98, and the other is the swc/serde build break. #230 fixes both.

Claude, an AI assistant, helped me build this, and I reviewed it.