gokdl2 is a Go library for the KDL Document Language.
This repository is an actively maintained fork of sblinch/kdl-go. The original project remains the historical upstream reference, while ongoing development for this fork lives here in njreid/gokdl2.
Comments survive a round trip, and the parser is much harder to trip up.
- Comments (#43): with
ParseComments, every comment is kept in place: trailing, end-of-block and end-of-document comments, comments between arguments and after}, multi-line/* */comments with their indentation, and slash-dashed nodes, entries and blocks. Output is stable on a second pass. See USAGE.md. - Hang fixed (#45): a number with a huge exponent, such as
1e-10210200, made parsing or generating run for minutes. It now takes well under a millisecond and round-trips exactly. - Parsing edge cases (#44): values at the very end of the input are no longer dropped. Unclosed blocks, a dangling
/-, duplicate child blocks, entries after a block, and text after a\continuation are rejected./* */works as a separator everywhere.
Every name gokdl2 generates now parses back, in both KDL versions.
- Generation quotes names that would be misread: v2 keywords and number-like names such as
true,inf,+1and.5(#5); v1true,false,null, sign-plus-digit names and names starting withr#(#39); and, in both versions, names starting with a backtick (#40) or containing U+FEFF. - Unmarshal: repeated keys in
,multiplemap nodes no longer panic (#3, #4). - Parsing: KDL v2 identifiers may start with a comma (#41), and v2 rejects
-.5x-style identifiers (#5). - Testing: a fuzz target checks that any string round-trips as a node name, property key and argument.
Faster, stricter, and much closer to the spec. Full details are in PR #29 and the v0.7.0 release.
Parsing is about 2× faster, generation about 3× faster, and all of them allocate far less. These are medians of 5 alternating runs of v0.6.1 against v0.7.0, on a 1 MB document, using bench_test.go:
| Benchmark | Time | Memory per op | Allocations |
|---|---|---|---|
| Parse v2 (auto) | 76 → 39 ms (−49%) | 21.8 → 11.8 MB | 273k → 181k |
| Parse v1 (auto) | 66 → 35 ms (−46%) | 30.4 → 11.8 MB | 286k → 181k |
| Parse, 668 B document | 78 → 28 µs (−64%) | 80 → 12 KB | 206 → 139 |
| Generate | 14.3 → 5.2 ms (−64%) | 4.2 → 0.07 MB | 295k → 835 |
| Unmarshal | 99 → 71 ms (−28%) | 29.3 → 19.4 MB | 514k → 421k |
| Marshal | 43 → 33 ms (−23%) | 18.2 → 12.9 MB | 601k → 309k |
Where the gains come from:
- Tokenizer: ASCII fast paths for whitespace, identifiers and quoted strings (#24), and no allocations per token (#20).
- Parser: a flattened
[state][token]dispatch table instead of two map lookups per token (#25). - Input: read once instead of buffered and copied twice (#21).
- Memory: nodes and values come from small on-demand chunks instead of a large up-front arena (#22), and node properties are a slice instead of a map plus a slice (#23).
- Generator: builds each node in one buffer and writes it once (#26).
- Marshaler: caches struct type information across calls (#27).
Parsing now accepts valid KDL it used to reject:
- numbers directly followed by
},{,;, a comment,/-or\. For example,n {x 1}andn 0xff;failed before (#32); - in v2, a type-annotated argument after another argument, as in
node 1 (u8)2(#28); - signed hexadecimal, octal and binary numbers such as
-0xff(#14); - a literal U+FFFD character (#34);
- in v1, a comment directly after an argument (#32).
…and rejects invalid KDL it used to accept:
- a property left without a value, as in
node { a b= }(#14); \u{zz}and other non-hex Unicode escapes (#14);- a bare identifier as a v1 property value, as in
node x=bad(#33); - in v2, values not separated by whitespace, such as
1(u8)2and adjacent expression strings (#28).
Parsing behaviour:
- A failing
io.Readeris reported instead of silently producing an empty document (#6). - A
/- kdl-versionmarker is authoritative, and when a document parses as neither version the v2 error is reported (#11). - Quoted type annotations are decoded, so
("u8")and(u8)are equal (#17).
Error messages:
- They point at the start of the offending token, as kdl-rs does; an unclosed block is reported at its
{(#35). - Line numbers are correct in files with CRLF line endings (#30), and an off-by-one in columns after the first line is fixed.
- Strings report "unexpected newline" and "unterminated string" precisely (#15).
Generation:
- Generated KDL always parses back to the same document, including multi-line and raw strings,
\u{…}escapes and negative hex (#12, #31). - Hex values keep their type annotation (#13).
- Write errors are no longer dropped (#18).
Marshaling and unmarshaling:
UnmarshalWithOptionshonours all of its options (#7).encoding.TextMarshaleroutput is no longer truncated to its first word (#8).- Types with the same name no longer share field tables (#9).
- Structs stored in
interface{}no longer panic (#10). - Concurrent marshals no longer share scanner state (#19).
- All 563 official kdl-org v1 and v2 test cases run on every
go test ./...; the v2 suite used to be opt-in (#36). - 118 cases ported from kdl-rs, the official Rust implementation, including expected error locations.
- Fuzz tests check that the tokenizer's fast paths match its general path, and that strings survive a generate-and-parse round trip.
- CI runs
go vet, the tests and the race detector, and pushing av*tag creates a GitHub release (#16).
- Stricter parsing: documents that relied on the invalid forms listed above now fail to parse.
- Error positions: error lines and columns differ, and an unclosed block's message starts with
unclosed child block:. - Go version: Go 1.22 or later is required.
- New API:
document.Node.EmptyTypeanddocument.Value.EmptyType, which record an explicit("")annotationtokenizer.DetectVersiontokenizer.ScanOneN
Document.Retainis now a no-op.
- KDL v2 parsing
- KDL v1 parsing
- automatic parsing mode that tries v2 first and falls back to v1
- generating, marshaling, and unmarshaling KDL documents
- active KDL v2 support alongside KDL v1 compatibility
- automatic version selection through
ParseVersionAuto, with explicitParseVersionV1andParseVersionV2controls - backtick expression strings, preserved as
document.Expressionvalues for CEL or similar runtimes - friendlier parse errors with line, column, and source excerpts
- fast: about twice the parse speed of v0.6 (details)
- passes all 563 cases of the official KDL v1 and v2 compliance suites, plus 118 cases ported from kdl-rs
- deterministic output for generated documents and marshaled maps/properties
- familiar API and tag syntax, similar to
encoding/json - marshaling and unmarshaling for Go structs, maps, and custom
(Un)Marshalinterfaces - support for
encoding/json/v2-styleformatoptions fortime.Time,time.Duration,[]byte, andfloat32/64
import "github.com/njreid/gokdl2"Parse() uses automatic version selection: it tries KDL v2 first, then falls back to KDL v1.
data := "node .child\n"
doc, err := kdl.Parse(strings.NewReader(data))
if err != nil {
panic(err)
}
fmt.Println(doc.Version)
fmt.Println(doc.Nodes[0].Name.NodeNameStringV2())2
node
gokdl2 adds backtick-delimited expression strings to KDL for embedding code, such as CEL, JMESPath or SQL, without escaping every quote. They parse to document.Expression, a string type distinct from plain strings:
rule (cel)`request.auth.claims.role == "admin"`
deny ```
request.path.startsWith("/admin")
&& !request.auth.mfa
```expr := doc.Nodes[0].Arguments[0].Value.(document.Expression)- Single backticks hold a one-line expression, and triple backticks a multi-line one, dedented like KDL v2
"""strings. - Escapes are those of KDL quoted strings, with
\`for a backtick. - They work as arguments and property values, with optional type annotations, in v1, v2 and auto mode.
- They round-trip through generation and marshaling, and unmarshal into
document.Expression,stringorinterface{}.
This is an extension, not part of the KDL spec, so other parsers will reject documents that use it. The full reference is in USAGE.md.
For fuller examples covering parsing modes, expressions, generating, marshaling, unmarshaling, and relaxed nginx-style syntax, see USAGE.md.
Detailed marshaling and unmarshaling behavior is documented in docs/marshal.md and docs/unmarshal.md.
The repository vendors the official compliance suites for both KDL versions (kdl-org-v1/, and kdl-org-v2/ at the commit recorded in kdl-org-v2/VENDORED.md). Both run as part of the normal test suite, together with the cases ported from kdl-rs in upstream_cases_test.go:
go test ./...The broader kdl-org/kdl-test suite runs through TestKDLTestSuite when it is checked out at ../kdl-test, or at the path in KDL_TEST_SUITE_DIR.
To fuzz the tokenizer and the string round trip:
go test -run '^$' -fuzz FuzzFastPathMatchesRunePath ./internal/tokenizer
go test -run '^$' -fuzz FuzzStringRoundTrip .gokdl2 is actively maintained and tracks the newer work happening in this fork.
Issue reports and pull requests are welcome.
gokdl2 is released under the MIT license. See LICENSE for details.