Archived: kept as a reference implementation. boer was an experiment to find out whether a JSON Schema validator built for cold starts and low per-instance memory could justify existing next to ajv. The implementation works and the numbers are real, but the advantage only matters for a workload that very few people have. This README is the write-up of that result. Negative results are still results.
The question. Serverless platforms like AWS Lambda and Cloudflare Workers start the same code thousands of times, and they bill or cap memory per instance. Codegen validators like ajv do their work up front, either compiling schemas at startup or shipping generated JavaScript. Can a validator that compiles schemas to a compact binary and interprets it at runtime save enough startup time and memory to be worth adopting?
What was built. A complete JSON Schema validator that passes 100% of the official JSON Schema Test Suite for drafts 4, 6, 7, 2019-09 and 2020-12, including $dynamicRef and unevaluated*. It uses no eval and no code generation. A compiled schema is a few flat typed arrays that can be dumped to a binary and loaded back in milliseconds.
The answer. It does what it set out to do, and the effect grows with schema size:
- For typical schemas, boer saves about 0.5–9 MB and 5–20 ms per instance. That's real, but far too small to change a memory tier or be noticed next to a Node Lambda's own cold start (100+ ms) and its ~45 MB baseline.
- For very large schemas, the savings are dramatic. On AWS CloudFormation, boer needs ~26 ms and ~5 MB, against ~3 s and 143–440 MB for ajv standalone.
- But almost nobody validates very large schemas inside short-lived or memory-capped instances. The tools that use such schemas (linters, editors, CI) run in long-lived processes. There, ajv's one-time startup cost doesn't matter and its 2–3× faster validation does.
So boer wins on its own axis, but that axis doesn't move a real-world decision for almost anyone. That isn't enough to justify the work of making a validator trustworthy enough for production (see What it would take).
All figures come from bench/memory (see Reproducing): Node 22 on a cloud Linux container, with a fresh process per run, reporting the median of 7 runs (3 for compile()).
- "Ready + first validate" is the time to load or compile the validator plus validating one small valid document.
- "Memory added" is the RSS growth over that same window, with a forced GC before and after. The run-to-run noise floor is about 0.6 MB.
- ajv standalone ranges. V8 compiles generated functions lazily, so ajv standalone is shown as a range: the low end is one small document, and the high end (
--no-lazy) has every function compiled, which is roughly full schema coverage. boer's cost doesn't depend on coverage.
| Schema | Size | ajv compile() at startup |
ajv standalone | boer |
|---|---|---|---|---|
| GitHub workflow | 99 KB | 176 ms | 14 ms | 7 ms |
| GitLab CI | 113 KB | 304 ms | 25 ms | 7 ms |
| AWS CloudFormation | 7.3 MB | 18.7 s | 3.0 s | 26 ms |
| Schema | ajv compile() at startup |
ajv standalone | boer |
|---|---|---|---|
| GitHub workflow | 22 MB | 1.6–4.5 MB | 1.2 MB |
| GitLab CI | 35 MB | 4.8–9.7 MB | 1.2 MB |
| AWS CloudFormation | 354 MB | 143–440 MB | 4.6 MB |
| Schema | Schema JSON (gzipped) | ajv standalone, minified (gzipped) | boer binary (gzipped) |
|---|---|---|---|
| GitHub workflow | 99 KB (14 KB) | 262 KB (31 KB) | 7 KB (3 KB) |
| GitLab CI | 113 KB (17 KB) | 508 KB (51 KB) | 12 KB (5 KB) |
| AWS CloudFormation | 7.3 MB (337 KB) | 25 MB (1.5 MB) | 691 KB (204 KB) |
boer interprets instead of generating code, so it's slower on a hot loop. On a ~15 KB payload (bench/runtime, Intel i7-14700, Node 25.9):
| Library | Avg (µs/iter) | Allocation (bytes) |
|---|---|---|
| ajv | ~2.1 | ~1,080 |
| boer | ~6.1 | ~161 |
| zod | ~19 | ~8,500–63,000 |
- Shipping size was never the problem. ajv's 36 MB of generated code for CloudFormation looks alarming, but it's so repetitive that it gzips to 1.5 MB, comfortably within Lambda's and Workers' upload limits. boer's binary is also not much smaller than the gzipped schema JSON itself (204 KB vs 337 KB), so "smaller artifact" isn't a selling point either.
- For realistic schemas, the savings are small in absolute terms. 4–9 MB and ~20 ms per instance for a GitLab-CI-sized schema are clear relative wins. But they're a few percent of a 128 MB function and a fraction of Node's own cold start, so nobody switches validators for them.
- The big wins need a rare workload. You only see 100× faster startup and 30–95× less memory with multi-megabyte schemas validated in many short-lived, memory-capped instances. Most big-schema users run long-lived processes, where ajv wins on throughput.
- Throughput is 2–3× below ajv. That's inherent to interpreting rather than generating code, and long-lived services care about it more than about startup.
- A validator is only useful if people trust it, and earning that trust is expensive. In one review session, three separate bugs surfaced where a fixed-size buffer overflowed silently and produced wrong answers. One of them made an earlier version of the CloudFormation benchmark meaningless (all three are fixed and covered by
packages/schema/tests/wide.test.js). There are two interpreters (validateanddiagnose) that must agree, and the code was written largely with AI assistance. Making that production-grade is months of work for a small audience.
If you have large schemas, many short-lived instances, a memory cap, and no eval all at once, boer's trade-off is the right one. The most plausible case is an edge API gateway validating requests against a large OpenAPI spec on Cloudflare Workers. boer doesn't parse OpenAPI documents yet, and it hasn't been benchmarked against the other eval-free validator, @cfworker/json-schema. Both are the first things to check before building on it.
- Correctness: differential fuzzing against ajv, running the compliance suite against
diagnose()as well asvalidate(), and an audit of every remaining fixed-size buffer. - Format: a versioned, documented binary format, with the root typedef stored in the binary.
- Tooling: a CLI for the build step, and OpenAPI input.
- Evidence: benchmarks on real Workers and Lambda, not just Node.
These findings apply beyond this repo.
Design ideas that worked:
- Types as integer pointers. Every type is a plain JS number that stays within V8's small-integer range, so passing types around never allocates. Small validators (
maxLength: 20, number bounds, short enums) are packed into the number itself and need no heap entry at all. - Schemas as flat typed arrays. A compiled catalog has no object graph, closures or generated code. That makes serialization a straight copy of typed-array sections, and
load()runs in milliseconds even for CloudFormation. - Interning property names. Every property name becomes an integer
keyId, and objects are stored as sorted[keyId, typeId]pairs. That gives binary-search lookup, and repeated names across a huge schema are stored once. - Keeping errors out of the hot path.
validate()only returns a boolean and allocates almost nothing; error paths come from a separatediagnose()pass. That's the reason boer allocates ~161 bytes per validation where ajv allocates ~1 KB.
Pitfalls worth knowing about:
- Typed arrays fail silently. Writes past the end are dropped and reads return
undefined, so an undersized buffer produces wrong answers, not crashes. Every buffer written through a moving pointer needs a capacity check and growth. Test with wide inputs, such as ananyOfwith thousands of branches, not just deep ones. - Measuring V8 memory is easy to get wrong:
- The first function V8's optimizing compiler handles costs a one-time ~5 MB RSS, charged to whatever code triggers it first. Warm the optimizer up before measuring.
- V8 compiles generated functions lazily, so measuring codegen validators with one tiny document understates them. Report a range up to
--no-lazy. - Total process RSS hides both effects; measure the change across a GC'd window.
- Size alone doesn't argue for a validator. Generated validator code compresses extremely well. Startup time and resident memory are the costs to look at.
bun install
git submodule update --init # JSON Schema Test Suite, needed for the tests
bun make # build every package into dist/
bun test # ~15,700 tests, targets the built bundles
node bench/memory/prepare.js # build ajv standalone + boer artifacts (~4 min, mostly minifying CloudFormation)
node bench/memory/run.js 7 --with-compile # startup and memory tables
node --expose-gc bench/runtime/bench_large.js # throughputCompile at build time:
import fs from 'fs';
import { catalog } from '@boer/validate';
import { parseJSONSchema } from '@boer/schema';
import { compile } from '@boer/compiler';
import { dump } from '@boer/inspect';
const cat = catalog();
const root = compile(cat, parseJSONSchema(schema, 'draft-07'))[0].schema;
fs.writeFileSync('schema.boer', dump(cat));
fs.writeFileSync('schema.root.json', JSON.stringify(root)); // the binary doesn't store the rootLoad at runtime:
import { catalog } from '@boer/validate';
import { load } from '@boer/inspect';
import { createDiagnose } from '@boer/diagnose';
const cat = catalog(load(new Uint8Array(fs.readFileSync('schema.boer'))));
const root = JSON.parse(fs.readFileSync('schema.root.json', 'utf8'));
if (!cat.validate(data, root)) {
console.log(createDiagnose(cat)(data, root));
// [{ path: 'Resources.B', message: '...' }]
}There is also a Zod-style DSL (object, array, union, or, literal, optional, ...) exported from boer, which compiles into the same catalog:
import { object, string, number, or, optional, validate, diagnose } from 'boer';
const User = object({ id: or(string(), number()), name: string(), bio: optional(string()) });
if (!validate(data, User)) {
console.log(diagnose(data, User));
}Bit 0: COMPLEX (0 = primitive, 1 = complex pointer)
Bit 1: NULLABLE (type accepts null)
Bit 2: OPTIONAL (type accepts undefined)
Bits 3+: payload (meaning depends on COMPLEX bit)
When COMPLEX is 0, bits 3–7 are primitive flags (ANY, STRING, NUMBER, INTEGER, BOOLEAN), and small validators are packed into the upper bits. When COMPLEX is 1, bits 3+ index the KINDS vtable. Pointers stay within bits 0–29 so V8 never boxes them.
- KINDS (
Uint32Array) holds stride-2 entries of[kind_header, slab_offset]. The header's low 4 bits are the kind (object, array, anyOf, tuple,$dynamicRef, ...), and its high bits are flags likeK_STRICT(additionalProperties: false). Entries with validators grow to stride 4:[header, slab_offset, validator_header, validator_ptr]. - SLAB (
Uint32Array) is bulk storage, made of length-prefixed runs. An object is[length, keyId, typeId, keyId, typeId, ...], sorted bykeyId. - VALIDATORS (
Float64Array) holds numeric payloads (minimum,maxLength, ...). A bitmask in the KINDS entry says which validators are present, and a SWAR popcount finds each one's slot.
dump() writes a 96-byte header with a CRC32 checksum, then the used regions of SLAB, KINDS and VALIDATORS, followed by the dictionary, regexes and enum constants. load() verifies the checksum and copies each section back into fresh typed arrays. The only real work is decoding dictionary strings and building regexes.
- The binary format is unversioned and may change.
- The root typedef isn't stored in the binary.
diagnose()is a separate interpreter, and the compliance suite only runs againstvalidate().- There is no CLI and no OpenAPI input.
| Package | What it does |
|---|---|
boer |
DSL entry point, re-exports the builder, validate and diagnose |
@boer/core |
Bit constants, type definitions, shared utilities |
@boer/builder |
DSL allocators (object, array, union, etc.) |
@boer/validate |
Catalog and validation engine |
@boer/schema |
JSON Schema parser (all drafts) |
@boer/compiler |
Compiles parsed JSON Schema AST into the catalog |
@boer/diagnose |
Error reporting with paths |
@boer/inspect |
Binary dump / load and debug printing |
Apache-2.0