Jelly RDF Binary Codec

The fastest RDF serialization format for Dart.
Streaming binary encoding based on Protocol Buffers — significantly smaller and faster than Turtle or N-Triples.

The Challenge

Text-based RDF formats like Turtle and N-Triples are human-readable, but they pay a steep performance price for that readability.

Text Formats Are Slow

  • Every IRI is repeated in full on every triple
  • UTF-8 parsing is CPU-intensive at scale
  • No streaming: must buffer the entire document
  • Large output size means more I/O overhead

Jelly Solves This

  • IRI lookup tables — referenced by compact integer ID
  • Protobuf wire format — minimal parsing overhead
  • Frame-level streaming — process data as it arrives
  • 75% of Turtle's output size on real datasets

On the large benchmark (17.2k triples), Jelly encodes in 81% of Turtle's time and decodes in just 7% of Turtle's time, while producing output that is 25% smaller. See the full benchmark results.

The Solution

A complete Dart implementation of the Jelly RDF specification — batch and streaming, graphs and datasets.

Batch API — drop-in replacement for text codecs

import 'package:locorda_rdf_jelly/jelly.dart';

// Encode a graph to compact binary Jelly format
final bytes = jellyGraph.encode(graph);

// Decode back
final decoded = jellyGraph.decode(bytes);

✓ Same API as Turtle and other codecs
✓ Plugs into RdfCore for content-type dispatch
✓ 82 conformance tests pass (RDF 1.1)

Getting Started

Install the package

dart pub add locorda_rdf_jelly locorda_rdf_core
// Quick start: batch encode/decode with the pre-configured global codecs
import 'dart:typed_data';
import 'package:locorda_rdf_core/core.dart';
import 'package:locorda_rdf_jelly/jelly.dart';

void main() {
  // Build a small graph
  final graph = RdfGraph(triples: [
    Triple(
      IriTerm('http://example.org/alice'),
      IriTerm('http://xmlns.com/foaf/0.1/name'),
      LiteralTerm.string('Alice'),
    ),
    Triple(
      IriTerm('http://example.org/alice'),
      IriTerm('http://xmlns.com/foaf/0.1/knows'),
      IriTerm('http://example.org/bob'),
    ),
  ]);

  // Encode to compact binary Jelly format
  final Uint8List bytes = jellyGraph.encode(graph);
  print('Encoded ${bytes.length} bytes (vs ~200 bytes as Turtle)');

  // Decode back
  final decoded = jellyGraph.decode(bytes);
  print('Decoded ${decoded.size} triples');
}
// Frame-level streaming — encode and decode a stream of triple batches
import 'dart:async';
import 'package:locorda_rdf_core/core.dart';
import 'package:locorda_rdf_jelly/jelly.dart';

Future<void> main() async {
  // A stream of triple batches (e.g. from a database or file in pages)
  final Stream<Iterable<Triple>> triplePages = Stream.fromIterable([
    [
      Triple(
        IriTerm('http://example.org/s1'),
        IriTerm('http://example.org/p'),
        LiteralTerm.string('first batch'),
      ),
    ],
    [
      Triple(
        IriTerm('http://example.org/s2'),
        IriTerm('http://example.org/p'),
        LiteralTerm.string('second batch'),
      ),
    ],
  ]);

  // Encode — lookup tables are shared across frames for better compression
  final encodedStream = JellyTripleFrameEncoder().bind(triplePages);

  // Collect the encoded frames
  final frames = await encodedStream.toList();
  print('Encoded ${frames.length} Jelly frames');

  // Decode — each frame emits a List<Triple>
  final byteStream = Stream.fromIterable(frames);
  final decoded =
      JellyTripleFrameDecoder().bind(byteStream).expand((frame) => frame);

  final triples = await decoded.toList();
  print('Decoded ${triples.length} triples across all frames');
}
// Integration with RdfCore for content-type-based dispatch
import 'package:locorda_rdf_core/core.dart';
import 'package:locorda_rdf_jelly/jelly.dart';

void main() {
  // Register Jelly alongside the built-in text codecs
  final rdfCore = RdfCore.withStandardCodecs(
    additionalBinaryGraphCodecs: [jellyGraph],
    additionalBinaryDatasetCodecs: [jelly],
  );

  // Suppose we already have a graph (e.g. parsed from Turtle)
  final graph = RdfGraph(triples: [
    Triple(
      IriTerm('http://example.org/s'),
      IriTerm('http://example.org/p'),
      LiteralTerm.string('hello'),
    ),
  ]);

  // Encode to Jelly via content-type dispatch
  final bytes = rdfCore.encodeBinary(graph, contentType: jellyMimeType);
  print('Jelly bytes: ${bytes.length}');

  // Decode back — codec selection is automatic
  final decoded = rdfCore.decodeBinary(bytes, contentType: jellyMimeType);
  print('Decoded ${decoded.size} triples');
}

Key Features

Fastest in the Suite

Direct protobuf wire-format writing — no intermediate GeneratedMessage allocation on either encode or decode. O(1) LRU lookup tables, IRI term caching, and repeated-term delta encoding compound to give Jelly its lead. See the benchmarks.

🌊 Frame-Level Streaming

JellyTripleFrameEncoder / JellyTripleFrameDecoder (and quad equivalents) are idiomatic StreamTransformerBase instances. Compose with .bind() and .expand() just like any other Dart stream transformer.

🔗 Cross-Frame Table Sharing

In streaming mode, lookup tables accumulate across frames. IRIs and datatypes seen in early frames are reused in later ones — giving better compression for continuous streams than independent per-frame encoding.

📊 Graphs & Datasets

Batch and streaming APIs for both single graphs (TRIPLES physical type) and full datasets (QUADS or GRAPHS physical type). Named graph boundaries are preserved in GRAPHS mode.

Conformance Tested

82 official Jelly-RDF conformance tests pass (51 decode + 31 encode), executed via the jelly-protobuf test suite. All RDF 1.1 test cases pass.

🔌 RdfCore Plugin

Register JellyGraphCodec and JellyDatasetCodec with RdfCore.withStandardCodecs for unified content-type dispatch alongside Turtle, JSON-LD, and RDF/XML.

Use Cases

📡 High-Throughput Pipelines

Process millions of triples with minimal CPU and memory overhead. Jelly's binary format eliminates the bottlenecks of text parsing at scale.

💾 Compact Storage

Store RDF graphs in databases or files with up to 25% less space than Turtle. Lookup-table compression is especially effective for datasets with repeated IRIs.

🌊 Streaming Ingestion

Feed triples into a consumer as they are decoded, without buffering the entire dataset. Frame-level streaming keeps end-to-end latency low.

Ready to Go Fast?

Read the documentation or explore other RDF packages.