A read-only indexed binary file format optimized for range queries over time-series keys with small fixed-size record values.
Golf files are created by appending records and then sealing the file, after which they are immutable and efficient for random range reads.
- Range queries via binary search over a block index -- O(log B + log N)
- Block-level compression (LZ4 or Zstd) -- only touched blocks are decompressed
- mmap + standard I/O read strategies
- Fixed-size records for O(1) random access within decompressed blocks
- CRC32C checksums at header, block, and index levels
- Configurable timestamp resolution (nanoseconds, microseconds, milliseconds)
- Multi-language: Rust, Go, Python, TypeScript, Swift, Kotlin
Package names below are local module names, not published packages -- nothing is on crates.io, npm, PyPI, or any Go module registry yet; build from source (see Installation).
| Language | Package / Module | Write | Read (LZ4) | Zstd | In-memory Reader | mmap Reader |
|---|---|---|---|---|---|---|
| Rust | golf crate |
✅ | ✅ | ✅ | ✅ | ✅ |
| Go | golf-format/golf-go |
✅ | ✅ | ✅ | ✅ (bytes.Reader) |
✅ |
| Python | golf-format |
✅ | ✅ | ✅ | ✅ | ✅ |
| TypeScript | golf-format (npm) |
✅ | ✅ | ❌¹ | ✅ | n/a² |
| Swift | Golf package |
✅ | ✅ | ❌¹ | ✅ | planned |
| Kotlin/JVM | golf package |
✅ | ✅ | ❌¹ | ✅ | planned |
- These implementations raise a clear error when encountering Zstd instead of bundling a decoder; see docs/COMPATIBILITY.md.
- The TypeScript reader is buffer-based, which plays the same role as mmap on the JS runtime.
All six implementations are byte-compatible with each other and validated against shared fixtures on every test run -- a file written by any language can be read by every language that supports its codec.
See SPEC.md for the complete byte-level specification.
┌──────────────────────┐ offset 0
│ Header │ 64 bytes + optional metadata
├──────────────────────┤
│ Data Block 0 │ compressed
│ Data Block 1 │
│ ... │
│ Data Block N-1 │
├──────────────────────┤
│ Block Index │ 40 bytes per block descriptor
├──────────────────────┤
│ Footer │ 32 bytes (at EOF)
└──────────────────────┘
use golf::{GolfWriter, GolfReader};
use golf::writer::WriterConfig;
use golf::header::{Compression, TimestampResolution};
// Write
let mut writer = GolfWriter::new(WriterConfig {
record_value_size: 8,
compression: Compression::Lz4,
block_capacity: 8192,
..Default::default()
});
writer.append(1000, vec![1u8; 8]).unwrap();
writer.append(2000, vec![2u8; 8]).unwrap();
let file = std::fs::File::create("data.golf").unwrap();
writer.seal(file).unwrap();
// Read
let mut reader = GolfReader::open_path("data.golf").unwrap();
let records = reader.query(1000, 2000).unwrap();More: rust/README.md
import golf "github.com/golf-format/golf-go"
// Write
w := golf.NewWriter(golf.WriterConfig{
RecordValueSize: 8,
Compression: golf.CompressionLz4,
BlockCapacity: 8192,
})
w.Append(1000, []byte{1,1,1,1,1,1,1,1})
w.Append(2000, []byte{2,2,2,2,2,2,2,2})
f, _ := os.Create("data.golf")
w.Seal(f)
// Read
reader, _ := golf.OpenFile("data.golf")
records, _ := reader.Query(1000, 2000)More: go/README.md
from golf import GolfWriter, GolfReader, WriterConfig, Compression
# Write
writer = GolfWriter(WriterConfig(record_value_size=8, compression=Compression.LZ4))
writer.append(1000, b'\x01' * 8)
writer.append(2000, b'\x02' * 8)
with open('data.golf', 'wb') as f:
writer.seal(f)
# Read
with GolfReader.open('data.golf') as reader:
records = reader.query(1000, 2000)Install from this repo: pip install ./python.
More: python/README.md
import { GolfWriter, GolfReader, Compression } from 'golf-format';
// Write
const writer = new GolfWriter({ recordValueSize: 8, compression: Compression.Lz4 });
writer.append(1000n, Buffer.alloc(8, 1));
writer.append(2000n, Buffer.alloc(8, 2));
fs.writeFileSync('data.golf', writer.seal());
// Read
const reader = GolfReader.open('data.golf');
const records = reader.query(1000n, 2000n);Timestamps are JavaScript BigInt. More: typescript/README.md
import Golf
// Write
let writer = GolfWriter(config: WriterConfig(recordValueSize: 8, compression: .lz4))
try writer.append(timestamp: 1000, value: [UInt8](repeating: 1, count: 8))
try writer.append(timestamp: 2000, value: [UInt8](repeating: 2, count: 8))
try writer.seal(to: URL(fileURLWithPath: "data.golf"))
// Read
let reader = try GolfReader.open("data.golf")
let records = try reader.query(start: 1000, end: 2000)Zero third-party dependencies (software CRC32C + built-in LZ4 block codec). More: swift/README.md
import golf.*
// Write
val writer = GolfWriter(WriterConfig(recordValueSize = 8, compression = Compression.LZ4))
writer.append(1000UL, ByteArray(8) { 1 })
writer.append(2000UL, ByteArray(8) { 2 })
writer.sealTo(File("data.golf"))
// Read
val reader = GolfReader.open("data.golf")
val records = reader.query(1000UL, 2000UL)Timestamps are ULong; zero third-party dependencies (JDK CRC32C + built-in
LZ4 block codec). More: kotlin/README.md
Golf packages are not yet published -- there is nothing on crates.io, PyPI, npm, or any Go module registry. For now, install from source and verify each implementation by running its test suite:
git clone https://github.com/gregnazario/golf.git
cd golf
# Rust
cd rust && cargo test
# Go
cd ../go && go test ./...
# Python
cd ../python
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]" && pytest
# TypeScript
cd ../typescript && npm install && npm run build && npm test
# Swift
cd ../swift && swift test
# Kotlin (uses kotlinc directly -- no Gradle/network needed)
bash scripts/test.shA passing test suite means the implementation is built and usable locally from
its directory (e.g., import golf from the activated Python virtualenv, or
import from typescript/dist/ after building). Each language directory has its
own README with package-level install instructions. See
Running Tests for details, including the full cross-language
compatibility suite.
# Rust
cd rust && cargo test
# Go
cd go && go test ./...
# Python
cd python && python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]" && pytest
# TypeScript
cd typescript && npm install && npm test
# Swift
cd swift && swift test
# Kotlin (uses kotlinc directly -- no Gradle/network needed)
bash kotlin/scripts/test.sh
# Full cross-language compatibility suite (writes fixtures from every
# implementation, then verifies every implementation reads them all)
bash testdata/run_compat_tests.shThe compat suite regenerates everything under testdata/*.golf; see
docs/COMPATIBILITY.md for what each fixture exercises.
| Document | Contents |
|---|---|
| SPEC.md | Canonical byte-level file format specification |
| docs/COMPATIBILITY.md | Cross-language parity guarantees + fixture suite |
| CONTRIBUTING.md | How to change the format/implementations safely |
rust/README.md, go/README.md, ... |
Per-language install, API reference, examples |
golf/
SPEC.md -- Canonical binary format specification
rust/ -- Rust implementation (reference)
go/ -- Go implementation
python/ -- Python implementation
typescript/ -- TypeScript/Node.js implementation
swift/ -- Swift package (zero dependencies)
kotlin/ -- Kotlin/JVM library (zero dependencies)
testdata/ -- Cross-language test fixtures + compat runner
docs/ -- Design & compatibility documentation
Apache-2.0 -- see LICENSE.