Lib.rs

arrow

Apache Arrow

arrow-select

Selection kernels for arrow arrays

arrow-schema

Defines the logical types for arrow arrays

datafusion

in-memory query engine that uses Apache Arrow as the memory model

polars

DataFrame library based on Apache Arrow

arrow-cast

Cast kernel and utilities for Apache Arrow

lance

A columnar data format that is 100x faster than Parquet for random access

geoarrow-array

GeoArrow array definitions

pyo3-arrow

Arrow integration for pyo3

arrow-odbc

Read/Write Apache Arrow arrays from/to ODBC data sources

vortex

file format with all builtin codecs and a sampling compressor

arrow-ipc

Support for the Arrow IPC format

orc-rust

Apache ORC file format using Apache Arrow in-memory format

arrow-buffer

Buffer abstractions for Apache Arrow

arrow-array

Array abstractions for Apache Arrow

arrow-arith

Arrow arithmetic kernels

arrow-json

Support for parsing JSON format to and from the Arrow format

lance-datafusion

Internal utilities used by other lance modules to simplify working with datafusion

arrow-data

Array data abstractions for Apache Arrow

arrow-ord

Ordering kernels for arrow arrays

arrow-string

String kernels for arrow arrays

polars-arrow-format

Unofficial flatbuffers and tonic code of Apache Arrow spec

datafusion-cli

Command Line Client for DataFusion query engine

arrow-csv

Support for parsing CSV format to and from the Arrow format

datafusion-sqllogictest

DataFusion sqllogictest driver

arrow-digest

Stable hashes for Apache Arrow

narrow

Apache Arrow

lance-bitpacking

Vendored copy of https://github.com/spiraldb/fastlanes for use in Lance

ballista

Distributed Compute

arrow-avro

Support for parsing Avro format into the Arrow format

geodatafusion

Spatial extensions for Apache DataFusion

polars-arrow

Minimal implementation of the Arrow specification forked from arrow2

tsdb_timon

Efficient local storage and Amazon S3-compatible data synchronization for time-series data, leveraging Parquet for storage and DataFusion for querying, all wrapped in a simple and intuitive API

datafusion-physical-expr

Physical expression implementation for DataFusion query engine

pyo3-introspection

Introspect dynamic libraries built with PyO3 to get metadata about the exported Python types

fsst

FSST string compression for Lance

vortex-datafusion

Apache Datafusion integration for Vortex

polars-parquet

Apache Parquet I/O operations for Polars

xml2arrow

Efficiently convert XML data to Apache Arrow format for high-performance data processing

lance-datagen

A columnar data format that is 100x faster than Parquet for random access

lance-file

Lance file format

geoarrow-schema

GeoArrow geometry type and metadata definitions

vortex-scalar

Vortex Scalars

datafusion-catalog

vortex-datetime-parts

Vortex physical encoding that compresses temporal components individually

lance-index

Lance indices implementation

connector_arrow

Load data from databases to Apache Arrow, the fastest way

datafusion-datasource-parquet

ar_row

Row-oriented access to Arrow arrays

lance-namespace-impls

Lance Namespace Implementations

geoarrow

amalgamation crate

lance-table

Lance table format

lightstream

Composable, zero-copy Arrow IPC and native data streaming for Rust with SIMD-aligned I/O, async support, and memory-mapping

lance-geo

Lance's geospatial extension providing geospatial UDFs

arrow-integration-test

Support for the Apache Arrow JSON test data format

arrow_extendr

Enables the use of arrow-rs in R using extendr and nanoarrow

datafusion-physical-optimizer

DataFusion Physical Optimizer

lance-namespace

Lance Namespace Core APIs

sea-clickhouse

ClickHouse Client with SeaQL integration

datafusion-datasource

lance-linalg

A columnar data format that is 100x faster than Parquet for random access

lance-arrow

Arrow Extension for Lance

sorting-parquet-writer

writing sorted Parquet files using Apache Arrow

datafusion-substrait

DataFusion Substrait Producer and Consumer

ballista-executor

Ballista Distributed Compute - Executor

stringtape

A tape class for strings arrays compatible with Apache Arrow

minarrow

Apache Arrow-compatible, Rust-first columnar data library for high-performance computing, native streaming, and embedded workloads. Minimal dependencies, ultra-low-latency access, automatic 64-byte SIMD alignment…

datafusion-pruning

DataFusion Pruning Logic

datafusion-spark

DataFusion expressions that emulate Apache Spark's behavior

lance-io

I/O utilities for Lance

dora-record

dora goal is to be a low latency, composable, and distributed data flow

datafusion-catalog-listing

lance-encoding

Encoders and decoders for the Lance file format

datafusion-physical-expr-common

Common functionality of physical expression for DataFusion query engine

geoarrow-expr-geo

GeoArrow

riskless

A pure Rust implementation of Diskless Topics

datafusion-datasource-csv

datafusion-datasource-json

lance-testing

A columnar data format that is 100x faster than Parquet for random access

pgpq

Encode Apache Arrow RecordBatches to Postgres’ native binary format

json2arrow

Convert JSON files to Arrow

vortex-datetime-dtype

Vortex datetime extension dtype

ballista-cli

Command Line Client for Ballista distributed query engine

datafusion-datasource-arrow

expman

Core logic and storage engine for expman

datafusion-datasource-avro

datafusion-session

vortex-roaring

Vortex roaring bitmap arrays

mcapdecode

MCAP decoding library with optional Arrow integration, protobuf, and ROS 2 decoders

lance-jni

JNI bindings for Lance Columnar format

ptars

Fast conversion from protobuf to Apache Arrow and back

transmcap

CLI for converting MCAP messages into JSONL, CSV, and Parquet via Arrow

lance-examples

Lance examples in Rust

geoparquet

reader and writer

mcaptui

Terminal UI for browsing MCAP topics and decoded messages via mcapdecode

vortex-runend-bool

Vortex run end encoded boolean array, strictly better than runend for bool arrays

flarrow-layout

flarrow (flow + arrow) is a rust runtime/framework for building dataflow applications

flarrow-builtins

flarrow (flow + arrow) is a rust runtime/framework for building dataflow applications

perspective

A data visualization and analytics component, especially well-suited for large and/or streaming datasets

arrow-format

Unofficial flatbuffers and tonic code of Apache Arrow spec

vortex-dict

Vortex dictionary array

csv2parquet

Convert CSV files to Parquet

json2parquet

Convert JSON files to Parquet

arrow-udf-js

JavaScript runtime for Arrow UDFs

csv2arrow

Convert CSV files to Arrow

flarrow-runtime

flarrow (flow + arrow) is a rust runtime/framework for building dataflow applications

vcf-arrow

A VCF data parser using Appache Arrow as standard format

llkv-column-map

Column mapping utilities for the LLKV toolkit

pyspark-arrow-rs

Derive macros to be used to add some helper functions to Rust structs to make them useable in Pyspark's mapInArrow

tpchgen-arrow

TPC-H data generator into Apache Arrow format

flarrow-api

flarrow (flow + arrow) is a rust runtime/framework for building dataflow applications

vortex-compute

Compute functions that operator over Vortex vectors, buffers, and masks

arrow-tools

packages

adbc-driver-flightsql

ADBC FlightSQL driver native library distribution for Rust

lance-encoding-datafusion

Encoders and decoders for the Lance file format that rely on datafusion

polars-cli

CLI interface for running SQL queries with Polars as backend

lance-arrow-scalar

Arrow scalar type with Ord, Hash, and Eq support

spatialbench-arrow

SpatialBench data generator into Apache Arrow format

geodatafusion-geojson

GeoJSON TableProvider for DataFusion

mcap2arrow-ros2-common

Shared ROS 2 type system and CDR decoding helpers for mcap2arrow

geodatafusion-geoparquet

GeoParquet TableProvider for DataFusion

mcap2arrow

MCAP to Arrow conversion library with protobuf and ROS 2 decoders

geoarrow-csv

CSV reader and writer for GeoArrow

zarr-datafusion

Extending DataFusion to do SQL queries on Zarr data

lance-tools

Tools for interacting with Lance files and tables

tpctools

generating and converting TPC-H and TPC-DS data sets

lance-namespace-datafusion

Lance namespace integration with Apache DataFusion catalogs and schemas

geoarrow-cast

Functions for converting from one GeoArrow geometry type to another

strawboat

A native storage format based on Apache Arrow

lance-core

Lance Columnar Format -- Core Library

arrow-udf-python

Python runtime for Arrow UDFs

geoarrow-test

Test data for GeoArrow data

lance-tokenizer

Tokenizer abstractions and implementations for Lance

flarrow-message

flarrow (flow + arrow) is a rust runtime/framework for building dataflow applications

evolution-slicer

Data slicing components for evolution

vortex-schema

Vortex file schema abstraction

minarrow-pyo3

PyO3 bindings for MinArrow - zero-copy Arrow interop with Python via PyArrow

evolution-parser

Data parsing functionality for evolution

geoarrow-flatgeobuf

Reader and writer for FlatGeobuf files to GeoArrow memory

flarrow-url

flarrow (flow + arrow) is a rust runtime/framework for building dataflow applications

mcapdecode-ros2idl

ROS 2 IDL schema decoder for mcapdecode CDR payloads

evolution-target

Output targets for evolution

evolution-mocker

Mocking components of evolution

flarrow-file-ext

flarrow (flow + arrow) is a rust runtime/framework for building dataflow applications

flarrow-url-scheme

flarrow (flow + arrow) is a rust runtime/framework for building dataflow applications

datafusion-row

Row backed by raw bytes for DataFusion query engine

flarrow-flows

flarrow (flow + arrow) is a rust runtime/framework for building dataflow applications

bodkin

Proc macro to simplify the integration of Arrow Data

duckdb-server

DuckDB Server for Mosaic

foreign_vec

Unofficial implementation of Apache Arrow spec in safe Rust

mcapdecode-arrow

Arrow conversion utilities built on top of mcapdecode-core schemas and values

llkv-threading

Thread pooling utilities for the LLKV toolkit

evolution-common

Common util components of evolution

kapot

Distributed Compute

evolution-schema

Schema implementations for evolution

llkv-compute

Compute kernels and math operations for LLKV

flarrow-url-default

flarrow (flow + arrow) is a rust runtime/framework for building dataflow applications

re_arrow_store

An in-memory time series database for Rerun log data, based on Apache Arrow

arrow-flightsql-odbc

An Apache Arrow Flight SQL server which proxies requests over ODBC

mcap2arrow-arrow

Arrow conversion utilities built on top of mcap2arrow-core schemas and values

hungerdb

serverless, low-latency vector database for AI applications

convergence-arrow

Utils for bridging Apache Arrow and PostgreSQL's wire protocol

mcapdecode-ros2msg

ROS 2 .msg schema decoder for mcapdecode CDR payloads

ballista-cache

Ballista Cache

datafusion-data-access

General data access layer currently mainly based on the object store interfaces

range-reader

Converts low-level APIs to read ranges of bytes to Read + Seek

geopolars

Geospatial extensions for Polars

ballista-core

Ballista Distributed Compute

llkv-types

Common data types for the LLKV toolkit

polars_arrow_rvsry99dx

Apache Arrow

mongodb-arrow-connector

MongoDB connector that reads and writes data to/from Apache Arrow

re_data_store

An in-memory time series database for Rerun log data, based on Apache Arrow

mcap2arrow-ros2msg

ROS 2 .msg schema decoder for mcap2arrow CDR payloads

arrowmax

High-performance Arrow data stack: columnar storage, zero-copy streaming, and schema codegen

lance-test-macros

A columnar data format that is 100x faster than Parquet for random access

gandiva_rust_udf

gandiva rust udfs

tiders-core

Core library for tiders blockchain data framework

alloy-rs

Static Rust library for working with the Apache Arrow ffi using any C supported language

csv2pq

CSV to Apache parquet converter

re_arrow2

Unofficial implementation of Apache Arrow spec in safe Rust

valu3-parquet

Parquet and Arrow encoding and decoding for valu3

flarrow-runtime-core

flarrow (flow + arrow) is a rust runtime/framework for building dataflow applications

vercel_blob

client for the Vercel Blob Storage API

tiders-rpc-client

A tiders RPC client for fetching EVM blockchain data from any standard JSON-RPC provider

katniss-pb2arrow

WIP