Lib.rs

parquet

Apache Parquet implementation in Rust

arrow

Apache Arrow

arrow-select

Selection kernels for arrow arrays

odbc2parquet

Query an ODBC data source and store the result in a Parquet file

parquet2

Safe implementation of parquet IO

datafusion-python

Apache DataFusion DataFrame and SQL Query Engine

tsdb_timon

Efficient local storage and Amazon S3-compatible data synchronization for time-series data, leveraging Parquet for storage and DataFusion for querying, all wrapped in a simple and intuitive API

parquet-variant

Apache Parquet Variant implementation in Rust

dataprof

High-performance data profiler with ISO 8000/25012 quality metrics for CSV, JSON/JSONL, and Parquet files

parquet-variant-compute

Apache Parquet Variant Batch Processing

polars-view

A fast and interactive viewer for CSV, Json and Parquet data

dms-cdc-operator

Rust-based utility for comparing the state of a list of tables in an Amazon RDS database with data stored in Parquet files on Amazon S3, particularly useful for change data capture (CDC) scenarios

nu_plugin_parquet

nu plugin to add parquet support

dms-cdc-operator-client

Rust-based client for comparing the state of a list of tables in an Amazon RDS database with data stored in Parquet files on Amazon S3, particularly useful for change data capture (CDC) scenarios

nail-parquet

Lightning-fast CLI for data analysis: explore, filter, transform Parquet/CSV/Excel files with SQL-powered operations

sas7bdat

+ CLI for decoding SAS7BDAT datasets and streaming them to modern formats

parquet-geospatial

Apache Parquet Geometry and Geography implementation in Rust

parquet-variant-json

Apache Parquet Variant to/from JSON

ironbeam

A batch processing clone of Apache Beam in Rust

excelstream

High-performance streaming Excel & CSV library with S3/GCS cloud support and Parquet conversion - Ultra-low memory usage

aisle

Metadata-driven Parquet pruning for Rust: Skip irrelevant data before reading

polars-parquet

Apache Parquet I/O operations for Polars

pg2parquet

Command line tool for exporting PostgreSQL tables or queries into Parquet files

datafusion-datasource-parquet

cc2p

Convert a CSV to parquet file format

dply

A command line data manipulation tool inspired by the dplyr grammar

tabiew

A lightweight TUI application to view and query tabular data files, such as CSV, TSV, and parquet

cloudfront-logs

AWS CloudFront log line parser

ushcn

US Historical Climatology Network data downloader

alimentar

Data Loading, Distribution and Tooling in Pure Rust

dataset-writer

write CSV/Arrow/Parquet files concurrently

tass

A pager for tabular data

tonbo

Embedded database for serverless and edge runtimes, storing data as Parquet on S3

otlp2parquet

Stream OpenTelemetry logs, metrics, and traces to Parquet files

supertable-core

Core library for SuperTable, a next-generation open table format

tpchgen-cli

Blazing fast pure Rust TPC-H data generator command line tool and library

datu

data file utility

helios-sof

complete implementation of the SQL-on-FHIR specification for Rust, enabling the transformation of FHIR resources into tabular data using declarative ViewDefinitions. It supports all major FHIR versions (R4…

archive-to-parquet

Recursively convert archives to parquet files

pulse-ops

Built-in operators and combinators for Pulse dataflows — map, filter, join, window, and custom processing logic

tinyetl

Fast, zero-config ETL in a single binary for transforming data between formats and databases

csvdb

Convert between SQLite/DuckDB databases and CSV directories

nc-to-pq

A CLI tool to convert 1D variable NetCDF4 files to Apache Parquet format

csv2parquet

Convert CSV files to Parquet

dsq-formats

File format support for dsq - handles reading and writing various data formats

json2parquet

Convert JSON files to Parquet

readervzrd

A generic reader for csv and json data

datacell

powerful CLI tool and library for spreadsheet manipulation with pandas-style operations. Supports CSV, Excel (XLSX, XLS, ODS), Parquet, and Avro formats with formula evaluation, data transformation…

pulse-io

Input/output connectors for Pulse — integrates with external systems such as Kafka, Arrow, and Parquet

parquet2json

A command-line tool for streaming Parquet as line-delimited JSON

smooth-json

opinionated, customizable utility to flatten serde_json Value variants into serde_json Objects ready for use in columnar or table-like usages

liveplot

Realtime interactive plotting library using egui/eframe, with optional gRPC and Parquet export support

allsource-core

High-performance event store core built in Rust

liquid-cache-storage

10x lower latency for cloud-native DataFusion

evolution

Efficiently evolve your old fixed-length data files into modern file formats

pulse-state

State management utilities for Pulse — provides windowing, aggregations, and persistent operator state

fusio-parquet

Parquet reader and writer implementations for Fusio

icepick

Experimental Rust client for Apache Iceberg with WASM support for AWS S3 Tables and Cloudflare R2

parquet-viewer

command-line tool to view Apache Parquet files

parquet-record

High-performance Rust library for moving structs to/from disk using Parquet format. Abstracts complex Arrow/Parquet usage while providing batch writing and parallel reading capabilities for maximum performance.

prestige

file reading and writing utilities and tools

pulse-core

Core runtime and dataflow engine for Pulse — defines execution graph, operators, and streaming primitives

xdl-dataframe

DataFrame module for XDL - pandas/Spark-style data manipulation with support for CSV, TSV, Parquet, Avro

parq

A blazingly-fast tool for exploring and analyzing Apache Parquet file: inspect schema, view statistics, browse data, and dissect structures

parquet_to_excel

convert parquet file(s) to an/a excel/csv file with constant memory in rust

parquetry

Runtime library for Parquet code generator

midas_processor

High-performance Rust tool for converting UK Met Office MIDAS weather datasets from BADC-CSV to optimized Parquet format

daemon_rs

High-performance structured logging daemon with Parquet storage

dremio-rs

Dremio Rust client

polars-cli

CLI interface for running SQL queries with Polars as backend

mdb-cli

Command line client for the MarpleDB API

pqrs

Apache Parquet command-line tools and utilities

nc2parquet

High-performance NetCDF to Parquet converter with cloud storage support

dbt-fusion-workspace-hack

workspace-hack package, managed by hakari

compact-thrift-parquet

Parquet metadata structures generated from thrift definitions

geoparquet

reader and writer

seqtable

High-performance parallel FASTA/FASTQ sequence counter

parquet-key-management

Implements the Parquet Key Management Tools API in Rust to enable integration with a Key Management Server when using Parquet modular encryption

compact-thrift-runtime

Runtime library for compact-thrift code generator

datasynth-output

Output sinks for CSV, Parquet, JSON, and streaming formats

datatui

fast, keyboard-first terminal data viewer

getmeta

Not just gold builds anymore!

cottas-rs

working with compressed RDF files in the COTTAS format. COTTAS stores triples as a triple table in Apache Parquet. It is built on top of DuckDB and provides an HDT-like interface.

datasetq

A data processing tool with a jq-like syntax for structured data formats, including CSV, JSON, Parquet, Avro, and more

liquid-cache-common

10x lower latency for cloud-native DataFusion

pq-utils

reading parquet files

parquet_aramid

Query engine using Parquet tables as a Key-Value store

parquet-format-safe

Safe Parquet and Thrift reader and writer (sync and async)

shaha

Hash database builder and reverse lookup tool

parquet-format

Apache Parquet Format - thrift definition and generated Rust file

shema

Schema generation macros

tesser-ledger

Ledger primitives for Tesser accounting

csvs_convert

Some Datapackage Conversion

liquid-cache-parquet

10x lower latency for cloud-native DataFusion

rangebar-io

I/O operations for rangebar data (CSV, Parquet, Arrow)

parquet_opendal

parquet Integration for Apache OpenDAL

ecad-processor

High-performance multi-metric weather data processor for European Climate Assessment & Dataset (ECA&D) archives with Parquet output

tonbo-predicate

Predicate evaluation for Tonbo embedded database

parquet-format-async-temp

Temporary crate containing thrift library + parquet definitions compiled to support read+write async

tenrso-ooc

Out-of-core processing with Arrow/Parquet for TenRSo

sbbf-rs

Split block bloom filter implementation

wardenclyffe

A tiny Rust query engine that supports SQL-like filters, CSV scanning, projections, and a custom DSL powered by Pest

parquetry-gen

Parquet code generator

tpctools

generating and converting TPC-H and TPC-DS data sets

otlp2parquet-proto

OTLP protobuf definitions for otlp2parquet

liquid-cache-server

10x lower latency for cloud-native DataFusion

deepbiop-fq

Deep Learning Preprocessing Library for Fastq Format

liquid-cache-client

10x lower latency for cloud-native DataFusion

bdt

viewing, querying, converting, and comparing files in popular data formats (CSV, Parquet, JSON, Avro)

deepbiop-fa

Deep Learning Preprocessing Library for Fastq Format

otlp2parquet-writer

Parquet writer for otlp2parquet

otlp2parquet-handlers

Stream OpenTelemetry logs, metrics, and traces to Parquet files

parquetry-sort

Runtime sorting library for Parquet code generator

prestige-cli

CLI interface for manually fetching and reading Prestige-parquet files

polars-parquet-format

Safe Parquet and Thrift reader and writer (sync and async)

slim-runner

Run SLiM simulation grid runs in parallel

sqlite2parquet

Generate parquet files from sqlite databases

metriken-query

PromQL query engine and TSDB for metriken parquet files

tbl-core

reading and modifying tabular files

otlp2parquet-common

Stream OpenTelemetry logs, metrics, and traces to Parquet files

liquid-cache-local

10x lower latency for cloud-native DataFusion

datagen

An easy to use tool to generate fake data in bulk and export it as Avro, Parquet or directly into your database as tables

pgparquet

High-performance CLI tool for streaming Parquet files from Google Cloud Storage into PostgreSQL

daft-parquet

Parquet processing for the Daft project

evolution-builder

Builder implementations for evolution

parqeye

Parquet viewer for the command line

warc-parquet

converting WARC to Parquet

prql-query

pq: query and transform data with PRQL

parquet2lance

Convert parquet files to lance

tbl-cli

tbl is a tool for reading and editing tabular data files

query-fuse

An interactive SQL query engine for local columnar files (Parquet, Arrow, Feather)

dr

Command-line data file processing in Rust

motif-scanner

Command line tool for scanning DNA sequences for transcription factor binding sites

glaredb_ext_parquet

Apache Parquet extension for GlareDB. Originally forked from github.com/apache/arrow-rs

azof

Lakehouse format with event time travel

amadeus-parquet

An Apache Parquet implementation in Rust

evolution-writer

Output target writers for evolution

tabler

📊 Tabler: A lightweight TUI tool to view, query, and navigate CSV, TSV, and Parquet data files

xpq

command line tool for analyzing parquet files

gosh-adaptor

Adaptor for chemical model

evolution-converter

Converter implementations for evolution

range-reader

Converts low-level APIs to read ranges of bytes to Read + Seek

innofile

InnoFile

depyler-knowledge

Sovereign Type Database for Python library type extraction

rs-ints2parquet

Converts the integers to a parquet

csv2pq

CSV to Apache parquet converter

athort

Assortment of Parquet and other items

bazof

Lakehouse format with event time travel

datahobbit

that generates CSV or Parquet files with synthetic data based on a provided JSON schema

csv_generator

that generates CSV or Parquet files with synthetic data based on a provided JSON schema

xpq2

command line tool for analyzing parquet files

rs-csv2parquet

Converts a csv file to a parquet

jlcpcb-to-parquet

convert JLCPCB Parts Library to Parquet

otlp2parquet-core

Core OTLP to Arrow/Parquet conversion logic

rs-parquets2count

Computes the total number of rows of the parquet files

pack-it

Packer for Parquet tables

azof-cli

CLI utility for azof lakehouse format

rs-splited2parquet

Converts csv-like lines to rows and saves as a parquet

hypersync-client

client library for hypersync

rs-zips2meta2parquet

Converts the metadata values of the zip files to a parquet

glaciers

decode raw EVM logs into decoded events

parquet-py

command-line interface & Python API for parquet

dendritic-datasets

Prebuilt datasets that can be imported for ML model training

parquet-flamegraph

program to generate flamegraph and investigate parquet storage

ethl-cli

Tools for capturing, processing, archiving, and replaying Ethereum events

parquet-lru

Implement LRU cache reader for parquet::arrow::async_reader::AsyncFileReader