Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

Extremely fast Query Engine for DataFrames, written in Rust

Fastest library to load data from DB to DataFrames in Rust and Python

Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!

This extension makes vscode seamlessly work with dbt™: Auto-complete, preview, column lineage, AI docs generation, health checks, cost estimation etc

Build, Manage and Deploy AI/ML Systems

Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics

Fast, zero-config ETL in a single binary

This is a repo with links to everything you'd ever want to learn about data engineering