Stars
A repository to learn basic data processing techniques (Wikipedia processing, feature selection), and use them for some basic Web query classification.
Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. The next cohort starts in January 2026. Join the course here 👇🏼
A curated list of free public APIs — searchable, community-maintained, with a free JSON API.
Microsoft Authentication Library (MSAL) for Python makes it easy to authenticate to Microsoft Entra ID. General docs are available here https://learn.microsoft.com/entra/msal/python/ Stable APIs ar…
Learn how to design large-scale systems. Prep for the system design interview. Includes Anki flashcards.
Analyzes resource usage and performance characteristics of running containers.
Tools for deploying Data Factory (v2) in Microsoft Azure
XML data source for Spark SQL and DataFrames
A python package to help work with the apache atlas REST APIs
cryptography is a package designed to expose cryptographic primitives and recipes to Python developers.
Focalboard is an open source, self-hosted alternative to Trello, Notion, and Asana.
dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
The Missing Semester of Your CS Education 📚
A Node.js style checker and lint tool for Markdown/CommonMark files.
An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
Command-line program to download videos from YouTube.com and other video sites
A personal knowledge management and sharing system for VSCode
Roadmap to becoming a data engineer in 2021
Flash OS images to SD cards & USB drives, safely and easily.
Tracking and measuring neighborhood and district-level eviction rates in the city of San Francisco.
A cd command that learns - easily navigate directories from the command line
A Python script that requests a spot EC2 instance
A delightful community-driven framework for managing your bash configuration, and an auto-update tool so that makes it easy to keep up with the latest updates from the community.
🙃 A delightful community-driven (with 2,500+ contributors) framework for managing your zsh configuration. Includes 300+ optional plugins (rails, git, macOS, hub, docker, homebrew, node, php, python…