etl

Beginner data engineering project - batch edition

大数据采集,抽取平台,zdh_web是zdh系列服务的可视化管理平台，包含数据采集,调度,权限,审批流,私域营销等模块

📘 Making Data, the DataMade Way

Data Engineering Pilipinas is a community for data engineers, data analysts, data scientists, developers, AI / ML engineers, and users of closed and open source data tools and methods / techniques in the Philippines. Data Engineering Pilipinas is a PyData group.

Nextract is a Extract Transform Load (ETL) platform build on top of Node.js streams

Ethereum Analytical Database - Ethereum data access solution that can be used for analytics and application development. The solution works on a fast DB - Clickhouse.

dbt tutorial using a local PostgreSQL database

Integrating Apache Airflow, dbt, Great Expectations and Apache Superset to develop a modern open source data stack.

Socialytics is a social network data acquisition (ETL), analytics and intelligence tool.

Formação de Data Analytics - Turmas Alex Souza

Solo-built a senior-level analytics platform in 5 weeks: 471 messy government files → 320+ tested dbt models → 18 marts with 100+ CFO & equity KPIs → public BigQuery + 3 boardroom-ready Power BI dashboards. From zero knowledge to production-grade warehouse in 4 months — core build done in 5 weeks

In this project, dbt, Great Expectations, Python and Pandas were used to transform and validate the "Inside Airbnb" dataset. The tools ensure quality data, ready for analysis.

Tutorial using Great Expectations library, validating and profiling data on a local PostgreSQL database.

Node-RED ndoes for streaming data - utilising NodeJS Stream APIs

Aula magna do Future Tech Leaders Brazil (AEB + SDG AI Lab): pipeline ETL geoespacial que extrai precipitação por satélite (CHIRPS via Google Earth Engine), calcula anomalias e publica um WebMap interativo. Notebook didático, pacote Python e apostila.

An open-source archive of campaign finance and lobbying disclosure data from the California Secretary of State’s CAL-ACCESS database

Name frequency data from the US, Spain, Argentina, Chile and Italy — scraped, cleaned, and normalized into one consistent format with Python. Published on Kaggle.

Yu-Gi-Oh! Forbidden & Limited Lists or Limit Regulations ("banlists") as JSON. Part of YAML Yugi.