Beginner data engineering project - batch edition

Data Engineering Pilipinas is a community for data engineers, data analysts, data scientists, developers, AI / ML engineers, and users of closed and open source data tools and methods / techniques in the Philippines. Data Engineering Pilipinas is a PyData group.

dbt tutorial using a local PostgreSQL database

📘 Making Data, the DataMade Way

Nextract is a Extract Transform Load (ETL) platform build on top of Node.js streams

Ethereum Analytical Database - Ethereum data access solution that can be used for analytics and application development. The solution works on a fast DB - Clickhouse.

Wurzel is an open-source Python framework for advanced ETL pipelines in Retrieval-Augmented Generation (RAG) systems.

Socialytics is a social network data acquisition (ETL), analytics and intelligence tool.

In this project, we built a database that demonstrates the changes in American top fastest-growing private companies through time. The database is built on by ingesting, combining, and restructuring data from three main data sources into a conformed one Postgresql database, and deploy into the Flask app.

Integrating Apache Airflow, dbt, Great Expectations and Apache Superset to develop a modern open source data stack.

An open-source archive of campaign finance and lobbying disclosure data from the California Secretary of State’s CAL-ACCESS database

Various Data Analytics Projects based On Statistics in form of Notebook.

For this mini project we performed ETL (Extract,Transform,Load) on video game based data from Steam. This data was then loaded into a SQL database.

Um visualizador para tendências de animes

In this project, dbt, Great Expectations, Python and Pandas were used to transform and validate the "Inside Airbnb" dataset. The tools ensure quality data, ready for analysis.

Discover London GeoJson data with our interactive Graph & Map Visualization Solution using D3. View data with map, graph, & timeline visualizations. Advanced interactivity: pan, zoom, select, brush, link.

This project is an automated ETL pipeline (Extract, Transform, Load) designed to extract commodity data (specifically Gold and Silver) from a free public API.

In this project, the entire 3D models were created by Revit Autodesk and the ETL was used to convert the Revit file format to the Unity Supports .fbx file format. Through Unity3D the AR app was created for Android devices, using this app anyone can able to view the entire UNOM Guindy campus in AR mode by simply installing the .apk on your device