Open Source software code for use with PCIe card-based hardware AI accelerators catering to both inference and training use cases

Data Engineering Pipeline extracting and transforming data from rewe shop API.

This repository contains my work completed during the Data Engineering Zoomcamp 2025

GitHub Issue Auto-Triage & Notification System

A self-hosted, lightweight ETL pipeline for orchestrated exports from REST API data to S3 with zero infrastructure overhead. Intelligent API batching, S3 optimization, retry logic, automatic S3 gap backfilling, customizable logging.

GCP Batch Data Pipeline

The World Disaster Pipeline is an ETL system that processes global disaster data from the EM-DAT database. It automates ingestion, transformation, storage, and visualization, all in the cloud.

Multi-Agent System for Autonomous, Verifiable Knowledge Synthesis

Built a complete end-to-end data platform to ingest, process, and analyze complex, multi-source public datasets for business intelligence.

The Data Engineering Zoomcamp covers essential skills in containerization, workflow orchestration, data warehousing, analytics engineering, batch, and streaming processing. It includes tools like Docker, Terraform, BigQuery, dbt, Spark, Kafka, Kestra, Postgres, Google Data Studio, and Metabase.

ELT for AEMET weather data.