Multi-Agent System for Autonomous, Verifiable Knowledge Synthesis
-
Updated
Dec 15, 2025 - Python
Multi-Agent System for Autonomous, Verifiable Knowledge Synthesis
Open Source software code for use with PCIe card-based hardware AI accelerators catering to both inference and training use cases
A self-hosted, lightweight ETL pipeline for orchestrated exports from REST API data to S3 with zero infrastructure overhead. Intelligent API batching, S3 optimization, retry logic, automatic S3 gap backfilling, customizable logging.
Built a complete end-to-end data platform to ingest, process, and analyze complex, multi-source public datasets for business intelligence.
GitHub Issue Auto-Triage & Notification System
The World Disaster Pipeline is an ETL system that processes global disaster data from the EM-DAT database. It automates ingestion, transformation, storage, and visualization, all in the cloud.
The Data Engineering Zoomcamp covers essential skills in containerization, workflow orchestration, data warehousing, analytics engineering, batch, and streaming processing. It includes tools like Docker, Terraform, BigQuery, dbt, Spark, Kafka, Kestra, Postgres, Google Data Studio, and Metabase.
GCP Batch Data Pipeline
Add a description, image, and links to the kestra topic page so that developers can more easily learn about it.
To associate your repository with the kestra topic, visit your repo's landing page and select "manage topics."