Senior Data Engineer with 7+ years building scalable data infrastructure.
What I do:
- Build & automate ETL/ELT pipelines that turn raw data into actionable insights
- Design cloud-native data architectures on AWS
- Implement data quality frameworks and observability
Core stack: Python, SQL, Airflow, dbt, AWS
Currently focusing on: Real-time streaming architectures Β· Apache Kafka Β· Apache Flink
πΉ crypto-elt-pipeline
Modern ELT pipeline for cryptocurrency market analysis.
- Ingestion Layer: Extract data from multiple crypto APIs via PyAirbyte
- Orchestration Layer: Manage workflows with Dagster
- Transformation Layer: dbt for data modeling
- Quality Layer: Validate data with Pandera + dbt tests
- Storage Layer: DuckDB for in-process analytics
- Visualization Layer: Streamlit dashboards
- Serving Layer: FastAPI serving transformed data
End-to-end data engineering pipeline for gaming analytics.
- Ingestion Layer: Extract data from RAWG API using dlt
- Orchestration Layer: Manage workflows with Prefect
- Quality Layer: Validate data with Soda Core + SQLMesh tests
- Transformation Layer: Transform data with SQLMesh
- Storage Layer: MotherDuck for analytics queries
- Visualization Layer: Present insights with Marimo and Rill dashboards