Welcome to my GitHub profile. My name is Marcia and here you'll find my studies and projects.
About me:
Data Engineer with 5 years across the data lifecycle — I started in data science and moved into data engineering. I work end to end: ingestion, modeling, transformation, deployment, and monitoring.
Recently completed the Erasmus Mundus Master's in Software Engineering (SE4GD), a joint degree across Italy, Finland, and the Netherlands, concluding with thesis research on data interoperability across systems and organisations, conducted with a data consultancy in Finland.
I hold a Bachelor's degree in Software Engineering and a specialization in Artificial Intelligence, both focused on industrial and applied technologies.
Day to day I work in Python, SQL, Airflow, dbt, Git, Docker, CI, Linux/bash — plus in-pipeline testing and data quality, and monitoring.
Documenting: dbt docs for lineage, ADRs for decisions, Mermaid for architecture, AGENTS.md.
Reach me at marcia.rrdg@gmail.com or connect on LinkedIn. — Always happy to connect, exchange ideas, or collaborate.
- Pipelines: Python · SQL · PySpark · Airflow · dbt Core · dlt
- Storage: PostgreSQL · DuckDB · Delta Lake / Iceberg · S3
- Cloud & DevOps: AWS · Azure · Docker · Terraform · GitHub Actions (CI/CD)
- Quality & Observability: Great Expectations · dbt tests · Grafana · Prometheus
- AI & ML: Scikit-learn · PyTorch · Hugging Face · LangChain · LangGraph · RAG pipelines · pgvector
- BI & Visualization: Metabase · Streamlit
- Agentic Systems & AI Automation — coding agents for scoped tasks, automated PR review in the pipeline, agentic CI/CD workflows, and context engineering to make agent behaviour predictable
- Agent observability — building a telemetry layer to track how agentic workflows actually behave in practice: traces, costs, failure modes
- Lakehouse & Distributed Compute — Delta Lake / Iceberg table formats, Spark (PySpark), and ELT orchestration with Airflow and dbt Core
- End-to-End Data Platform — REST API and document ingestion into object storage, dbt modeling into star schemas, Airflow orchestration, and Great Expectations validation
- Streaming & Event-Driven Systems — Kafka for real-time ingestion
- Semantic layer — modeling the layer analysts query, without a warehouse underneath · DuckDB · dbt
- Orchestrated Databricks workloads — pipelines on Airflow 3.0 · Databricks · PySpark · Airflow
- Databricks Certified Data Engineer Associate (in progress)