• I'm 22 years old
• Final year Master’s in Computer Science @ Instituto Superior Técnico (Lisbon)
• Research: Model Merging, specifically On-Policy Distillation solutions on Knowledge Distillation-based fusion
• Currently passionate with the full model creation pipeline: pre-training → training -> post-training → evaluation → deployment → monitoring
• Based in Portugal 🇵🇹

Check out my CV

Current theme: Moving Model Fusion from Static Imitation to Dynamic Exploration

I am investigating On-Policy Multi-Teacher Distillation, a framework designed to fix the "exposure bias" limitation in current methods like FuseLLM and InfiFusion. Instead of training a student to passively mimic static teacher logits (Off-Policy), I force the student to generate its own rollouts and receive dynamic feedback from an ensemble of experts.

Key Hypotheses:

• Dynamic vs. Static: Training on student-induced distributions (On-Policy) allows the model to learn error-recovery strategies that static distillation misses.
• Consensus Stability: Using Generalized Jensen-Shannon Divergence ($JS_\pi$) creates a bounded, symmetric loss landscape, preventing gradient explosions when the student explores "long-tail" trajectories.
• Expert Allocation: Applying Max-Margin Ensemble Aggregation enables the student to dynamically attend to the most confident domain expert (e.g., Math vs. Code teacher) for every token it generates.

Keywords: multi-teacher distillation • on-policy learning • jensen-shannon divergence • model fusion • max-margin aggregation

Currently reading through this repository for Model Merging / Model Fusion literature.

• 📍 EVBuddy - EVBuddy is a reproducible ML pipeline for EV charging-station availability forecasting in Barcelona.
• 🧊 CUBOS — It is open source and free to use game engine for any purpose. Written in modern C++ (20) and with data-oriented design in mind, its goal is to be both performant and highly flexible.
• 🧪 LEMAC-Website — Website for the laboratory LEMAC (Computer Assisted Mechanical Engineering Laboratory).

• 🗣️ Natural Language Project - Natural Language's course project.
• 🧠 Deep Learning Homeworks - Deep Learning's course Homework.
• 🤖 Social Robotics and Human Interaction - LLM as a judge engagement evaluator.
• 💻 Operative Systems - Operating System's course Project: TécnicoFS, simplified file system in user mode.

If you’re into model merging, distillation, or practical ML, I’d love to chat.

• 💼 LinkedIn: https://www.linkedin.com/in/diogomsmiranda/
• 📫 Email: diogomsmiranda@tecnico.ulisboa.pt