I am an engineer focused on training systems for large language models and multimodal models.
During my master's studies, I focused on AI Infrastructure and worked on the development and maintenance of large language model inference systems and large-scale model training frameworks. My current focus is on building and optimizing training infrastructure for LLMs, vision-language models, and multimodal models, with particular interests in distributed training, parallelism strategies, compute and communication efficiency, memory management, and heterogeneous hardware adaptation.
I started exploring front-end and back-end development in high school. This experience gave me a broader engineering perspective and helps me build practical tools and systems around machine learning workflows.
📍 Personal Homepage: dyedd.cn
Contributed to the development of LoongForge, a unified training framework for large language models, vision-language models, diffusion models, and embodied models. It covers pre-training, continued pre-training, and supervised fine-tuning, and combines Megatron and Torch-Native training stacks to support large-scale distributed training, hybrid parallelism, and performance optimization across different models and workloads.
Contributed to the maintenance of Chitu, a high-performance inference framework for large language models, with a focus on efficiency, flexibility, and availability.
Built a diffusion model training project that natively integrates DeepSpeed, Diffusers, and PEFT, with support for distributed training, LoRA fine-tuning, EMA, and single-node or multi-node training through Slurm.
A personal LLM API aggregation system for connecting to and managing multiple model providers and API protocols through a unified interface.
Thanks for visiting!