Fine-Tuning LLMs (Gemma, LLaMA, Mistral, and More)

Welcome to this curated repository showcasing the fine-tuning of various open-source large language models (LLMs) such as Gemma, LLaMA, Mistral, and others using Hugging Face Transformers, PEFT (LoRA/QLoRA), and other modern libraries.

This repo is designed for researchers, ML engineers, and enthusiasts looking to explore or build on top of custom fine-tuned LLMs.

• Fine-tuning with Hugging Face Trainer and PEFT (LoRA / QLoRA)
• Dataset loading and preprocessing
• Tokenization and model configuration
• Evaluation with custom metrics
• Easy-to-edit configs for reproducibility
• Support for mixed precision (fp16, bf16)

Got improvements, additional models, or tips? Contributions are welcome! Just open an issue or submit a pull request.