Fine-tune LLMs on your Mac with Apple Silicon
Prototype locally, scale to cloud. Same code, just change the import.

Quick Start · Training Methods · Examples · Status

Bringing the Unsloth experience to Mac users via Apple's MLX framework.

• 🚀 Fine-tune LLMs locally on your Mac (M1/M2/M3/M4/M5)
• 💾 Leverage unified memory (up to 512GB on Mac Studio)
• 🔄 Same API as Unsloth - your existing code just works!
• 📦 Export anywhere - HuggingFace format, GGUF for Ollama/llama.cpp

# Unsloth (CUDA)                        # Unsloth-MLX (Apple Silicon)
from unsloth import FastLanguageModel   from unsloth_mlx import FastLanguageModel
from trl import SFTTrainer              from unsloth_mlx import SFTTrainer

# Rest of your code stays exactly the same!

This is NOT a replacement for Unsloth or an attempt to compete with it. Unsloth is incredible - it's the gold standard for efficient LLM fine-tuning on CUDA.

This IS a bridge for Mac users who want to:

• 🧪 Prototype locally - Experiment with fine-tuning before committing to cloud GPU costs
• 📚 Learn & iterate - Develop your training pipeline with fast local feedback loops
• 🔄 Then scale up - Move to cloud NVIDIA GPUs + original Unsloth for production training

Local Mac (Unsloth-MLX)     →     Cloud GPU (Unsloth)
   Prototype & experiment          Full-scale training
   Small datasets                  Large datasets
   Quick iterations                Production runs

🚀 v0.3.5 - Merged model save + load_adapter fixed!

• SFT: Show train/val loss + perplexity in the GUI during training (and save a small metrics log)
• SFT: Add a small built-in prompt evaluation set runner (before/after fine-tune comparison)
• RL: Add clearer dataset schema validation + sample preview per method (DPO/ORPO/GRPO/KTO/SimPO)
• RL: Add built-in reward function templates/presets for GRPO (math, formatting, length, etc.)
• RL: Add better progress reporting (loss components, reward stats, win-rate-like summaries)
• RL: Improve checkpointing/resume support (continue training from an output folder)
• Export: Add optional quantization + export presets (quality/speed) and better compatibility notes per model family
• Packaging: Publish to PyPI and add a simple versioned release workflow

# Using uv (recommended - faster and more reliable)
uv pip install unsloth-mlx

# Or using pip
pip install unsloth-mlx

# From source (for development)
git clone https://github.com/ARahim3/unsloth-mlx.git
cd unsloth-mlx
uv pip install -e .

Want to fine-tune without writing code? Use our Gradio-based GUI!

# Install GUI dependencies
pip install -e .

# Launch the GUI
python gui.py

# Open http://127.0.0.1:7860 in your browser

The GUI provides tabs for:

• Loading models from HuggingFace
• Chatting with models
• Configuring LoRA adapters
• SFT and RL training
• Exporting models

See GUI_README.md for detailed instructions.

Export notes:

• Browse… buttons are available in the Export tab to pick output locations without typing paths.
• Save LoRA Adapters exports a small folder containing adapters.safetensors + adapter_config.json.
• Save Merged Model produces a fused MLX model folder suitable for tools like LM Studio (MLX backend).
• GGUF export is only supported by mlx_lm for model families: llama, mistral, mixtral. Some model types (e.g. lfm2) cannot be exported to GGUF via mlx_lm.

LEAP GGUF export (Liquid AI):

• If you're using LEAP-supported model architectures (LFM2 / LFM2-VL / Qwen), you can create an iOS-ready .gguf using the LEAP GGUF Bundling section in the GUI.
• Install the bundling CLI: pip install leap-bundle.
• Authenticate once with leap-bundle login <api-key> (credentials are persisted to ~/.liquid-leap). After that you can leave the GUI API key field blank.
• Recommended flow: Save Merged Model → LEAP: Validate Directory → Create Bundle Request → Download GGUF.
• On iOS, you can load the downloaded .gguf locally with LEAP SDK via Leap.load(url:).

Dataset notes (SFT):

• SFT training expects JSONL rows in the {"messages": [...]} chat format.
• If your dataset is not already in messages format, use the GUI button Convert to messages JSONL in the SFT Training tab.
• Supported input schemas for conversion:
  • messages: already-correct chat rows
  • conversations: ShareGPT-style (from/value) and variants
  • Alpaca-style: instruction + optional input + output
  • Prompt/completion: prompt + completion (or response)
• Converted datasets are written to data/converted/ and the GUI auto-fills the dataset path to the converted file.

from unsloth_mlx import FastLanguageModel, SFTTrainer, SFTConfig
from datasets import load_dataset

# Load any HuggingFace model (1B model for quick start)
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="mlx-community/Llama-3.2-1B-Instruct-4bit",
    max_seq_length=2048,
    load_in_4bit=True,
)

# Add LoRA adapters
model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
    lora_alpha=16,
)

# Load a dataset (or create your own)
dataset = load_dataset("yahma/alpaca-cleaned", split="train[:100]")

# Train with SFTTrainer (same API as TRL!)
trainer = SFTTrainer(
    model=model,
    train_dataset=dataset,
    tokenizer=tokenizer,
    args=SFTConfig(
        output_dir="outputs",
        per_device_train_batch_size=2,
        learning_rate=2e-4,
        max_steps=50,
    ),
)
trainer.train()

# Save (same API as Unsloth!)
model.save_pretrained("lora_model")  # Adapters only
model.save_pretrained_merged("merged", tokenizer)  # Full model
model.save_pretrained_gguf("model", tokenizer)  # GGUF (see note below)

from unsloth_mlx import get_chat_template, train_on_responses_only

# Apply chat template (supports llama-3, gemma, qwen, phi, mistral, etc.)
tokenizer = get_chat_template(tokenizer, chat_template="llama-3")

# Or auto-detect from model name
tokenizer = get_chat_template(tokenizer, chat_template="auto")

# Train only on responses (not prompts) - more efficient!
trainer = train_on_responses_only(
    trainer,
    instruction_part="<|start_header_id|>user<|end_header_id|>\n\n",
    response_part="<|start_header_id|>assistant<|end_header_id|>\n\n",
)

Check examples/ for working code:

• Basic model loading and inference
• Complete SFT fine-tuning pipeline
• RL training methods (DPO, GRPO, ORPO)

• Hardware: Apple Silicon Mac (M1/M2/M3/M4/M5)
• OS: macOS 13.0+ (15.0+ recommended for large models)
• Memory: 16GB+ unified RAM (32GB+ for 7B+ models)
• Python: 3.9+ (3.12 recommended)

The Issue: GGUF export (save_pretrained_gguf) doesn't work directly with quantized (4-bit) base models. This is a known limitation in mlx-lm, not unsloth-mlx.

What Works:

• ✅ Training with quantized models (QLoRA) - works perfectly
• ✅ Saving adapters (save_pretrained) - works
• ✅ Saving merged model (save_pretrained_merged) - works
• ✅ Inference with trained model - works
• ❌ GGUF export from quantized base model - mlx-lm limitation

Workarounds:

1. Use a non-quantized base model (recommended for GGUF export):

   # Use fp16 model instead of 4-bit
   model, tokenizer = FastLanguageModel.from_pretrained(
       model_name="mlx-community/Llama-3.2-1B-Instruct",  # NOT -4bit
       max_seq_length=2048,
       load_in_4bit=False,  # Train in fp16
   )
   # Train normally, then export
   model.save_pretrained_gguf("model", tokenizer)  # Works!

2. Dequantize during export (results in large fp16 file):

   model.save_pretrained_gguf("model", tokenizer, dequantize=True)
   # Then re-quantize with llama.cpp:
   # ./llama-quantize model.gguf model-q4_k_m.gguf Q4_K_M

3. Skip GGUF, use MLX format: If you only need the model for MLX/Python inference, just use save_pretrained_merged() - no GGUF needed.

Related Issues:

• mlx-lm #353 - MLX to GGUF conversion
• mlx-examples #1382 - Quantized to GGUF

Contributions welcome! Areas that need help:

• Custom MLX kernels for even faster training
• More comprehensive test coverage
• Documentation and examples
• Testing on different M-series chips (M1, M2, M3, M4, M5)
• VLM training improvements

Apache 2.0 - See LICENSE file.

• Unsloth - The original, incredible CUDA library
• MLX - Apple's ML framework
• MLX-LM - LLM utilities for MLX
• MLX-VLM - Vision model support

Community project, not affiliated with Unsloth AI or Apple.
⭐ Star this repo if you find it useful!