Repository navigation
Generated unsloth_compiled_cache file cause Indentation Error when use unsloth with smolvlm2 #2179
Description
Activity
It maybe a bit annoy but will there be any fix soon?
Based on my conversation on discord with @hermeschen1116 . Here is some additional information:
- This is the offending code snippet:
import os from argparse import ArgumentParser from typing import Any, Dict, List import datasets import evaluate import torch import yaml from unsloth import FastVisionModel from datasets import load_dataset from dotenv import load_dotenv from transformers import GenerationConfig, TextStreamer model, processor = FastVisionModel.from_pretrained( "HuggingFaceTB/SmolVLM2-500M-Video-Instruct", load_in_4bit=False, # Use 4bit to reduce memory use. False for 16bit LoRA. #use_gradient_checkpointing="unsloth", # True or "unsloth" for long context _attn_implementation="flash_attention_2", use_cache=True, dtype=None, device_map="cuda", low_cpu_mem_usage=True, ) FastVisionModel.for_inference(model)
- This is happening with the SmolVLM2-500-Video-Instruct model which can be found at https://huggingface.co/HuggingFaceTB/SmolVLM2-500M-Video-Instruct
- His environment is up to date. He's running the code on a local jupyter notebook on an RTX 3080 machine.
- HIs version of unsloth, unsloth_zoo, transformers, trl, xformers are all up to date.
just pushed an unsloth-zoo patch PR which fixes the issue in the models' forward method which misses explicit device casting.
- added a commit that references this issue
on Jun 8, 2025 - added a commit that references this issue
on Jun 25, 2025 - added a commit that references this issue
on Jul 7, 2025
I try to use unsloth with smolvlm2 but it keep throwing out "unexpected indentation error".
The cause as the error message tells is in 481th line of the generated file unsloth_compiled_cache/unsloth_compiled_module_smolvlm.py.
It looks like below.
I tried to fix it by manually modified the generated file but it will be overwritten next time I execute the python script.
The post in unsloth DC server I made