PEFT model doesn't update params when having changed LoRA config #2295
Description
System Info
I have noticed that when updated the target_modules settings in the LoRA config, the PEFT model params remain unchanged. Might affect other PEFT settings too.
My assumption is that get_peft_model() does not re-instantiate/update its settings once it has been initialized before.
System: Windows 11
Python: 3.11
peft: 0.14.0
Who can help?
Information
- The official example scripts
- My own modified scripts
Tasks
- An officially supported task in the
examplesfolder - My own task or dataset (give details below)
Reproduction
For reproduction in a Jupyter Notebook:
from peft import LoraConfig, get_peft_model, TaskType
from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch
label_list = ['B-LOC', 'B-MISC', 'B-ORG', 'B-PER', 'I-LOC', 'I-MISC', 'I-ORG', 'I-PER', 'O']
# Initialize tokenizer
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-1B")
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "right"
model = AutoModelForTokenClassification.from_pretrained(
"meta-llama/Llama-3.2-1B",
pad_token_id=tokenizer.eos_token_id,
torch_dtype=torch.bfloat16,
device_map="auto",
num_labels=len(label_list)
)
for name, module in model.named_modules():
print(name)lora_config = LoraConfig(
task_type=TaskType.TOKEN_CLS,
r=16,
lora_alpha=32,
target_modules=["q_proj", "v_proj"],
lora_dropout=0.1
)model = get_peft_model(model, lora_config)
model.print_trainable_parameters()This outputs
trainable params: 1,722,377 || all params: 1,237,555,218 || trainable%: 0.1392
But when changing the above code without restarting the kernel to:
lora_config = LoraConfig(
task_type=TaskType.TOKEN_CLS,
r=16,
lora_alpha=32,
target_modules=["layers.0.self_attn.q_proj", "layers.0.self_attn.v_proj"], # changed to specific heads
lora_dropout=0.1
)and retrieving the trainable params again:
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()it outputs again
trainable params: 1,722,377 || all params: 1,237,555,218 || trainable%: 0.1392
but after the update it should be
trainable params: 124,937 || all params: 1,235,957,778 || trainable%: 0.0101
Expected behavior
When having updated lora_config, get_peft_model() should retrieve the current config.