The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

I think compute_loss_func is actually never used in SFTTrainer. I recommend removing it (deprecate and remove in next version)

I think compute_loss_func is actually never used in SFTTrainer. I recommend removing it (deprecate and remove in next version)

I think it is used because it is passed to the transformer.Trainer's constructor which allows the user to pass in a custom loss function.

The Trainer calls the passed compute_loss_func in its own compute_loss function.
https://github.com/huggingface/transformers/blob/25b4a0d8aef515ed309c935607473da319d9291c/src/transformers/trainer.py#L4123-L4125

I actually think having this is a nice way to allow users to experiment with their own loss functions, so would refrain from getting rid of it. This PR in progress also serves as an example of how one can use the compute_loss_func for POCs.
#3960

compute_loss is overwritten in SFT, so this function is never user

compute_loss is overwritten in SFT, so this function is never user

Yes but even the overwritten version calls the trainer's implementation of compute_loss which in turn uses compute_loss_func

Relevant code in Trainer's compute_loss

if self.compute_loss_func is not None:
                loss = self.compute_loss_func(outputs, labels, num_items_in_batch=num_items_in_batch)
            elif model_name in MODEL_FOR_CAUSAL_LM_MAPPING_NAMES.values():
                loss = self.label_smoother(outputs, labels, shift_labels=True)

My bad you're right!