Make sure to install the requirements for these examples:

pip install -r requirements-examples.txt

Examples for training control vectors from a model's hidden states that can be used to influence the behavior and generated output during inference.

LoRA training Mistral-7B-Instruct-v0.2 with the Nvidia HelpSteer dataset.

Run train.sh in the helpsteer directory to download the dataset & model from HuggingFace and start the LoRA training. You can customize the training configuration by editing config.yml.

DPO training Qwen1.5-7B-Chat with the DPO Mix 7K dataset. The training consists of a supervised fine tuning (SFT) followed by direct preference optimization (DPO).

Run train.sh in the dpo-mix-7k directory to download the dataset & model from HuggingFace and start the training. You can customize the training configuration by editing the config files sft.yml and dpo.yml.

Implementation of the "Massive Multitask Language Understanding" benchmark using the MMLU dataset.

Run mmlu.py with the model you would like to evaluate.

Implementation of the MMLU-Pro benchmark using the MMLU-Pro dataset.

Run mmlu-pro.py with the model you would like to evaluate.

Calculating perplexity scores for a sample dataset of entry paragraphs from Wikipedia articles.

Run perplexity.py with the model you would like to evaluate. Add quantization options to evaluate perplexity with quantized models.