Fix code example in quicktour.md #1181
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The documentation is not available anymore as the PR was closed or merged. |
pacman100
approved these changes
Nov 24, 2023
Contributor
pacman100
left a comment
There was a problem hiding this comment.
Thank you @merveenoyan for fixing the docs! ✨
Member
|
Thanks a lot Merve for fixing this issue, which surely many users stumbled over. Question: Should we fork the repo and move it to https://huggingface.co/peft-internal-testing? Theoretically, that user (or someone else with access to their HF account) could at any point change the checkpoint to be some arbitrary pickle file. |
Contributor
Author
|
@BenjaminBossan makes sense, or we could have @pacman100's repository adapted to recent changes. However, I'd be in favor of a fork given it's a more recent model. |
Contributor
Author
|
@BenjaminBossan I duplicated repository under my namespace and changed it in the code as well. |
Member
Thanks @merveenoyan |
Guy-Bilitski
pushed a commit
to Guy-Bilitski/peft
that referenced
this pull request
May 13, 2025
cyyever
pushed a commit
to cyyever/peft
that referenced
this pull request
Sep 4, 2025
* initial file * initial tokenizer * UnpairedPreferenceBatchSampler * use batch_sampler * use interleave_datasets * add loss * fix imports * use SequentialSampler when training * formatting * add other helpers * add prediction_step * fix the kto pair docs * tests * compute_reference_log_probs * add get_eval_dataloader * fix typo * kto with is_encoder_decoder true * Update docs/source/dpo_trainer.mdx Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com> * Update trl/trainer/kto_trainer.py Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com> * Update trl/trainer/kto_trainer.py Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com> * Update trl/trainer/kto_trainer.py Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com> * Update trl/trainer/kto_trainer.py Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com> * Update trl/trainer/kto_trainer.py Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com> * Update trl/trainer/kto_trainer.py Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com> * Update trl/trainer/kto_trainer.py Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com> * Update trl/trainer/kto_trainer.py Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> * Update trl/trainer/kto_trainer.py Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> * Update trl/trainer/kto_trainer.py Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> * fixed typo * Update trl/trainer/kto_trainer.py Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> * Update trl/trainer/kto_trainer.py Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> * Update docs/source/kto_trainer.mdx Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> * Update docs/source/kto_trainer.mdx Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> * renamed KTO dataset keys * use DPOTrainer's get_batch_logps * add get_batch_samples * typo * Handle last token in prompt * Create KTOConfig class that subclasses transformers.TrainingArguments * Update KTO tests to handle KTOConfig * Update KTO script to use KTOConfig * formatting * Update docs/source/dpo_trainer.mdx Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> * Update docs/source/kto_trainer.mdx Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> * Update trl/trainer/kto_trainer.py Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> * Update docs/source/kto_trainer.mdx Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> * Update trl/trainer/training_configs.py Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> * Update examples/scripts/kto.py Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> * Update examples/scripts/kto.py Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> * use max_completion_length * Update examples/scripts/kto.py Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> * add back get_batch_logps * use max_completion_length * move config to its own file * Check tokenize params on Trainer init * Clone labels for end-dec model to solve RuntimeError * formatting * fix enc-dec later * completion_decoder_input_ids is optional for enc-dec * fix breaking test * add a kl key for KL estimation with shuffled completion * add loss ad weights * fix bug in chosen_idx * add back metrics * fix typos * fix kto_loss docs * typo * set loss to None when there is no target completions in batch * use nan tensor instead of none * fix reference_logps test * fix logits * a bit more robust options * log only the correct prompt-completion during eval * Update trl/trainer/kto_trainer.py Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> * Update examples/scripts/kto.py Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> * Update examples/scripts/kto.py Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> * Update docs/source/kto_trainer.mdx Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> * Update docs/source/dpo_trainer.mdx Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> * add docs for desirable_weight and undesirable_weight args * dropout is always disabled * remove DDP hack * formatting * move more arguments of trainer to config * comment out T5 test for now * Add docstring to KTOTrainer * moved Config docstrings to the appropriate class * add autodoc to markdown * formatting * updated copyright year * add model tags * do not add BOS to start of completion * Move data_collator to KTOTrainer * formatting * data_collator is not in args * shuffle_completion with specific input_columns * remove all but the needed columns * Update docs/source/dpo_trainer.mdx Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> * Update examples/scripts/kto.py Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> * Update tests/test_kto_trainer.py Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> * moved more args to kto_config * fjx test * use all_exhausted strategy and shuffle after * use KTOConfig in HfArgumentParser * use ModelConfig --------- Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com> Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> Co-authored-by: Pablo Vicente Juan <p.vicente.juan@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hello 👋
The model in the code snippet in
quicktour.mdhas a key in it'sconfigfile that's likely either deprecated (breaks backwards compatibility?).The code is as follows 👇
The model's config has
enable_lorakey which raises this error and it fails when initializing theLoraConfigobject. So I changed the code snippet to add a more recent model that doesn't error out (and it's currently trending with good amount of likes too so might be useful).