Commits

Branch selector

main

User selector

All users

All time

Commit history

Commits on Jan 26, 2025

• 🏷️ Add model tags to model trained with GRPO (#2663)

  qgallouedec

  authored
  1123bd0View commit details

  Copy full SHA for 1123bd0

  Browse repository at this point

• 🌀 Fix GRPO default completion length doc (#2662)

  andyl98

  authored
  55a329eView commit details

  Copy full SHA for 55a329e

  Browse repository at this point

Commits on Jan 25, 2025

• 📏 Log completion length in GRPO (#2659)

  qgallouedec

  authored
  4720656View commit details

  Copy full SHA for 4720656

  Browse repository at this point

• 📍 Disable caching when grad checkpointing enable in GRPO (#2653)

  Show description for 807046b
  qgallouedec

  authored
  807046bView commit details

  Copy full SHA for 807046b

  Browse repository at this point

• 🔎 Finegrained reward logging for GRPO (#2651)

  qgallouedec

  authored
  317d2d4View commit details

  Copy full SHA for 317d2d4

  Browse repository at this point

• 👐 DeepSpeed integration for GRPO (#2652)

  qgallouedec

  authored
  aeb03cfView commit details

  Copy full SHA for aeb03cf

  Browse repository at this point

Commits on Jan 24, 2025

• 🚛 Provide all columns of the dataset to the reward function (#2650)

  Show description for 2578e95
  qgallouedec

  authored
  2578e95View commit details

  Copy full SHA for 2578e95

  Browse repository at this point

• 🥞 Fix KTO gradient accumulation loss scaling (#2648)

  qgallouedec

  authored
  6f99f42View commit details

  Copy full SHA for 6f99f42

  Browse repository at this point

• 🥞 Fix GRPO gradient accumulation loss scaling (#2647)

  qgallouedec

  authored
  d14f7f3View commit details

  Copy full SHA for d14f7f3

  Browse repository at this point

• 🥞 Fix CPO gradient accumulation loss scaling (#2645)

  qgallouedec

  authored
  8e65825View commit details

  Copy full SHA for 8e65825

  Browse repository at this point

• Update grpo_trainer.md

  qgallouedec

  authored
  5e4d7beView commit details

  Copy full SHA for 5e4d7be

  Browse repository at this point

Commits on Jan 23, 2025

• 🌯 Fix context manager runtime error when gather is disabled (#2639)

  Superskyyy

  authored
  f34b70aView commit details

  Copy full SHA for f34b70a

  Browse repository at this point

• 🍭 Custom reward function for RLOO (#2612)

  Show description for 0e216f7
  August-murr

  authored
  0e216f7View commit details

  Copy full SHA for 0e216f7

  Browse repository at this point

• 🥞 Fix BCO gradient accumulation loss scaling (#2638)

  qgallouedec

  authored
  59c2014View commit details

  Copy full SHA for 59c2014

  Browse repository at this point

• 🥞 Fix DPO gradient accumulation loss scaling (#2615)

  Show description for 40c2383
  

  winglian

  and

  qgallouedec

  authored
  40c2383View commit details

  Copy full SHA for 40c2383

  Browse repository at this point

• 🏆 Custom reward function for GRPO and shiny doc (#2606)

  Show description for a1d2955
  

  qgallouedec

  and

  lewtun

  authored
  a1d2955View commit details

  Copy full SHA for a1d2955

  Browse repository at this point

• 💎 Rename an inner var in GRPO to improve clarity (#2616)

  Show description for 887c1f3
  qgallouedec

  authored
  887c1f3View commit details

  Copy full SHA for 887c1f3

  Browse repository at this point

• 👋 Drop MDX (#2611)

  qgallouedec

  authored
  949db23View commit details

  Copy full SHA for 949db23

  Browse repository at this point

Commits on Jan 22, 2025

• ✂️ Reintroduce `truncation_mode` in `DPOTrainer` (#2551)

  Show description for fe4b5ef
  

  anakin87

  and

  qgallouedec

  authored
  fe4b5efView commit details

  Copy full SHA for fe4b5ef

  Browse repository at this point

• 🫷 Include stop token in policy model's generation_config (#2528)

  Show description for a9b54a8
  

  dawidm

  and

  qgallouedec

  authored
  a9b54a8View commit details

  Copy full SHA for a9b54a8

  Browse repository at this point

Commits on Jan 21, 2025

• 🧩 PPO/RLOO/OnlineDPO sequence generation: make deepsped 3 weight gathering optional (#2557)

  Show description for d4222a1
  

  dawidm

  and

  qgallouedec

  authored
  d4222a1View commit details

  Copy full SHA for d4222a1

  Browse repository at this point

• ⚡ Add uv installation instructions (#2601)

  Show description for a5c88d6
  

  stevhliu

  and

  qgallouedec

  authored
  a5c88d6View commit details

  Copy full SHA for a5c88d6

  Browse repository at this point

• 💾 Reduce memory peak in GRPO by adding `max_prompt_length` and loop usage in logp computation (#2598)

  Show description for b6a084c
  qgallouedec

  authored
  b6a084cView commit details

  Copy full SHA for b6a084c

  Browse repository at this point

• 🧰 Tool fine-tuning support DPO (#2479)

  Show description for d9f0568
  

  August-murr

  and

  qgallouedec

  authored
  d9f0568View commit details

  Copy full SHA for d9f0568

  Browse repository at this point

Commits on Jan 20, 2025

• Fix merge error (#2595)

  qgallouedec

  authored
  3d2c1e4View commit details

  Copy full SHA for 3d2c1e4

  Browse repository at this point

• 🫣 Ignore CLI test for Python 3.9 (#2592)

  Show description for 5fd7836
  qgallouedec

  authored
  5fd7836View commit details

  Copy full SHA for 5fd7836

  Browse repository at this point

• 👨‍👨‍👧‍👧 GRPO (#2565)

  Show description for 0f5ffad
  qgallouedec

  authored
  0f5ffadView commit details

  Copy full SHA for 0f5ffad

  Browse repository at this point

Commits on Jan 18, 2025

• Update reducing_memory_usage.md

  qgallouedec

  authored
  88514d5View commit details

  Copy full SHA for 88514d5

  Browse repository at this point

• 🎞️ Fix documentation SFT -`max_seq_length` instead of `max_length` (#2590)

  skandermoalla

  authored
  76837e8View commit details

  Copy full SHA for 76837e8

  Browse repository at this point

Commits on Jan 17, 2025

• 🫢 Add `max_prompt_length` parameter in tests (#2588)

  Show description for 3555393
  qgallouedec

  authored
  3555393View commit details

  Copy full SHA for 3555393

  Browse repository at this point

• ✂️ Truncate by default (#2587)

  Show description for fd4b283
  qgallouedec

  authored
  fd4b283View commit details

  Copy full SHA for fd4b283

  Browse repository at this point

• [RLOO] fix token_level_kl (#2575)

  Show description for 1b1140a
  kashif

  authored
  1b1140aView commit details

  Copy full SHA for 1b1140a

  Browse repository at this point

• 🐛 Simplify bug report template (#2585)

  qgallouedec

  authored
  4c7eb6fView commit details

  Copy full SHA for 4c7eb6f

  Browse repository at this point

• Update issue_auto_labeller.yml [ci skip]

  qgallouedec

  authored
  564fc86View commit details

  Copy full SHA for 564fc86

  Browse repository at this point

• Update issue_auto_labeller.yml

  qgallouedec

  authored
  3215a1cView commit details

  Copy full SHA for 3215a1c

  Browse repository at this point