Commits

Branch selector

main

User selector

All users

All time

Commit history

Commits on Feb 8, 2025

• fix: typos in documentation files (#2804)

  maximevtush

  authored
  55e680eView commit details

  Copy full SHA for 55e680e

  Browse repository at this point

Commits on Feb 7, 2025

• ⛰️ Reduce peak vram consumption with efficient selective log_softmax (#2799)

  Show description for 09eefa7
  

  tyler-romero

  and

  qgallouedec

  authored
  09eefa7View commit details

  Copy full SHA for 09eefa7

  Browse repository at this point

• ➖ Fix GRPO example in README (#2800)

  qgallouedec

  authored
  7fdb69aView commit details

  Copy full SHA for 7fdb69a

  Browse repository at this point

• 🔬 SFT simplification (#2405)

  Show description for 5b9236d
  

  qgallouedec

  and

  kashif

  authored
  5b9236dView commit details

  Copy full SHA for 5b9236d

  Browse repository at this point

• 📠 Log completions for GRPO (#2772)

  Show description for 82d12eb
  

  qgallouedec

  and

  lewtun

  authored
  82d12ebView commit details

  Copy full SHA for 82d12eb

  Browse repository at this point

• 🎯 [SFT] add token accuracy metric (#2597)

  Show description for 84d73fd
  

  kashif

  and

  qgallouedec

  authored
  84d73fdView commit details

  Copy full SHA for 84d73fd

  Browse repository at this point

• 🆚 Distinguish padding and eos when they differ (#2793)

  binary-husky

  authored
  2241f17View commit details

  Copy full SHA for 2241f17

  Browse repository at this point

Commits on Feb 6, 2025

• 📉 Optimize GRPO memory usage by redefining `per_device_batch_size` as generations per device (#2776)

  Show description for cf97133
  qgallouedec

  authored
  cf97133View commit details

  Copy full SHA for cf97133

  Browse repository at this point

• 💡 Add 'Post training an LLM for reasoning with GRPO in TRL' tutorial (#2785)

  sergiopaniego

  authored
  724acb9View commit details

  Copy full SHA for 724acb9

  Browse repository at this point

• Revert "Before the first training step, the model has no optimizer: fix ds3"

  Show description for 7134a1e
  qgallouedec

  committed
  7134a1eView commit details

  Copy full SHA for 7134a1e

  Browse repository at this point

• Before the first training step, the model has no optimizer: fix ds3

  qgallouedec

  committed
  bf6e7edView commit details

  Copy full SHA for bf6e7ed

  Browse repository at this point

• 🙃 Fix reward function in GRPO example (#2777)

  junuMoon

  authored
  e95f9fbView commit details

  Copy full SHA for e95f9fb

  Browse repository at this point

• 💡 GRPO vram-efficiency improvement; only compute relevant logprobs (#2773)

  tyler-romero

  authored
  a85768fView commit details

  Copy full SHA for a85768f

  Browse repository at this point

Commits on Feb 5, 2025

• ↔️ GRPO: Set max_model_len when initializing vLLM instance (#2728)

  Show description for 78c5ce2
  

  mirceapricop

  and

  qgallouedec

  authored
  78c5ce2View commit details

  Copy full SHA for 78c5ce2

  Browse repository at this point

Commits on Feb 4, 2025

• 🚧 Add Optional ZeRO-3 Weight Gathering for GRPO in Sequence Generation (#2667)

  Show description for af4ad47
  

  SeungyounShin

  and

  qgallouedec

  authored
  af4ad47View commit details

  Copy full SHA for af4ad47

  Browse repository at this point

• 🔁 🦈 Support iterative GRPO (#2700)

  Show description for b2ae999
  

  shirinyamani

  and

  qgallouedec

  authored
  b2ae999View commit details

  Copy full SHA for b2ae999

  Browse repository at this point

• 🤖 Properly unwrap torch.compile-ed models in GRPO (#2750)

  Show description for bd946f9
  

  winglian

  and

  qgallouedec

  authored
  bd946f9View commit details

  Copy full SHA for bd946f9

  Browse repository at this point

• 🔎 Add missing script argument in PPO documentation (#2720)

  JohnConnor123

  authored
  f42e34eView commit details

  Copy full SHA for f42e34e

  Browse repository at this point

• 📖 Clarification max len in Reward documentation (#2740)

  Show description for 338fbd5
  

  ParagEkbote

  and

  qgallouedec

  authored
  338fbd5View commit details

  Copy full SHA for 338fbd5

  Browse repository at this point

• 📐 Add vLLM dtype configuration for GRPO trainer (#2738)

  Show description for 32f8fa8
  

  joey00072

  and

  qgallouedec

  authored
  32f8fa8View commit details

  Copy full SHA for 32f8fa8

  Browse repository at this point

• 📌 vLLM >= 0.7.1 for device fix (#2766)

  Show description for 1a22764
  ctjlewis

  authored
  1a22764View commit details

  Copy full SHA for 1a22764

  Browse repository at this point

• 💔 Decouple loss computing and generation in GRPO (#2762)

  qgallouedec

  authored
  1f344c9View commit details

  Copy full SHA for 1f344c9

  Browse repository at this point

• 🔂 Use vLLM prefix caching for speedup (#2757)

  Show description for 85121fc
  

  winglian

  and

  qgallouedec

  authored
  85121fcView commit details

  Copy full SHA for 85121fc

  Browse repository at this point

Commits on Feb 2, 2025

• ⚠️ Fix attention masking in GRPO (#2708)

  Show description for bbdd6db
  

  3 peopleauthored
  bbdd6dbView commit details

  Copy full SHA for bbdd6db

  Browse repository at this point

• docs: Fix typos in alias descriptions (#2729)

  defiberrys

  authored
  6e088d1View commit details

  Copy full SHA for 6e088d1

  Browse repository at this point

Commits on Feb 1, 2025

• fix: Fix typo in filename in ultrafeedback-prompt.py (#2716)

  brawncode

  authored
  a325a0eView commit details

  Copy full SHA for a325a0e

  Browse repository at this point

• 💰 Fix incorrect calculation in Olivia's baguette spending logic (#2727)

  defiberrys

  authored
  0ec1ccdView commit details

  Copy full SHA for 0ec1ccd

  Browse repository at this point

Commits on Jan 31, 2025

• 🏰 `num_logits_to_keep` to `logits_to_keep` (#2721)

  qgallouedec

  authored
  1c35a48View commit details

  Copy full SHA for 1c35a48

  Browse repository at this point

• 📖 Nit fix in SFT Documentation (#2722)

  ParagEkbote

  authored
  2ce36aeView commit details

  Copy full SHA for 2ce36ae

  Browse repository at this point

• Improve GRPO example (#2717)

  lewtun

  authored
  bf69191View commit details

  Copy full SHA for bf69191

  Browse repository at this point

• 📖 Add GRPOTrainer to README.md (#2713)

  Show description for 265663a
  

  burtenshaw

  and

  qgallouedec

  authored
  265663aView commit details

  Copy full SHA for 265663a

  Browse repository at this point

• fix: Fix typo in filename Update ultrafeedback.py (#2699)

  brawncode

  authored
  5ab15d3View commit details

  Copy full SHA for 5ab15d3

  Browse repository at this point

Commits on Jan 30, 2025

• 📋 Add eval loss logging during prediction in GRPO (#2694)

  Show description for fecaa99
  

  kashif

  and

  qgallouedec

  authored
  fecaa99View commit details

  Copy full SHA for fecaa99

  Browse repository at this point

• 💡 Add "Mini-R1: Reproduce Deepseek R1 „aha moment“ a RL tutorial" (#2697)

  Show description for ab30a01
  qgallouedec

  authored
  ab30a01View commit details

  Copy full SHA for ab30a01

  Browse repository at this point

• ☠️ Remove deprecated (#2692)

  Show description for 6dc278a
  qgallouedec

  authored
  6dc278aView commit details

  Copy full SHA for 6dc278a

  Browse repository at this point