Commits

Branch selector

 main

User selector

crowsonkb

All time

Commit History

Commits on Oct 11, 2025

• Update ring_attn.py for flash attention 2.8.3

  crowsonkb

  committed
  ad41aba

  Copy full SHA for ad41aba

Commits on May 26, 2025

• Add validation loss script

  crowsonkb

  committed
  a91584b

  Copy full SHA for a91584b

Commits on Mar 30, 2025

• Add Mistral 3 support to long context trainer

  crowsonkb

  committed
  299cba1

  Copy full SHA for 299cba1

Commits on Mar 28, 2025

• Add long context preference tuner

  crowsonkb

  committed
  f6c6710

  Copy full SHA for f6c6710

Commits on Feb 27, 2025

• Update trainer for transformers 4.49.0

  crowsonkb

  committed
  7688abb

  Copy full SHA for 7688abb

Commits on Dec 15, 2024

• Add Qwen2 model type to long context trainer

  crowsonkb

  committed
  8fbd73b

  Copy full SHA for 8fbd73b

Commits on Oct 16, 2024

• Correct normalization for cross-entropy loss

  crowsonkb

  committed
  b02fbcb

  Copy full SHA for b02fbcb

Commits on Oct 15, 2024

• Correctly do ring attention for multinode training

  crowsonkb

  committed
  8f730ba

  Copy full SHA for 8f730ba

Commits on Oct 14, 2024

• Do sequence parallel across each node separately in long context trainer

  crowsonkb

  committed
  872ff77

  Copy full SHA for 872ff77

Commits on Oct 12, 2024

• Add long context trainer

  crowsonkb

  committed
  64a04f2

  Copy full SHA for 64a04f2

Commits on Sep 4, 2024

• Update requirements.txt

  crowsonkb

  committed
  c8eecd3

  Copy full SHA for c8eecd3

• Update Dockerfile

  crowsonkb

  committed
  ede6f73

  Copy full SHA for ede6f73

Commits on Dec 4, 2023

• Fix application menu items on macOS

  crowsonkb

  committed
  5567f1f

  Copy full SHA for 5567f1f

Commits on Dec 2, 2023

• Use evaluator logit scale argument

  crowsonkb

  committed
  3c0c2b9

  Copy full SHA for 3c0c2b9

• Add the Direct Preference Optimization RLAIF generator script

  crowsonkb

  committed
  518696e

  Copy full SHA for 518696e

Commits on Nov 24, 2023

• Fix gradient checkpointing in RLAIF script

  crowsonkb

  committed
  135fccc

  Copy full SHA for 135fccc

Commits on Jul 21, 2023

• Add dice-mc to requirements.txt

  crowsonkb

  committed
  6a54135

  Copy full SHA for 6a54135

Commits on Jul 19, 2023

• Add generator LoRA support

  crowsonkb

  committed
  4c1ae01

  Copy full SHA for 4c1ae01

Commits on Jul 14, 2023

• Use RiversHaveWings/minihf_evaluator_openllama_7b as evaluator

  crowsonkb

  committed
  bd909b7

  Copy full SHA for bd909b7

Commits on Jul 12, 2023

• Generate using batch size 1

  crowsonkb

  committed
  032fb03

  Copy full SHA for 032fb03

Commits on Jul 6, 2023

• Add progress bar on console for individual calls to the generate function

  crowsonkb

  committed
  0e38387

  Copy full SHA for 0e38387

• Fix evaluation of root node in Weave

  crowsonkb

  committed
  ff08d70

  Copy full SHA for ff08d70

Commits on Jul 4, 2023

• Use OpenLLaMA 7B as generator, LoRA of it as evaluator

  crowsonkb

  committed
  aa9f166

  Copy full SHA for aa9f166

Commits on Jun 5, 2023

• Tune in 8-bit, with gradient checkpointing, on multiple GPUs

  crowsonkb

  committed
  590fbcf

  Copy full SHA for 590fbcf

Commits on May 28, 2023

• Add .gitignore

  crowsonkb

  committed
  35fdb10

  Copy full SHA for 35fdb10

• Don't try to open directories in a zip as files

  crowsonkb

  committed
  9db6d77

  Copy full SHA for 9db6d77

Commits on May 27, 2023

• Fix overflow in fp16 when computing mean embedding during training

  crowsonkb

  committed
  47edacf

  Copy full SHA for 47edacf

Commits on May 26, 2023

• Exclude padding tokens from mean embedding when training reward models

  crowsonkb

  committed
  78ae67a

  Copy full SHA for 78ae67a