Commits

Branch selector

pjit-t5x

User selector

All users

All time

Commit history

Commits on Jan 26, 2022

• feat(train): overhead from 70% to 1% 🥳

  borisdayma

  committed
  2b7f5f1View commit details

  Copy full SHA for 2b7f5f1

  Browse repository at this point

• feat(pjit): follow t5x style

  borisdayma

  committed
  7b5868fView commit details

  Copy full SHA for 7b5868f

  Browse repository at this point

Commits on Jan 25, 2022

• fix(train): grads spec

  borisdayma

  committed
  00710bcView commit details

  Copy full SHA for 00710bc

  Browse repository at this point

• feat(train): improve pjit speed

  borisdayma

  committed
  f254058View commit details

  Copy full SHA for f254058

  Browse repository at this point

• fix(train): consider correct batch size

  borisdayma

  committed
  b7c7458View commit details

  Copy full SHA for b7c7458

  Browse repository at this point

Commits on Jan 24, 2022

• feat(train): custom start_preconditioning_step

  borisdayma

  committed
  8149924View commit details

  Copy full SHA for 8149924

  Browse repository at this point

• feat(train): handle distributed_shampoo in pjit

  borisdayma

  committed
  032f623View commit details

  Copy full SHA for 032f623

  Browse repository at this point

• feat: update distributed_shampoo + fix None spec

  borisdayma

  committed
  8a9e367View commit details

  Copy full SHA for 8a9e367

  Browse repository at this point

• feat(train): distributed_shampoo with pjit

  borisdayma

  committed
  cc34d07View commit details

  Copy full SHA for cc34d07

  Browse repository at this point

Commits on Jan 23, 2022

• feat(train): use pjit (#125)

  borisdayma

  authored
  f5239e1View commit details

  Copy full SHA for f5239e1

  Browse repository at this point

• style: unsused import

  borisdayma

  committed
  7a176b9View commit details

  Copy full SHA for 7a176b9

  Browse repository at this point

• fix style

  borisdayma

  committed
  f044cb8View commit details

  Copy full SHA for f044cb8

  Browse repository at this point

• feat(train): restore opt_state efficiently

  borisdayma

  committed
  1bfc1b5View commit details

  Copy full SHA for 1bfc1b5

  Browse repository at this point

Commits on Jan 22, 2022

• feat(model): clean way to load on cpu

  borisdayma

  committed
  12f323dView commit details

  Copy full SHA for 12f323d

  Browse repository at this point

• feat(train): load model on CPU

  borisdayma

  committed
  3d43591View commit details

  Copy full SHA for 3d43591

  Browse repository at this point

• feat(train): different rng per node

  borisdayma

  committed
  2d212d8View commit details

  Copy full SHA for 2d212d8

  Browse repository at this point

• feat(train): no batch dimension with pjit

  borisdayma

  committed
  df1fe19View commit details

  Copy full SHA for df1fe19

  Browse repository at this point

• feat(train): progress on pjit

  borisdayma

  committed
  49597a2View commit details

  Copy full SHA for 49597a2

  Browse repository at this point

Commits on Jan 21, 2022

• feat(train): start pjit support

  borisdayma

  committed
  0081723View commit details

  Copy full SHA for 0081723

  Browse repository at this point

Commits on Jan 20, 2022

• feat: use_artifact if run existing

  borisdayma

  committed
  a5ed112View commit details

  Copy full SHA for a5ed112

  Browse repository at this point

• Load from wandb artifact (#121)

  Show description for f69b21b
  borisdayma

  authored
  f69b21bView commit details

  Copy full SHA for f69b21b

  Browse repository at this point

• Style (isort).

  pcuenca

  committed
  f9d51f7View commit details

  Copy full SHA for f9d51f7

  Browse repository at this point

Commits on Jan 19, 2022

• feat(train): update sweep config

  borisdayma

  committed
  bbbf7c8View commit details

  Copy full SHA for bbbf7c8

  Browse repository at this point

Commits on Jan 18, 2022

• Use DalleBartTokenizer. State restoration reverted to previous method:

  Show description for ae983d7
  pcuenca

  committed
  ae983d7View commit details

  Copy full SHA for ae983d7

  Browse repository at this point

• Tokenizer, config, model can be loaded from wandb.

  Show description for 7e48337
  pcuenca

  committed
  7e48337View commit details

  Copy full SHA for 7e48337

  Browse repository at this point

• fix(train): variable not defined

  borisdayma

  committed
  4c87adfView commit details

  Copy full SHA for 4c87adf

  Browse repository at this point

• feat(train): cleanup args

  borisdayma

  committed
  a2bf605View commit details

  Copy full SHA for a2bf605

  Browse repository at this point

Commits on Jan 17, 2022

• Merge pull request #122 from borisdayma/feat-acccum

  Show description for c91ceb7
  borisdayma

  authored
  c91ceb7View commit details

  Copy full SHA for c91ceb7

  Browse repository at this point

• feat(data): support accumulation in non-streaming

  borisdayma

  committed
  88c8e06View commit details

  Copy full SHA for 88c8e06

  Browse repository at this point

• refactor(train): cleanup

  borisdayma

  committed
  274ba73View commit details

  Copy full SHA for 274ba73

  Browse repository at this point

• feat: custom gradient accumulation

  borisdayma

  committed
  2d07559View commit details

  Copy full SHA for 2d07559

  Browse repository at this point

Commits on Jan 16, 2022

• fix: style

  borisdayma

  committed
  df01fa8View commit details

  Copy full SHA for df01fa8

  Browse repository at this point

• feat(train): use MultiSteps for gradient accumulation

  borisdayma

  committed
  4fa53a5View commit details

  Copy full SHA for 4fa53a5

  Browse repository at this point

• Change import order again.

  Show description for 2b2be9b
  pcuenca

  committed
  2b2be9bView commit details

  Copy full SHA for 2b2be9b

  Browse repository at this point

• Fix import order to make isort happy.

  pcuenca

  committed
  64d99b2View commit details

  Copy full SHA for 64d99b2

  Browse repository at this point