Commits

Branch selector

 master

User selector

hjlee1371

Datepicker

All time

Commit History

Commits on Dec 9, 2024

• remove unnecessary files

  hjlee1371

  committed
  f31ab3c

  Copy full SHA for f31ab3c

• fix moe

  hjlee1371

  committed
  6c9faaf

  Copy full SHA for 6c9faaf

• change moe implementaion to use megablocks

  hjlee1371

  committed
  b1869c6

  Copy full SHA for b1869c6

Commits on Nov 3, 2024

• add deepseekv2-like mla

  hjlee1371

  committed
  e55ebc2

  Copy full SHA for e55ebc2

Commits on Oct 19, 2024

• add moe aux loss

  hjlee1371

  committed
  79e2018

  Copy full SHA for 79e2018

• add moe

  hjlee1371

  committed
  1f21d28

  Copy full SHA for 1f21d28

Commits on Oct 11, 2024

• update README

  hjlee1371

  committed
  b083975

  Copy full SHA for b083975

• update README

  hjlee1371

  committed
  6b676c8

  Copy full SHA for 6b676c8

• update README

  hjlee1371

  committed
  ff444fc

  Copy full SHA for ff444fc

• update README

  hjlee1371

  committed
  887fcfe

  Copy full SHA for 887fcfe

• add mup-simple

  hjlee1371

  committed
  802a8e0

  Copy full SHA for 802a8e0

Commits on Oct 10, 2024

• fix multi-gpu logging

  hjlee1371

  committed
  5a41836

  Copy full SHA for 5a41836

• add qknorm argument

  hjlee1371

  committed
  8446bdf

  Copy full SHA for 8446bdf

Commits on Oct 9, 2024

• add qknorm

  hjlee1371

  committed
  21dc8b6

  Copy full SHA for 21dc8b6

• add initialization

  hjlee1371

  committed
  ce377bc

  Copy full SHA for ce377bc

• remove swp

  hjlee1371

  committed
  0f08b7e

  Copy full SHA for 0f08b7e

• improve logging

  hjlee1371

  committed
  975f43f

  Copy full SHA for 975f43f

• fix z-loss

  hjlee1371

  committed
  28dc39a

  Copy full SHA for 28dc39a

• update to llama2 tokenizer & add openwebtext-10k

  hjlee1371

  committed
  8bb267c

  Copy full SHA for 8bb267c

• improve loss logging

  hjlee1371

  committed
  5b893fa

  Copy full SHA for 5b893fa

• add tensorboard

  hjlee1371

  committed
  c5428ba

  Copy full SHA for c5428ba

• remove fp16 support

  hjlee1371

  committed
  2ce47ff

  Copy full SHA for 2ce47ff

• modify default config for small-scale proxy

  hjlee1371

  committed
  f7711d2

  Copy full SHA for f7711d2

• add mfu calculation

  hjlee1371

  committed
  2c94d39

  Copy full SHA for 2c94d39

• add license

  hjlee1371

  committed
  1fdc3b5

  Copy full SHA for 1fdc3b5

• add z-loss

  hjlee1371

  committed
  d72b219

  Copy full SHA for d72b219

Commits on Oct 5, 2024

• add llama

  hjlee1371

  committed
  b2e8a50

  Copy full SHA for b2e8a50