Commits

Branch selector

 main

User selector

All users

Datepicker

All time

Commit History

Commits on Jun 3, 2025

• 0.2.16

  lucidrains

  committed
  ac234d9

  Copy full SHA for ac234d9

• address https://github.com/lucidrains/nGPT-pytorch/issues/12

  lucidrains

  committed
  67779ed

  Copy full SHA for 67779ed

Commits on Mar 19, 2025

• complete ability to breed two normalized transformers, add another function for normal transformers based on svd later

  lucidrains

  committed
  166bbe7

  Copy full SHA for 166bbe7

Commits on Mar 16, 2025

• refactor evo functions to ready for full transformer

  lucidrains

  committed
  753dfce

  Copy full SHA for 753dfce

Commits on Mar 15, 2025

• prepare to crossover entire transformer

  lucidrains

  committed
  9748367

  Copy full SHA for 9748367

Commits on Mar 14, 2025

• evolution will be part of the final equation.

  lucidrains

  committed
  d42b793

  Copy full SHA for d42b793

Commits on Mar 13, 2025

• update

  lucidrains

  authored
  9acfe26

  Copy full SHA for 9acfe26

• cross over the feedforward scales as well

  lucidrains

  committed
  b2acfa0

  Copy full SHA for b2acfa0

• prepare for breeding the nGPT feedforwards

  lucidrains

  committed
  79d9d95

  Copy full SHA for 79d9d95

Commits on Nov 7, 2024

• add enable_cudnn

  lucidrains

  committed
  5208aad

  Copy full SHA for 5208aad

Commits on Nov 3, 2024

• fix returning zeros when row is all masked out for non-causal, and remove mask value

  lucidrains

  committed
  f4a6c0d

  Copy full SHA for f4a6c0d

Commits on Nov 2, 2024

• protect against row being all masked out in sdpa

  lucidrains

  committed
  d46e9cc

  Copy full SHA for d46e9cc

• oops, got it backwards

  lucidrains

  committed
  899b414

  Copy full SHA for 899b414

• address https://github.com/lucidrains/nGPT-pytorch/issues/11 with a hparam

  lucidrains

  committed
  fcc4b16

  Copy full SHA for fcc4b16

• 0.2.2

  lucidrains

  committed
  6129dee

  Copy full SHA for 6129dee

• Revert "address https://github.com/lucidrains/nGPT-pytorch/issues/9"

  Show description for 328cf61
  lucidrains

  committed
  328cf61

  Copy full SHA for 328cf61

• address https://github.com/lucidrains/nGPT-pytorch/issues/9

  lucidrains

  committed
  7aa502f

  Copy full SHA for 7aa502f

Commits on Oct 31, 2024

• line things up a bit more accurately for follow up research

  lucidrains

  committed
  0522845

  Copy full SHA for 0522845

• prep for kv cache

  lucidrains

  committed
  1f80078

  Copy full SHA for 1f80078

• fix value residual

  lucidrains

  committed
  eb4d271

  Copy full SHA for eb4d271

• fix experimental features on Residual

  lucidrains

  committed
  2462985

  Copy full SHA for 2462985

• default scale on Residual to None

  lucidrains

  committed
  cfbd064

  Copy full SHA for cfbd064

Commits on Oct 30, 2024

• fix a bug when model dimension != head dimension x heads

  lucidrains

  committed
  f6b5082

  Copy full SHA for f6b5082

Commits on Oct 29, 2024

• allow for model to auto norm weights on optimizer step

  lucidrains

  committed
  8be11d3

  Copy full SHA for 8be11d3

• default to norm qk false

  lucidrains

  committed
  c03b634

  Copy full SHA for c03b634

Commits on Oct 28, 2024

• add value residual learning

  lucidrains

  committed
  44f20aa

  Copy full SHA for 44f20aa

• trust @jfpuget and default example to using parametrize

  lucidrains

  committed
  61e484b

  Copy full SHA for 61e484b

• just default working example to mixed precision

  lucidrains

  committed
  6f72109

  Copy full SHA for 6f72109

• value residual for the experimental version

  lucidrains

  committed
  e35d871

  Copy full SHA for e35d871

• able to turn off rotary for nTransformer

  lucidrains

  committed
  6fcd3c6

  Copy full SHA for 6fcd3c6

• address https://github.com/lucidrains/nGPT-pytorch/issues/3

  lucidrains

  committed
  9c1f52c

  Copy full SHA for 9c1f52c

• fix eps for nTransformer

  lucidrains

  committed
  50f9a5e

  Copy full SHA for 50f9a5e

Commits on Oct 27, 2024

• address https://github.com/lucidrains/nGPT-pytorch/issues/9

  lucidrains

  committed
  f56477a

  Copy full SHA for f56477a

Commits on Oct 25, 2024

• add an experimental version

  lucidrains

  committed
  42660fc

  Copy full SHA for 42660fc

Commits on Oct 21, 2024

• address https://github.com/lucidrains/nGPT-pytorch/issues/4

  lucidrains

  committed
  5e76aac

  Copy full SHA for 5e76aac

Pagination

Previous

Next