Stars
🔥 LeetCode for PyTorch — practice implementing softmax, attention, GPT-2 and more from scratch with instant auto-grading. Jupyter-based, self-hosted or try online.
PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models(NeurIPS 2024 Spotlight)
A low-bitrate single-codebook 16 / 24 kHz speech codec based on focal modulation
RWKV infctx trainer, for training arbitary context sizes, to 10k and beyond!
Evaluating LLMs with Dynamic Data
A fast RWKV Tokenizer written in Rust
This repository contains an implementation of SliceFine, a parameter-efficient fine-tuning (PEFT) method proposed in the paper titled "SliceFine The Universal Winning-Slice Hypothesis For Pre-train…