rope

Official implementation for "RIFLEx: A Free Lunch for Length Extrapolation in Video Diffusion Transformers" (ICML 2025) , UltraViCo (ICLR 2026) and UltraImage

[ICML 2026] Official implementation for "DyPE: Dynamic Position Extrapolation for Ultra High Resolution Diffusion".

Customized fork of Rope Deepfake software featuring live streaming capabilities and support for Deepfacelive models

[ECCV 2026] Official implementation of "HRDiT: Training-Free High-Resolution Image Generation with Off-the-Shelf Diffusion Transformer Models"

从 MiniMind 源码读起,再延伸到现代大模型技术体系的中文学习笔记。主线逐行精读预训练 / SFT / DPO / PPO / GRPO 与训练机制;附录 17 篇进阶卷覆盖量化、投机解码、RLHF 全景、模型代际史等 MiniMind 没涉及、但进阶绕不开的主题。

[CVPR 2026] Official PyTorch implementation of WaDi: Weight Direction-aware Distillation for One-step Image Synthesis

Extended refactoring capabilities for python-lsp-server using Rope

Generates dynamic rope setup from spline. c4d python plugin.

This repository contains an implementation of the LLaMA 2 (Large Language Model Meta AI) model, a Generative Pretrained Transformer (GPT) variant. The implementation focuses on the model architecture and the inference process. The code is restructured and heavily commented to facilitate easy understanding of the key parts of the architecture.

[ECCV 2026] Official implementation of "MemRoPE: Training-Free Infinite Video Generation via Evolving Memory Tokens"

Rope skipping real-time counting

A skill that teaches LLM agents how to use rope for python codebase refactors

한국어 사전학습과 SFT를 위한 1.09B 파라미터 풀스크래치 LLM — RoPE, KV Cache, BF16·8-bit AdamW 최적화, 데이터셋 캐싱 및 학습 모니터링 지원.

Context-aware Biases for Length Extrapolation

🧮 Algebraic Positional Encodings.

Tiny-MoE is a lightweight Mixture-of-Experts language model built entirely from scratch in native PyTorch and trained end-to-end on Kaggle using free 2× NVIDIA T4 GPUs. The project implements modern LLM techniques—including MLA, RoPE, YaRN, streaming pre-training, and efficient inference—without relying on existing model implementations.

IP Adapter FaceID demo webui

使用Decoder-only的Transformer进行时序预测，包含SwiGLU和RoPE(Rotary Positional Embedding)，Time series prediction using Decoder-only Transformer, Including SwiGLU and RoPE(Rotary Positional Embedding)

PyTorch implementation of RoFormer's Rotary Position Embeddings (RoPE) with comprehensive tests, benchmarks, and documentation. Used in LLaMA, PaLM, GPT-NeoX.

Various data structure implementations in Python