PyTorch DataLoader for many VQA datasets

A PyTorch implementation of the Hyena-Y model, a convolution-based multi-hybrid architecture optimized for edge devices.

Predicting a subgraph alongside the answer in a graph based VQA model

MHA, MQA, GQA, MLA 相关原理及简要实现

Vision-Language, Solve GQA(Visual Reasoning in the Real World) dataset.

This is a multimodal model design for the Vision Question Answering (VQA) task. It integrates the Llama2 13B, OWL-ViT, and YOLOv8 models.

A RAG-based question-answering system that processes user queries using local documents. It extracts relevant information to answer questions, falling back to a large language model when local sources are insufficient, ensuring accurate and contextual responses.

Reference Flash Attention implementation in PyTorch with V1/V2, GQA/MQA, Triton kernels, benchmark and docs.

A code deep-dive on one of the key innovations from Deepseek - Multihead Latent Attention (MLA)

Source code for my honours thesis: "Graph Attention Networks for Compositional Visual Question Answering"

A from-scratch PyTorch LLM implementing Sparse Mixture-of-Experts (MoE) with Top-2 gating. Integrates modern Llama-3 components (RMSNorm, SwiGLU, RoPE, GQA) and a custom-coded Byte-Level BPE tokenizer. Pre-trained on a curated corpus of existential & dark philosophical literature.

Modern LLM Attention from Scratch — MHA, GQA, MQA, RoPE, and KV-Cache implemented in pure PyTorch.

「Pytorch 기반 GPT 모델 및 모듈 라이브러리」에 대한 내용을 다루고 있습니다.

This repository contains an implementation of Group Query Attention (GQA), an efficient variant of multi-head attention used in modern transformer models like LLaMA.

Case study of multi-layer perceptron and random forest techniques as applied to a subset of the GQA dataset.

Compare LLM architectures without downloading weights — structural fingerprint & proxy-test advisor for vLLM, TensorRT-LLM, SGLang, ONNX Runtime

Korean 3B LLM (pure Transformer) pretrained from scratch on 8× NVIDIA B200 GPUs with SFT + ORPO alignment

Implementing and benchmarking four attention variants from scratch in PyTorch trained on Lichess chess games.