Welcome to my personal playground for mastering Artificial Intelligence and Machine Learning from first principles.
The goal of this repository is strictly educational: to deeply understand how modern AI architecture works by building core components from scratch in C++ without relying on external ML libraries.
This repository contains C++ implementations of foundational neural network architectures, built step-by-step to understand the underlying mathematics, forward/backward propagation mechanisms, and gradient dynamics.
-
Multi-Layer Perceptron (MLP)
- Fully custom forward & backpropagation implementation from scratch.
- Solves classic non-linear problems like XOR.
- Implements Sigmoid activations, Mean Squared Error (MSE) loss, and Stochastic Gradient Descent (SGD).
-
Mini-Transformer / Decoder Language Model
- Minimal C++ autoregressive language model inspired by GPT architectures.
- Key components implemented manually:
- Character-level Tokenizer & Embedding layer (with positional signal injection).
- Causal Self-Attention mechanism with causal masking.
- Feed-Forward Neural Networks (FFN).
- Layer Normalization (
LayerNorm) to stabilize deep residual streams. - Gradient Clipping to prevent exploding gradients.
- Language Model Head with Cross-Entropy Loss & Softmax.
Alongside the source code, this repo serves as my personal lab notebook. You will find written post-mortems and technical breakdowns of experiments performed during training, such as:
- Training Dynamics & Stability: Analyzing gradient explosions in deep residual networks and how Layer Normalization & Gradient Clipping resolve them.
- Overfitting & Sampling: Observing token loops and deterministic behavior in micro-datasets using greedy decoding versus sampling techniques.
All you need is a C++ compiler supporting modern C++ (C++11 or higher, g++ or clang++). No external dependencies required!