This is just me trying to understand some concepts. Feel free to ignore. Or not.

# Run demo
python3 demo.py

# Test
python3 -m unittest

# Type-check
uv run mypy .

# Notebooks
uv jupyter lab

What I have looked at so far:

1. Backpropagation
2. Artificial neuron
   • Inputs, weights, and bias
   • Activation functions
3. Neuronal network
   • Layers
   • Multi-layer perceptron
4. Training
   • Loss function
   • Backpropagation
   • Parameter tuning (gradient descent)

Things I want to learn more about:

• Transformers
• Attention
• Multi-modal models
• Vision language models

• Andrej Karpathy: Deep Dive into LLMs like ChatGPT
• Andrej Karpathy: Zero to Hero, YouTube playlist
• Standford CS230: Deep Learning (YouTube playlist)
• 3Blue1Brown: Neural networks

• LLM Visualization, GitHub
• Tiktokenizer, GitHub: Visualize how different models split inputs into tokens