Hello, I'm Shivendra. I like to code and make videos. Check out some of my previous work at Vakya
1- SmallLanguageModel [completed, closed]: Making a LLM from scratch all the way from generating raw training data to tokenizing it, creating a model & then training it.
2- Enigma-1.5b [completed, closed]: Transformer model trained on raw DNA data to predict the next letter of the DNA.
3- Axon [completed, updating]: Numpy from scratch in python without any external library. Also with a tiny scalar level aut0grad axon.micro
4- Axgrad [in progress]: Pytorch from scratch, mostly python based but soon will support c/c++ based backend.
5- Axon.drop [in progress]: Scalar level autograd engine written in c/c++ projected to use as tensor & accessed via python
6- WebGraze [completed, updating]: A Python-based library for webscraping & generating/downloading data from various sources on the internet for training ml models.
7- Synapse [completed, to be updated]: A free platform for streaming music & audio/podcasts, based on Youtube V3 API.
8- Micrograd.C [completed, no updates]: Micrograd by Karpathy written in C & C++.
9- AIVA-4x500M [in progress, backlogged]: A multimodal ai system inspired by AVA from Ex-Machina, but currently more like OpenAi's 4o, (I started working on it prior to the model launch).
10- Shredword [in progress, private]: A tokenizer library just like TikToken by OpenAi, using C based code at the backend for fast execution speeds with Python wrapper.
11- Enigma2 [in progress, backlogged]: Second version of Enigma with a different approach this time with the model & tokenization process.
Instagram | Twitter | LinkedIn | Youtube