Aleksa Gordić’s Post

Well, it's official. YugoGPT 7B significantly beats Mistral and LLaMA 2 and is now officially the best open-source LLM in the world for Serbian & other HBS (Croatian, Bosnian, Montenegrin) languages. Earlier this summer I was frustrated when I saw how poor the situation is as soon as you leave the English NLP space (w/ few exceptions). Exactly 57 days ago I came up with this idea while working on the Open-NLLB project and training a machine translation system for HBS. Why not train an LLM for HBS languages? And so yugoGPT was born. The problem? I had 1 RTX 3090 GPU on my custom built rig. :) (kindly provided to me by NVIDIA back in London!) Fast forward to today I have 16 80 GB A100s from Together AI (whom I seriously recommend - not just for their hardware but also for their support & speed & dedication) and 16 more coming in a week (from a different company -> will share more details then). I had to learn a ton of new stuff as I was basically a single guy (w/ Nikola Ljubešić's tremendous help that saved me a ton of time on the data front) building this whole complex system (shout-out to my Discord community was essential for the Serbian LLM eval project). And I must say: I learned more over the past 6 months than I did during my whole time back at DeepMind. Unless you're very very senior - you just don't get the level of agency I'm enjoying right now. If I could do this bootstrapped with 16 A100s, give me a cluster of 1000 GPUs and I'll show you how things are getting done :) (hah!) well more on that later, things are cooking in the background. I'll just say that I'm already working on the 2nd iteration that'll be significantly better. As for yugoGPT I’ll be red teaming it and I'll need help from the community over the next period to test it. more updates soon :) keep an eye out Again big thanks to: * Together AI - for generous help with compute * Weights & Biases - for generous donation of GPT-4 credits for the Serbian LLM eval project * Slobodan Marković - for spreading the word!! * Nikola Ljubešić - for sharing the data he's been working on over the past many years * and to many individuals & companies from the local region (mostly Serbia) that recently supported me! --- Results obtained via Serbian LLM eval that I've recently released: https://lnkd.in/diQDTeh2