Last Monday IDA Embedded ran our first "Get started running LLM local" hands on workshop.
To take us through the jungle we had Asbjørn Olling and Marek Hradil to show the easy path using NobodyWho's Open Source library.
They solve the problem embedding LLMs to your application, website fully no strings attached.
We took a deep into how the open weight models are constructed and how local inference work and can be setup on you laptop or servers.
For that part we had setup an isolated network with own firewall, server and some AI compute server. One was your sons gamer machine with full RGB ligth on. And NobodyWho had a brand new Nvidia DGX Spare connected to our network for every body to try models running on that one.
Very positive feedback from the participants and request from some to dive even deeper. There are still a lot of things to explore.
Since Monday I have updated my AI workstation with a used RTX 3090 TI, experimented with different models and optimized the token throughput with all the tricks Mads Henrichsen pulled of in Verbose Podcast #119.
Tested Codex , Opencode and ended up with Pi and Gwen3.8-27b.
And for my daily task it gets the job done. Sure my context window is smaller, but that just a reminder to work smarter. Split task in to smaller chuncks, build better skills.
It just works will all my custom Claude Code skills that describe my IT setup, mail access, hardware lab equipment etc.
The change over will not be hard ...
Thanks to all who showed up - to Asbjørn and Marek for helping out and to the Verbose team Kasper Junge, Jonas Høgh Kyhse-Andersen for showing the way.