Bogdan Buduroiu

Code of Conduct ⚡ Zig Programming Language

Thu, 30 Apr 2026 12:24:10 +0800

Strict No LLM / No AI Policy

No LLMs for issues.

No LLMs for pull requests.

No LLMs for comments on the bug tracker, including translation. English is encouraged, but not required. You are welcome to post in your native language and rely on others to have their own translation tools of choice to interpret your words.

Reading this on the Zig project page, I brushed it off as Zig sticking their head in the sand and making an ideological, but misguided call. It was only after reading The Zig project’s rationale for their firm anti-AI contribution policy - Simon Willison’s Weblog that I started to engage with the rationale of Zig’s choice, and I have to admit that I was the one that sutck my head in the sand and made an ideological, but misguided call.

Mechanize

Sun, 12 Apr 2026 09:36:30 +0800

At the end of my AI Lent wrap up post post I wrote:

I was sure that I would renounce the use of AI tools for good, and proudly proclaim my newfound wisdom and virtue to the world. Now, I’m not so sure.

I recently stumbled upon this company’s landing page and hiring pitch:

Your job is to build software environments that frontier AI fails at. You’ll do most of the building through coding agents; writing code by hand is too slow now. The models are good enough that finding something they genuinely can’t do is not easy.

Pi Zero Bikecomputer

Sat, 11 Apr 2026 16:43:53 +0800

Planning on making a proper build around this, which has some sort of weatherproofing, and can be mounted out-front.

A Mac Studio for Local AI - 6 Months Later

Sat, 11 Apr 2026 16:38:52 +0800

Interesting insight that MoE models might be better suited for self-hosting, since apps like Claude Code require multiple models to work in parallel for different tasks.

AI Lent - Wrap up

Thu, 09 Apr 2026 00:00:00 +0000

Easter weekend has come and gone in a blitz, and seemingly, so has this challenge. As a reminder for those coming here for the first time:

Refresher on ‘AI Lent’

In Christian tradition, Lent is a 40-day fast and repentance preceding Easter, beginning on Ash Wednesday and ending before Maundy Thursday or Easter. In that same vein, I chose to embark on a 40-day fast from any AI use.

AI Lent - Day 8-26 - Back to coding agents

Mon, 16 Mar 2026 00:00:00 +0000

It almost feels like writing a journal entry detailing some sort of relapse on the road to an addiction I’m trying to break. It felt like something I knew I shouldn’t do, but was too tempting not to indulge in. I felt my brain freed up from thinking about hard problems, and able to just scroll the internet while the agents did the work for me.

After a couple of days, I was rationalising this “relapse” as “aid” during a time of many conflicting deadlines. I can clearly say now, this was probably a false belief, and that it probably took me more time to complete tasks due to the amount of fragmentation my attention was subject to.

The Rime of the Ancient Maintainer

Wed, 04 Mar 2026 19:21:47 +0800

Article argues that we should value maintainers as guardians, and that maintaining is an unappreciated job, because if maintaining is done right, nothing happens.

brainrot is a radical act

Wed, 04 Mar 2026 19:18:44 +0800

Hoard things you know how to do

Wed, 04 Mar 2026 19:03:57 +0800

A big part of the skill in building software is understanding what’s possible and what isn’t, and having at least a rough idea of how those things can be accomplished.

At some point, I will release my post on why I think, regardless of LLM-assisted coding (but even more because of LLM-assisted coding), taste is one of the most important skills a Software Engineer can hone.

Zohran Mamdani Wants to Reclaim Efficiency From the Right

Wed, 25 Feb 2026 23:35:02 +0800

These “reforms” have predictably eroded the capacity of governments to carry out basic functions while increasing their dependence on profit-driven companies and contractors that often cost significantly more than government employees. […] contractors now outnumber federal employees by more than two to one (the size of the federal workforce has remained largely unchanged over the last half century, […])

Romania has its own version of this problem. The consultancy and contracting ecosystem around EU fund absorption in Romania is enormous, and there’s well-documented waste in how procurement operates at the local level.

It took me almost a year to exit Apple's Walled Garden

Wed, 25 Feb 2026 19:11:00 +0800

TLDR at the end

For someone with a lot of free time to tinker with tech, Apple feels kinda meh nowadays. Don’t get me wrong, I think the technical innovations that allow the M Series chips to be this ridiculously powerful, battery efficient, silent, and cool is nothing short of impressive. The iPhone is also an incredibly well built product with an amazing camera, crisp Facetime calls, and great media playback. The extreme multi-tenancy of the FoundationDB-backed iCloud is an unsung engineering feat. Also, the seamless iCloud sync and backups will never get in the way of your 12hr+ doomscroll session…

Issue #8 (Feb'26): Page-based buffer overflow

Tue, 24 Feb 2026 21:47:41 +0800

I love this format, I like to see more human curation of content.

slop capitalism

Tue, 24 Feb 2026 18:15:30 +0800

AI is the Best Thing to Happen to Art

Tue, 24 Feb 2026 16:58:10 +0800

I haven’t watched any of the Marvel movies, the last Disney movie I enjoyed was the 1998 version of Mulan. However, I feel like I’ve watched all the Avengers movies by just seeing the billboard ads that appeared in Taipei, as well as through the plastic slop that gets shipped to movie theatres with the release.

I’m not sure about Hollywood, but for STEM disciplines, a lot of brilliant individuals get poached by finance, even though their degrees are in theoretical physics. It’s hard to say no to the bag, especially when you see the number of 0s on your first paycheck, and your brain hasn’t fully developed yet. From there, most brilliant minds fall to lifestyle creep, to the point where they can’t downgrade their lives to work on something more noble, because their kids now go to private school and they’re paying mortgage on a house that’s way bigger than they need.

The Age-Verification Trap: Verifying user’s ages undermines everyone’s data protection

Tue, 24 Feb 2026 16:54:50 +0800

Age verification is obviously a “casus belli” that a bunch of (mostly Western, mostly European) leaders adopt because they realise the next 10-20 years will be marked by having to lead from positions of deep unpopularity.

Brussels though, chooses the path of authoritarianism and crushing dissent, to unite in a “totalitarian war union” (to borrow from Varoufakis), instead of understanding the reason for their deep unpopularity and reforming.

Moreover, Europe needs a post-EU future.

AI Lent - Day 7 - Almost failed

Tue, 24 Feb 2026 16:34:44 +0800

Ok, * cracks knuckles * I know what I need to do to refactor this. It’s a bit hairy, at times containing anti-patterns, and quite highly coupled, but it’s nothing I haven’t tackled before.

Oh wait… actually I spoke too soon, I think my initial assessment of the problem space was wrong, and there is a reason we ended up with this spagetto.

I still have claude installed on my machine, what’s so bad with just using it to get a birds’ eye view of the problem and use it as a rubber duck?

What Do Large Language Models Tell Us about Ourselves?

Mon, 23 Feb 2026 16:56:28 +0800

What if instead of measuring AI by the standard of human intelligence, we measured human intelligence by the standard of AI. We have a clearer understanding of how AI works, than how the human brain works. Could this exercise actually help us derive some insights into ourselves?

The paper argues that we’ve been actually doing this for decades, starting with how superhuman abilities demonstrated by Deep Blue made us reconsider what it means to be “good at chess”, and more recently reconsidering the most intimate parts of human existence, language.

AI Lent - Day 6

Mon, 23 Feb 2026 16:34:44 +0800

Still really really hard to work with large codebases without AI, it’s really mentally draining, but I think it’s because I’ve relied on Claude Code for so long. This is so ironic, since refactoring large codebases should’ve been the Achilles heel of coding agents.

I went back to my vim strategy to understand where I’m going wrong and why it’s hard for me to keep more than 3 pieces of code referencing eachother in my mind.

The dawn of the post-literate society

Mon, 23 Feb 2026 16:25:31 +0800

Long read about how the reading revolution of the 18th century was “a catastrophe for the ultra-privileged and exploitative aristocrats of the European aristocratic ancient regime”, how the rapid expansion of literacy helped destroy the old system of government with kings at the top, lords and clergy underneath and peasants squirming at the very bottom, and how our modern loss of literacy is incompatible with democracy.

How Google Maps quietly allocates survival across London’s restaurants - and how I built a dashboard to see through it

Mon, 23 Feb 2026 16:22:35 +0800

Google Maps is not a directory, it’s a market maker. A lot of interesting findings in this one, including that Google Maps is surprisingly bad at categorising cuisines and restaurants (I’ve seen this as well in Taiwan), as well as the author’s own algorithm for finding underrated gems (and avoiding overrated restaurants) in London.

you also need the stuffed monkey

Mon, 23 Feb 2026 12:53:25 +0800

KPIs for digital products trend upwards and to the right when they provide us with the cloth mother to comfort our monkey brains. Despite the internet ridiculing Punch-kun’s use of a stuffed orangutan to comfort itself when being rejected by other macaques, this behaviour shows a socially positive way of dealing with loneliness and rejection.

about

Mon, 23 Feb 2026 00:00:00 +0000

it was the summer before my final year in university that a mate of mine introduced me to Ian Goodfellow’s “Explaining and Harnessing Adversarial Examples”. we were sitting in a Spoons on Southampton’s High Street when he showed me the infamous panda to gibbon via adversarial noise photo.

as my final year started, i urged my supervisor to allow me to pick my own subject to research for my dissertation, and so i wrote Outlier Detection using Generative Adversarial Networks, earning myself a Zepler Project Prize for best dissertation in my department that year.

AI Lent - Day 4 & 5

Sun, 22 Feb 2026 13:50:44 +0800

Weekend time, spent it reading and engaging with various communities on Mastodon.

Reading a lot more about the theoretical and machine learning part of training and deploying LLMs.

Metrics	Score	Note
Satisfaction
Mental fatigue / cognitive load
Number of PRs merged
Time to complete desired features
Confidence in released features

What to blog about

Sat, 21 Feb 2026 14:04:26 +0800

I’m treating this blog as future self-reference first. I consider keeping this digital garden well kept for myself will make it useful for others too.

PostgreSQL 18 RETURNING Enhancements: A Game Changer for Modern Applications

Sat, 21 Feb 2026 13:20:34 +0800

I make use of RETURNING heavily when writing Postgres-based worker queues. It allows me to claim a job and immediately scan the returned row into a struct, unmarshall the payload based on the operation type, and dispatch it without a second round-trip to the database.

Hold on to Your Hardware

Fri, 20 Feb 2026 20:10:14 +0800

ThePrimeagen gloated about buying Micron stock when he learned that AI is very memory hungry, I didn’t really have my finger on the pulse to that degree.

As someone that appreciates digital sovereignty, moving back to thin clients is something I fear. We can play around in the FOSS inflatable pool, but unless we manage to open source fabbing our own silicon, it’s nothing more than LARP-ing as tech autarkists.

How Generative and Agentic AI Shift Concern from Technical Debt to Cognitive Debt

Fri, 20 Feb 2026 17:58:51 +0800

My second day without AI, it took me 4 hours to write a simple Express API

I Sold Out for $20 a Month and All I Got Was This Perfectly Generated Terraform

Fri, 20 Feb 2026 14:37:21 +0800

Most software engineering involves plumbing together the same 4 SaaS tools that corporate approved and negotiated contracts with. Most software engineers are mercenaries, typing away for cash. Most of the frustration software engineers face in the industry comes from the deadly impression that they’re actually artists.

AI is not mid

Fri, 20 Feb 2026 14:03:08 +0800

Good balance between hype and doom in this article responding a NYT’s op-ed calling AI ‘mid’.

A piece stuck out to me is how the author describes AI changing the constraint landscape, especially around how we interact with information:

Transformer-based AI has fundamentally changed how we interact with information […]. Being able to skip, summarize and compare content - we’re no longer bound to consume information in the packages it was created.

AI Lent - Day 3 - Rediscovery

Fri, 20 Feb 2026 13:50:44 +0800

Rediscovering the beauty of computation primitives - loops, templates, etc.

I mainly did some edits to my dotfiles and this blog. Introduced the /links archetype (inspired by Simon Willison) and fixed some broken Tailwind CSS.

Metrics	Score	Note
Satisfaction	8
Mental fatigue / cognitive load	3	mostly templating work today
Number of PRs merged	0
Time to complete desired features	8
Confidence in released features	8

My approach to running a link blog

Thu, 19 Feb 2026 14:10:41 +0800

I find a lot of links, many of them end up in Linkhut, which is quite nice for social sharing, but I’d like to surface them here too.

AI Lent - Day 2 - Thawing

Thu, 19 Feb 2026 13:50:42 +0800

Today I wrote a small Express-based API.

When letting my mind flow freely, a lot of ideas bubbled up naturally, as opposed to forcing half-baked ones out through a plan mode session where I iterate with Claude.

I ended up not going the route of a managed API Gateway like Apigee, and rolling my own auth.

The hard part has been finding information. Websearch is terrible, I’m spending 5-10 seconds proving to Cloudflare that yes, I AM HUMAN.

AI Lent - Day 1 - Rusty

Wed, 18 Feb 2026 13:50:40 +0800

Cancelled Claude Max, uninstalled Claude Code, as I won’t be needing them for the next 40 days.

Wrote a small Nix package, noticed vim motions feeling rusty.

Also feels much harder to think how to structure a piece of code. Definitely feeling the atrophy

Metrics	Score	Note
Satisfaction	6	mostly just frustrated that everything feels hard again
Mental fatigue / cognitive load	8	learning to effectively search docs without an agent is hard
Number of PRs merged	0	Lunar New Year
Time to complete desired features	4	Took me 45 minutes to write a simple nix package
Confidence in released features	8	Pretty sure my nix package is correct

阿姨的蛋捲佈局

Mon, 16 Feb 2026 13:53:34 +0800

陪女友返鄉為了不要把手夾在屁股裡拜訪要買一些伴手禮。

到北車的時間已經有點趕，滿座的火車10分鐘以內就會離開月台。好啊買兩盒六月初一蛋捲絕對不會出錯。

我乖乖去排隊，前一位買9盒蛋捲的阿姨準備結帳。浪費的每一秒都感覺離錯過火車更近了一步。

同時，阿姨開始猶豫9盒蛋捲夠不夠…「再買3盒比較好，你們最受歡迎的口味是哪一個… 好那再3盒芝麻好了」。

店員：「現在買5送一，免費送你再兩盒，想要什麼口味呢」。

這句話使阿姨重新考慮她全部的蛋捲佈局，「啊啊啊那這6盒芝麻要換3個原味3盒巧克力」。

最後總共買了15盒。店員問她：「要不要買袋子」

「不要」

40 days of AI Lent

Sat, 14 Feb 2026 13:05:38 +0800

The 40 days of Lent are typically associated with the days Jesus spent fasting in the desert, resisting the temptation of Satan. It has become a religious tradition in Christian religions to also fast from Ash Wednesday to Easter.

I think it’s unfair to compare the Devil to AI, but I digress.

'My Child will X' meme with a take on AI

However, to keep in tune with the story of Lent, starting this Ash Wednesday (18th Feb 2026, coincidentally my 30th birthday), I will keep a 40 day AI Lent. No AI help, no Claude Code, Copilot, Google AI search.

I build my static Hugo site with Nix

Mon, 09 Feb 2026 13:38:28 +0800

I chose Hugo for setting up my static blog. I like using vim, and I want to just use the tools I already use for coding to write markdown blog posts, run git add . && git commit -m "new" && git push origin main and have a new article land on my blog. I want my repo to be my CMS, my declarative configuration of all Hugo parameters and plugins, as well as the declarative configuration of the build and deploy process.

Vim Be Fast

Mon, 09 Feb 2026 12:59:42 +0800

I like to be fast. I like to code fast. I like to press two keys and refactor the name of a function across an entire repository. I like Treesitter. I also like LSPs. And most importantly, I use vim btw.

As an engineer, one of your main jobs is keeping multiple concepts in your mind, letting them slosh around up there until eureka strikes, and ultimately, knowing precisely where to find or insert those nebulous concepts as code. Being able to locate and fire those fresh concepts with the precision of a AGM-114R9X Hellfire missile is a core capability of being an effective engineer. If people argue that engineers should spend time honing soft skills, I can also squeeze in my argument that there’s room for engineers to improve their mastery of their IDE/text editor¹.

Using microvm.nix to sandbox Openclaw

Fri, 06 Feb 2026 00:00:00 +0000

My mate organised a “setup Openclaw” hangout today. I’m not really thrilled about ~~Clawdbot~~, ~~Moltbot~~, Openclaw, but what the hell, my theme for 2026 is community, hanging out with friends is part of community, and I thought I could give myself a nice challenge here, so I joined, with the idea of setting up Openclaw my own way.

Since Sprites and exe.dev came on the scene, everyone’s been talking about stateful sandboxes for AI agents. I wanted to know what it takes to do lightweight virtualisation and sandboxing in the year of the lord 2026.

'The old order is not coming back,' Carney says in provocative speech at Davos

Wed, 21 Jan 2026 12:38:14 +0800

Mike Carney leading with Thucydides’ quote on “strong do what they can, the weak suffer what they must”. Thucydides does not offer the line, “The strong do what they can,” as a neutral analysis of how international affairs operate. He offers it as an expression of the reckless arrogance that brought about the destruction of the Athenian Empire.

Impressive seeing western world leaders challenging the “international rules-based order”, and embracing quite Third Worldist talking points related to “middle powers uniting”.

This website doesn't render U+1F1F9 U+1F1FC

Tue, 20 Jan 2026 12:19:43 +0800

When people talk about controversial emojis, they usually think of:

the finger-pinch emoji U+1F90F (🤏) that was at the center of anti-feminist conspiracy theories alleging it’s use for body-shaming men.
the water gun emoji U+1F52B (🔫) which replaced the revolver emoji following US school shootings
the pregnant man, pregnant person emoji U+1FAC3 U+1FAC4 (🫃 🫄) which are used to represent pregnancy in gender non-binary individuals

One controversial emoji that usually doesn’t get talked about enough is U+1F1F9 U+1F1FC. While the controversy around the emojis above is mainly driven by cuture war, the controversy around U+1F1F9 U+1F1FC stems from geopolitics, history and restistance. We could approach this controversy from a multitude of different angles, including denial of national identity, Sinocentrism or just plain bad takes.

Taming the Boox Go Color 7 Gen 2: Root, Debloat, and Firewall

Mon, 12 Jan 2026 00:00:00 +0000

Warning - Please read before proceeding

Following the steps outlined in this blog post might brick your device

My new Framework setup made me go touch grass more

Thu, 03 Jul 2025 00:00:00 +0000

When I bought my first Linux laptop, the Framework 13, I decided I wanted to do something different. Similar to how I wanted to forego prebuilt blog themes for my personal blog and instead let the style reflect my person, I also wanted to opt for a completely bare distro that would allow me to configure my laptop just how I saw fit. Nothing more, nothing less.

So I took the leap from the pristine, pre-configured, predictable (and pedantic) macOS experience, and embarked on a journey to a barren land where I could re-experience that feeling I had discovering Minecraft Creative Mode for the first time.

Bogdan's Taipei Spots

Wed, 02 Jul 2025 00:00:00 +0000

Updated frequently, keep checking…

父母

Beats, eats and seats. Mix between an American diner and a British pub.

Using non-Talos nodes with a Talos K8s Cluster

Sun, 09 Mar 2025 00:00:00 +0000

Talos Linux is an operating system meant to run one thing: Kubernetes. After using kubeadm on Debian, Talos is, in my opinion the quickest way to bootstrap a Kubernetes cluster on bare metal.

When running a Talos cluster however, you might want to join worker nodes that can’t run Talos, like when running a Kubelet node on Raspberry Pi 5 (at the time of writing, RPi 5 does not have uboot support required for booting into Talos).

Using Hetzner Object Storage as Terraform S3 backend

Thu, 27 Feb 2025 00:00:00 +0000

Hetzner’s Object Storage is an S3-compatible service, meaning it can be used for configuring Terraform backends with.

Here’s how you do that, assuming you’ve gone through the process of creating a bucket as well as generating S3 keys for it

backend "s3" {
 bucket = "your-bucket"
 endpoints = {
 s3 = "https://hel1.your-objectstorage.com" # Hetzner's endpoint
 }
 key = "your-project.tfstate" # Name of the tfstate file

 region = "main" # this is required, but will be skipped!
 skip_credentials_validation = true # this will skip AWS related validation
 skip_metadata_api_check = true
 skip_region_validation = true
 skip_requesting_account_id = true # skips checking STS 
 use_path_style = true # Ceph-S3 compatibility
 skip_s3_checksum = true # Ceph-S3 compatibility
}

Notes:

一位歌手如何揭發了羅馬尼亞反羅姆的歧視

Thu, 03 Oct 2024 00:00:00 +0000

2024年6月12日，酷玩樂團在布加勒斯特登台。欣喜若狂的人群最終以噓聲趕他們下舞台酷玩樂團的粉絲已習慣了樂團通常介紹當地的藝術家上台。布加勒斯特的演出也不例外，他們介紹的歌手是一個年輕的藝人叫「Babasha」。 22歲的Babasha是羅馬尼亞家喻戶曉的明星，他的單曲「Păi Naa」在羅馬尼亞目前的單曲日榜頂部，YouTube上的觀看次數達到 5,500 萬次。然而Babasha登台時，他竟然被噓了。據他說，這之所以是因為他是羅姆人。

Manele是一種與羅姆人息息相關的流派。該流派起源阿拉伯跟巴爾幹音樂。它的歌詞表達的感受小自戀愛，大至困難與歧視。如果你會參加一個羅馬尼亞的婚禮，你一定會聽到Manele活潑歡快的節奏。儘管（或是因此）它很受歡迎，Manele向來很分裂的。批評家抱怨Manele是個唯物主義、過度性化和地位低下的流派，而支持者說Manele受到的批評是從歧視來的。在2010年，Cluj-Napoca的市議會禁止了計程車司機聽Manele，還有在2023年，Timișoara的市長也禁止了Manele，甚至取消了另一位羅姆藝人的演出。

歷史上羅姆人被羅馬尼亞政治迫害，包括種族清洗，奴隸制跟私刑。作為是一種種族同質的國家，因此羅馬尼亞從未就種族主義進行對話。目前羅馬尼亞是歐盟最大經濟成長體之一。照理說隨者它的興盛，社會裡的歧視應該減少。但諷刺的是，羅馬尼亞也是歐洲人權法院收到種族歧視案件最多的國家之一。如今，羅姆人佔羅馬尼亞口的3.3%。他們仍然經常面對勞工歧視、被醫療體系拒絕、種族隔離、還有受到警察暴力。Manele的邊緣化也是這個反羅姆人情緒的延伸。

好的事是，隨著抖音與羅姆音樂的崛起，人們對反羅姆歧視的認識逐漸上升趨勢。值得讚揚的是，酷玩樂團在面對粉絲的歧視行為時並沒有退縮。Babasha在布加勒斯特與樂隊進行了第二次演出，Chris Martin告訴粉絲：「如果你不喜歡，就去廁所吧，或者做任何你想做的事，但要友善」

RAG systems have a recall problem, not a hallucination one

Wed, 02 Oct 2024 00:00:00 +0000

RAG or Retrieval-Augmented Generation is the de-facto standard for information retrieval in LLM-assisted applications such as chatbots and “AI” assistants. The TL;DR of RAG is that, by adding grounded context to an LLM generation, you can guide said LLM and significantly reduce the chance of hallucinations.

The issue, however, is that LLMs have finite context lengths, meaning that we need to be picky about what we in-scope as the context for the LLM generation.

Obsidian Templater Workflow for Chinese Language Learning

Sat, 07 Sep 2024 00:00:00 +0000

People have been raving about SRS (Spaced Repetition Systems) as a tool for language learning for ages. The gist is that, by spacing out flashcards for vocab, repeating the cards you get wrong more frequently, and the ones you get right exponentially less frequently, you can solidify that knowledge in your meat hard-disk.

If SRS is the goat of vocab acquisition, then Anki is the goat of SRS systems. The only issue being, to be able to use an SRS system, you need to have flashcards to review. And while solutions for clipping text, movies, etc. exist, this doesn’t fit the way I acquire vocab, which is through oral conversations, from class and other materials.

Semantic Router: GPT-4o API video sampling via semantic chunking

Tue, 14 May 2024 00:00:00 +0000

On May 13 2024, OpenAI released their newest multimodal model, GPT-4o. Based on an end-to-end multimodal architecture, this new model, dubbed “omni” is able to seamlessly handle text, visual and audio input in a single neural network.

While the OpenAI Demos showed GPT-4o being used with OpenAI native apps, users looking to use the multimodal capabilities via the API will be met with this OpenAI statement:

GPT-4o in the API supports understanding video (without audio) via vision capabilities. Specifically, videos need to be converted to frames (2-4 frames per second, either sampled uniformly or via a keyframe selection algorithm) to input into the model

Setup a Remote NVIDIA AI Workbench Node using EC2

Mon, 01 Apr 2024 00:00:00 +0000

Semantic Router: Postprocessing LLM output using Semantic Splitters

Mon, 18 Mar 2024 00:00:00 +0000

In the previous iteration of our Semantic Router series, we looked at how we can use Routes to steer decision making in AI applications by comparing the semantics of LLM input.

Today, we’re going to look at Semantic Splitters, a new mechanism that allows us to chunk large amounts of multimodal unstructured data based on its semantic encoding, but also postprocess LLM output without requiring extensive pattern matching or additional LLM calls.

Semantic Router: Steer local LLMs for decision-making and more

Tue, 16 Jan 2024 00:00:00 +0000

At Aurelio Labs, we’ve open sourced our Semantic Router tool, a superfast decision-making layer that allows the steering of LLMs based on the semantic meaning of input fed to them. And with the release of Semantic Router v0.0.16, we’ve enabled users to steer open source LLMs (like Mistral-7B-Instruct-v0.2) for chat, function calling and more.

I’d like to demonstrate how I use Semantic Router with Trending on Weibo to guide models such as Baichuan-7B and Mistral-7B to generate news articles with different tone of voice, depending on the article being reported on. We will only be relying on open source models, and consumer hardware (M1 Pro chip) to achieve this.

Formal Grammars for Large Language Models

Thu, 05 Oct 2023 00:00:00 +0000

Let’s jump in and explore a world where Large Language Models speak to us not in prose, but in JSON, XML, and where we can deterministically restrict their outputs so that they act according to the Unix philosophy:

Write programs that do one thing and do it well.
Write programs to work together.
Write programs to handle text streams, because that is a universal interface.

Why the Unix philosophy?

We have LLMs and LORAs that do one thing very well
We want to plug the output of one model into another (or loop it back into itself)
We can pipe the output of LLMs into non-LLM tools like sed, jq, for complex pipelines

Throughout this article, I will be using the new Mistral-7B-Instruct model, as it is small, fast and VERY capable.

Quick classification of Taiwan's Indigenous Weapons Programmes

Fri, 29 Sep 2023 00:00:00 +0000

This August, I had the privilege to join the first Ching Chuan Kang Air Base Annual Open Day after the COVID Pandemic. I’ve had an absolute blast looking at all the ELINT planes, ASW helicopters and fighters.

The most impressive part though, was that I would walk up to an exhibit, and upon reading the information card, I’d find out it’s actually indigenously produced.

So, with the unveiling of the Hai Kun-class Indigenous Defense Submarine, here’s a quick rundown of (some of) Taiwan’s current and future indigenous defense programmes.

Deploying HuggingFace models on NVIDIA-enabled EKS nodes

Thu, 31 Aug 2023 00:00:00 +0000

Installing NVIDIA drivers is pain. Pure pain. Then throw in Auto-Scaling Groups, MIME-type multipart uploads for bootstrap scripts and NVIDIA specific containerd configuration and you might as well call it a day and sign up for Replicate or HuggingFace’s Inference endpoints and pay a hefty premium.

But first, why?

Setting aside cost savings on on-demand instances (AWS A10G instances are 30% cheaper than the same instance on HuggingFace), even bigger cost savings leveraging Spot instances in your K8s node pools, even bigger savings on S3 egress costs (if you roll your own LLMs).

Dagster ❤️ Llama - orchestration for modern LLM pipelines

Wed, 19 Jul 2023 00:00:00 +0000

Creating pipelines of LLM-based transformations is a standard in increasingly many data pipelines. Given the rise of tools such as LangChain, you can really squeeze more out of your LLM than just prompt engineering endlessly.

And while I have my own beef with LangChain (and I’m not the only one), I wanted to write a blog to detail how you can use industry standard, tried-and-tested Data Engineering tools to achieve a robust LLM pipeline you can trust, inspect and rely on.

How to delete all of your Tweets - using Go

Thu, 02 Feb 2023 00:00:00 +0000

"Software needs to be built such that it will allow us to forget." - Alex Schroeder

Taiwan or Thailand? A primer to gaffes, misinformation and warfare

Sat, 28 Jan 2023 00:00:00 +0000

On New Year’s Eve 2022, CNN ran a segment on how different Asian cities celebrate the occasion. A technical gaffe caused a wider outrage on social media, and became ammunition for the more hawkish voices reporting on China.

Correspondents Will Ripley, based out of Taipei, and Kristie Lu Stou, out of Hong Kong, were reporting on the passage into the New Year, as celebrated in Asia.

Keeping in sync with the different timezones starting from GMT+9 (Japan) and going backwards, Will and Kristie aimed to report on how different cities in Asia celebrate New Year’s Eve, just as they crossed the 00:00 mark into the new year.

Outlier Detection using Generative Adversarial Networks

Tue, 01 May 2018 00:00:00 +0000

Outlier detection techniques are widely used for solving tasks such as fraud detection, anomalous tissue detection in medical imaging data, or improving voice recognition models. By cleaning training datasets of outlying data points, unsupervised machine learning algorithms also benefit from a significant performance increase.

This report investigates the Generative Adversarial Network model, a technique for training generative models to learn the underlying probability distribution of an arbitrary training dataset through discirminative means. The work carried on in this project shows how the latent space of a Generative Adversarial Network model can be exploited for the task of outlier detection in Computer Vision tasks, as well as providing an extension that allows a Generative Adversarial Network to detect outliers in generic, high-dimensional datasets.

Alai Spark