Read more

That stopped me, because it happened in the middle of a benchmark, and the refusal turned out to be the most cautious thing Fable did all day. On every task it did answer, it wrote some of the least memory-safe C++ in the test: three times the bounds violations of its own sibling Opus 4.8, down at the bottom of the chart next to GPT-5. The one model paranoid enough to treat a fast buffer-fill as an attack is also the one whose output you would least want anywhere near a buffer. The safety filter and the code generator are not talking to each other.

Full Article: https://hftuniversity.com/post/one-sentence-doubles-the-safety-violations-in-ai-generated-c

LLDB hasn’t just caught up with GDB. It has lapped it. And the gap is widening every year. The architecture is superior, the expression evaluator is in a different league, and the development velocity makes GDB look like a maintenance-mode project. Apple, Google, Microsoft, and FreeBSD have all standardized on LLDB. The industry has voted. The holdouts are running out of excuses.

Let me show you why.

Full article: https://hftuniversity.com/post/why-the-llvm-debugger-has-already

Cloned the repo. Tried to build. Hit a few glitches — nothing unusual, most GitHub projects don’t compile on the first try. But as I dug in, it became clear that this project needed the latest CMake, the latest compiler, the latest standard, the latest everything. I think it even required some new feature from the latest bash. After upgrading to Clang 14.0.6, CMake 3.24, and satisfying every standard requirement I could find, I still couldn’t compile it. At that point I was patching the actual source code trying to make it build, so I gave up. I gave it an honest shot and I failed — and I am not exactly a rookie in building systems.

The question that stuck with me: why would someone squeeze their public project into such a tiny compatibility corner that even an experienced systems engineer can’t build it after hours of trying? How many nanoseconds is the latest compiler saving over the well-supported Clang that ships with Ubuntu LTS? How much more readable is their code in C++23 versus C++17? Are concepts really going to make that code better enough to justify being unbuildable by 90% of potential users?

Full article: https://hftuniversity.com/post/latest-standards-no-thank-you

Article: https://hftuniversity.com/post/the-c-documentary-won-t-show-you-a-number-i-will

Anyone trying to apply the original post’s lesson to their own code needs to know that. If the bottleneck is the predicate’s branch unpredictability, AVX2 with a LUT gets you most of the way there, no AVX-512 required. If the bottleneck is specifically vpcompressd m, k, zmm on Zen 4, the fix in the post is the right one and you should use it. If you’re on Zen 5, the question doesn’t arise; pick either form and move on. If you’re calling std::copy_if directly and expecting any of this to apply, you’re in a different conversation, because the standard library hasn’t shipped any of it.

Full article:

https://hftuniversity.com/post/the-copy-if-speedup-that-wasn-t-about-copy-if-or-avx-512

A standard library API written in 2003, ratified in 2011, and frozen by ABI in roughly 2013 cannot reflect the cache hierarchy that the field standardized on by 2017. The hardware moves faster than the standard, the standard moves faster than the implementations, and the implementations move faster than the deployed binaries. By the time the consensus shifts, the deployed-binary layer is the binding constraint, and the committee cannot fix the design without breaking the linker contract that holds the ecosystem together. The Prague 2020 vote did not create this situation; it ratified the situation that already existed.

Full article: https://hftuniversity.com/post/all-further-discussion-will-assume-chaining-how-a-2003-sentence-locked-c-out-of-fast-hash-maps

The methodology is sound and the numbers are real. But the headline noun is wrong. That benchmark holds one static branch PC and stretches its outcome history. It measures per-PC history depth, not “how many branches a CPU can predict.”

The number of distinct branch sites, K, is pinned at 1. Nobody asked what happens on the rest of the (K, M) plane.

So I ran the whole surface on my benchmark box, isolated cores, eleven runs per cell, in-process mispredict counter for the verify gate, predictions written down before the first measurement.

Two regimes show up, split around K=16 to 64: below it you are bounded by per-PC history depth at roughly 1,500, above it total capacity grows almost linearly with K.

The punchline is that the same Zen 2 silicon gets you 49% mispredicts at K=1, M=30,000, and 0.2% at K=1,024, M=32 with more total branch instances in flight. One sentence of “Processor X predicts N branches” hides about 1.5 orders of magnitude, and it may quietly reorder the vendor ranking too.

Article: https://hftuniversity.com/post/the-lemire-branch-predictor-benchmark-is-k-1-here-is-the-k-m-surface

#cpp #cplusplus #programming

Full article: https://hftuniversity.com/post/the-lemire-branch-predictor-benchmark-is-k-1-here-is-the-k-m-surface

The complaint that POSIX returns EINTR to userspace is, in this telling, a complaint that the world is not the world the textbook described. The right answer is not to redesign the world. The right answer is to update the textbook.

We have been having this argument since 1988. Dijkstra has been dead for twenty-three years, and the “doomed discipline” he tried to indict has shipped every kernel that matters.

Full article: https://hftuniversity.com/post/there-is-a-mountain-lion-in-the-datacenter

https://hftuniversity.com/post/two-kinds-of-knowledge

In both cases, the single-cause story buries the real one. Troy fell at the chokepoint of the Dardanelles during a Late Bronze Age systems collapse, not because of one runaway princess. C++ won because of C ABI compatibility, the Unix distribution channel, the AI winter, the collapse of the Lisp machine market, and a zero-overhead cost model that no GC language could match in 1985.

I wrote up the full deconstruction, including timestamped quotes from the video, the four essays where Gabriel himself walked back his own thesis, and the stacked fallacies (mono-causal, false dichotomy, narrative, appeal to authority by anecdote) that make this kind of content stick when it should not.

https://hftuniversity.com/post/blame-helen-of-troy-blame-new-jersey-how-one-youtuber-recycled-a-3-000-year-old-fallacy-to-explain-c

With the explosive increase in market data volume driven by algorithmic trading over the past 20 years, compression has become essential infrastructure, not a nice-to-have, but a cost driver. A single day of full-depth CME capture can run into hundreds of gigabytes. Multiply that by the number of exchanges a global firm monitors, multiply again by the years of history your quant team needs for backtesting, and you’re staring at petabytes. The difference between a 4:1 and a 7:1 compression ratio is the difference between a manageable storage bill and a painful one.

Compression formats and their associated software have become critical tools for trading firms, and yet the choice is almost always made on vibes rather than data. “We’ve always used gzip” is the most common justification I hear. This study provides actual numbers, first for the default presets of the four classic Unix tools, and then for the modern toggles (zstd’s --long, --patch-from, Intel’s ISA-L igzip, and LZ4) that reshape the conclusions in places.

Full article:

https://hftuniversity.com/post/raw-data-compression-of-cme-globex

There is a reason the modern C++ codebase that ships in any tier-one trading firm, search engine, or browser is built mostly out of non-standard libraries. Boost, abseil, Folly, EASTL, Chromium’s //base, hand-rolled containers, custom allocators, Hana’s CTRE, Niall Douglas’s Outcome, Lewis Baker’s coroutine library. The standard library is the thing you avoid as much as you use. Every senior engineer learns this. The juniors find out the first time they reach for std::list in a code review and the senior gently asks them to use std::vector instead.

Dargo’s line is exactly right. std::function is legacy. Avoid it in new code. The point of this article is that the sentence is fifteen years old and counting, that the C++ standard library has been making it true about its own contents for the entire post-C++11 era, and that the next time the committee ships a shiny new replacement for something it shipped a decade ago, the honest read of the announcement is that the older design did not work and now there is a paper saying so. The new feature is welcome. The pattern is what you should learn to read.

Full article: https://hftuniversity.com/post/the-c-standard-library-has-been-walking-itself-back-for-fifteen-years-and-the-receipts-are-public

Read more

Read more

Read more

Read more

Read more

And HFT developers? We shrugged.

Not because PREEMPT_RT is bad. It’s excellent for what it does. But if you’re running a trading system, the kernel is not your bottleneck. Your data structures are.

Before I show you the benchmarks that prove this, you need to understand what PREEMPT_RT actually does and why it took two decades to get here.

The Twenty-Year Journey

Back in 2004, real-time Linux was a mess. Several groups were working on incompatible approaches to the same problem: how do you make Linux guarantee response times? The efforts were uncoordinated, the patches incomplete, the mailing list discussions legendary in their toxicity.

Then Ingo Molnar did something unexpected. The RedHat kernel scheduler maintainer took pieces from various patch sets, rewrote most of them from scratch, and combined them into something coherent. Thomas Gleixner, who had been working with Dr. Doug Niehaus from Kansas University on the KURT real-time extension, joined forces with Molnar. Steven Rostedt (of ftrace fame) became another major contributor. Within months, they had a “halfway usable Real-Time solution.”

The goal was never “as fast as possible.” As Gleixner puts it: “Real-Time is not as fast as possible; it is as fast as specified.” The difference matters. Real-time means guaranteed completion within defined timeframes. Miss that deadline in a car’s braking system or a medical device, and people die.

The companies who funded this work read like a who’s-who of embedded systems: ARM, BMW, Intel, National Instruments, Texas Instruments, RedHat. Former supporters included Google, IBM, and NXP. Gleixner has been candid about the funding model resembling Jenga: “companies build key products depending on these technologies...seem to take the availability...for granted.”

Why twenty years to merge? Not because the code didn’t work. Because the changes touch core kernel infrastructure. Spinlocks. Interrupt handling. Scheduler internals. Every patch had to be designed so it “does not get in the way of other interests.” Incremental improvements went in quickly. Foundational changes required years of polishing and community alignment.

Why Normal Kernels Have Unbounded Latency

To understand what PREEMPT_RT fixes, you need to understand why standard Linux has terrible worst-case latency.

The latency formula is deceptively simple:

Each term can explode. Here’s why.

Spinlocks disable preemption. When kernel code acquires a spinlock, it disables interrupts on that CPU. Any pending interrupt, including your trading thread’s timer, waits until the spinlock is released. The problem: critical section duration is unbounded. Some driver holding a spinlock for 500 microseconds delays everything behind it.

Interrupt handlers run with interrupts disabled. Linux splits interrupt handling into two parts. The top-half runs immediately when hardware triggers an interrupt, with that interrupt line masked so nothing else can fire on it. The bottom-half (softirq, tasklet, or workqueue) runs later to do the heavy lifting. The problem is the top-half: if it takes 200 microseconds, nothing else on that interrupt line runs for 200 microseconds. Priority doesn’t matter. Your SCHED_FIFO thread with priority 99 waits for a USB driver’s interrupt handler to finish its top-half.

The kernel is not preemptible by default. When your high-priority thread wakes up, it can’t run until the current kernel code path reaches a voluntary preemption point. System calls run to completion. A memcpy() of 10MB in kernel space? Your thread waits.

Priority inversion destroys guarantees. Thread A (high priority) needs a lock. Thread B (low priority) holds it. Thread C (medium priority) preempts B. Now A is blocked behind C, even though A has higher priority. Without priority inheritance, your carefully tuned priorities are meaningless.

The result: a standard kernel might respond in 5 microseconds under no load, then spike to 500+ microseconds when a USB device enumerates or a filesystem flushes. You cannot bound the worst case.

What PREEMPT_RT Actually Does

PREEMPT_RT attacks each source of unbounded latency.

Spinlocks become sleeping locks. In a PREEMPT_RT kernel, spinlock_t doesn’t spin at all. It’s implemented on top of rt_mutex, which means a task waiting for a “spinlock” goes to sleep instead of burning CPU cycles. When the lock is released, the waiter wakes up. Preemption is not disabled while holding these locks.

Interrupt handlers become kernel threads. Almost all interrupt handlers in PREEMPT_RT run in process context, not hardware interrupt context. They’re scheduled like any other thread, with priorities you control. A USB driver’s interrupt handler can’t preempt your trading thread if your thread has higher priority.

Read more