Five self-contained benchmarks that show how CPU cache behavior affects real program performance, and how to fix each problem. Every example includes before/after code and measured results on an Apple M4 Pro (arm64).
Modern CPUs never fetch a single byte from RAM. They fetch 64 bytes at a time — a cache line — and store it in L1/L2/L3 cache. Subsequent reads to any byte within that 64-byte block are served from cache at ~1–4 ns. Reads that miss all cache levels go to DRAM at ~80–150 ns — a 20–40× penalty.
┌─────────────────────────────────────────────────────────────────────┐
│ Memory hierarchy on Apple M4 Pro │
│ │
│ CPU Core │
│ ┌─────────────────┐ │
│ │ Registers │ < 1 ns │
│ ├─────────────────┤ │
│ │ L1 cache 192KB │ ~1–2 ns │
│ ├─────────────────┤ │
│ │ L2 cache 16 MB │ ~4–8 ns ← working set target │
│ └────────┬────────┘ │
│ │ shared │
│ ┌────────┴────────┐ │
│ │ L3 cache 24 MB │ ~20–40 ns │
│ └────────┬────────┘ │
│ │ │
│ ┌────────┴────────┐ │
│ │ DRAM (unified) │ ~80–150 ns ← avoid for hot paths │
│ └─────────────────┘ │
│ │
│ Cache line size: 64 bytes (universal on x86-64 and arm64) │
└─────────────────────────────────────────────────────────────────────┘
The five examples each exercise a different failure mode. Run all of them with:
go test ./... -bench=. -benchtime=3sDirectory: 01_false_sharing/
Two goroutines each write their own independent counter, but both counters live inside the same 64-byte cache line. To the CPU's cache coherence protocol, a write to any byte in a line marks the entire line as modified, invalidating all copies held by other cores. The other core must fetch a fresh copy before it can write its own byte — even though the two writes never touch the same bytes.
NaiveCounters in memory (16 bytes total):
┌────────────────────────────────────────────────────────────────────┐
│ ◄──────────────────── 64-byte cache line ───────────────────────► │
│ a (int64, 8 bytes) │ b (int64, 8 bytes) │ (unused 48 bytes) │
└────────────────────────────────────────────────────────────────────┘
goroutine 1 writes ▲ goroutine 2 writes ▲
they share one cache line → coherence storm
Every write by goroutine 1 forces goroutine 2's core to invalidate its copy, and vice versa. This creates a continuous round-trip of cache-line ownership between cores (~100 ns each), making what looks like two independent operations into a de facto mutex.
type NaiveCounters struct {
a int64 // offset 0 ─┐
b int64 // offset 8 ─┘ same 64-byte cache line
}Pad each field so it occupies its own cache line. Writes to a and writes to b now touch completely separate lines — zero coherence traffic.
const cacheLineSize = 64
type PaddedCounters struct {
a int64
_ [cacheLineSize - 8]byte // 56 bytes: fills the rest of this cache line
b int64
_ [cacheLineSize - 8]byte // 56 bytes: fills the rest of this cache line
}PaddedCounters in memory (128 bytes total):
┌────────────────────────────────────────────────────────────────────┐
│ ◄────────── cache line 0 ─────────────────────────────────────► │
│ a (8 bytes) │ padding (56 bytes) │
└────────────────────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────────────────────────┐
│ ◄────────── cache line 1 ─────────────────────────────────────► │
│ b (8 bytes) │ padding (56 bytes) │
└────────────────────────────────────────────────────────────────────┘
goroutine 1 ▲ goroutine 2 ▲
each goroutine owns a private cache line → no contention
BenchmarkFalseSharing 156 ms/op ← cache line bounces between cores
BenchmarkPaddedCounters 50 ms/op ← each goroutine owns its cache line
Improvement: 3.1×
Each c.a++ in the naive version secretly costs ~100 ns of cache coherence overhead. After padding, both goroutines run in parallel with zero interference.
Any struct with multiple fields written concurrently by different goroutines. Common examples: per-shard statistics, worker-local accumulators, ring buffer head/tail pointers.
Directory: 02_struct_layout/
The Go compiler inserts invisible padding bytes between struct fields to keep each field aligned to its natural alignment boundary (an int64 must start at a multiple of 8, an int32 at a multiple of 4, etc.). When fields are ordered carelessly, padding holes inflate the struct size, reducing how many structs fit in a cache line.
// Bad: bool first forces 7 bytes of padding before int64
type BadLayout struct {
flag bool // 1 byte
// ← 7 bytes padding (to align int64 to 8-byte boundary)
value int64 // 8 bytes
count int32 // 4 bytes
// ← 4 bytes padding (to align float64 to 8-byte boundary)
ratio float64 // 8 bytes
} // total: 32 bytes (11 bytes wasted)BadLayout in memory (32 bytes):
┌──┬───────┬────────┬────┬────┬────────┐
│ f│ pad×7 │ value │cnt │pad4│ ratio │
│1B│ 7B │ 8B │ 4B │ 4B │ 8B │
└──┴───────┴────────┴────┴────┴────────┘
←─── 32 bytes ───────────────────────→
Only 2 structs fit in a 64-byte cache line.
Order fields largest to smallest by size. The compiler still inserts trailing padding to keep the total size a multiple of the largest alignment, but eliminates all internal holes.
// Good: largest to smallest — no internal padding
type GoodLayout struct {
value int64 // 8 bytes
ratio float64 // 8 bytes
count int32 // 4 bytes
flag bool // 1 byte
// ← 3 bytes trailing padding (harmless)
} // total: 24 bytesGoodLayout in memory (24 bytes):
┌────────┬────────┬────┬─┬───┐
│ value │ ratio │cnt │f│pad│
│ 8B │ 8B │ 4B │1│ 3B│
└────────┴────────┴────┴─┴───┘
←── 24 bytes ────────────────→
2+ structs fit in a 64-byte cache line.
Size comparison at runtime:
BadLayout size: 32 bytes (wastes 8 bytes of padding)
GoodLayout size: 24 bytes
BenchmarkBadLayout 3.86 ms/op
BenchmarkGoodLayout 3.86 ms/op ← same speed for sequential SIMD loops
The throughput difference is negligible here because: (a) the loop is auto-vectorized — the compiler treats it as a bulk SIMD operation where the bottleneck is compute throughput, not individual struct size; (b) M4's prefetcher fully saturates memory bandwidth for sequential scans.
The benefit is real but shows up differently:
- Capacity: with 10 M elements,
BadLayoutallocates 320 MB vsGoodLayout's 240 MB — 33% more memory, 33% more cache pressure in pointer-chasing workloads. - Cache utilization: a pointer-heavy graph of
GoodLayoutnodes fits more nodes per cache line, reducing miss rate. - Measurable on x86: server CPUs with narrower OOO windows and lower memory bandwidth show 15–25% throughput differences on this pattern.
go vet -fieldalignment ./...
# or
fieldalignment -fix ./... # golang.org/x/tools/go/analysis/passes/fieldalignmentDirectory: 03_aos_vs_soa/
When a hot loop reads only one field from a large struct, every cache-line fetch drags in all the other fields — wasted bandwidth that crowds out useful data.
Array of Structs (AoS) — each entity is 64 bytes (one full cache line):
Entity[0] Entity[1]
┌────────┬─────────────────────────────────────────────┐
│ PosX │ PosY VelX VelY Health pad[24] │
│ 8B │ 56 bytes of dead weight │
└────────┴─────────────────────────────────────────────┘
↑ used ↑ fetched from DRAM, not used
Loop reads PosX only → 8 of 64 bytes per cache line are useful = 12.5% efficiency
With pointer chasing (random walk through the entity array), every hop is a DRAM miss loading 64 bytes and using only 8. Working set: 1M × 64 bytes = 64 MB.
Separate fields into their own contiguous arrays. The hot loop only touches the slice it needs; cold slices never enter cache.
Struct of Arrays (SoA):
PosX[] [f f f f f f f f] ← 8 float64s per cache line, all used
PosY[] [f f f f f f f f] ← untouched when loop only reads PosX
VelX[] [f f f f f f f f]
...
Working set for PosX loop: 1M × 8 bytes = 8 MB → fits in L3 cache
Cache line efficiency: 64/64 = 100%
A simple for i := range entities loop doesn't show the difference on modern CPUs: the hardware prefetcher detects the stride-1 pattern and pre-fetches lines before the code asks for them. Cache-miss latency is hidden entirely.
Pointer chasing (next = nodes[next].next) creates a data-dependent load: the address of the next memory access is stored inside the current one. The CPU cannot know the next address until the current load completes. Every hop serializes:
hop 1: load nodes[0] → wait for DRAM (~100 ns) → get next=583421
hop 2: load nodes[583421] → wait for DRAM (~100 ns) → get next=...
hop 3: ...
No speculation, no prefetching. Cache miss latency is fully exposed.
BenchmarkFat_PointerChase 120 ms/op ← 64-byte nodes, 64 MB working set (DRAM)
BenchmarkCompact_PointerChase 13 ms/op ← 4-byte indices, 4 MB working set (L3)
Improvement: 9.2×
Per-hop latency: Fat ≈ 120 ns, Compact ≈ 13 ns
The compact version fits in L3 cache; the fat version spills to DRAM. The 9× gap is the real-world L3→DRAM latency ratio.
- Game entity systems: position/velocity updated every frame, health/inventory accessed rarely — split into position SoA + attribute AoS.
- Database row stores vs column stores: queries over one column benefit from columnar layout.
- Any struct where hot loops touch a strict subset of fields.
Directory: 04_sequential_vs_random/
The gap between cache levels is not gradual — it is a cliff. Crossing from L2 into L3 costs 5–10×; crossing from L3 into DRAM costs another 3–5×. A data structure that just barely exceeds a cache boundary can be orders of magnitude slower than one that just fits.
A common misconception is that data[randomIndex[i]] is slower than data[i]. On modern out-of-order CPUs it often isn't, because randomIndex is read sequentially — the prefetcher handles it — and the CPU's OOO window can issue 100+ speculative loads into data simultaneously, hiding their latency behind each other.
This looks random but is actually parallel-load:
for _, idx := range perm { // perm is sequential → prefetched
sum += data[idx] // 100+ of these fire in parallel in the OOO window
}
True sequential dependency requires pointer chasing:
cur = chain[cur] // cannot fire until cur is known, which requires the previous loadBoth benchmarks perform 1,000,000 pointer-chasing hops. The only difference is the size of the chain (its working set):
| Benchmark | Chain size | Working set | Fits in |
|---|---|---|---|
BenchmarkChase_L2 |
32,768 entries | 128 KB | L2 cache |
BenchmarkChase_DRAM |
16,000,000 entries | 64 MB | DRAM |
BenchmarkChase_L2 4.6 ms/op → 4.6 ns/hop (L2 cache hit)
BenchmarkChase_DRAM 99.8 ms/op → 99.8 ns/hop (DRAM miss)
Ratio: 21.7× slower
Latency by level (measured):
L2 ████ 4.6 ns
│
│ (5–10× cliff)
│
L3 ██████████████████████████ ~20–40 ns (estimated)
│
│ (3–5× cliff)
│
DRAM ████████████████████████████████████████████████████████ ~100 ns
A data structure's hot working set (the bytes touched in the critical loop) must fit in L2 to avoid the cliff. Measure with perf stat -e cache-misses (Linux) or Instruments (macOS) before optimizing.
Directory: 05_sharded_counter/
sync/atomic operations are often described as "lock-free" — but they still serialize at the cache level. atomic.AddInt64 issues a read-modify-write instruction that acquires exclusive ownership of the cache line for the duration of the operation. When multiple cores issue this instruction simultaneously, ownership bounces between them:
4 goroutines, 1 atomic counter:
Core 0: atomic.Add → acquire line → +1 → release → wait...
Core 1: waiting... → acquire line → +1 → release → wait...
Core 2: waiting... → acquire → +1 → ...
Core 3: waiting...
The effective throughput is one increment per ~100 ns cache-line round-trip, regardless of how many cores are trying. Apparent concurrency is actually serialized at the hardware level.
Allocate one int64 per CPU, each on its own cache line. Each core increments only its own shard — zero cross-core contention. Reads sum all shards.
const cacheLinePad = 64 - unsafe.Sizeof(int64(0)) // = 56
type paddedCell struct {
val int64
_ [cacheLinePad]byte // isolates val on its own cache line
}
type ShardedCounter struct {
cells []paddedCell // one per GOMAXPROCS
}Memory layout (3 CPUs):
cell[0]: [val int64 | padding 56B] ← cache line 0, owned by CPU 0
cell[1]: [val int64 | padding 56B] ← cache line 1, owned by CPU 1
cell[2]: [val int64 | padding 56B] ← cache line 2, owned by CPU 2
The simplified version still has contention when goroutines outnumber CPUs or migrate between CPUs. Using sync.Pool assigns each goroutine a private cell from the pool. The Go runtime maps pool entries to OS threads (Ps), so contention is minimal even under high goroutine counts.
type PoolCounter struct {
mu sync.Mutex
cells []*paddedCell
pool sync.Pool
}
func (c *PoolCounter) Inc() {
cell := c.pool.Get().(*paddedCell) // get a private cell
atomic.AddInt64(&cell.val, 1) // no contention: this cell is ours
c.pool.Put(cell) // return it
}
func (c *PoolCounter) Get() int64 {
// sum all cells — O(GOMAXPROCS) but reads are rare
var total int64
for _, cell := range c.cells {
total += atomic.LoadInt64(&cell.val)
}
return total
}BenchmarkAtomicCounter_4G 76 ms/op ← 4 goroutines hammer one cache line
BenchmarkShardedCounter_4G 147 ms/op ← simplified: still contends on cells[0]
BenchmarkPoolCounter_4G 11 ms/op ← pool assigns private cells
Pool improvement over atomic: 6.7×
The naive ShardedCounter is slower than even the single atomic because the implementation still writes to cells[0] from all goroutines — the struct has padding but the index doesn't rotate. This is intentional: it shows that padding alone is not enough; you must also ensure goroutines don't share a cell.
This pattern is used by:
- prometheus/client_golang:
nativeHistogramshards across CPUs - go-metrics: per-P counters with padding
- sync.Pool itself: internally shards by P to avoid cross-core contention
# All benchmarks
go test ./... -bench=. -benchtime=3s
# One package
go test ./01_false_sharing/... -bench=. -benchtime=3s -v
# With memory stats
go test ./... -bench=. -benchmem
# Compare before/after with benchstat
go test ./01_false_sharing/... -bench=. -count=5 > before.txt
# (make a change)
go test ./01_false_sharing/... -bench=. -count=5 > after.txt
benchstat before.txt after.txt| Example | Before | After | Speedup |
|---|---|---|---|
| False sharing (2 goroutines, 100M iters each) | 157 ms | 50 ms | 3.1× |
| Struct layout (10M elements, sequential scan) | 3.86 ms | 3.86 ms | — (size: 32B→24B) |
| AoS vs SoA (1M pointer-chase hops, 64B vs 4B nodes) | 120 ms | 13 ms | 9.2× |
| Cache latency cliff (1M hops, DRAM vs L2) | 100 ms | 4.6 ms | 21.7× |
| Sharded counter (4G, atomic vs pool) | 76 ms | 11 ms | 6.7× |
Cache line = 64 bytes — the atomic unit of memory transfer.
Problem Root cause Fix
────────────────────── ──────────────────────────────── ───────────────────────────
False sharing Two goroutines write different Pad each hot field to its
bytes in the same cache line own 64-byte cache line
Padding waste Compiler inserts holes between Order fields largest →
misaligned fields, inflating smallest; use go vet
struct size -fieldalignment
AoS cache waste Fat structs drag cold fields Split hot fields into
into cache on every miss separate slices (SoA)
Latency cliff Working set overflows a cache Profile working set size;
level, each hop hits DRAM keep hot path in L2
Atomic contention All cores hammer the same Shard counter per CPU/
cache line with RMW goroutine, pad each shard
Latency reference:
Register < 1 ns
L1 cache ~1–2 ns
L2 cache ~4–8 ns ← design hot loops to fit here
L3 cache ~20–40 ns
DRAM ~80–150 ns ← a 20–40× penalty vs L2
The biggest wins come from reducing the working set of your hot path until it fits in L2 — not from micro-optimizing the code itself.