Houses

• Ptmalloc2/glibc malloc is the default heap allocator for many GNU/Linux distributions. Stack exploitation has largely been mitigated by protections, e.g., ASLR and canaries. Heap corruption plays a central role in exploitation across modern native software.
• The CTF community has produced a large, high-quality corpus of heap-exploitation challenges driven by the competition selection, where the technique is most developed. It carries a dedicated vocabulary (heap feng shui) and a taxonomy of version-specific methods named House of Orange, House of Apple, and so on, which is where this benchmark gets its name. Allocator internals are documented exhaustively: unlinking during consolidation, _IO_FILE/vtable hijacking (FSOP) for control-flow capture.
• glibc heap exploitation is standardized and highly observable: allocator state, memory contents, and kernel activity can all be instrumented (e.g., via gdb). That is what lets us build a deterministic, dense reward that CVEs can't.
• The skill transfers: heap-exploitation competence generalizes to exploit development against harder targets and custom allocators like V8/Oilpan and jemalloc.

Most exploit-development RL environment grades out-of-band, and the problem sharpens the shorter the task gets. Rewarding at primitive granularity cannot credit the situational links real chains are made of. E.g., an unlink or a partial overwrite scores nothing, because it's not on the list. Worse, it perturbs the rollout: the policy stops mid-way to satisfy the oracle instead of going for the flag, and the trajectory it leaves behind is contaminated with work that was never about exploitation.

We learned this the hard way. First ladder of houses was fourteen rungs, L0 to L4, named after the techniques: leak_libc, write_what_where, pc_control, overlapping_chunk. An LD_PRELOAD shim watched allocations, a signal handler watched faults, a socket tee watched I/O, and a warden negotiated a target address with the agent so we could tell a real primitive from a lucky crash. It broke on both axes at once, and those two axes frame everything below.

Semantically it was out-of-band. Playbook rungs mean we were grading imitation of known writeups. And negotiating a target address is worse than it sounds: it tells the agent the ladder exists, the reward becomes a task specification, and we watched a negotiated target hand out leak_libc for free.

Mechanically it perturbed. Every LD_PRELOAD, even a no-op DSO, adds a mapping and shifts ld.so, the stack and environ relative to each other. Our own shim's mmap footprint made the mmap-adjacency houses unexploitable. We were losing challenges to our own instrument. No writeup exploit could light the ladder without first being rewritten to take a --targets flag.

That last failure became the test we now hold ourselves to: an unmodified reference exploit has to light the ladder end to end. If the exploit has to know the oracle exists, the oracle is out-of-band.

The move that made the rest possible is that we did not have to invent a vocabulary, because glibc already wrote one. do_check_chunk, do_check_free_chunk, do_check_inuse_chunk, do_check_malloc_state: a full arena consistency walk, all #defined to nothing unless you build with MALLOC_DEBUG, sitting beside the ~50 malloc_printerr sites cheap enough to survive the performance argument. The allocator authors already wrote down, in code, a model of what must be true, and every heap technique anyone has named is a path through violations of that model. So we hoist it out of the process where the attacker cannot corrupt it, complete it, and extend it with an external ledger that can decide what the in-process checker structurally cannot, such as whether the pointer malloc just returned sits inside a live object.

Two ledgers live outside the guest. L_API is intent: (ptr, req, usable) recorded at every allocator API return and removed on free. It holds what the program believes it owns, untouchable because it is not in the guest. L_MEM is belief: chunk headers, tcache_perthread_struct, fastbinsY, bins[], top and secondary arenas parsed from the real in-guest structures with safe-linking decoded. It holds what the allocator believes. A benign run keeps them in bisimulation, and this is where the design falls out: every exploitation step is by construction an induced divergence, because lying to the allocator about memory the program does not own is the attacker's only leverage. The rung set falls out of that as a classification of divergences, which is why no technique name appears anywhere in it.

We check at malloc and free entry and exit, the same points glibc calls check_malloc_state, because ptmalloc's invariants are only required to hold at API quiescence. That is exactly where the contract applies, and it costs O(live chunks + bin entries) per call. Observation must not change what is observed, since heap layout, address-space layout and fault semantics are the object of the exploit, which disqualifies essentially the whole sanitiser literature. We use a ptrace observer breakpointing libc code, hardware watchpoints as a derived fast path, and a QEMU/TCG plugin that sees every load, store and syscall as a second, mechanically unrelated arm. With ASLR pinned the service's entire output byte stream is byte-identical monitored and unmonitored, including on challenges whose output carries raw heap and libc pointers, and the two arms independently agree on the per-allocation chunk sequence.

The rungs form a lattice on three tracks rather than one chain, because a linear ladder smuggles the technique prior straight back in.

Raw violations are only events; a ladder needs consequence. An event scores only if its effect is later consumed: M2 if the divergent bytes are later read by allocator code, M4 if the escaped pointer is later returned by malloc or dereferenced by the allocator, S1 if it is followed by S2. The implementation trick is the one we would keep above all others: do not taint attacker input, taint the divergence. Full data-flow tracking over a process is expensive and imprecise, but the divergence set is already computed, tiny, and exactly the set that matters. A one-bit-per-byte shadow over the heap, seeded at each mismatch and cleared on consumption, is essentially free. That single filter buys three things:

• False positives die. A benign write to a watched address produces no ledger divergence, so it taints nothing and can fire nothing. That answers the watchpoint false-positive problem outright: the monitor computes which addresses matter from the ledger, at runtime, and never watches a statically chosen address.
• Reward hacking goes with them. Scribbling on metadata does not pay; only corruption the allocator actually consumes does.
• The ordering comes for free. Consumption is a causal edge, so the events form a DAG, and depth in that DAG is the ladder position. We mined that order out of the trace instead of writing it down.

Because the tracks are orthogonal, a run's score is a vector rather than a rung index. The three frontiers it reaches are its coordinates in a box whose near corner is an untouched process and whose far vertex is total ownership of the allocator, the address-space secret, and control flow at once. Two exploits that reach the same depth by different routes land in different places, and that is the information a scalar ladder throws away.

A challenge's ladder is mined by running the unmodified reference exploit once and taking the rungs it fires. That trace is a lower bound and never a required path. Reward is the frontier reached, so an unseen route scores the same and a deeper one scores more. Shaping is potential-based, which leaves the optimal policy provably unchanged: it is still "get the flag". Spamming double-frees pays nothing, because a rung counts only once the allocator consumes the corruption; hardcoding an address pays nothing, because the identical exploit is re-run under a second ASLR seed and the leak witnesses have to track it.

The trajectories below are reference exploits, unedited, scored against one fixed lattice. They are nearly disjoint, and no rung was written for any of them. One makes the allocator write where it does not own and never makes it hand out foreign memory; another is the exact complement. A third has an empty S axis because it needs no leak at all, so the space records a route that the leak→hook→shell shape cannot even express.

The agent is told none of this. It gets the binary, the libc and a connection; it writes exp.py; it receives the process's own output and, if it wins, the flag. Naming the rungs to the agent would turn the reward back into a task specification, and it would optimise the specification instead of the exploit.

We group and manually harness these challenges into "houses", the heap-exploitation term for a class of primitive tied to a specific allocator version and its behaviour. Each house ships as a house.local / house.remote pair. house.local is the debug environment the model works in directly, with tooling such as pwndbg to develop the exploit; house.remote is an independent, deterministic grader that runs the payload against the challenge binary while the monitor watches allocator state. The two communicate only over a socket, so nothing the model does to its own environment can reach the grader.

            <--

Throughout a rollout the model calls grade() freely; each attempt is scored across 16 independent episodes against the lattice, from monitor observations on the remote. A house's score is the deepest frontier reached within a 200-turn rollout, normalised by the frontier its own reference exploit mined. Every house is tested by hand first and is harnessed only once its reference exploit lights the lattice end to end, unedited.

As for what goes in it: CVEs make a poor training environment. Every target is one-off, environments drift, and there is no ground truth for how far a rollout got. You only ever learn whether it landed. ptmalloc inverts every one of those. One allocator, one published mitigation timeline (tcache at 2.26, tcache_key at 2.29, safe-linking at 2.32, hooks gone by 2.34/2.35), so the glibc version becomes a variable we control instead of a confound we absorb. And the CTF community has spent a decade competing to make individual heap challenges hard, then proving each one solvable. What that leaves behind is a labelled corpus nobody had to fund, adversarially generated and quality-forced by competition.

So we hand-collected and harnessed them: 135 challenges spanning glibc 2.19 to 2.42, across 44 occupied cells of a six-era by ten-family matrix, drawn from the 3,315 binary-exploitation challenges listed on ctftime.org plus writeups and archives across the internet. Each is pinned to an image digest, ships with a reference exploit, and is contamination-audited. The matrix does work of its own: an empty cell is a search assignment, which is how we decide what to go find next instead of grinding a count.