Avi Fenesh
Applied ML research, written as it runs

Models should be measured where they run.

I study how to make large-model inference smaller, faster, and still trustworthy: train the compact head, prune the experts, assign the precision—then measure the claim in a runtime built to expose it. Everything here is a working record; the results are the signal.

Systems software engineer, AWS ElastiCache · Maintainer, Valkey GLIDE · Founder, agent-sh
Research / 01

Working papers, before the polish hides the uncertainty.

Each page separates the question, protocol, evidence, negative results, and what remains unresolved. PDFs arrive when the experiment is complete—not before.

WP–01 · speculative decoding Study concluded

Small-vocabulary MTP heads for memory-bound speculative decoding

Can a compact, separately trained MTP head replace the inherited full-width block and 248k-row output projection without surrendering the acceptance that makes speculation useful? The audit answered: no—function cuts pay an off-distribution tax that fidelity cuts do not, and the zero-training trimmed-vocabulary recipe is the one that survives.

Current signal 1.82× measured wall time at 9B; 2.88× best composed cell at 27B Measured · bw24 engine · trimmed NVFP4 head · quantized-base compose
Open study ↗
WP–02 · MoE compression Artifact published

Prune, heal, quantize: composing expert pruning with low-bit MoE deployment

A study of compression order: remove experts with router-weighted activation evidence, heal the surviving function, then assign low precision—rather than asking quantization or pruning to carry the full loss alone.

Current signal 128 → 64 experts; final ARC within 0.012 of baseline Preliminary · Qwen3-30B-A3B · NVFP4 simulation
Open study ↗
Research index and methodology
Evidence ledger / 02

The result can be yes, no, or not yet.

I keep the failed arms because they define the research more honestly than a clean success narrative does.

0.507 / 0.593 StudentSV vs teacher chain acceptance at ~13% of draft FLOPs
2.61 vs 2.99 perplexity after 50% expert prune + heal, against the unpruned baseline
−6.0% PPL imatrix-aware NVFP4 scale search at 9B, in review at llama.cpp
Refuted

A public REAP50 mask is a usable starting point for Hy3.

At the bounded screening gate the community mask collapsed—0.239 domain-macro against 0.727 for the unpruned ceiling on the same screen. Layer-balanced pruning with private calibration holds 83.5% retention at the 100GB cap, and the published Layer103.5 overlay finishes 76/115 against matched Layer100's 73/115.

16 Jul 2026
Refuted

A distilled compact head can match the trimmed-vocabulary drafter off-distribution.

Function cuts—pruning, low-rank, distillation—keep 97% of their acceptance on calibration-register text and pay 10–19 retention points under real shift; fidelity cuts pay at most 1.9. The mechanism is statistical, not a size problem: parameters serving rare content accumulate no mass on any finite calibration sample. The zero-training recipe stands.

19 Jul 2026
Supported

The compact head pays for itself in measured wall time.

End-to-end in the bw24 engine, the trimmed NVFP4 head reaches 1.82× on MT-Bench at 9B, and composes with a quantized base to 137.5 tok/s at 27B—2.88×, the best measured cell so far.

19 Jul 2026
Supported

A smaller MTP head can learn useful draft behavior at a fraction of the compute.

Chain-distilled StudentSV reaches 0.457 recursive chain acceptance against the teacher's 0.593—77%—at roughly an eighth of the draft compute, with all four slots live.

12 Jul 2026
Refuted

NVFP4 compression alone degrades the co-trained MTP head.

It did not. Across forced replay, generation-distribution replay, and seeded sampled runs, two clean quantization arms showed no acceptance loss versus BF16. The original healing premise was closed.

10 Jul 2026
Supported

Draft heads should train on the model's own generations.

Own-output training reached 0.548 held-out acceptance versus 0.376 for a generic corpus at one third of the steps. The training distribution has to match the serving distribution.

11 Jul 2026
Open

Should surviving REAP experts receive different precision budgets?

The next MoE study asks whether routing frequency, activation impact, and spill cost can predict which surviving experts deserve higher precision—and whether the policy survives a generative evaluation.

Protocol design
Experimental systems / 03

The apparatus is part of the argument.

Research claims about inference depend on loaders, kernels, caches, evaluation code, and hardware behavior. I build enough of that stack to know what the number includes.

1.0× parity 1.5× 2.0× Qwen3.5-9B Qwen3.6-27B Qwen3.6-35B MoE Plain decode · 1.07× Speculative decode · 1.93× 1.07× 1.93× Plain decode · 1.08× Speculative decode · 1.24× 1.08× 1.24× Plain decode · 1.06× Speculative decode · 1.16× 1.06× 1.16× ● speculative decode ○ plain decode
bw24 decode speed relative to llama.cpp's best config—same rig, same prompts, interleaved runs, exactness-gated. Short-code prompt class shown; preliminary, one machine.
01 / engine bw24

A from-scratch Rust + CUDA inference engine used as an experimental instrument for MTP speculative decoding—greedy and sampled—NVFP4, MoE spill, and exactness gates on Blackwell.

public · active ↗ 02 / study hqmtp

The research harness for quantized-trunk MTP agreement and compact StudentSV heads: extraction, training, replay evaluation, corpus controls, and negative-result ledgers.

public · released ↗ 03 / compress REAP × quant

Expert saliency, pruning, full-function healing, low-bit simulation, and the loader work needed to carry REAP50 artifacts into a constrained inference runtime.

working paper 04 / migrate CRIU live migration

Copy-on-write live migration of a 200 GB loaded in-memory datastore with under 50 ms of freeze: userfaultfd page prefetch, priority-ordered VMA transfer, and lazy-page lifecycle fixes.

research fork ↗ 05 / systems Valkey GLIDE

Production systems work at a different scale: an official multi-language Valkey client. 131 authored pull requests merged and 410 reviewed—reliability, releases, and the support that starts after an implementation works once.

maintainer ↗ 06 / desktop ChatGPT desktop for Linux

Bringing OpenAI's desktop app to Linux: Wayland input, accessibility-tree automation, packaging across five formats, and the updater plumbing on a 2k-star community project.

collaborator ↗ 07 / tools agent-sh

Open-source agent tooling organization: agnix, a 432-rule linter and language server for agent configuration; computer-use-linux and agent-workspace, isolated desktop control for agents on Linux; plus code review and repository intelligence.

founder ↗
Notes / 04

Long-form writing, with room to think.

The paper pages are formal and evidence-heavy. The notes are where systems lessons, design arguments, and open-source experience can breathe.

How AI workloads changed the queue I was already building

What streaming, spending, waiting, and long-lived work revealed about the queue's model of execution.

5 min read

Your AI agent configs are probably broken

Why silent failures in skills, hooks, memory, MCP, and instruction files needed source-backed validation.

5 min read
All notes, talks, and answers
About / 05

A nontraditional route into research, built in public.

Avi Fenesh smiling at a table
Avi Fenesh · Tel Aviv

I am an independent ML researcher and a systems software engineer at AWS ElastiCache. My route into research runs through implementation: reproduce the baseline, expose the hidden assumption, train the missing comparison, and let the result change the plan.

I study computer science at The Open University of Israel while maintaining open-source systems and running research outside a traditional lab. That path has made method unusually important to me. Claims need matched baselines, saved artifacts, explicit noise floors, and a visible record of negative results.

The research is trained, not only measured: compact MTP draft heads with their own vocabularies, healed pruned MoEs, a fine-tuned ColBERTv2 retriever with a LoRA relevance judge to score it. Before focusing on efficient inference I worked deeply in datastores, language clients, queues, retrieval, and developer tools; I still maintain Valkey GLIDE, stay active across the valkey-io ecosystem, and help build desktop developer tools, and that record—review, releases, support—runs alongside the studies. Model research becomes real software surprisingly quickly.

I also volunteer as a mentor with Yotzim LaShinui, helping a junior developer from a formerly ultra-Orthodox background find their way into software.

Collaboration · replication · careful disagreement

Working on efficient models under real constraints?

aviarchi1994@gmail.com ↗ GitHub / avifenesh ↗ LinkedIn ↗