Avi Fenesh: research and notes https://avifenesh.ai/ Working papers and field notes on efficient inference, speculative decoding, model compression, systems, and open source. en Wed, 05 Aug 2026 00:00:00 +0300 Result note: layer-loop + ε-scaling beats FLOPs-matched vanilla in all three mixer families https://avifenesh.ai/research/#note-recipe-lab https://avifenesh.ai/research/#note-recipe-lab Tue, 04 Aug 2026 23:00:00 +0300 Eleven pre-registered rounds. In the data-constrained regime the looped+scaled model won on best-achievable val loss in attention (3/3 seeds), pure SSM (2/2), and hybrid (2/2): seven paired runs, zero sign flips, 26-34% fewer parameters. Rule isolated: loop the state-mixer, never the retriever. Repo: github.com/avifenesh/recipe-lab Result note: the memory-layer retrofit is free; its forgetting advantage never reached significance https://avifenesh.ai/research/#note-mem-retrofit https://avifenesh.ai/research/#note-mem-retrofit Mon, 03 Aug 2026 20:00:00 +0300 Product-key memory grafted post-hoc onto dense Qwen3-4B, five sequential update rounds vs LoRA. At lr/10 the retrofit costs nothing; the published forgetting advantage failed 12/12 confidence intervals, and power analysis shows n=3 could never resolve it. Repo: github.com/avifenesh/mem-retrofit Result note: block-routed SwiGLU: the kernel is nearly free, the capability isn't there https://avifenesh.ai/research/#note-block-routed-swiglu https://avifenesh.ai/research/#note-block-routed-swiglu Wed, 22 Jul 2026 18:00:00 +0300 Input-dependent gate/up re-pairing inside SwiGLU: the Triton kernel costs +0.73-0.83% on H100 and is CUDA-graph safe, but the equal-parameter capability test refuted the hypothesis (63.4% vs 66.1% held-out, 5 seeds). Clean negative. Repo: github.com/avifenesh/block-routed-swiglu Small-vocabulary MTP heads for memory-bound speculative decoding https://avifenesh.ai/research/small-vocabulary-mtp/ https://avifenesh.ai/research/small-vocabulary-mtp/ Sat, 11 Jul 2026 12:00:00 +0300 Working paper, concluded (v0.5): the distilled student hit 85.5% of teacher chain acceptance and still lost: function cuts pay a 10-19 point off-distribution tax fidelity cuts do not. The zero-training trimmed-vocabulary NVFP4 recipe wins at 1.82x/2.88x measured. Not peer reviewed. Prune, heal, quantize: composing expert pruning with low-bit MoE deployment https://avifenesh.ai/research/prune-heal-quantize/ https://avifenesh.ai/research/prune-heal-quantize/ Sat, 11 Jul 2026 11:00:00 +0300 Working paper (v0.2): 50% expert prune + heal + NVFP4 within 0.012 ARC of baseline; first physical artifact public (Hy3-REAP-Layer103p5); 83.5% retention at the 100GB cap; measured per-projection damage feeds a global precision allocator. Not peer reviewed. How AI workloads changed the queue I was already building https://avifenesh.ai/writing/how-ai-workloads-changed-the-queue/ https://avifenesh.ai/writing/how-ai-workloads-changed-the-queue/ Sat, 28 Mar 2026 15:50:18 +0000 AI jobs stream, spend, wait, and fail differently. A field note on moving budgets, token limits, live output, and human pauses into the queue itself. Your AI agent configs are probably broken (and you don't know it) https://avifenesh.ai/writing/your-ai-agent-configs-are-probably-broken/ https://avifenesh.ai/writing/your-ai-agent-configs-are-probably-broken/ Thu, 12 Feb 2026 00:51:42 +0000 Skills, hooks, memory, MCP, and agent instructions fail silently. A field note on treating AI coding configuration like executable code.