Pathway

Transformer-based AI, the architecture behind most frontier LLMs today, has a memory problem. Its pretrained weights do not learn after it ships. Every request begins from the same fixed state, with no native way to internalize the last session into the model. Pathway and Amazon Web Services (AWS) recently authored a blog that describes this to customers as a Groundhog Day loop. https://lnkd.in/euHjetCk For enterprise workflows that span months, depend on proprietary context, or have to reason across non-standard cases, that memory gap is the difference between a demo and a system of record. Pathway’s BDH is built for the ‘sticky inference’ segment of the market that serves this gap. Think, use cases tied to your proprietary data, where context is everything and the model must continuously learn from the business it serves. AI that learns your business becomes quickly extremely valuable. BDH is the first post-transformer frontier model with memory and continuous learning on the fly, built to be enterprise-native. We partnered with AWS to make BDH accessible, bringing frontier AI innovation to the use cases where context, memory, and long-horizon reasoning matter most.   Read our blog to uncover six concrete enterprise applications of BDH that show what this looks like in practice.  If one of these is on your 2026 roadmap, dragon@pathway.com is the inbound for a first BDH deployment with Pathway on AWS.

Said it would be a real fight. IT WAS!! 🥊 Adrian Kosowski: “Transformers think in language. They do not think in latent thought.” 🥊 Mathias Lechner: “I am convinced that the Transformer will find its own replacement.” 🥊 Llion Jones: “Lukasz is going to be correct up until that day, and then he is going to be wrong forever.” 🥊 Lukasz Kaiser: “Do not be scared of being 50-times slower! If you show me a model that is fifty times slower but on a better slope, you win.” Oh well, good thing I told them to give me a "clean fight" 😂 Transformer Vs Post Transformer: Deciding Round, By Pathway

Have you ever seen AI scientists in a boxing ring? Last night in SF: "Transformers vs. Post-Transformers", organized by Pathway. Four contenders, real gloves. The special part: two co-inventors of the original Transformer paper, on opposite sides. Lukasz Kaiser (OpenAI) defended the Transformer. Llion Jones (Sakana AI) argued against his own invention. On the post-Transformer side: Adrian Kosowski (Pathway) and Mathias Lechner (Liquid AI). One line stays with me. Llion about Lukasz: "He will be right. Until the day he is wrong forever." Three things I take from the night. Local minimum. The success of the Transformer keeps us locked inside it. Especially with hardware mafia. The next paradigm will not come from reshuffling attention, MLPs and residuals. It will come from someone ready to question every assumption, even backprop. Strong words, from one of its own authors. The "PageRank moment". Adrian reminded us: indexing the web took one equation and one company to change everything. We do not yet have such a moment for intelligence. BDH is one attempt to find it. Scaling law challenge. Lukasz put the bar very clearly. Show me a sharper slope on the loss curve, going below Transformer, and I concede. Not "comparable at small scale". Sharper. A closing proposal I really liked: keep a private dataset, mixed text and code, that nobody trains on. Benchmark perplexity there. Honest. Hard to game. And somehow we still do not do it. What is clear: Transformer architecture has already changed how the world operates. And it will definitely help us find the next breakthrough. Thank you, Zuzanna Stamirowska and Dexter Horthy, for hosting.

🥊 Taking ideas into the ring! Pathway just hosted its most epic event to date: a literal boxing match of ideas, with Lukasz Kaiser, Adrian Kosowski, Llion Jones, and Mathias Lechner sparring over the future of AI and the rise of the Post-Transformer era! From finding a boxing ring (yes, it's a legit one), to kicking off collaborations with fantastic partners in SF, and vibe-coding a website, it has been a blast! Huge thanks to everyone who made it possible! Pathway's behind-the-scenes fighters: Ludovic, Mudit, Saksham, Trisha, Weronika, Zuzanna Videography and photography: Frédéric Neema (pics below) and Tom Sparks & his team, Marcus and Keith Venue: Hamidah Poplus, Lizette Rodriguez, Micah Dayag, INSEAD DJ: Bisi Obateru

🥊 Just watched the nerdiest boxing match in SF history. Forget about the Met Gala, THIS is the invite you want! Lukasz Kaiser and Llion Jones, co-authors of “Attention Is All You Need”, the paper that invented the architecture behind ChatGPT, Claude, Gemini, and basically every major AI model today, stepped into the ring with Mathias Lechner (CTO, Liquid AI) and Adrian Kosowski (Cofounder, Pathway). Wall Street Journal just predicted that 2026 is the year post-Transformers will rule AI. I got to watch the debate in real time, Transformers vs. Post-Transformers. People were picking sides like it was a UFC fight, minus the actual fighting, just technical trash talking, peak SF.  A few things I learned: 🥊 Post-Transformers in plain English: architectures trying to fix Transformer’s biggest weaknesses, like expensive attention compute, fixed context windows, and inference inefficiency. 🥊 “It runs on a Raspberry Pi.” Mathias Lechner casually mentioned Liquid models fit on a Pi. Meanwhile Transformers need a data center and a small power plant. The future might not be bigger. It might be smaller, faster, and sitting on your desk. 🥊 Continual learning came up, and nobody had a clean answer. Today’s ChatGPT, Claude, Gemini are basically snapshots. Train, ship, freeze, retrain. No real-time learning. Post-Transformer folks argued their architectures are built to keep learning. Transformer folks said, “don’t worry, we’ll get there too.” 🥊 Are we at a local maxima with Transformers? That was the real fight in the ring. Not “is attention all you need,” but “is it enough from here?” Scaling is getting expensive. Are we sleeping on the next wave? 🥊 Hybrid is quietly winning. Pure post-Transformer hasn’t unseated GPT-class models. But hybrids are sneaking into prod. My takeaway? The smart money isn’t picking a side, but bet on both. Are you Team Transformer or Team Post-Transformer? Sakana AI Pathway Liquid AI #AI #Transformers #SanFrancisco

I just went to what was probably the coolest AI meetup ever. I never imagined I’d see the creators of Transformers (yes, the "T" in ChatGPT) debating inside a LITERAL boxing ring!!! 🥊 Gotta love living in SF! The night’s battle: 𝗧𝗿𝗮𝗻𝘀𝗳𝗼𝗿𝗺𝗲𝗿𝘀 𝘃𝘀. 𝗣𝗼𝘀𝘁-𝗧𝗿𝗮𝗻𝘀𝗳𝗼𝗿𝗺𝗲𝗿𝘀. 🥊 In one corner: Lukasz Kaiser, co-author of Attention Is All You Need, defending the Transformer. 🥊 In the other corner: Adrian Kosowski, Mathias Lechner, and... plot twist... Llion Jones, also one of the original Transformer authors, arguing for what comes next. If you haven't heard of 𝗽𝗼𝘀𝘁-𝘁𝗿𝗮𝗻𝘀𝗳𝗼𝗿𝗺𝗲𝗿𝘀 before, it's not one specific architecture. It is an umbrella term for AI model designs that try to move beyond the standard Transformer as the dominant recipe for intelligence. A few takeaways that stuck with me: 𝟭. 𝗧𝗿𝗮𝗻𝘀𝗳𝗼𝗿𝗺𝗲𝗿𝘀 𝗮𝗿𝗲 𝘀𝘁𝗶𝗹𝗹 𝗵𝗮𝗿𝗱 𝘁𝗼 𝗯𝗲𝗮𝘁. Łukasz described the Transformer less as “just attention” and more as a kind of differentiable memory system: it stores keys and values, retrieves relevant information, and scales beautifully when given enough data and compute. 𝟮. 𝗧𝗵𝗲 𝗳𝘂𝘁𝘂𝗿𝗲 𝗶𝘀 𝗽𝗿𝗼𝗯𝗮𝗯𝗹𝘆 𝗻𝗼𝘁 𝗼𝗻𝗲 𝗢𝗥 𝘁𝗵𝗲 𝗼𝘁𝗵𝗲𝗿. 𝗜𝘁’𝘀 𝗧𝗿𝗮𝗻𝘀𝗳𝗼𝗿𝗺𝗲𝗿𝘀 𝗔𝗡𝗗 𝗽𝗼𝘀𝘁-𝗧𝗿𝗮𝗻𝘀𝗳𝗼𝗿𝗺𝗲𝗿𝘀. Mathias had the most engineering-pragmatic take of the night. Models are not designed in a vacuum. They are designed for hardware, latency, memory, deployment constraints, use cases, and available data. So the question might not be "Which architecture wins forever?" but "Which combination of building blocks works best for this job?" 𝟯. 𝗧𝗿𝗮𝗻𝘀𝗳𝗼𝗿𝗺𝗲𝗿𝘀 𝗳𝗲𝗲𝗹 𝗯𝗿𝘂𝘁𝗲-𝗳𝗼𝗿𝗰𝗲 𝗮𝗻𝗱 𝗱𝗮𝘁𝗮-𝗵𝘂𝗻𝗴𝗿𝘆. Llion's critique really stuck with me: humans don’t need to read the entire internet multiple times to become intelligent. And maybe the success of Transformers is trapping the field in a local minimum. 🌉 That last part  connects to a lot of Bay Area chatter I’ve been hearing lately around post-training. For example: 🔹 Recently, I had the opportunity to chat with Risto Miikkulainen about his paper that explores evolution strategies as a backpropagation-free way to fine-tune LLMs at billion-parameter scale. What if some of the next progress does not come from better backprop alone, but from optimization methods that explore the landscape differently? 🔹 This also made me think of Sara Hooker's essay "On the Slow Death of Scaling", and the broader argument that scaling alone has started to crowd out other levers of progress. So definitely a lot of chatter around what's next after Transformers? 🏆 So... who won? In the most scientific ending possible, the winner was decided by a live decibel meter measuring crowd cheers. The numbers were so close, but Transformers took the belt this time. Will that still be true in a year? In a month? Thank you Zuzanna Stamirowska and Pathway for organizing such a wonderful event!!!

Tonight in San Francisco – Pathway brings the inventors of AI to settle it in the ring. 🥊 Transformer vs Post-Transformer. May 5, SF. Lukasz Kaiser, Adrian Kosowski, Llion Jones, Mathias Lechner Pathway, Sakana AI, Liquid AI

Boxing match tomorrow night in SF!! Transformers vs Post-Transformers, hosted by Pathway. This is going to be a blast. My dear friend Zuzanna Stamirowska is turning this into a real debate. Are we actually in an AI bubble, or just early in something where the core paradigm hasn’t really been challenged yet? Feels like this is the tension right now: do we keep stretching Transformers, or does something new need to emerge? She’s putting some of the people who built this field into a boxing ring: Lukasz Kaiser, co-inventor of the Transformer; Adrian Kosowski, inventor of BDH, Pathway CSO; Llion Jones, another co-inventor of the Transformer and co-founder and CTO of Sakana AI; and Mathias Lechner, co-inventor of liquid neural networks. The point is to force the architecture question into the open: can memory, continual learning, and long horizon reasoning be solved by systems around the Transformer, or do they require a new architecture? You don’t often get the people who created these ideas in the same room arguing it out. Definitely not in a boxing ring. If you’re building or investing in AI, this should be a fun one. Event link in comments.