Most of the worries about an AI bubble involve investments in businesses that built their large language models and other forms of generative AI on the concept of the transformer, an innovative type of neural network that eight years ago laid the foundations for the current boom.
But behind the scenes, artificial-intelligence researchers are pushing into new approaches that could pack an even bigger payoff.
One early-stage startup developing a transformer alternative, Palo Alto, Calif.-based Pathway, is announcing today that its “Dragon Hatchling” architecture now runs on NVIDIA AI infrastructure and Amazon Web Services (AWS) cloud and AI tech stack.
The company has shipped Dragon Hatchling architecture, but doesn’t plan to release the commercial models trained on it until next year. Once that happens, its Nvidia and AWS compatibility means companies would be able to put it into production “the next day,” Pathway told me in an exclusive story for the WSJ Leadership Institute.
Dragon Hatchling imbues AI with memory that large language models can’t match, according to Pathway, theoretically enabling a new class of continuously learning, adaptive AI systems. The company also casts its approach as a potentially faster way to get to AGI.
Pathway isn’t alone in this quest. It regards well-established Anthropic as its biggest obstacle. And it must convince people who have just learned one set of AI skills that they should adopt something new.
Regardless of whether Pathway fulfills its ambitions, it will at least get a chance to make its case to the market. Its arrival also reinforces the intense scientific effort driving AI forward, even as big deals, big valuations and big personalities command the attention.
Pathway Chief Commercial Officer Victor Szczerba said the technology could be useful in solving complex supply chain variability, fusion research, space exploration and optimization of global trading networks.
The key is a need for real innovation. To come up with a new spaceship design, an AI model can’t just access lots of data on other spaceships and learn. It requires a model capable of generalizing or learning to reason, rather than pattern matching.
“We will speed up innovation cycles dramatically,” Zuzanna Stamirowska, co-founder and CEO at Pathway told me. “The problem with current transformer-based models is that they need a lot of data, and they don’t generalize outside….of what they have seen.”
The company said it has raised more than $16.2 million in VC funding and about $3.8 million in grants. Backers include Lukasz Kaiser, one of eight Google researchers who kicked off the transformer era with the paper “Attention Is All You Need,” and TQ Ventures.
While a bubble may have formed around the concept of the large language model, that need not be true of AI itself, according to Martin Farach-Colton, chair of the Computer Science and Engineering Department at NYU Tandon School of Engineering.
The Wall Street Journal