Future AGI

Future AGI is now available on Google Cloud Marketplace. AI teams across industries already rely on Future AGI as the self-improving layer for their agents in production. With this listing, organizations building on Google Cloud can add it to their stack the same way they adopt the rest of their infrastructure: through their existing Google Cloud account and through a procurement process their teams already know. For us, this is about reach. Many of the teams building the most ambitious AI systems today do it on Google Cloud, and we want Future AGI to be available wherever that work happens. Grateful to the Google Cloud partner team for their support in getting here. #GoogleCloud #AIAgents

A voice agent can get a perfect transcript score and still deliver a terrible call. That is the problem with treating voice evaluation as chatbot evaluation with an audio file attached. The words may be right while the caller was interrupted twice, an account number was heard incorrectly, or the agent took just long enough to make them say “hello?” Voice needs its own evaluation stack: audio quality, speech recognition, model behavior, voice output, and the conversation as a whole. When a call fails, you need to know which layer owns it. We wrote a practical guide to what to test, instrument, and trace before they become someone’s least favorite phone call.

If coding agents and data workflows are already on your roadmap, this is a good chance to hear how another experienced builder thinks about the problem and compare notes with people working through it too. AI engineers, data teams, platform engineers, and technical leads are all welcome. Register here: https://luma.com/uktqv4zk

Typical Monday, a voice agent somewhere is failing the exact call it failed last week. Same interruption, same wrong tool, same stretch of dead air. And nothing is stopping it from doing it again tomorrow. Relatable? That's the real problem with production voice: failures repeat, because nothing turns a bad call into a fix. So we built the loop that does, and it works on whatever you run: LiveKit, Vapi, Retell, or your framework on your own infrastructure. Take the calls that failed. Turn them into simulations and run hundreds of variations. Evaluate each on the audio and the outcome. Let optimization rework the agent, compare the versions head to head, and ship the one that actually wins. The bug you kept hitting becomes a test every future version has to pass. And you don't have to wait for a bad call. Run the same loop before launch: simulate the hard scenarios up front and catch the failures before a customer ever hits one. Here's a quick demo using LiveKit but the same applies across any orchestration platform 👇 then run it on your own agent, free and self-hosted. Start free → https://lnkd.in/gaxv4MRG

On Sept 17, we’re hosting Josh Reini from Snowflake to answer a practical question: what needs to sit around a coding agent before you can trust it with a real data workflow? A capable model can generate the code, but it still needs the right tools, reusable skills, and enough semantic context to understand the data it is working with. And when the result is wrong, you need traces that show whether the problem came from the agent, its context, or the harness itself. Rishav & Josh will work through how those pieces fit together and why changing the prompt is not always the fix, despite its excellent availability. Join us: https://luma.com/uktqv4zk

"Which eval should I use" has one real answer: it depends on how correctness can be established for that specific check. A rule can decide it → deterministic check Valid JSON, code passes its tests, correct tool call. Exact and free. Closeness to a reference is enough → similarity metrics Score overlap with the gold answer. Cheap, but lexical metrics miss paraphrase, so treat them as a proxy. Meaning is what matters → model-based judge Coherence, tone, faithfulness to a source, or semantic match to a gold answer. A judgment, so calibrate against human labels. Only running it confirms it → agent-as-judge Run the code, read the row, hit the API. The evaluator acts and verifies. No single check is enough → composite eval Bundle several of the above into one verdict, so a pass means every dimension held, not just the one that was easy to check. Different kinds of correctness need different evaluators. Default everything to an LLM judge and you get an expensive, non-deterministic opinion where a one-line check would have been exact. We support all of them in one built-in eval harness, so you can run the right check for each case instead of forcing everything through a single judge. The carousel below is the full map: which one, and when. 👇 PS: it's part of our open-source repo https://lnkd.in/ggseJpsj

ASR is one of those unglamorous voice-agent problems that everyone remembers after a call goes sideways. The transcript has the wrong name. Or a number is off by one digit. Or a customer says “I don’t want to…” and the important word simply disappears. From there, the rest of the stack does its job brilliantly - on the wrong information. We wrote a field guide to the seven ASR failure modes that matter most in production plus what to instrument, evaluate, and fix for each. 👇

Today we pause to honor the people whose work makes everything else possible. So much of what we rely on each day happens because someone showed up and did it well, often without notice, and always with more care than we see. To everyone whose hands and hours keep the world going: thank you, sincerely. We are better for it. Happy Labor Day!

We spent years telling AI teams to trace everything. Now they have a new problem: they traced everything. A trace is excellent evidence. It is a terrible unit of work. It shows what happened in one run. It does not tell the team whether the same failure keeps repeating, which part of the system owns the fix, or where an engineer should start. That distinction shaped Error Feed. Each detected failure is tied back to trace evidence and mapped to the likely fix layer: Prompt, Tools, Orchestration, or Guardrails. Repeated instances are grouped into one issue instead of becoming a backlog of near-identical traces. The raw traces stay attached because they are the proof. But the issue becomes the unit of work: one recurring problem, one fix surface, one place to begin. This is the next problem AI observability needs to solve. Not collecting more telemetry, but turning it into engineering work without losing the evidence or inventing certainty. Are traces still serving as both evidence and backlog for your team?

Every coding-agent leaderboard measures one thing: can the agent make a failing test pass. Your team measures something harder: would a reviewer actually merge this. Those are different tests, and the space between them is where agents that look strong on paper come apart. Public benchmarks also run on clean open repos the agent may have trained on. Your repo has internal libraries, house conventions, and a stricter review bar, so a high score tells you the agent can code, not that it can code here. This week's piece is the eval that runs on your code instead: replay the pull requests you have already merged, and score whether the agent's diff would have passed your review. Read it before you pick a coding agent off a leaderboard.