LangChain

LangSmith now supports decision models including Jev and SemIf, giving you visibility into every step. Now, you can: ✅ Debug faster ✅ Understand model behavior 🚀 Confidently improve your apps Try it today -> smith.langchain.com

Healthcare organizations are using agents to transform patient care across workflows. But one persistent bottleneck is that clinical expertise is a scarce resource, and these experts are typically ones that must vet the accuracy of these agents. LangSmith is helping transform expert review into durable + reusable evaluations, compounding the value of human judgment. Read on for two real-world examples of this evaluation loop in action: ✅ Abridge uses LangSmith to run evaluations, helping to cut its release cycle from one to two months down to a few days. ✅ Included Health uses LangSmith to track routing quality and safety for its AI health guide, which identified over 99% of high-risk situations correctly. https://lnkd.in/gpeQMXJx

With Managed Deep Agents, you can build an agent that lives in Slack and writes GitHub PRs with just a few lines of code and one deploy command. We built one and called it Patch. Tag it in a thread, describe the feature you want, and it reads the repo and opens a PR with the diff, ready to review. Here’s a demo from Ishaan Lal.

ICYMI -- yesterday I shared a quick video walk through of TypeSafe AI's new Jev model, system one models, and how to build them into your agent harness! Video walkthrough: https://lnkd.in/d7xXmGXj Accompanying blog: https://lnkd.in/gqrxcZUv

Clay evaluates every agent on three things: ✅ Quality ✅ Throughput ✅ Cost Jeff Barg on how LangSmith's evals measure quality and benchmark it against cost and throughput. 

We’re one week away from the LangSmith Roadshow in Boston! ✅ Get product deep dives into LangSmith Engine and Deep Agents ✅ Hear how Censys uses LangSmith to build, debug, and monitor agents running in production ✅ Go hands-on with LangSmith Engine during our workshop Interested in attending? RSVP at https://lnkd.in/gyE_fiN8

Sydney Runkle, Product Manager, LangChain OSS, Allie Laabs, Founding Developer Advocate at TypeSafe AI, and Hunter Lovell, Technical Lead at LangChain are hosting a livestream on all things Jev! https://lnkd.in/gFeDH7VC ✅ Get up to speed with an introduction to Jev ✅ Learn about harness design patterns, and how they can be applied to your own agent systems ✅ Get your questions answered by the LangChain and Typesafe teams

Jev-as-a-judge is available in LangSmith today. Evaluate open-ended agent behavior and turn the results into structured feedback faster and cheaper. ✅ Score every production trace instead of a sample.  ✅ Check more criteria per trace without the cost climbing.  ✅ Catch safety or security issues fast enough to trigger an automated response. Jev scores multiple questions in parallel and is cheap enough to run on everything, so all three become possible at once. In our testing, Jev matched a human reviewer's judgment call for call, while running faster and far cheaper than the LLM judges we compared it against. How to get started: 1️⃣ Sign in or sign up for LangSmith 2️⃣ Open the Evaluators tab in any tracing project 3️⃣ Add an LLM-as-a-Judge evaluator 4️⃣ Select TypeSafe AI as the provider https://lnkd.in/d26DmG_B

We're making SemIf, an open-source decision model, free for the next week in LangSmith LLM Gateway.

Need a 10 minute walkthrough of Jev? Start here ⤵️