Okahu

Your agent already ran. Now make it a test. 🧪 Most agent "tests" start with fixtures, mocks, and a blank file. Ours start with something you already have: a real trace. The Okahu VS Code extension turns any agent run into a ready-to-edit test in one click. Right-click a trace → Generate Test. Done. Then you assert on what actually happened: ✅ Which agents and tools ran (and which never should) ✅ Exact / substring / negative checks on inputs and outputs ✅ Span attributes, events, and scope (session, user, tenant) ✅ Quality gates: hallucination + answer relevancy judges ✅ Cost and latency limits (token budget, agent-turn duration) Swipe through the carousel for the full assertion cheat sheet 👉 Want to see it live? We run a bi-weekly demo series, and the next one is coming up: 📅 Wednesday, July 29 @ 8:30am PST 🎟️ RSVP: https://lnkd.in/g7px_3jd If you have cool projects or demos, please join and share with the community. #AIAgents #LLM #AgentEvaluation #VSCode #AIEngineering #Observability #Okahu Special thanks to the team Prasad Mujumdar Carey James Hoc Phan ASHOKAN PERUMAL Karmehr Arora Mohammed Ansari

Evals shouldn't live in a separate tool. They should live where you already work: VS Code! Video: https://lnkd.in/gn3ch7vi We just shipped the Okahu AI Observability extension for VS Code. It lets you: 🔎 Submit custom evals against your AI agent traces 📊 See results right inside VS Code without context switching 🧩 Trace GenAI apps & agents with Monocle2AI (open-source) ✅ Move from "it seems to work" to eval-driven development If you're building agents and tired of guessing whether your changes actually made things better, this closes the loop between writing code and knowing it works. 👉 Install it here: https://lnkd.in/gkyNvbXA Big thanks to ASHOKAN PERUMAL for walking through it on our community call.

REMINDER: Monocle2AI Office Hours is TOMORROW! Date/Time: July 15 @ 8:30am PST RSVP: https://lnkd.in/g7px_3jd Prasad Mujumdar Carey James Hoc Phan Mohammed Ansari Karmehr Arora

Come join our next Monocle2AI Office Hours Date/Time: July 15 @ 8:30am PST RSVP: https://lnkd.in/g7px_3jd Bring your question and code! Let's demo. 👏 Thanks to our awesome speakers: Hoc Phan Prasad Mujumdar Carey James Mohammed Ansari Karmehr Arora

𝗧𝘂𝗿𝗻 𝘁𝗿𝗮𝗰𝗲𝘀 𝗶𝗻𝘁𝗼 𝘁𝗲𝘀𝘁𝘀 𝗳𝗮𝘀𝘁𝗲𝗿 You captured a great trace from your agent. Replay is easy. Turning it into a real regression test? That's usually where most teams stall. A GitHub Star ⭐ goes a long way: https://lnkd.in/gEMrBJcf 𝗠𝗼𝗻𝗼𝗰𝗹𝗲'𝘀 𝗧𝗲𝘀𝘁 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗼𝗿 just got friendlier: → Generate pytest assertions straight from any captured trace → Load traces from JSON files, Okahu, or directly from a running agent → One command in a real test file out  python -m monocle_test_tools.generate_test trace.json > test_generated.py 𝗪𝗵𝘆 𝗶𝘁 𝗺𝗮𝘁𝘁𝗲𝗿𝘀: regression tests for AI agents are notoriously brittle when written by hand. Tests derived from actual traces capture the real call shape, the real tool sequence, and the real I/O so they break only when the behavior genuinely changes. Big thanks to Shambhavi Goel for shipping this. Prasad Mujumdar Mohammed Ansari Karmehr Arora for reviewing and additional features. #OpenSource #AIObservability #Testing #LLMOps #Monocle2AI

𝗔𝘂𝘁𝗼𝗺𝗮𝘁𝗲𝗱 𝗧𝗲𝘀𝘁 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻 𝘂𝘀𝗶𝗻𝗴 𝗠𝗼𝗻𝗼𝗰𝗹𝗲𝟮𝗔𝗜 Testing agentic apps is hard. You're not just checking outputs. You need to assert that the right agent ran, called the right tool, with the right input, under a token budget, with no hallucinations. We've been building two Claude Code skills that do this end-to-end on top of 𝗠𝗼𝗻𝗼𝗰𝗹𝗲𝟮𝗔𝗜 traces. https://lnkd.in/gms28MVR Full open source (would love a Star): https://lnkd.in/gEMrBJcf Thanks Prasad Mujumdar for the great demo. #GenAI #AgenticAI #LLMOps #Observability #Monocle #ClaudeCode #Testing

Most Claude Code users have no idea what their agent is actually doing under the hood. Which tools it called. How many tokens it burned. Where the prefix cache hit or missed. Monocle 0.8.3 cut that to two commands:   $ uv tool install monocle-apptrace   $ monocle-apptrace claude-setup Would love a star ⭐ : https://lnkd.in/gEMrBJcf Video: https://lnkd.in/gbywGiMG Thanks Mohammed Ansari and Prasad Mujumdar for such an awesome demo! #ClaudeCode #AIAgents #Monocle #Okahu #AIObservability #Productivity

We are starting a series of weekly demo Friday @ 9am PST Dates: June 5, 12 and 19 RSVP: https://lnkd.in/ghwjvHsg This week, we will walk through Monocle 0.8.3 Give us a ⭐: https://lnkd.in/gEMrBJcf > pip install monocle_apptrace