Jev Evaluation Agent
A personal-assistant agent uses Jev to select one tool from a 100-tool catalog before each model step.
Added to Jevfast
How it uses Jev
In classifier mode, Jev receives the user request, completed tool actions and results, and prior assistant text. A Choice selects one of 100 tools or `respond_to_user`; a Noul asks whether all requested actions are complete. A gate blocks premature final responses before the model receives the selected tool.
What Jev decides
Are all requested actions complete?Example answers · not a recorded Jev response · Source ↗
Question 1 · done
YOUR APP
INSTRUCTION
Have all requested actions already been carried out successfully, or is a needed lookup still missing?
STATE
The user's request, successful tool actions and their results, and the assistant's prior replies.
JEV · NOUL
YesNo
App workflow
- Build agent state
The router compacts conversation history into the request, actions already taken, and assistant responses.
- Choose the next step
Jev selects a tool or the respond option and separately estimates whether the task is complete.
- Gate completion
If the completion judgment is below the configured threshold, the router exposes the next-best tool instead of ending.
- Run and compare
The agent executes the tool and records decisions; evaluation runs compare classifier and direct-selection modes.
Why it is interesting
The repository holds prompts, catalog, mock world, and evaluations constant while switching who selects the next tool, making the routing comparison reproducible.