⚡ todo-jev

todo-jev classifies a request and recommends a handling path: a local rule, a structured decision or specialist skill, or a foundation model. It combines when a skill applies and when to exclude it with basic environment checks performed locally.

한국어 소개: 사용자의 요청을 분석해 처리 경로와 적합한 스킬을 추천하는 실험용 라우터입니다. 스킬의 적용·제외 조건과 기본 실행 환경을 함께 살펴봅니다.

Current scope: Classification, skill matching, and routing decisions are implemented. The execution handlers return example responses; performing the selected task requires connecting your own rule functions, skill runner, or model client.

An agent may have dozens of skills available, each with different strengths and prerequisites. Choosing a useful next step depends on both the request and the available environment.

This project explores whether giving Jev structured selection criteria can improve that choice:

• Three routing tiers for local rules, focused decisions or skills, and foundation-model work.
• 20 bundled skill profiles with application conditions, exclusions, and source metadata.
• Basic preflight checks that report environment evidence and influence skill routing.
• Local skill discovery to include additional installed skill descriptions in live classification.
• An offline fallback for trying the CLI without an API key.

Requires Python 3.10+ and Git. Run the examples from the repository root; environment checks inspect your current working directory.

git clone https://github.com/maker-KK/todo-jev.git
cd todo-jev
python -m venv .venv

Activate the virtual environment:

# macOS / Linux
source .venv/bin/activate

# Windows PowerShell
.venv\Scripts\Activate.ps1

Then install the dependencies:

python -m pip install -r requirements.txt

Use python3 if that is the Python command on your system.

Leave TYPESAFE_API_KEY unset or empty in both .env and your shell to use the keyword-based fallback:

python -m app.cli route "15 + 27 계산해줘"
python -m app.cli route "PDF 정답지와 문항 번호를 매칭해줘"
python -m app.cli route "사진 속 도형을 분석해줘"

These examples produce the following routing recommendations in offline mode:

Inspect a bundled skill and its environment check:

python -m app.cli route "tlc-spec-driven: 신규 기능의 요구사항을 정리해줘" --format detailed
python -m app.cli catalog

Offline matching uses literal keywords and profile text. Including a skill ID makes it easier to try a specific profile.

Create a .env file in the repository root with your TypeSafe API key:

TYPESAFE_API_KEY=your_typesafe_api_key_here

Run the same commands to classify through Jev. The client uses jev-latest at https://api.typesafe.ai/v1/systemone. Network errors and unsuccessful HTTP responses fall back to the local heuristic.

Set these variables in .env or your shell:

Thresholds apply to the general task categories in live mode. Skill matches follow the checks below; offline mode uses fixed heuristic scores and routing rules.

flowchart TD
    P["User request"] --> M{"API key configured?"}
    M -->|Yes| J["Jev classification"]
    M -->|No| H["Keyword fallback"]
    J -->|Network or HTTP failure| H
    J --> D["Task type, score and applicable skill checks"]
    H --> D
    D --> T1["Tier 1: Local rule"]
    D --> T2["Tier 2: Decision or skill"]
    D --> T3["Tier 3: Foundation model"]

In live mode, Jev returns a task choice and a local-suitability score. The classifier calculates:

matching_rate = task_type_confidence × can_handle_locally

This is a routing score. Treat it as a decision aid; it has not been calibrated as a probability of successful execution.

Preflight affects the selected tier, separately from the live score. Live classification calls Jev before choosing a tier, including for tasks ultimately assigned to Tier 1.

The canonical profile catalog contains 20 curated profiles. Each records application and exclusion conditions, expected inputs, preparation notes, a preflight contract, and source metadata.

Profile mode sends the first two application conditions and first two exclusions per profile to Jev. After a bundled skill is selected, the classifier also checks for literal exclusion-text matches in the request.

What the checks mean: Each profile runs one broad presence check, such as finding a Git repository, manifest, or Node executable. The CLI badge Verified & Ready means that check passed. Actual skill installation, tool availability, credentials, and output quality need validation by the execution layer. Recorded SHA256 fingerprints are metadata; identity verification is not enforced during routing.

The registry scans one level of */SKILL.md files under these default locations:

• .agent/skills/
• ../cjbs-backend/.agent/skills/
• ~/.gemini/antigravity/skills/

With auto-sync enabled, live criteria consider up to the first 15 discovered skills, using their names and shortened descriptions. Bundled profiles take precedence when names overlap. You can supply different directories through SkillRegistry(search_paths=[...]) in Python.

Access the structured classification result to integrate a routing decision into your own agent:

import asyncio
from app.classifier import JevClassifier

async def main():
    decision = await JevClassifier().classify("PDF 정답지와 문항 번호를 매칭해줘")
    print(decision.model_dump_json(indent=2))

asyncio.run(main())

The result includes task_type, matched_skill, matching_rate, recommended_tier, preflight_details, and rationale. The example handlers in app/router.py show where to connect downstream execution.

For the separate Aside agent integration, see aside/README.md.

The committed report, dated September 18, 2026, compares description-based criteria with profile-based criteria on 60 Korean prompts marked eval in the 80-prompt dataset.

Profile mode passed one additional case overall: +1.7 percentage points across all 60 cases and +6.7 percentage points across the 15 negative cases.

Read these results using the evaluator's scoring rules: a negative case passes if no skill is selected, a veto is reported, or the expected tier is selected. That is a broader criterion than requiring an exact routing match. This is one saved run; assessing general accuracy or latency improvements requires further evaluation. The report records no per-call live/fallback status.

To run your own comparison with a configured TypeSafe key:

python scripts/run_eval_comparison.py

The script evaluates both modes, attempts 120 classifications, and overwrites data/eval_comparison_report.json. Without a key, it uses the offline fallback.

Ensure TYPESAFE_API_KEY is absent or empty in both .env and your shell, then run the existing tests from the repository root:

python -m pytest

The canonical-identity test currently requires a separately installed tlc-spec-driven skill matching the recorded fingerprint. A fresh clone without that skill fails that test.

Useful contributions include reproducible routing mistakes, more varied evaluation prompts, stronger prerequisite checks, and execution adapters. When reporting a routing issue, include the prompt, expected path, actual decision, live/offline mode, and relevant environment details.

Open an issue or submit a pull request.

MIT © 2026 maker-KK