Repository navigation
Sorting which projects are Laya-compatible #17
Description
Activity
Good request, and it turned out to be two questions rather than one, so I have split them in #18.
Routing Jev through OpenRouter or Vercel's AI Gateway is not an alternative model. It is the same paid model with a different bill and one extra hop. A lot of entries advertise it, and it changes nothing about cost per decision or about licensing. That distinction was doing most of the confusing here.
Pointing a tool at a different model is the part worth marking, so the new section marks it — but only where we have actually done it, by running the tool against a server of our own rather than reading a README:
Tool Points at a non-Jev endpoint? Jevvy Yes — provider: "custom", any endpointjev-use Yes — TYPESAFE_BASE_URL, plus a keyless mock backendJCR Yes — the SDK's own TYPESAFE_BASE_URLhermes-jev-approvals Yes — custom base_urlwith optionalkey_env(read in code; we did not drive it)Jev Skills No — an allow-list of exactly two endpoints, both Jev routes Jev Agent Skill Router No — one hardcoded endpoint Jev DSH No — endpoint constant, no override jev-seo No — hardcoded constant, no override jevscan-evm No — endpoint constant, no override A project missing from that table means we have not checked it, not that the answer is no. We can only mark what we ran, and we run roughly ten a day, so it will fill in.
On the underlying worry — paying for Jev — the honest answer is that the projects in Open Models & Reproductions are the real alternative, because they replace the model rather than the route. Two things to check there, which I have put in the section as well:
- The weights' licence is a separate question from the repository's. OpenThai-SystemOne and PlayJev are Apache-2.0 on both, with the base model credited. Von's published weights carry no licence tag at all. NanoJev declares none on either the model or the dataset. A model with no licence on the Hub is not more reusable than a paid API — it is unclear instead of expensive. (Same answer I gave you on Argos1111/jev_local#1, with the table.)
- Several are a library, not an API. choosekit reads probabilities straight out of your own llama.cpp but ships no HTTP server; OpenThai-SystemOne and others speak
POST /v1/systemone, which existing SDK code reaches with a base-URL change.
If you would rather see a marker on every entry than a separate section, say so — the reason I did not is that we would be marking 200 projects from their READMEs, and README claims about endpoints have not survived contact with the code often enough for me to want to publish them unchecked.
- added a commit that references this issue
on Sep 21, 2026 Do you think that all the top Jev alternatives will agree on a certain API design such that there will be cross-compatibility? https://benchmarkheaven.com/jev-models
Short answer: they already have agreed, and the agreement is thinner than it looks. It holds for the one request everybody tests and falls apart everywhere else.
(On the link — we don't republish other people's measurements, so I won't quote anything from it. It's also a different axis: how good a model is and what its server's API looks like are unrelated, and the second one is the one you asked about.)
There is a de-facto standard, and it is TypeSafe's own route
POST /v1/systemone. Of the alternatives we have actually run, these serve it: Von, OpenThai-SystemOne, LLM2Jev and djev-run. For Von and OpenThai we pointed the officialtypesafe-sdkat them with nothing but a base-URL change and it worked. That is real cross-compatibility, and it happened without anyone writing a spec, for the ordinary reason: the SDK is the specification, and the body is what the SDK sends.It stops at the endpoint the SDK exercises
Everything around that one call diverges, and each case cost us a bug report:
Model listing. OpenThai has no
GET /v1/modelsat all, soclient.models.list()returns 404. Von had one, but returned the OpenAI shape —{"object":"list","data":[…]}— where the SDK expects{"models":[…]}; it now returns both shapes so either client works. Two implementations, two different failures, one endpoint. Anything that enumerates before it calls trips on this.What the
modelfield means. OpenThai echoes back whatever model name the caller sent, so the response cannot tell you what actually answered. You can point it at a model that isn't loaded and get a confident 200. (Same class of bug as choosekit's benchmark naming a model llama.cpp never ran — llama.cpp accepts unknown model names too.)How strict the response has to be. Driving djev-run's client, the Vercel AI SDK's TypeSafe provider rejects the entire response if a
scoreanswer omitsprobabilities. It also translates abooleanquestion into anoulon the wire and back again. Neither is written down anywhere obvious, and a hand-written server will omit that field and never notice, becausecurllooks fine.Errors. No two agree. This is where every "compatible" API ends up, and nobody's test suite covers it.
And a lot of them have no HTTP surface at all
Laya for Node is ONNX Runtime in-process — no Python, no server. open-alternative-jev is a library with no HTTP layer. choosekit talks to a llama.cpp or Ollama server you already run, and its unit is one question rather than a state with a question set. Open Jev serves
/decide, its own shape.For that group "cross-compatible" is the wrong question. What is portable there is the answer shape — a typed choice, a probability per option, a confidence — not the route. And that part has converged much more completely than the HTTP layer, because it is what the whole idea is about.
So, the prediction
Convergence on the request and response body: already happened, will hold, because the SDK enforces it and everyone tests it. Convergence on model listing, error shapes, streaming, batching and auth: no, not without someone writing a conformance suite, and probably not even then. That is the normal life of a de-facto standard — compare how many OpenAI-compatible servers implement chat completions faithfully versus how many implement
/v1/modelsthe same way.The two-line test
If you want to know whether a given server is actually compatible rather than approximately compatible, point the official SDK at it and call both:
client = TypeSafe(base_url="http://your-server/v1", api_key="...") client.models.list() # this is the one that fails client.system_one(state=..., questions=...) # this is the one everybody passes
The second call sorts the projects into "serves the route" and "doesn't". The first sorts them into "implemented the API" and "implemented the example".
Happy to add a wire-compatibility marker to the Running Without Jev table — something like speaks
/v1/systemone/ library only — since that is the split that actually decides whether you can swap something in. Say the word and I'll send it, tested the same way rather than read off READMEs.Reacted by TomLucidor
Since Jev is paid, there needs to be clearer delineation between those that can handle alternative models vs those that are Jev-exclusive