A small, measured experiment that let us trust a new class of model before we applied it anywhere else.
The TL;DR
We replaced our entire routing infrastructure that we used for our AI chat and content generation, including the context processing of conversation history, with Jev, a TypeSafe System One model that returns decisions instead of text. A small experiment showed the routing decision dropping from roughly 6 to 10 seconds of sequential processing to 300 to 500ms in one parallel pass, a roughly 12 to 20x gain in responsiveness and consistency, at significantly lower cost. It validated our decision to adopt the technology and gave us the confidence to apply Jev more broadly in our stack, starting with reranking.
What Is Jev?
Jev is a new class of model from TypeSafe, in a family they call System One. Where a normal LLM is a generator, producing free text, Jev is a decision function: you give it typed questions and structured state, and it returns a typed answer, a Choice, a Score, or a Noul (a boolean value encoding true or false), with a calibrated probability attached. It cannot invent a schema, because the schema is fixed. It cannot ramble, because it is not generating prose. It samples all outputs in parallel, and it is fast and cheap.
The potential is what drew us in. If a model can make a reliable decision, you can put it where you would otherwise write a hand-coded branch. The router of Mirror Ai's Multi Agent stack router was exactly that kind of place. But we were not going to rip out our routing infrastructure on a hunch. We ran a deliberately small experiment to prove the value first.
Why Bother?
Every question into Mirror Ai has to be routed to the right specialist agent before it can be answered. For a long time our routing was the weakest link in that path. It was an embedding model supported by a regular expression (regex) pattern matcher. As a final circuit breaker, we used a an LLM text model fallback when the first two components could not confidently determine user intent, to prevent a drop off in Mirror Ai response quality.
Overall, it worked well enough, but was slower and more expensive when we had to fall back to using the LLM. Moreover, it was brittle and sometimes inconsistent when challenged with questions where all of the existing router components were weak. This made quality enforcement and troubleshooting tricky, and performance tuning a nightmare... not a great situation.
Why do we even care at all?
Because we want users to experience a natural, multi-turn conversation with a highly capable AI system that provides a consistent user experience. After all, what's the use of creating an amazing Tech Stack if it can't deliver value to the users.
Therefore, we needed a routing layer that we could trust, regardless of the question, regardless of conversation context. And we wanted to be able to prove that any change we made was actually an improvement, not just a different set of mistakes.
The Technical Challenge
As mentioned before, our router was two layers. First, an embedding model scored how similar a question was to our example utterances, and if the score fell below a threshold, an LLM fallback took over. Embedding similarity can be brittle on ambiguous and elliptical follow-ups, depending on the quality of the model used and the semantics that it encoded. The LLM fallback, was slow, variable and non-deterministic. Two layers also meant two calls when we had to fall back, each with its own latency and failure mode.
On top of that, a regex fast-path was supposed to handle the most obvious cases. That was fine for direct queries, but it was bound to be hit and miss for situations where the regex lacked the structure and context to properly determine the user's intent, resulting in reduced response relevance and quality.
The deeper issue was that the work was sequential. The routing decision does not just pick a bucket. It has to understand what the user is asking in the context of the conversation so far. If someone says "How does it regulate crypto?", the router has to resolve what "it" means from the history, then decide which agent should answer. Our old path did that work by walking the conversation history step by step: context enrichment, then follow-up resolution, then the routing call, each as a separate step when the LLM fallback fired. That is slow by design, and it compounds.
What We Changed
To start with, we focused our Jev integration on routing only, there's no sense in going all in until the new tech proves itself in the stack. We replaced our entire routing infrastructure with a Semantic Router powered by the capabilities of jev.
One could say... we went "full Jev" ;)
One batched Jev call does context enrichment, follow-up resolution and routing together. That is the whole point. Instead of walking the conversation history step by step, Jev takes the typed state and the question and returns the routing decision in a single parallel pass. All of that activity that used to be an inherently sequential process now happens in one pass.
The regex fast-path stays enabled, so Jev only handles the ambiguous cases that need the full context, giving us the best of both worlds. The beauty of it was that we were able to achieve this with our existing AI model provider infrastructure, with minima additional configuration, since we previously embraced the flexibility of OpenRouter.
We kept our experiment small on purpose. A single-turn set of 27 questions and a multi-turn set of 10 conversations was enough to see whether the routing held up. The idea was to prove value cheaply before we considered the change anywhere broader.
The Results
Two summary tables are worth putting up front. The first is what Jev actually is, against a normal text model:
| A text model | Jev | |
|---|---|---|
| Output | Free text, can hallucinate | Typed decision: Choice / Score / Noul |
| Confidence | Not always | Always calibrated |
| Sampling | One token at a time | All outputs in parallel |
| Latency | 3 to 329 seconds | 70ms to 500ms |
| Input cost | $0.20 to $10 per MTok | $0.042 per MTok |
| Output cost | ~5x input | effectively free |
| Hallucination | Possible | unlikely by construction |
The second is the delta we measured between the old semantic and regex routing system and Jev:
| Metric | Before | With Jev | Delta |
|---|---|---|---|
| Multi-turn routing | 6/10 (60%) | 7/10 (70%) | +1 correct |
| Single-turn routing | 23/27 (85.2%) | 23/27 (85.2%) | no change |
| Routing latency (P95) | variable | 568ms tight | tighter spread |
| Input cost | $0.20 to $10 per MTok | $0.042 per MTok | far cheaper |
The multi-turn result is the one that matters most, because it is where routing usually fails. Full Jev correctly routed "Any recent news about it?" to news, which the hybrid system missed, and it held single-turn accuracy flat, so there was no regression in exchange for the gains.
Faster, and more consistent. The responsiveness gain is the headline. The old path was inherently sequential: context enrichment, follow-up resolution and routing each ran as a separate text-LLM pass, and routing-level text-LLM calls take seconds. Our own measurements put text-model latency from 3 up to 329 seconds. Jev does all of that work in one parallel pass, in 70 to 500ms. The table below is the arithmetic: a routing decision that used to take 6 to 10 seconds now resolves in a few hundred milliseconds. We measured roughly a 12 to 20x reduction in decision time, and the spread is far tighter, so responses are more predictable and timeout handling is easier.
| Routing decision | Old (sequential) | With Jev | Gap |
|---|---|---|---|
| Context enrichment + follow-up + routing | ~6 to 10 seconds | 300 to 500ms | ~12 to 20x |
Significantly cheaper. Cost follows the question and the conversation history you send. Jev charges $0.042 per million input tokens, and output is effectively free. A text-LLM router charges $0.20 to $10 per million input tokens, with output at about five times the input rate. For a fixed request, a question plus its context at around 1,500 input tokens and a routing decision of around 200 tokens, the old path costs up to a couple of cents depending on the model; Jev costs a few hundredths of a cent. On the per-token rate alone, Jev is roughly 5x to nearly 240x cheaper.
What We Learned
Our small experiment showed that we could get roughly 12 to 20x faster and more consistent routing using a more reliable system that was significantly cheaper. In comparison to our old routing strategy, it was an order-of-magnitude change in responsiveness and reliability at lower cost.
The juice was definitely worth the squeeze
The result also validated that Jev is worth the hype when used appropriately. It is not a general-purpose replacement for a text model. It is a decision function. Used that way, in the places where you want a reliable verdict rather than prose, it delivers.
That confidence is what let us take the next step. We are applying Jev more broadly in our tech stack, in places where we feel that it can maximise the quality and speed of Mirror Ai, and the next target was reranking, where the same decision-function framing applies to judging which retrieved evidence is actually useful.
References
- TypeSafe: "Introducing System One Models & Jev" blog
- TypeSafe: "SystemOne" docs
- Mirror Ai: Mirror Ai production routing, Jev routing in the chat and writer services. (Internal research)
- Mirror Ai: The Mirror Ai Stack: High-Level Architecture
