Key Info
OpenRouter tested TypesafeAI's Jev decision model against four popular LLMs on a tightly scoped classification task: labeling incoming requests as one of 30 task types across 200 identical cases. Jev was more than 5x faster than the next fastest model, came in second cheapest, and matched the accuracy of the other models.
Highlights
- All five models were evaluated sequentially and statelessly on the same 200 cases, a realistic production-style setup where latency and cost matter.
- Jev was over 5x faster than the next fastest model and was the second cheapest option, just behind Qwen3.8 Flash and under the other three LLMs.
- Accuracy did not suffer for speed: Jev matched the most popular LLMs used for classification, with all five models landing within a handful of cases of each other.
- The result supports Typesafe's thesis that decision models can be fast, accurate, and cheap for narrow, high-frequency production tasks.