Jev vs. LLMs: Is the 200x speedup real?

TypeSafe AI claims Jev can make certain AI workflows dramatically faster and cheaper than traditional LLMs. But how much of that advantage holds up in practice? I put Jev to the test against several popular models using real bank customer support messages to compare their speed, cost, accuracy, and reliability.

In this article, we’ll explore how Jev works, how it compares with Claude and DeepSeek, and whether its performance claims make it a practical alternative to general-purpose LLMs for classifying and routing customer support messages.

What is Jev AI, and what does TypeSafe claim?

On September 15, TypeSafe AI released Jev and broke the internet. For days I read about nothing else, and honestly, at first I didn’t understand what it was or how to use it, because it’s different from the LLMs we’re used to.

First of all, you cannot talk with Jev as you’d do with Claude or ChatGPT; it doesn’t speak back or write code; it just decides. Jev is the master decision maker, the ruler, the judge.

TypeSafe calls Jev a “System One model”. It is built for the quick decisions an app makes thousands of times a day: which team should get this ticket, is this review spam, does this message need a human. You give it a question and the possible answers, and it picks one with a confidence score. Because it never writes free text, TypeSafe says it can’t hallucinate.

The headline claim is speed. The TypeSafe homepage says Jev is 193.6x faster and 444.6x cheaper than LLMs, measured on TypeSafe’s own workflows against the average of GPT-6 Astra and Fable 5.1. The launch post puts the range at 40x to 200x. I test the simpler case most apps have: one question, one answer. Input costs $0.042 per million tokens, and output is free.

Those numbers come from TypeSafe’s own tests, and the company admits “our number is not empirical.” So I tested it: a small Next.js app that sorts real bank customer messages with Jev, DeepSeek V4.1 Flash, and two Claude models, measuring accuracy, speed, cost, and reliability. Short answer: when it’s busy, Jev is about 10x faster than a frontier model. That’s far from 200x. After a few seconds of quiet, it’s no faster than Claude Haiku. The 200x shows up in the bill, and only against the frontier model.

How does Jev work? Choice, score, and noul explained

Jev is just fundamentally different from the LLMs we know today. When an angry customer sends a ticket, traditional code cannot understand or reason about the situation. Jev reads it for your code and answers one of three kinds of questions: choice, score, and noul.

message = "The app crashes every time I upload a photo."

questions = {
  "team":   Choice("Which team owns this issue",
                   billing / technical / sales),
  "anger":  Score("How angry the writer sounds",
                  Calm / Annoyed / Furious),
  "urgent": Noul("The message is time-sensitive"),
}

This is shortened from the TypeSafe quickstart. The real code is Python with a few more brackets.

Choice

Choice picks one answer from your list: is it billing, technical, or sales?

Imagine a mail room: a post office worker looks at each letter and drops it in one of the few bins. The worker does not read the letters or write back. It just decides one thing: Which bin?

Score

Score picks a level on a ladder: calm, annoyed, furious. Furious is more than annoyed.

Let’s think of a waiter reading the table. A good waiter glances at a table and knows immediately whether the customers are happy, impatient, or about to ask for the manager.

Noul

Noul tells you how likely a sentence is to be true. “This message is urgent”: how sure are we?

You want to check the forecast; it doesn’t say it definitely will rain, it says there’s a 60% chance of rain. Noul works in this manner.

Jev vs. LLMs: What’s the difference?

If we sent the user ticket to an LLM, it might’ve replied with something like “I think this is probably a technical issue because the user mentions…”, while Jev would give technical. A single word from our list.

An LLM can also, in theory, make up an answer. It could send this ticket to a support team we don’t have. Jev will tick one of the boxes we have given it: billing, technical, or sales; no hallucinating support team here.

So the case for Jev is simple: if your app makes the same kind of decision over and over, Jev promises to make it faster and cheaper than a general model, and always in a shape your code can use. Let’s see if that holds up with real messages.

How I tested Jev against LLMs using real bank messages

Enough theory, let’s see Jev in action. I built a small app that does one job: a bank customer writes in, and the app decides which of 77 support categories the message belongs to. Lost card, refund not showing up, wrong exchange rate, etc.

The messages are real. They come from BANKING77, a public dataset of 13,083 bank customer messages, each one already labelled with the right category. That label is our answer key, so I don’t have to judge who’s right myself.

Four sorters take the same exam:

• Jev, asked a choice question with the 77 categories as options
• DeepSeek V4.1 Flash, a cheap general-purpose model, the kind many developers reach for first
• Claude Haiku 5.5, a small and fast LLM, the one a developer would actually pick for a job like this
• Claude Fable 5.1, a frontier model, and one of the two models TypeSafe compared Jev against

Every sorter gets the same question, the same 77 categories, and the same message. No extra hints for anyone, and every model runs with its default settings.

There are two sets of messages. The main set is 25 random messages. The hard set is 10 messages from pairs of categories that mean almost the same thing, like card_arrival and card_delivery_estimate, or top_up_failed and top_up_reverted. Both sets went through every sorter twice, so I could see if anyone changed their answer.

The app has one text box and a Sort button. You paste a message, and the four answers show up side by side, with how long each one took and what it cost. Behind it, a small script runs the same messages through all four, one call at a time, and counts the right answers. I ran it from Istanbul. The LLMs go through OpenRouter, and Jev goes straight to the TypeSafe API.

The demo app is sorting one real BANKING77 message. Real recording: Jev had been idle, so this call took 1,008 ms.

The code, the messages, and my results are all in this GitHub repo, so you can run the same test with your own keys.

How LLMs classify customer support messages

With an LLM, you sort by asking nicely. You write a prompt, put the 77 categories in it, and ask the model to answer with one name only:

const SYSTEM_PROMPT = `${QUESTION}
Reply with exactly one category name from this list and nothing else:
${labels.join("\n")}`;

The same prompt goes to DeepSeek, Haiku, and Fable. The model can still answer with a full sentence, add a full stop, wrap the answer in quotes, or come up with a category that isn’t on the list. So my code cleans up the answer and checks it against the list. If the answer isn’t one of the 77, it counts as wrong.

How Jev classifies customer support messages

We hand Jev the 77 categories as the options of a choice question, and that’s it:

const result = await typesafe.systemOne({
  state: message,
  questions: { category: choice(QUESTION, criteria) },
});

result.answers.category.choice;     // "card_payment_wrong_exchange_rate"
result.answers.category.confidence; // 0.99

We don’t need cleaning; we don’t need checking. Jev can only answer with one of the names we gave it, and it tells us how sure it is. Remember the mail room? This is the worker with 77 bins.

Jev vs. LLMs: Accuracy, speed, cost, and reliability

Here’s what happened with 25 real customer messages, each one sent twice:

How fast is Jev compared with LLMs?

When Jev gets calls back-to-back, it is the fastest by far: about 10x faster than Fable, about 4x faster than DeepSeek, and about 3.5x faster than Haiku. That’s fast, but it’s far from 200x.

Then I noticed something odd. In my first mixed run, where each message went to all four sorters in turn, Jev’s median was 1,095 ms, slower than Haiku. So I tested it on its own. With a pause of 3 seconds or less between calls, Jev answered in about 300 ms. With a pause of 4 seconds or more, every call took about 1,050 ms, and some took 2 seconds. A busy app gets the fast Jev. A quiet one gets a Jev that is about as fast as Haiku.

Real medians, slowed down 3x. Jev, Haiku, and DeepSeek ran back-to-back on 25 messages; “after a pause” is Jev after 4 to 15 seconds idle.

Does Jev really cost 200x less than an LLM?

Where the 200x does show up is the bill. Jev costs about 200 times less than Fable. Against the small models, though, the savings were only about 2x (Haiku) and 4x (DeepSeek), because small models are already cheap.

DeepSeek surprised me here. It’s the cheapest provider on OpenRouter, at $0.048 per million input tokens, almost the same as Jev’s $0.042. But OpenRouter spread my calls across 10 different providers, most of them pricier, and DeepSeek thinks for about 190 tokens before it answers by default. So it ended up costing more than Haiku.

Is Jev as accurate as general-purpose LLMs?

Accuracy was too close to call. Fable got 44, DeepSeek 43, Jev 42, Haiku 41. With 50 answers, one answer is worth two points, so these four are tied.

And the funny part: two of the wrong answers were the same for all four sorters. “Can I have a non-virtual card?” got get_physical_card from everyone, while the answer key says order_physical_card.

How consistent is Jev compared with LLMs?

Jev never changed an answer between runs, and neither did Fable. Haiku and DeepSeek each changed one in the main set.

The hallucination problem from earlier didn’t really show up: no model made up a category. But one slip did happen. For “How can I transfer money to my account?”, the right category is transfer_into_account, all lowercase. In the first run, Haiku answered Transfer_into_account, with a capital T. That isn’t exactly one of the 77 names, so my code counted it wrong. In the second run, Haiku got it right. The prompt told it to reply with a name from the list, and it still bent that once.

Jev’s confidence score is also useful. On “I received my new card, but I don’t see it in the app anywhere”, Jev got it right but said it was only 55% sure, then 50% the second time. Haiku got that message wrong both times. Overall, Jev’s right answers averaged 93% confidence and its wrong ones 79%. A low score is your signal to send the message to a human.

How do Jev and LLMs handle ambiguous messages?

The hard set is where the categories overlap. Here are four of the ten, from the first run:

Over both runs, Haiku got 17 of 20, Fable 16, DeepSeek 15, and Jev 14. Jev’s only miss that the others didn’t share was the friend transfer, and it was less sure about that one (66%). The other misses were the answer key’s fault more than the models’. Nobody would call “So what about the card PIN?” a request for a physical card.

DeepSeek also had the worst moment of the whole test here: one call hung until my 2-minute timeout ran out.

So on truly ambiguous messages, the general models did a little better. Ten messages is a small test, so treat it as an early sign. Still, it’s worth knowing before you hand Jev your trickiest decisions.

When should you use Jev instead of an LLM?

Jev decides; it doesn’t write. If you need a reply to the customer, a summary, or some code, you still need an LLM. A good setup is Jev sorting every message in milliseconds, and an LLM writing a reply only for the ones that need it.

Jev also needs a fixed list of answers. If you don’t know the possible answers in advance, it’s the wrong tool. A choice question can have up to 255 options, which is plenty for most apps, but it’s still a list.

A general model is the better pick when the categories overlap a lot, and you want the model to weigh the details, as in the hard set. It is also fine when your traffic is quiet: a Jev that waits seconds between calls is about as fast as Haiku.

And if your volume is small, the money doesn’t matter much. My whole two-run test on Fable cost 43 cents. The savings only become real when you sort thousands of messages a day.

Should you switch from LLMs to Jev?

So, is the 200x speedup real? In my test, no. Busy Jev was about 10x faster than a frontier model and about 3.5x faster than a small one, and after a pause it was no faster than Haiku. The 200x showed up only in the price, and only against Fable.

Should you switch then? If your app makes the same kind of decision over and over, like sorting tickets, flagging spam, or routing messages, Jev is faster, cheaper, and just as consistent. If you need words, or your categories blur into each other, keep a general model in the loop.

I use Jev every day, mixed into my workflow wherever a quick decision is needed. I’ve been on the $5 free quota TypeSafe gave at launch, and it still hasn’t run out.

Three takeaways:

• Test the claim yourself: a vendor’s benchmark is only a starting point. Twenty-five real messages told me more than the homepage did
• Speed and price are different stories: Jev’s speed lead was single digits, and it shrinks when your app is quiet, but its price lead against a frontier model was 200x
• Use the confidence score: it’s free with every answer, and it tells you which messages need a human

The repo is public, so clone it, swap in your own messages, and see what Jev does with them.