On this page
- What does the newer DecideBench run say?
- Which should you pick, Jev or Tev?
- At a glance: Jev and Tev side by side
- How did Jev and Tev do on our tests?
- Is Tev faster and cheaper than Jev?
- Can you trust their confidence scores?
- What is Tev1, and how was it made?
- How do frontier models compare on the same tasks?
- What are the limits of this test?
- FAQ
- Sources
Jev is TypeSafe's decision model: it picks one answer from your list, such as approve this refund or not. Tev is a small open model that Together AI trained, for about $17, to answer like Jev. On DecideBench, our public test of 400 decisions, Jev got 98.0% right and Tev 92.8%. DecideBench was updated on 9 October 2026. Tev is faster. Which costs less depends on your prompt.
Worked examples, meaning one solved example for each answer option sent with every question, make Tev's prompts much longer. Tev's input grows from about 246 to 1,195 tokens per request. Jev's grows only from 468 to 768. With them, Tev costs $50.20 per million decisions and Jev $32.26. Without them, Tev costs about half as much as Jev.
Need the right answer on close reading? Pick Jev. Need to run it yourself, or need speed on easy tasks? Pick Tev.
- Tev has open weights, so you can download and run it yourself. Together's launch post on 23 September 2026 gave the recipe but no scores.
- Jev is TypeSafe's decision model, launched on 15 September 2026 as a hosted service. It answers questions about text and writes no text.
- Accuracy: Jev 98.0%, Tev 92.8% on DecideBench. Our first run, without worked examples, had Jev 7 points ahead too.
- Pairs: each task comes with a twin that differs by one detail that flips the answer, such as a return asked for inside or outside a 30-day window. On DecideBench Jev got both twins right 96.0% of the time, Tev 86.0%.
- Speed: Tev answered in 197 milliseconds on Together, Jev in 639.
- Cost: both list $0.042 per million tokens read, with output free. Without worked examples Tev costs about half as much per decision. With them Tev costs more.
- Our first test cost $1.10. Its data and code are on GitHub.
What does the newer DecideBench run say?
Jev still wins, by a slightly wider margin, and Tev is no longer the cheaper one once you add worked examples. DecideBench is our own benchmark. It uses the same 400 tasks as the test below, item for item, and adds one change: every task now comes with worked examples.
| Accuracy | Both halves of a pair right | Accuracy without worked examples | Cost per million decisions | Typical reply time | |
|---|---|---|---|---|---|
| Tev on Together | 92.8% | 86.0% | n/a | $50.20 | 197 ms |
| Tev on your own GPU | 92.8% | 86.0% | 90.0% | $46.82 | 823 ms, no network |
| Jev | 98.0% | 96.0% | 98.2% | $32.26 | 639 ms |
Source: DecideBench v1.1, measured between 28 September and 9 October. Jev was called through AI Space, our gateway, and its reply time includes that hop.
The changelog and the published result files explain each change.
Worked examples lifted both models. Tev rose from 90.0% to 92.8%. Jev rose from 97.2% to 98.0%. Run again without worked examples, Tev scored 90.0% and gave the same answer on all 400 tasks as in September. Jev scored 98.2% without examples. It gave four new answers on the same Jev version, and all four were right.
Prices did not change; the prompts got longer. Both are still billed at $0.042 per million input tokens. Tev takes the worked examples as earlier turns of a chat. Jev takes them inside each answer's description, as TypeSafe documents. Tev's average task grew from 246 billed tokens to 1,195. Jev's grew from 468 to 768. So Tev's cost per million decisions rose to $50.20, and Jev's to $32.26.
Tev on your own GPU is a new row. It is priced by GPU time, not tokens. On 30 September we moved the self-hosted runs to one rented NVIDIA L4 graphics card at $0.81 an hour. The 400 tasks took 83 seconds, or $46.82 per million. Its reply time is measured on the GPU itself, so it does not line up with the hosted times.
Both replied more slowly than in September. The changelog records no cause.
Rule-following stayed Tev's weak spot. On approving an AI agent's actions, Tev scored 86% and Jev 100%. The two now miss the same 6 tasks, and three of those labels could be argued.
Together's model page now shows $0.04 per million input tokens. DecideBench prices Tev at the $0.042 from Together's launch announcement. At either price, Tev costs more than Jev once the worked examples go in.
DecideBench covers 20 decision models. Best Jev alternatives ranks them. Our own open model, Yev, scored 94.2%, between Tev and Jev; see Yev, an open source Jev alternative. Cloudflare's Clef has its own comparison in Clef vs Jev.
From the at-a-glance table on, this post reports our original 25 September test, run without worked examples. Its findings still hold for that setup.
Which should you pick, Jev or Tev?
Pick Jev when a wrong answer costs you something. Pick Tev when you need to host the model, need the fastest reply, or your tasks are easy.
Pick Jev if your task turns on one word: a "not", a date, a number, who is speaking. On our hard tasks Jev got 95.9% right and Tev 84.1%. Pick Tev if your data cannot leave your servers, you want to fine-tune your own version, or you sort plain requests where both models score near 100%. Keep a frontier model in reserve for the cases neither handles; see decision model vs LLM for that trade.
New to Jev? Start with what is Jev.
At a glance: Jev and Tev side by side
Jev and Tev each pick one answer from a list you give them, at the same list price. They differ in who runs them, how fast they answer and how often they are right. A token is a small chunk of text, about three quarters of a word.
| Tev | Jev | |
|---|---|---|
| Who makes it | Together AI | TypeSafe |
| Run it yourself? | Yes: open weights on Hugging Face | No: hosted API |
| List price per million tokens | $0.042 to read, output free | $0.042 to read, output free |
| Cost per million decisions, our test (no worked examples) | $10.33 | $19.64 |
| Cost per million decisions, DecideBench (with worked examples) | $50.20 | $32.26 |
| Typical reply time, our test | 173 ms | 459 ms |
| Accuracy, our test | 90.0% | 97.2% |
| How you call it | A normal chat call; it replies with one option letter | A choice question; it returns the option plus a probability for each |
Prices come from Together on X and TypeSafe's launch post; the rest comes from our results and DecideBench.
How did Jev and Tev do on our tests?
Jev got 389 of 400 tasks right and Tev got 360. The gap is too large to be chance.
| Accuracy | Both halves of a pair right | |
|---|---|---|
| Tev | 90% | 80% |
| Jev | 97.2% | 95% |
Source: Our benchmark, 25 September 2026, github.com/choyiny/jev-vs-tev
We wrote 400 new tasks in 8 families: support requests, return policies, agent routing, content moderation, review sentiment, fact checking, approving agent actions and bug triage. They come in 200 pairs. The two halves of a pair differ by one small edit that changes the right answer. Here is one pair. A jacket was delivered on 2 March, and the policy allows returns within 30 days. In one half the customer asks on 20 March, so the answer is a full refund. In the other they ask on 15 April, so the answer is to decline. A model that skims for topic words gets one half right and the other wrong.
Pair accuracy, the second bar, catches that skimming: Tev got both halves right on 80.0% of pairs, Jev on 95.0%.
Head to head
Jev and Tev were both right on 356 tasks and both wrong on 7. Jev alone was right on 33. Tev alone was right on 4. A standard test for paired results (McNemar's) puts the chance of a gap this lopsided at about one in a million (p = 1.08e-06).
Hard tasks widen the gap
About 40% of pairs are marked hard: negations, sarcasm, rule edge cases, misleading words.
| Easy (230 tasks) | Hard (170 tasks) | |
|---|---|---|
| Tev | 94.3% | 84.1% |
| Jev | 98.3% | 95.9% |
Source: Our benchmark, 25 September 2026, github.com/choyiny/jev-vs-tev
On easy tasks the two are 4 points apart. On hard ones, nearly 12.
Where Tev slipped most
Tev's three weakest families all ask it to apply a rule, not spot a topic:
- Approving agent actions: 82% (Jev 100%). An AI agent proposes a shell command, a database query or an email. The model must approve it, send it to a person or block it. The difference is often one flag, such as a test run marker removed from a command, or an outside email address.
- Content moderation: 84% (Jev 98%). Quoting abuse to report it is allowed; posting it is not. The words are the same.
- Review sentiment: 84% (Jev 98%). Five levels, with "but" clauses and sarcasm.
Tev scored 100% on support requests, and so did Jev. Each family has only 50 tasks, so treat these family scores as a direction, not a precise number. Tev's weights are public, so a team could train it further on its own weak families.
Where each model's mistakes land
Tev leans toward caution and misses the rarer answers. Jev's few misses sit in one place: counting days on return policies.
- Tev rarely asks a clarifying question when it should. Four tasks called for one; Tev asked once.
- Tev sends safe actions to a person. It approved only 62% of the safe agent actions and passed the rest to human review. That errs toward caution, which is the cheaper kind of mistake.
- Tev misses negative reviews. It picked "negative" on only 40% of the reviews that were negative.
- Jev's misses cluster in return-policy day counting. It declined only 71% of the returns that fell outside the window. When it offered a full refund it was right 88% of the time, and store credit 75%.
One balanced score sums this up. It rates each possible answer on its own, then averages them, so a model that ignores rare answers scores lower. Out of 100, Tev scored 88.5 and Jev 95.5. Some answers appear only a handful of times, so read those per-answer rates as a direction.
Is Tev faster and cheaper than Jev?
Faster, yes. Cheaper, only without worked examples. In our 25 September run, which sent none, Tev's typical reply came back 2.7 times faster, and it cost about half as much per decision. With worked examples, as in DecideBench, Tev costs $50.20 per million decisions and Jev $32.26.
| Typical (p50) | Slow end (p95) | |
|---|---|---|
| Tev | 0.173s | 0.215s |
| Jev | 0.459s | 0.954s |
Source: Our benchmark, 25 September 2026, github.com/choyiny/jev-vs-tev
Latency is the wait between sending a task and getting the answer. Half of Tev's replies took 173 milliseconds or less; for Jev it was 459. The slowest 1 in 20 replies show the spread. Tev's took up to 215 milliseconds, Jev's up to 954. Tev is also steadier. One caveat favours Tev: we reached Jev through our AI Space gateway, which adds a hop. We called Tev on Together directly.
Why does Jev cost more at the same price? Both list $0.042 per million tokens read, and neither charges for output. But for the same task, Jev billed 468 input tokens on average and Tev 246. TypeSafe likely wraps its own prompt around your text. So a million decisions cost $19.64 on Jev and $10.33 on Tev. Jev pricing works through bigger jobs.
Can you trust their confidence scores?
Yes, for both, and Jev's are closer to the truth. Calibration means a model's stated confidence matches how often it is right: when it says 90% sure, it should be right about 9 times in 10.
Jev returns a probability for every option. For Tev we read the probability from its first reply letter. Both scored well on the usual calibration measure, expected calibration error. Stated confidence missed the real hit rate by 2.0 points on average for Jev. For Tev the miss was 3.1 points. A second measure, the Brier score, also rewards being right more often, so the accuracy gap shows up there too: Jev 0.041, Tev 0.158, where lower is better. In practice, both let you send low-confidence answers to a person. Jev's threshold will hold more reliably. How to use Jev shows that setup.
What is Tev1, and how was it made?
Tev, released as Tev1-4B-experimental, is a 4 billion parameter open model that Together AI trained to answer like Jev. Together built it as a small add-on (a LoRA fine-tune) on top of Alibaba's Qwen3.5 4B.
Together trained it on 37,840 training examples. The run took about 25 minutes and cost about $17. Those training examples come from well-known public datasets of yes/no questions, bank support requests, news headlines and film reviews. Together added generated tasks on policies and routing. The recipe is on GitHub. You call it with temperature 0 and a limit of 8 output tokens, and it replies with one option letter. It still writes that letter the way a chat model does. It does not use Jev's method of answering many questions in one pass.
Because Tev learned from those public datasets, testing it on them would reward memory. So we wrote new tasks. Claude drafted all 400, and none come from those datasets. How to test a classifier explains the method.
How do frontier models compare on the same tasks?
Two large chat models scored higher still, at far greater cost. GLM-5.3 is an open model from Z.ai; it got 99.0%. Claude Opus 5.5 from Anthropic got 99.2%. They cost $809.08 and $1,707.74 per million decisions. Each reply took 2.5 to 2.9 seconds. We ran them only as a reference point.
You can get close to them for far less. Jev's mistakes cluster on a few answers, so we asked GLM-5.3 again only when Jev gave one of three answers it gets wrong most often. That setup reached 98.1% at $86 per million decisions. With Tev first, the best setup reached 97.1% and sent 39% of tasks to the larger model, because Tev's mistakes spread over many answers. LLM cascade walks through the method. The trade between a decision model and a chat model has its own post: decision model vs LLM.
What are the limits of this test?
Our first benchmark was one small test, run once, on one day. DecideBench re-ran the same 400 tasks a few days later, with worked examples. Read the numbers with these limits in mind:
- Claude wrote all 400 tasks, and the same kind of model checked the labels. We spot-checked them. A careful person could still dispute an item or two.
- 400 tasks is small. The headline gap is solid. Family scores rest on 50 tasks each and could move a lot.
- Jev's timing includes our gateway hop; Tev's does not. Timing also depends on region and load.
- We tested only pick-one questions. Jev's yes/no and score types have no Tev match here.
- The two APIs count tokens differently, which is why the same list price gives different costs.
- GLM-5.3 and Opus 5.5 ran through AI Space with their default reasoning settings.
Jev and Tev also missed the same 7 tasks (6 in DecideBench). We reviewed each; three could be argued. We kept every label as written. Full details are in the methodology.
FAQ
Is Tev1 as good as Jev?
No. On DecideBench, with worked examples, Jev got 98.0% of 400 tasks right and Tev 92.8%. Our first run, without worked examples, gave 97.2% against 90.0%. On its hard tasks the gap grew to 95.9% against 84.1%. Tev matched Jev on simple support requests. It fell furthest behind on tasks that apply a rule, such as approving an AI agent's proposed action.
Is Tev cheaper than Jev?
Only without worked examples. Both list $0.042 per million input tokens, with output free. Without worked examples Jev bills 468 input tokens per decision and Tev 246. A million decisions then cost $19.64 on Jev and $10.33 on Tev. With worked examples, Tev bills 1,195 tokens and Jev 768. Then Tev costs $50.20 per million decisions and Jev $32.26.
Can I run Tev on my own servers?
Yes. Together published Tev's weights on Hugging Face and the training recipe on GitHub. Jev is a hosted service only. If your data cannot leave your own servers, Tev is the one of the two you can use.
Did Together publish accuracy numbers for Tev?
No. Together's 23 September 2026 launch post explained how Tev was trained and how to call it, with no accuracy scores. Our test on 25 September is the first independent one we know of, and DecideBench now tracks Tev alongside 19 other decision models. The data and code are public so anyone can rerun it.
Should I use Tev or a frontier model?
Choose by how costly a wrong answer is. GLM-5.3 and Claude Opus 5.5 scored 99.0% and 99.2% on the same tasks. In that first run they cost 78 to 165 times more per decision than Tev and took over two seconds per reply. One setup runs Jev first and sends only its riskiest answers to a larger model; in our test that reached 98.1% at $86 per million decisions.
Sources
- Together AI, How to train your own Jev for $17, 23 September 2026
- Together AI on X, Tev serverless pricing, 23 September 2026
- Together AI, Tev1-4B-experimental model card, accessed 25 September 2026
- Together AI, tev1 training recipe, accessed 25 September 2026
- TypeSafe, Introducing System One Models and Jev, 15 September 2026
- Cloudflare, GLM-5.3 on Workers AI, accessed 25 September 2026
- Together AI, Tev1-4B-experimental model page, accessed 14 October 2026
- XY Space, DecideBench dataset and results, v1.1, accessed 14 October 2026
- XY Space, DecideBench leaderboard, accessed 14 October 2026
- XY Space, DecideBench changelog, 9 October 2026
- XY Space, Jev vs Tev benchmark: results and methodology, 25 September 2026
Written by
Cho Yin Yong
Principal AI Solutions Engineer, XY Space
Principal AI Solutions Engineer at XY Space. University of Toronto lecturer for five years, co-author of two patents, winner of two competitive AI awards, and nine years of regulated engineering leadership.
More from Cho Yin YongShare this article
Work with us
We build the systems these posts describe, and we'll tell you in the first call whether yours is worth building.
Start a project