On this page
- Which open source System One models did DecideBench v2 test?
- What is the best open source System One model?
- What is the best small open source System One model for one GPU?
- How did Cloudflare's Clef weights do?
- Which open source System One models should you skip?
- What does self-hosting a System One model cost?
- How do you run an open source System One model?
- Open source or hosted: which should you pick?
- FAQ
- Sources
Open source System One models are decision models you download and run on your own hardware. On 9 November 2026, DecideBench v2 ran 19 of them on one graphics card, an NVIDIA RTX PRO 6000. Perplexity's pplx-decider 27B came top. It got 92.2 of every 100 decisions right. Drex Max, the best hosted API, got 94.4.
If you need the most accurate model you can run yourself, pick pplx-decider v1.2 27B.
If you want a small model that fits beside other work on one GPU, start with yev1-4b or pplx-decider-flash v1.2 4B.
If running cost matters most and you can accept about 1 wrong answer in 4, pick OpenJev Flash 9B.
- 19 open-weight decision models ran on one GPU, all on the same 450 decisions, on the same day.
- One open model, pplx-decider 27B, beats Jev (86.2%). Jev is TypeSafe's hosted model, the one that started the category. Cloudflare's Clef 27B lands 0.9 points under Jev.
- Seven of the 19 score below 45%. Four are close to guessing.
- Self-hosted cost here is GPU time at $3.36 an hour with 4 requests in flight. A busier server costs less per decision.
- Yev (yev0-4b, yev1-4b) and DecideBench are made by XY Space, the publisher of this post.
Which open source System One models did DecideBench v2 test?
All 19 are in the table below, ranked by accuracy. "Pair accuracy" needs a word first. DecideBench's 450 decisions come in 225 pairs. The two halves of a pair differ by one small edit that changes the right answer, such as a file attached to the chat instead of a database table. Pair accuracy counts the pairs where the model got both halves right. A model that matches on topic words scores well on accuracy and badly on pairs.
| Model | Maker | Size | Accuracy | Pair accuracy | Cost per 1M decisions | Median latency |
|---|---|---|---|---|---|---|
| pplx-decider v1.2 27B | Perplexity | 27B | 92.2% | 86.2% | $386.34 | 1,528 ms |
| Clef 27B | Cloudflare | 27B | 85.3% | 72.4% | $216.42 | 928 ms |
| yev1-4b (ours) | XY Space | 4B | 79.3% | 62.7% | $71.15 | 275 ms |
| pplx-decider-flash v1.2 4B | Perplexity | 4B | 78.2% | 62.2% | $125.34 | 284 ms |
| yev0-4b (ours) | XY Space | 4B | 78.2% | 60.0% | $162.63 | 408 ms |
| Bespoke-Nimble-9B | Bespoke Labs | 9B | 75.6% | 56.0% | $135.67 | 507 ms |
| OpenJev Flash 9B | OpenJev | 9B | 75.3% | 55.6% | $41.33 | 174 ms |
| TEV (Tev1-4B-experimental) | Together AI | 4B | 74.9% | 54.7% | $81.88 | 306 ms |
| imajev-4b | individual (mohit67890) | 4B | 74.0% | 53.8% | $57.59 | 248 ms |
| JevK5 v0.3 | individual (alibiserikbay) | 4B | 68.0% | 44.0% | $73.02 | 310 ms |
| Clef-Flash 9B | Cloudflare | 9B | 66.9% | 45.8% | $65.42 | 273 ms |
| Decider-4B | individual (Mapika) | 4B | 63.6% | 42.2% | $63.18 | 254 ms |
| Kev-4B | individual (jaredpalmer) | 4B | 41.8% | 17.8% | $100.81 | 374 ms |
| GLiNER2.5-Decide | Fastino | 340M | 40.7% | 7.6% | $94.79 | 432 ms |
| Laya typed-decisions | individual (convaiinnovations) | 421M | 39.1% | 10.7% | $13.96 | 58 ms |
| Kev-9B | individual (jaredpalmer) | 9B | 35.8% | 15.1% | $123.46 | 466 ms |
| Decider-2B | individual (Mapika) | 2B | 31.8% | 10.2% | $28.19 | 115 ms |
| Julia-1 | Supersonic Labs | 144M | 26.9% | 2.2% | $9.13 | 39 ms |
| CLM-v0.1-8B | individual (Contrastive-LM) | 8B | 26.4% | 1.8% | $16.65 | 67 ms |
Random guessing scores 25.1%. Every row ran on the same GPU, and latency was timed on that machine with no network in it. Makers in the "individual" rows are the Hugging Face or GitHub account that publishes the weights; DecideBench lists no company for them.
The decisions cover nine everyday jobs, such as approving an AI agent's proposed action, ruling on a return, gating a staged software release and triaging a bug report. Each one arrives in the form it would take in production, such as a raw email with headers or a release controller's metrics. If the category is new to you, read what is a System One model first, and the full ranking for the hosted models too.
| Accuracy | Both halves of a pair right | |
|---|---|---|
| pplx-decider 27B | 92.2% | 86.2% |
| Clef 27B | 85.3% | 72.4% |
| yev1-4b (ours) | 79.3% | 62.7% |
| pplx-decider-flash 4B | 78.2% | 62.2% |
| OpenJev Flash 9B | 75.3% | 55.6% |
| TEV 4B | 74.9% | 54.7% |
Source: DecideBench v2, measured 9 November 2026
What is the best open source System One model?
pplx-decider v1.2 27B is the best open source System One model on DecideBench v2. It got 415 of 450 decisions right. That is 10 fewer than Drex Max and 27 more than Jev. No self-hosted model is both cheaper and as accurate.
pplx-decider 27B is strong almost everywhere. It got 98% or better on agent actions, tool routing and claim checking, and every content moderation and customer intent decision right. Its weak spot is one every decision model shares: gating a staged software release. There the answer needs a computed crash-free rate, checked against an ordered list of rules. pplx-decider scored 64% there. Claude Opus 4.8, the reference LLM the benchmark was built against, scored 100%.
Running pplx-decider takes one extra step. Perplexity ships pplx-decider as a Python library, not a server. To test it like the others, we wrote `tools/pplx/serve.py`, which puts it behind Jev's API and scores queued requests together in one call. Batching changed the bill more than the score:
| pplx-decider 27B served | Accuracy | Cost per 1M decisions | Median latency |
|---|---|---|---|
| One request at a time | 92.4% | $875 | 3,443 ms |
Batched (serve.py) | 92.2% | $386 | 1,528 ms |
Batching cut cost and latency by more than half for one decision less in 450. If you deploy it, batch. What is pplx-decider covers the model itself.
What is the best small open source System One model for one GPU?
yev1-4b is the most accurate of the small models, at 79.3% for $71.15 per million decisions. XY Space makes it, so weigh that. pplx-decider-flash 4B is 5 decisions behind, too few to call a real gap.
| Small model | Accuracy | Cost per 1M decisions | Median latency |
|---|---|---|---|
| yev1-4b (ours) | 79.3% | $71.15 | 275 ms |
| pplx-decider-flash v1.2 4B | 78.2% | $125.34 | 284 ms |
| OpenJev Flash 9B | 75.3% | $41.33 | 174 ms |
| imajev-4b | 74.0% | $57.59 | 248 ms |
The difference between yev1-4b and pplx-decider-flash is cost. On this run yev1-4b cost 57% of what pplx-decider-flash did, at about the same speed.
OpenJev Flash 9B is the cheapest usable open model on the list: $41.33 per million decisions and 174 ms, the fastest median of any model above 70%. It got 75.3% right. Check its licence before you build on it (see the FAQ).
A small model gives up most of its points on the hard families. yev1-4b scored 42% on release gating and 70% on bug triage, against 64% and 90% for pplx-decider 27B. On routing an agent to a tool, it was 4 points behind.
How did Cloudflare's Clef weights do?
Cloudflare's published Clef weights scored 85.3% (Clef 27B) and 66.9% (Clef-Flash 9B) self-hosted. Through AI Space, XY Space's platform, the same model names scored 68.4% and 57.6% on these items. So v2 reports the published weights.
The published weights scored 16.9 points higher for Clef 27B and 9.3 for Clef-Flash. We do not know what the hosted path serves differently. What we can say is that the weights on Hugging Face are the version that scored higher, and they are what you get when you run Clef yourself. Clef 27B lands 4 decisions short of Jev on the same items. Clef vs Jev has the older, hosted comparison.
Which open source System One models should you skip?
On this test, skip the seven models below 45%. Four of them, Laya, GLiNER2.5-Decide, Julia-1 and CLM, are encoder-style or contrastive models that scored 26–41% where random guessing scores 25.1%. Kev-4B, Kev-9B and Decider-2B land in the same band, at 32–42%.
A low score here does not mean the model is useless for the job its makers built it for. Their own cards describe narrower work:
- Laya typed-decisions is fine-tuned on four workflows named on its card: agent-trace observability, customer service, invoice processing and security incidents, with a 1,024-token context.
- Julia-1 is, in Supersonic Labs' words, "the first released test of our training system". Its card reports results on short-label classification sets such as AG News. It answered in 39 ms here.
- Kev's card lists training sets such as Banking77 and AG News, which are short texts with fixed labels. DecideBench's inputs are long payloads with a written policy.
- Decider-2B: we ran v11. Its repository now holds v12, dated 20 October 2026, which we have not tested.
Three of these (Laya, Julia-1, CLM) run without worked examples, because they have no place to take them. If your decisions look like what a card describes, test the model on a few hundred of your own.
What does self-hosting a System One model cost?
Self-hosting cost here is GPU time, not a per-token price. DecideBench timed each model on all 450 decisions at 4 requests in flight, then priced that time at $3.36 an hour, the on-demand rate for an AWS g7e.2xlarge with one RTX PRO 6000.
Worked through for pplx-decider 27B: the 450 decisions took 186 seconds, so one hour on the card handles about 8,700 decisions. At $3.36 an hour, that is $386 per million. OpenJev Flash 9B took 20 seconds for the same 450, about 81,000 an hour.
Three things move that number:
- Load. Each model ran alone with 4 requests in flight. A server kept busier, or one that batches harder, costs less per decision.
- What you pay for the GPU. A reserved card, or one you already own, costs less than on-demand.
- The server. Batching halved pplx-decider's cost on the same card.
Compare that with hosted list prices: Drex Max cost $91 per million decisions on the same items, Jev $75.04 and s1-fast $29.82. At this load, only Laya, Julia-1, CLM and Decider-2B cost less per million than s1-fast, and none of them scores above 40%. Self-hosting buys control first and price second. System One model pricing has the hosted side.
How do you run an open source System One model?
Download the weights from Hugging Face and put them behind a server. Only Clef ships one that speaks Jev's /v1/systemone API; the others come with their own server, a chat endpoint, or a Python library, so a thin wrapper keeps your client code the same whichever model answers. These are the repositories DecideBench ran, each checked on 9 November 2026:
| Model | Weights | How it was served in v2 |
|---|---|---|
| pplx-decider v1.2 27B | perplexity-ai/pplx-decider-v1.2 | Perplexity's library behind our batching wrapper |
| pplx-decider-flash v1.2 4B | perplexity-ai/pplx-decider-flash-v1.2 | same wrapper |
| Clef 27B | Cloudflare/clef | SGLang (lmsysorg/sglang:dev-clef) at /v1/systemone |
| Clef-Flash 9B | Cloudflare/clef-flash | same SGLang image |
| yev1-4b, yev0-4b | choyiny/yev1-4b, choyiny/yev0-4b | yev serve (vLLM backend for yev1) |
| OpenJev Flash 9B | openjev/OpenJev-Flash-9B | vLLM 0.29 in FP8 plus the repo's helper/shim.py |
| TEV | togethercomputer/Tev1-4B-experimental | vLLM chat endpoint |
| Bespoke-Nimble-9B | bespokelabs/Bespoke-Nimble-9B | DecideBench's tools/nimble/serve.py |
| imajev-4b | mohit67890/imajev-4b | the imajev server |
| JevK5 v0.3 | alibiserikbay/JevK5 | the JevK5 runtime |
| Decider-4B, Decider-2B | Mapika/decider-4b, Mapika/decider-2b | decider.serve |
Every server command and setting is in the DecideBench self-hosting notes. Clef's card gives a Docker command for SGLang that serves /v1/systemone directly. If you would rather train one for your own decisions, how to build a decision model walks through it.
Open source or hosted: which should you pick?
Pick open weights when the data has to stay on your hardware. Pick a hosted API when accuracy per dollar and latency matter most. The gap at the top is now small: pplx-decider 27B is 2.2 points behind Drex Max.
| pplx-decider 27B, self-hosted | Drex Max, hosted | |
|---|---|---|
| Accuracy | 92.2% | 94.4% |
| Pair accuracy | 86.2% | 89.3% |
| Cost per 1M decisions | $386.34 (GPU time) | $91.00 (list price) |
| Median latency | 1,528 ms, no network | 258 ms, network included |
| Where the data goes | Your GPU | Nace.AI |
At this load Drex Max costs about a quarter as much and answers about six times faster, and its latency already includes the trip over the network. A busier pplx-decider server closes some of the cost gap, not all of it. What the open model buys is that every state, policy and customer email stays on a machine you control, and nobody can change or retire the model under you. Decision Models retired its hosted Quyet-1.0-Large on 3 November 2026; open weights stay where you put them.
AI Space, XY Space's platform, hosts Jev and open models on Western infrastructure, and Jev's v2 numbers were measured through it.
FAQ
Can I run a System One model locally?
Yes. All 19 models in this post have downloadable weights and ran on a single NVIDIA RTX PRO 6000 in DecideBench v2. Cloudflare's Clef ships a Jev-compatible /v1/systemone server; the rest need their own server or a thin wrapper, like the ones in the DecideBench repo. The most accurate, pplx-decider v1.2 27B, got 92.2% right. Looking for a local Jev alternative specifically? See our Yev write-up.
What GPU do I need for an open source decision model?
We ran every model on one NVIDIA RTX PRO 6000, which has 96 GB of memory on AWS's g7e.2xlarge. The model cards give their own needs: pplx-decider v1.2 asks for a CUDA GPU with room for about 52 GiB of 16-bit weights plus working memory, and pplx-decider-flash about 9 GiB. OpenJev Flash 9B is about 19 GB at 16 bits and is served in FP8. Clef's card says it was tested on a single H200.
Are open source decision models as good as Jev?
One is better. On DecideBench v2, pplx-decider v1.2 27B got 92.2% right against Jev's 86.2%, which is 27 more decisions in 450. Cloudflare's Clef 27B is 0.9 points under Jev, four decisions, which is within noise. Every other open model scored below 80%. Jev was called through AI Space for these runs.
Which open source System One model is on Hugging Face?
All the usable ones are. pplx-decider and its flash version are under perplexity-ai, Clef under Cloudflare, OpenJev Flash under openjev, TEV under togethercomputer and Yev under choyiny. The table in the "How do you run" section links every repository DecideBench ran. CLM is published on GitHub as Contrastive-LM/CLM.
What licence do open source System One models use?
Most of the ones we checked use Apache 2.0, which allows commercial use: pplx-decider v1.2, pplx-decider-flash v1.2, Clef, Clef-Flash, Yev, Bespoke-Nimble-9B, imajev-4b, JevK5 and Decider. OpenJev Flash 9B's card says CC BY-NC 4.0, a non-commercial licence. TEV's card lists no licence. Read the card for the exact version you download.
Sources
- DecideBench, "DecideBench v2 results, method and limitations", 9 November 2026: github.com/choyiny/decidebench
- DecideBench, "Changelog, v2.0", 9 November 2026: CHANGELOG.md
- DecideBench, leaderboard and dataset, 9 November 2026: leaderboard, dataset
- Perplexity, "pplx-decider-v1.2" model card, updated 19 October 2026: huggingface.co/perplexity-ai/pplx-decider-v1.2
- Perplexity, "pplx-decider-flash-v1.2" model card, updated 19 October 2026: huggingface.co/perplexity-ai/pplx-decider-flash-v1.2
- Cloudflare, "Clef" model card, updated 30 October 2026: huggingface.co/Cloudflare/clef
- OpenJev, "OpenJev Flash 9B" model card, updated 7 October 2026: huggingface.co/openjev/OpenJev-Flash-9B
- Amazon Web Services, "Amazon EC2 G7e instances", accessed 9 November 2026: aws.amazon.com/ec2/instance-types/g7e
Written by
Cho Yin Yong
Principal AI Solutions Engineer, XY Space
Principal AI Solutions Engineer at XY Space. University of Toronto lecturer for five years, co-author of two patents, winner of two competitive AI awards, and nine years of regulated engineering leadership.
More from Cho Yin YongShare this article
Work with us
We build the systems these posts describe, and we'll tell you in the first call whether yours is worth building.
Start a project