Benchmark Heaven JevBench (Capability) Leaderboard
decision185 models · Updated October 2026
On the Benchmark Heaven JevBench (Capability) benchmark, Perplexity Decider v1.1 (27B) ranks #1 with a score of 82.7, while Sage 1.3.0 offers the best score-per-dollar at $10.00/1M output tokens. The full ranking, with cost per million tokens, is below.
💰 Best value
Sage 1.3.0 — score 78.6 at $10.00/1M output tokens
| # | Model | Score | ECE | p50 latency | Input / 1M |
|---|---|---|---|---|---|
| 1 | Perplexity Decider v1.1 (27B)Perplexity | 82.7 | — | — | $0.02 |
| 2 | vLLM-SR d3 (27B)vLLM Semantic Router | 82.2 | — | — | — |
| 3 | Quyet-1.0-Largechinhnc | 81.7 | — | — | — |
| 4 | Jeff-1.0-Largejgeuter | 81.5 | — | — | — |
| 5 | Torchcast Decision 27BTorchcast AI | 80.2 | — | — | — |
| 6 | metask-jev-rain-31B-trainedMeTask-Rain | 79.6 | — | — | — |
| 7 | decisio v0.8.0 on gemma-4-31B-itaminry | 79.5 | — | — | — |
| 8 | Kev 27BJared Palmer | 79.0 | — | — | — |
| 9 | Sage 1.3.0Levanto Labs | 78.6 | — | — | $0.05 |
| 10 | SPX-CD-ProSurdAI | 78.5 | — | — | — |
| 11 | amos-1-largeScira (mukaddamzaid) | 78.0 | — | — | — |
| 12 | bobcat-1.3sanghwa-na | 77.8 | — | — | — |
| 13 | deck31b (Gemma 4 31B)krishna-gogineni-765 | 77.6 | — | — | — |
| 14 | torchcast-decision-4bTorchcast AI | 77.5 | — | — | — |
| 15 | Eikos-27Bcaiovicentino1 | 77.2 | — | — | — |
| 16 | JevTypeSafe AI | 77.1 | — | — | $0.04 |
| 17 | Fastino GLiDEFastino | 77.0 | — | — | $0.15 |
| 18 | metask-jev-rain-12B-trainedMeTask-Rain | 76.9 | — | — | — |
| 19 | wity-1Wity (AlphaNimble) | 76.9 | — | — | — |
| 20 | Wald-26B-A4BORG2 AI | 76.8 | — | — | — |
| 21 | rene-1-31b-fp8salfatigroup | 76.0 | — | — | — |
| 22 | OpenJevrazorback16 | 75.8 | — | — | — |
| 23 | Cloudflare clefCloudflare | 75.3 | — | — | $0.04 |
| 24 | h2o-lightning-4bH2O.ai | 75.0 | — | — | — |
| 25 | simple-jev · Qwen3.8-27B (featherless)featherless-ai | 74.9 | — | — | — |
| 26 | Liquid d1 (hosted)Liquid AI | 74.5 | — | — | $0.02 |
| 27 | SPX-CD-FlashSurdAI | 74.3 | — | — | — |
| 28 | JEV-27BAutoTrust AI Lab | 73.8 | — | — | — |
| 29 | Mercury DecideInception | 73.6 | — | — | $0.00 |
| 30 | GPT-6 Luna Decisions (OpenAI Decisions API)OpenAI | 73.5 | — | — | $0.10 |
| 31 | Surogate Rune 26B-A4B v3Surogate | 73.5 | — | — | — |
| 32 | RSI-Jev v6.1 VL 27B (effort auto)shgao | 73.5 | — | — | — |
| 33 | bobcat-flash-1.2sanghwa-na | 73.0 | — | — | — |
| 34 | Autoloops – Gemma 4 31B ITAutoloops | 72.7 | — | — | — |
| 35 | swanOneblockbrain | 72.4 | — | — | — |
| 36 | Microsoft-Decision-1Microsoft | 72.0 | — | — | $0.04 |
| 37 | torchcast-decision-12bTorchcast AI | 71.7 | — | — | — |
| 38 | autojev-27bdenis-pplx | 71.6 | — | — | — |
| 39 | decider-12bMapika | 71.3 | — | — | — |
| 40 | vLLM-SR Decision 2.0 Vega 27BvLLM Semantic Router | 71.3 | — | — | — |
| 41 | Jev-Omniakhilaaa3 | 71.3 | — | — | — |
| 42 | Winnow-12BEldanRing | 71.3 | — | — | — |
| 43 | Xor 26B-A4B v1.1Juspay Technologies | 71.2 | — | — | — |
| 44 | Cygnetblockbrain | 70.9 | — | — | — |
| 45 | decisio v0.8.0 on gemma-4-12B-itaminry | 70.8 | — | — | — |
| 46 | JPT-35B-A3Bkirp | 70.7 | — | — | — |
| 47 | Instinct (ZooWork)ZooWork | 70.5 | — | — | — |
| 48 | Decider chat · Gemma-4-31BMapika | 70.4 | — | — | — |
| 49 | jebadiah-27bFrontier Infra | 70.3 | — | — | — |
| 50 | GEV-26B-DecideAutoTrust AI Lab | 70.0 | — | — | — |
| 51 | Blink v0.3 26B-A4B (NVFP4)Pixilab.ai | 70.0 | — | — | — |
| 52 | decisio v0.9.0 on gemma-4-12B-itaminry | 70.0 | — | — | — |
| 53 | NInfer Qwen3.8-Flash-Next mixedigorls | 69.7 | — | — | — |
| 54 | attest-31B-v1.0attest.engineering | 69.5 | — | — | — |
| 55 | Wald-4BORG2 AI | 69.3 | — | — | — |
| 56 | Xor 1.2Juspay Technologies | 68.9 | — | — | — |
| 57 | vLLM-SR d3-mini (4B)vLLM Semantic Router | 68.8 | — | — | — |
| 58 | SimpleJev Qwen3.6-35B-A3B (Featherless)Featherless AI | 68.3 | — | — | $0.00 |
| 59 | Hopper 12B (trained)HopitAI | 68.2 | — | — | — |
| 60 | SPX-CD-OmniSurdAI | 68.2 | — | — | — |
| 61 | NInfer Qwen3.8-27B NVFP4igorls | 68.0 | — | — | — |
| 62 | Open-Jev-27B-v1.1ZefanCai | 68.0 | — | — | — |
| 63 | Aplomb 1EmpirioLabs AI | 67.0 | — | — | — |
| 64 | Bespoke Nimble 9B v3Bespoke Labs | 66.8 | — | — | — |
| 65 | strands-decider-26B-A4B-gemma4-v1-2610Strands Agents | 66.7 | — | — | — |
| 66 | LitJevzhengxuyu | 66.6 | — | — | — |
| 67 | BevReza2kn | 66.2 | — | — | — |
| 68 | Decider 35B-A3BMapika | 66.2 | — | — | — |
| 69 | Metask-Jev-4BMetask-AI-Lab | 66.0 | — | — | — |
| 70 | shisa-de-1Shisa.AI | 65.5 | — | — | — |
| 71 | plumb-4bcrh225 | 65.3 | — | — | — |
| 72 | Decider 4BMapika | 64.8 | — | — | — |
| 73 | JevK5 v0.3alibiserikbay | 64.2 | — | — | — |
| 74 | JPT-4Bkirp | 63.9 | — | — | — |
| 75 | Cloudflare clef-flashCloudflare | 63.9 | — | — | $0.02 |
| 76 | Vansa-3.4Vansa | 63.7 | — | — | $0.03 |
| 77 | deck-4B v1.0krishna-gogineni-765 | 63.6 | — | — | — |
| 78 | frontier-infra Jebadiah 9B v2.1Frontier Infra | 63.5 | — | — | — |
| 79 | seb-9bironbcc | 63.1 | — | — | — |
| 80 | janus-4bcmxu | 62.9 | — | — | — |
| 81 | amos-1-mediumScira (mukaddamzaid) | 62.9 | — | — | — |
| 82 | messier-oneagentmessier | 62.5 | — | — | — |
| 83 | imajev-9bmohit67890 | 62.4 | — | — | — |
| 84 | jqvOctalab-Inc | 61.9 | — | — | — |
| 85 | imajev-4bmohit67890 | 61.9 | — | — | — |
| 86 | Open-Jev-9BZefanCai | 61.9 | — | — | — |
| 87 | Kahn1-Qwen3.5-4BOkura66 | 61.8 | — | — | — |
| 88 | levInterfaze | 61.7 | — | — | — |
| 89 | vLLM-SR Decision 2.0 Lux 9BvLLM Semantic Router | 61.6 | — | — | — |
| 90 | jeb-35b-a3bszybkie-ai | 61.6 | — | — | — |
| 91 | Kev 9B v2Jared Palmer | 61.5 | — | — | — |
| 92 | blink-4bthegovind | 61.4 | — | — | — |
| 93 | malkuth-4bdhtocks | 61.1 | — | — | — |
| 94 | qwen3.5-9b-jev-data-mix-v2jsaurabh | 60.9 | — | — | — |
| 95 | decision-4b-v1.2FlyMy.AI | 60.4 | — | — | — |
| 96 | APUS-Reflex-35B-A3Bapus-ailab | 60.4 | — | — | — |
| 97 | jiwo 4B v2eljiwo | 60.3 | — | — | — |
| 98 | PurpleMIST-Flash-1.0Olaverse Lab | 60.1 | — | — | — |
| 99 | Standard One 8B SH (v3)Standard Thinking | 60.0 | — | — | — |
| 100 | Quyet-1.0-Mediumchinhnc | 59.3 | — | — | — |
| 101 | decision-4bevalengine | 59.1 | — | — | — |
| 102 | reflex 4Bkshetrajna12 | 58.9 | — | — | — |
| 103 | openjev-sglang (Qwen3.6-35B-A3B)Eric Zhang | 58.7 | — | — | — |
| 104 | StandardOne-8BStandard Thinking | 58.4 | — | — | — |
| 105 | Cloudflare clef-omni (30B-A3B)Cloudflare | 58.4 | — | — | $0.15 |
| 106 | Manchego v2.1nschlaepfer | 58.3 | — | — | — |
| 107 | Diffusion Jev | 58.1 | — | — | — |
| 108 | jev-localus | 57.5 | — | — | — |
| 109 | Instinct Dual 4BZooWork | 57.2 | — | — | — |
| 110 | JEV-Qwen3.5-9B-Base-NVFP4WIlfLin | 57.1 | — | — | — |
| 111 | decision-4b-v1.1FlyMy.AI | 57.1 | — | — | — |
| 112 | uniphore-4b-deciderUniphore | 57.1 | — | — | — |
| 113 | jebadiah-9b-v2Frontier Infra | 57.0 | — | — | — |
| 114 | CoCo-Decision-4B-KoCorners | 57.0 | — | — | — |
| 115 | APUS-Reflex-9Bapus-ailab | 56.9 | — | — | — |
| 116 | typecastlm-qwen3.5-3.8bmihailgribov | 56.5 | — | — | — |
| 117 | Bespoke-Nimble-9BBespoke Labs | 56.5 | — | — | — |
| 118 | pjev-2bKailune-AI | 56.5 | — | — | — |
| 119 | Standard One 3B SH (v2)Standard Thinking | 56.0 | — | — | — |
| 120 | SemIfTheoLeeCJ | 55.4 | — | — | — |
| 121 | ArseneLupin-v1.1IBOU-SEARCH | 55.1 | — | — | — |
| 122 | local-jev Qwen3.5-4Bamithgc | 55.0 | — | — | — |
| 123 | decision-2b-previewFlyMy.AI | 54.9 | — | — | — |
| 124 | bongard-miniAgentBull | 54.8 | — | — | — |
| 125 | nomaBlackdromeAILabs | 54.4 | — | — | — |
| 126 | imajev-2bmohit67890 | 54.0 | — | — | — |
| 127 | spark-s1-4b-v6abhishek085 | 54.0 | — | — | — |
| 128 | RYOTIDE-Qwen9csabag | 52.5 | — | — | — |
| 129 | system-one-open (Gemma 4 E2B LoRA)mithalouni | 51.9 | — | — | — |
| 130 | classone-qwen3.5-9bdevops-thiago | 50.5 | — | — | — |
| 131 | kev-8bJared Palmer | 50.0 | — | — | — |
| 132 | Nev-2B-GGUFEryriLabs | 49.9 | — | — | — |
| 133 | LiquidAI d1-3BLiquid AI | 49.5 | — | — | — |
| 134 | JADEtheunnecessarythings | 49.3 | — | — | — |
| 135 | open-alternative-jevikermoel | 49.1 | — | — | — |
| 136 | OpenSourceJevsabeel111 | 48.5 | — | — | — |
| 137 | Open-Jev-2BZefanCai | 47.7 | — | — | — |
| 138 | EXAONE-4.0-1.2B-JEV v0.3carrtesy | 47.6 | — | — | — |
| 139 | malkuth-2bdhtocks | 47.6 | — | — | — |
| 140 | Nemotron Diffusion 8Bpst2154 | 47.5 | — | — | — |
| 141 | jiwo 0.8B v2eljiwo | 47.2 | — | — | — |
| 142 | decision-machine-1milliseconds.ai | 46.1 | — | — | — |
| 143 | decisor-4bunderlabs | 45.8 | — | — | — |
| 144 | Decider 2BMapika | 44.6 | — | — | — |
| 145 | LAYAConvai Innovations | 44.3 | — | — | $0.00 |
| 146 | Kev 4B r10Jared Palmer | 44.0 | — | — | $0.04 |
| 147 | lev-350mFranck Verrot | 43.4 | — | — | — |
| 148 | certo-decision-modelaltslate | 43.1 | — | — | — |
| 149 | PurpleMIST-Mini-1.0Olaverse Lab | 43.0 | — | — | — |
| 150 | smalljev semantic-v9isHeSatoshi | 43.0 | — | — | — |
| 151 | jul-decision-minicpm5-2busejul | 42.6 | — | — | — |
| 152 | qwen3.5-0.8B-decision-modelmghafiri | 42.6 | — | — | — |
| 153 | krino-2bGuru0381 | 42.4 | — | — | — |
| 154 | gutsykouhxp | 42.3 | — | — | — |
| 155 | watt-flash-0.1zaitlabs | 41.6 | — | — | — |
| 156 | decision-fast-previewFlyMy.AI | 40.6 | — | — | — |
| 157 | RLCD ModernBERT 151Mheman10x | 40.4 | — | — | — |
| 158 | Quyet-1.0-Small-ENchinhnc | 39.5 | — | — | — |
| 159 | classone-gemma4-e4bdevops-thiago | 38.4 | — | — | — |
| 160 | classone-qwen3.5-4bdevops-thiago | 38.0 | — | — | — |
| 161 | OpenDecisiondeepanwadhwa | 38.0 | — | — | — |
| 162 | verdict-smallManav2op | 37.6 | — | — | — |
| 163 | Kev 0.6BJared Palmer | 37.0 | — | — | — |
| 164 | vega-08b-public-intentsnandakishorm | 36.6 | — | — | — |
| 165 | WaterSheepsamratduttaofficial | 35.5 | — | — | — |
| 166 | open-jev DeBERTa-v3-largecom-kotobalabs | 35.4 | — | — | — |
| 167 | modernbert-jev-v2ksonny4 | 34.6 | — | — | — |
| 168 | GLiNER2.5-DecideFastino | 34.5 | — | — | — |
| 169 | kev-0.5bJared Palmer | 32.9 | — | — | — |
| 170 | LiquidAI d1-omni-600MLiquid AI | 32.8 | — | — | — |
| 171 | Quyet-1.0-Smallchinhnc | 32.5 | — | — | — |
| 172 | Quyet-1.0-Tinychinhnc | 31.1 | — | — | — |
| 173 | SimpleJevfeatherless-ai | 31.1 | — | — | — |
| 174 | tde-general-large-v0.1tdelab | 30.9 | — | — | — |
| 175 | Open Jev JSON CanvasJoshuaSP | 30.6 | — | — | — |
| 176 | Phocinae-Largha-150M-v1Phocinae | 30.3 | — | — | — |
| 177 | system-onesgoedecke | 29.8 | — | — | — |
| 178 | JevAct (jev1-2b-v2)einptein | 28.8 | — | — | — |
| 179 | Mirror | 25.8 | — | — | — |
| 180 | tacet-sonatacodepawl | 24.3 | — | — | — |
| 181 | CLM-v0.1-8BContrastive-LM | 20.1 | — | — | — |
| 182 | classone-qwen3.5-2bdevops-thiago | 19.2 | — | — | — |
| 183 | deem-0.8-v1LibertAIDAI | 18.1 | — | — | — |
| 184 | classone-gemma4-e2bdevops-thiago | 15.2 | — | — | — |
| 185 | Julia 1Supersonic Labs | 9.8 | — | — | — |
What does Benchmark Heaven JevBench (Capability) test?
Dataset is used to resolve references in a question, perhaps to multiple input positions, and perform discrete operations over them (such as addition, counting, or sorting).
Frequently asked questions
What is the Benchmark Heaven JevBench (Capability) benchmark?
Dataset is used to resolve references in a question, perhaps to multiple input positions, and perform discrete operations over them (such as addition, counting, or sorting).
Which model is best on Benchmark Heaven JevBench (Capability)?
Perplexity Decider v1.1 (27B) (Perplexity) currently leads the Benchmark Heaven JevBench (Capability) leaderboard with a score of 82.7.
What is the best value model on Benchmark Heaven JevBench (Capability)?
Sage 1.3.0 offers the best score-per-dollar on Benchmark Heaven JevBench (Capability) — a score of 78.6 at $10.00 per 1M output tokens.
Pricing is indicative — confirm with the provider before production use. Updated October 2026.