Spaces:
Running
Running
Download index.html from TorkeHub/torki-2-benchmarks: direct link, hf CLI and curl.
- Browser
- Download file 69.2 kB
-
https://huggingface.co/spaces/TorkeHub/torki-2-benchmarks/resolve/main/index.html
- Command line
-
hf download hf://spaces/TorkeHub/torki-2-benchmarks/index.html
-
curl -L -o index.html https://huggingface.co/spaces/TorkeHub/torki-2-benchmarks/resolve/main/index.html
69.2 kB
| <html lang="en"><head><meta charset="utf-8"> | |
| <meta name="viewport" content="width=device-width, initial-scale=1"> | |
| <title>Torki 2 Benchmarks</title> | |
| <meta name="description" content="Torki 2 on 22 public benchmarks, compared with models from Anthropic, OpenAI, Google, Meta, xAI, DeepSeek and Sarvam."> | |
| <link rel="preconnect" href="https://fonts.googleapis.com"><link rel="preconnect" href="https://fonts.gstatic.com" crossorigin> | |
| <link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Inter:opsz,wght@14..32,400;14..32,500;14..32,600;14..32,700&display=swap"> | |
| <style> | |
| :root{--bg:#000;--fg:#f4f4f1;--mid:#a1a19c;--low:#62625e;--rule:#222220;--rule2:#33332f;--mark:#56564f;--white:#fff; | |
| --f:"Inter",system-ui,-apple-system,"Segoe UI",Roboto,sans-serif;color-scheme:dark} | |
| *{box-sizing:border-box}html{scroll-behavior:smooth}@media (prefers-reduced-motion:reduce){html{scroll-behavior:auto}} | |
| body{margin:0;background:var(--bg);color:var(--fg);font:16px/1.6 var(--f);font-feature-settings:"tnum" 1,"cv11" 1;-webkit-font-smoothing:antialiased} | |
| a{color:inherit}a:focus-visible,summary:focus-visible{outline:2px solid var(--white);outline-offset:3px} | |
| .w{max-width:1180px;margin:0 auto;padding-inline:24px} | |
| .top{position:sticky;top:0;z-index:10;background:rgba(0,0,0,.86);backdrop-filter:blur(10px);border-bottom:1px solid var(--rule)} | |
| .top .w{display:flex;align-items:center;gap:28px;height:56px;overflow-x:auto;scrollbar-width:none} | |
| .top .brand{font-weight:600;letter-spacing:-.01em;text-decoration:none;white-space:nowrap} | |
| .top nav{display:flex;gap:22px;margin-left:auto}.top nav a{color:var(--mid);text-decoration:none;font-size:14px;white-space:nowrap} | |
| .top nav a:hover{color:var(--fg)} | |
| .hero{padding-block:88px 0} | |
| .kicker{color:var(--mid);font-size:15px;margin:0} | |
| h1{font-size:clamp(56px,10.5vw,148px);line-height:.92;letter-spacing:-.055em;font-weight:600;margin:14px 0 0} | |
| .lede{max-width:640px;font-size:19px;line-height:1.55;color:var(--mid);margin:28px 0 0} | |
| .lede strong{color:var(--fg);font-weight:500} | |
| .strip{margin:64px 0 0;padding:24px 0 0;border-top:1px solid var(--rule2)} | |
| .strip ul{list-style:none;margin:0;padding:0} | |
| .strip li,.axis{display:grid;grid-template-columns:200px minmax(0,1fr) 88px;align-items:center;gap:20px} | |
| .strip li{height:46px} | |
| .sb{font-size:15px}.rk{text-align:right;font-size:14px;color:var(--mid)} | |
| .track{position:relative;height:100%;background: | |
| linear-gradient(90deg,var(--rule2) 1px,transparent 1px) 0 0/25% 100% repeat-x;border-right:1px solid var(--rule2)} | |
| .track i{position:absolute;top:50%;width:2px;height:16px;margin:-8px 0 0 -1px;background:var(--mark);border-radius:1px} | |
| .track b{position:absolute;top:50%;width:4px;height:30px;margin:-15px 0 0 -2px;background:var(--white);border-radius:2px;box-shadow:0 0 18px rgba(255,255,255,.5)} | |
| .track b em{position:absolute;right:10px;top:50%;transform:translateY(-50%);font-style:normal;font-weight:600;font-size:14px;color:var(--white);background:var(--bg);padding:0 4px} | |
| .axis .ticks{display:flex;justify-content:space-between;font-size:12px;color:var(--low)}.axis s{text-decoration:none} | |
| .axis .ticks s:first-child{transform:translateX(-2px)}.axis .ticks s:last-child{transform:translateX(8px)} | |
| .strip figcaption{color:var(--low);font-size:13.5px;max-width:720px;margin-top:16px} | |
| .sec{padding-block:104px 0} | |
| h2{font-size:clamp(30px,4.2vw,46px);letter-spacing:-.035em;line-height:1.05;font-weight:600;margin:0} | |
| .intro{color:var(--mid);max-width:680px;margin:14px 0 28px;font-size:17px} | |
| .tw{overflow-x:auto;border-top:1px solid var(--rule2);border-bottom:1px solid var(--rule2)} | |
| table{border-collapse:separate;border-spacing:0;width:100%} | |
| .cmp th,.cmp td{padding:13px 8px;text-align:right;white-space:nowrap;border-bottom:1px solid var(--rule);vertical-align:top} | |
| .cmp tbody tr:last-child>*{border-bottom:0} | |
| .cmp thead th{font-size:12.5px;font-weight:500;color:var(--mid);vertical-align:bottom;white-space:normal;line-height:1.3;min-width:62px} | |
| .cmp thead tr.labs th{font-size:12px;color:var(--low);font-weight:400;border-bottom:0;padding-bottom:6px;text-align:center} | |
| .cmp thead tr.labs th[colspan]{box-shadow:inset 0 -1px 0 var(--rule2)} | |
| .cmp .bh,.cmp tbody th{position:sticky;left:0;background:var(--bg);text-align:left;z-index:1;min-width:176px;white-space:normal} | |
| .cmp tbody th{font-weight:500}.cmp tbody th span{display:block;font-weight:400;font-size:13px;color:var(--low)} | |
| .cmp td small{display:block;font-size:11.5px;color:var(--low);margin-top:2px} | |
| .cmp td a{text-decoration:none;border-bottom:1px dotted var(--low)}.cmp td a:hover{border-color:var(--fg)} | |
| .cmp sup{font-size:10px;color:var(--mid);margin-left:1px} | |
| .cmp .t{color:var(--white);font-weight:600;background:rgba(255,255,255,.045)}.cmp thead th.t{color:var(--white)} | |
| .cmp td.best{font-weight:700;color:var(--white)}.cmp td.best a{border-bottom:2px solid var(--white)} | |
| .cmp td.t.best{background:rgba(255,255,255,.13)}.cmp td.t.best .tv{border-bottom:2px solid var(--white)} | |
| .cmp td.na{color:var(--low)} | |
| .notes{color:var(--low);font-size:13.5px;max-width:860px;padding-left:20px;margin:16px 0 0}.notes li{margin:4px 0} | |
| .cats{display:grid;grid-template-columns:repeat(2,minmax(0,1fr));gap:0 56px;border-top:1px solid var(--rule2)} | |
| .cat h3{font-size:15px;font-weight:600;margin:28px 0 8px;color:var(--mid)} | |
| .cat ul{list-style:none;margin:0;padding:0} | |
| .cat li{display:flex;justify-content:space-between;gap:16px;align-items:baseline;padding:11px 0;border-bottom:1px solid var(--rule)} | |
| .cat li strong{font-weight:500}.cat li span{display:block;font-size:13px;color:var(--low)} | |
| .cat li b{font-size:22px;font-weight:600;letter-spacing:-.02em;color:var(--white)} | |
| .flag{font-style:normal;font-size:11.5px;color:var(--mid);border:1px solid var(--rule2);border-radius:6px;padding:0 6px;margin-left:8px;vertical-align:1px} | |
| .build{border-top:1px solid var(--rule2);padding-top:22px} | |
| .build.feature{display:grid;grid-template-columns:minmax(0,1.6fr) minmax(0,1fr);gap:32px;align-items:start} | |
| .builds{display:grid;grid-template-columns:repeat(2,minmax(0,1fr));gap:48px 32px;margin-top:48px} | |
| .frame{aspect-ratio:16/10;border:1px solid var(--rule2);border-radius:10px;overflow:hidden;background:#000} | |
| .frame iframe{width:100%;height:100%;border:0;display:block} | |
| .bm h3{font-size:20px;letter-spacing:-.015em;margin:14px 0 6px}.feature .bm h3{margin-top:0;font-size:26px} | |
| .bm p{color:var(--mid);margin:0 0 12px;max-width:62ch} | |
| details{border-top:1px solid var(--rule);padding-top:10px;margin-top:6px}summary{cursor:pointer;font-size:14.5px} | |
| .prompt{font-size:14px;line-height:1.6;color:var(--fg);white-space:pre-wrap;margin-top:10px} | |
| .open{display:inline-block;margin-top:12px;font-size:14.5px;text-decoration:none;border-bottom:1px solid var(--mid)} | |
| .method{display:grid;grid-template-columns:minmax(0,1fr) minmax(0,1.4fr);gap:56px;border-top:1px solid var(--rule2);padding-top:28px} | |
| .method dl{margin:0}.method dt{font-weight:600;margin-top:18px}.method dt:first-child{margin-top:0}.method dd{margin:4px 0 0;color:var(--mid)} | |
| footer{margin-top:120px;border-top:1px solid var(--rule);padding-block:28px 48px;color:var(--low);font-size:14px} | |
| footer .w{display:flex;justify-content:space-between;gap:16px;flex-wrap:wrap}footer a{text-decoration:none} | |
| @media (max-width:860px){.strip li,.axis{grid-template-columns:110px minmax(0,1fr) 62px;gap:12px}.sb{font-size:13px}.rk{font-size:12px} | |
| .cats,.builds,.method,.build.feature{grid-template-columns:minmax(0,1fr)}.hero{padding-top:56px}.cmp .bh,.cmp tbody th{min-width:160px}} | |
| </style></head><body> | |
| <div class="top"><div class="w"><a class="brand" href="https://torkehub.com">TorkeHub</a> | |
| <nav aria-label="Sections"><a href="#frontier">Anthropic and OpenAI</a><a href="#google-meta-xai">Google, Meta, xAI</a><a href="#china-india">China and India</a><a href="#results">All results</a><a href="#built">Built by Torki 2</a><a href="#method">Method</a></nav></div></div> | |
| <main class="w"> | |
| <header class="hero"> | |
| <p class="kicker">Evaluation results, October 2026</p> | |
| <h1>Torki 2, measured.</h1> | |
| <p class="lede">We ran <strong>Torki 2</strong>, the model behind Torki, on 22 public benchmarks, the same model our users get, with thinking on and no tools, then set every result next to the numbers other labs publish. Every number links to its source.</p> | |
| <figure class='strip' aria-label='Where Torki 2 lands on each benchmark'><ul><li><span class='sb'>GPQA Diamond</span><span class='track'><i style='left:90.6%' title='Claude Opus 5.5: 90.6'></i><i style='left:93.7%' title='Claude Fable 5.1: 93.7'></i><i style='left:95.6%' title='Claude Sonnet 5.5: 95.6'></i><i style='left:73.0%' title='Claude Haiku 4.5: 73.0'></i><i style='left:96.0%' title='GPT-6 Astra: 96.0'></i><i style='left:95.4%' title='GPT-6.1 Sol: 95.4'></i><i style='left:90.5%' title='GPT-6 Luna: 90.5'></i><i style='left:80.1%' title='gpt-oss-120b: 80.1'></i><i style='left:94.3%' title='Gemini 3.1 Pro: 94.3'></i><i style='left:90.4%' title='Gemini 3 Flash: 90.4'></i><i style='left:86.9%' title='Gemini 3.1 Flash-Lite: 86.9'></i><i style='left:84.3%' title='Gemma 4 31B: 84.3'></i><i style='left:93.5%' title='Muse Spark 1.3: 93.5'></i><i style='left:83.5%' title='Muse Glimmer-30B: 83.5'></i><i style='left:69.8%' title='Llama 4 Maverick: 69.8'></i><i style='left:90.1%' title='Grok 4.3: 90.1'></i><i style='left:92.4%' title='DeepSeek-V4-Pro: 92.4'></i><i style='left:90.9%' title='DeepSeek-V4.1-Flash: 90.9'></i><i style='left:93.5%' title='Kimi K3: 93.5'></i><i style='left:91.72%' title='GLM-5.3: 91.7'></i><i style='left:92.93%' title='MiniMax-M3: 92.9'></i><i style='left:88.9%' title='Seed 2.0 Pro: 88.9'></i><i style='left:92.3%' title='Hunyuan Hy4 (preview): 92.3'></i><i style='left:91.0%' title='ERNIE 5.1: 91.0'></i><i style='left:93.5%' title='Step 5 (preview): 93.5'></i><i style='left:78.7%' title='Sarvam-105B: 78.7'></i><i style='left:66.5%' title='Sarvam-30B: 66.5'></i><b style='left:89.65%'><em>89.7</em></b></span><span class='rk'>19th of 28</span></li><li><span class='sb'>Humanity's Last Exam</span><span class='track'><i style='left:61.4%' title='Claude Opus 5.5: 61.4'></i><i style='left:59.1%' title='Claude Fable 5.1: 59.1'></i><i style='left:55.0%' title='Claude Sonnet 5.5: 55.0'></i><i style='left:10.4%' title='Claude Haiku 4.5: 10.4'></i><i style='left:54.7%' title='GPT-6 Astra: 54.7'></i><i style='left:52.9%' title='GPT-6.1 Sol: 52.9'></i><i style='left:38.5%' title='GPT-6 Luna: 38.5'></i><i style='left:19.6%' title='gpt-oss-120b: 19.6'></i><i style='left:48.7%' title='Muse Spark 1.3: 48.7'></i><i style='left:22.0%' title='Muse Glimmer-30B: 22.0'></i><i style='left:43.1%' title='Grok 4.7: 43.1'></i><i style='left:37.2%' title='Grok 4.3: 37.2'></i><i style='left:42.7%' title='DeepSeek-V4-Pro: 42.7'></i><i style='left:39.1%' title='DeepSeek-V4.1-Flash: 39.1'></i><i style='left:46.9%' title='Kimi K3: 46.9'></i><i style='left:42.26%' title='GLM-5.3: 42.3'></i><i style='left:38.97%' title='MiniMax-M3: 39.0'></i><i style='left:32.4%' title='Seed 2.0 Pro: 32.4'></i><i style='left:43.4%' title='Hunyuan Hy4 (preview): 43.4'></i><i style='left:46.48%' title='Step 5 (preview): 46.5'></i><b style='left:38.01%'><em>38.0</em></b></span><span class='rk'>16th of 21</span></li><li><span class='sb'>MMLU-Pro</span><span class='track'><i style='left:92.38%' title='Claude Fable 5.1: 92.4'></i><i style='left:78.72%' title='Claude Haiku 4.5: 78.7'></i><i style='left:79.17%' title='gpt-oss-120b: 79.2'></i><i style='left:85.2%' title='Gemma 4 31B: 85.2'></i><i style='left:80.5%' title='Llama 4 Maverick: 80.5'></i><i style='left:87.5%' title='DeepSeek-V4-Pro (preview): 87.5'></i><i style='left:87.97%' title='Kimi K3: 88.0'></i><i style='left:86.77%' title='GLM-5.3: 86.8'></i><i style='left:84.22%' title='MiniMax-M3: 84.2'></i><i style='left:87.0%' title='Seed 2.0 Pro: 87.0'></i><i style='left:84.3%' title='ERNIE 5.1: 84.3'></i><i style='left:81.7%' title='Sarvam-105B: 81.7'></i><i style='left:80.0%' title='Sarvam-30B: 80.0'></i><b style='left:88.07%'><em>88.1</em></b></span><span class='rk'>2nd of 14</span></li><li><span class='sb'>AIME 2025</span><span class='track'><i style='left:80.7%' title='Claude Haiku 4.5: 80.7'></i><i style='left:92.5%' title='gpt-oss-120b: 92.5'></i><i style='left:95.2%' title='Gemini 3 Flash: 95.2'></i><i style='left:98.3%' title='Seed 2.0 Pro: 98.3'></i><i style='left:88.3%' title='Sarvam-105B: 88.3'></i><i style='left:80.0%' title='Sarvam-30B: 80.0'></i><b style='left:96.25%'><em>96.3</em></b></span><span class='rk'>2nd of 7</span></li><li><span class='sb'>AIME 2026</span><span class='track'><i style='left:89.2%' title='Gemma 4 31B: 89.2'></i><i style='left:94.7%' title='Muse Glimmer-30B: 94.7'></i><i style='left:100.0%' title='DeepSeek-V4.1-Flash: 100.0'></i><i style='left:94.2%' title='Seed 2.0 Pro: 94.2'></i><i style='left:95.0%' title='ERNIE 5.1: 95.0'></i><b style='left:95.0%'><em>95.0</em></b></span><span class='rk'>2nd of 6</span></li><li><span class='sb'>MATH-500</span><span class='track'><i style='left:94.8%' title='gpt-oss-120b: 94.8'></i><i style='left:98.6%' title='Sarvam-105B: 98.6'></i><i style='left:97.0%' title='Sarvam-30B: 97.0'></i><b style='left:99.4%'><em>99.4</em></b></span><span class='rk'>1st of 4</span></li><li><span class='sb'>LiveCodeBench</span><span class='track'><i style='left:90.52%' title='Claude Fable 5.1: 90.5'></i><i style='left:61.5%' title='Claude Haiku 4.5: 61.5'></i><i style='left:83.23%' title='gpt-oss-120b: 83.2'></i><i style='left:72.0%' title='Gemini 3.1 Flash-Lite: 72.0'></i><i style='left:80.0%' title='Gemma 4 31B: 80.0'></i><i style='left:93.5%' title='DeepSeek-V4-Pro (preview): 93.5'></i><i style='left:87.8%' title='Seed 2.0 Pro: 87.8'></i><i style='left:71.7%' title='Sarvam-105B: 71.7'></i><i style='left:70.0%' title='Sarvam-30B: 70.0'></i><b style='left:77.1%'><em>77.1</em></b></span><span class='rk'>6th of 10</span></li><li><span class='sb'>IFBench</span><span class='track'><i style='left:54.3%' title='Claude Haiku 4.5: 54.3'></i><i style='left:69.0%' title='gpt-oss-120b: 69.0'></i><i style='left:77.0%' title='Muse Glimmer-30B: 77.0'></i><i style='left:81.3%' title='Grok 4.3: 81.3'></i><i style='left:82.86%' title='MiniMax-M3: 82.9'></i><b style='left:73.33%'><em>73.3</em></b></span><span class='rk'>4th of 6</span></li><li><span class='sb'>MMMU</span><span class='track'><i style='left:73.2%' title='Claude Haiku 4.5: 73.2'></i><i style='left:73.4%' title='Llama 4 Maverick: 73.4'></i><i style='left:85.4%' title='Seed 2.0 Pro: 85.4'></i><b style='left:84.67%'><em>84.7</em></b></span><span class='rk'>2nd of 4</span></li><li><span class='sb'>MILU</span><span class='track'><i style='left:93.1%' title='Claude Opus 5.5: 93.1'></i><i style='left:93.0%' title='Claude Fable 5.1: 93.0'></i><i style='left:91.6%' title='Claude Sonnet 5.5: 91.6'></i><i style='left:76.8%' title='Sarvam-30B: 76.8'></i><b style='left:85.09%'><em>85.1</em></b></span><span class='rk'>4th of 5</span></li></ul><div class='axis' aria-hidden='true'><span></span><span class='ticks'><s>0</s><s>25</s><s>50</s><s>75</s><s>100</s></span><span></span></div><figcaption>Each grey mark is another model's published score on the same benchmark; the white marker is Torki 2. Rank counts every model in the three tables below that has a number for that benchmark.</figcaption></figure></header><section class='sec' id='frontier'><h2>Against Anthropic and OpenAI</h2><p class='intro'>The current Claude and GPT models. Their makers publish few of these benchmarks for their newest models, so most numbers here are independent measurements.</p><div class='tw'><table class='cmp'><thead><tr class='labs'><th class='bh'></th><th></th><th colspan='4'>Anthropic</th><th colspan='4'>OpenAI</th></tr><tr><th class='bh'>Benchmark</th><th class='t'>Torki 2</th><th>Claude<br>Opus 5.5</th><th>Claude<br>Fable 5.1</th><th>Claude<br>Sonnet 5.5</th><th>Claude<br>Haiku 4.5</th><th>GPT-6<br>Astra</th><th>GPT-6.1<br>Sol</th><th>GPT-6 Luna</th><th>gpt-oss-120b</th></tr></thead><tbody><tr><th scope='row'>GPQA Diamond<span>Graduate-level science</span></th><td class='t'><span class='tv'>89.7</span><small></small></td><td class=''><a href='https://epoch.ai/benchmarks/gpqa-diamond' target='_blank' rel='noopener' title='Measured by Epoch AI'>90.6<sup>β </sup></a><small>max</small></td><td class=''><a href='https://artificialanalysis.ai/models/claude-fable-5-1' target='_blank' rel='noopener' title='Measured by Artificial Analysis'>93.7<sup>β </sup></a><small>max</small></td><td class=''><a href='https://epoch.ai/benchmarks/gpqa-diamond' target='_blank' rel='noopener' title='Measured by Epoch AI'>95.6<sup>β </sup></a><small>max</small></td><td class=''><a href='https://www.anthropic.com/news/claude-haiku-4-5' target='_blank' rel='noopener' title='Published by the model's maker'>73.0</a><small></small></td><td class='best'><a href='https://openai.com/index/gpt-6-astra/' target='_blank' rel='noopener' title='Published by the model's maker'>96.0</a><small>max</small></td><td class=''><a href='https://epoch.ai/benchmarks/gpqa-diamond' target='_blank' rel='noopener' title='Measured by Epoch AI'>95.4<sup>β </sup></a><small>max</small></td><td class=''><a href='https://epoch.ai/benchmarks/gpqa-diamond' target='_blank' rel='noopener' title='Measured by Epoch AI'>90.5<sup>β </sup></a><small>max</small></td><td class=''><a href='https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf' target='_blank' rel='noopener' title='Published by the model's maker'>80.1</a><small>high</small></td></tr><tr><th scope='row'>Humanity's Last Exam (text-only)<span>Expert questions, text only</span></th><td class='t'><span class='tv'>38.0</span><small>sample</small></td><td class='best'><a href='https://artificialanalysis.ai/evaluations/humanitys-last-exam' target='_blank' rel='noopener' title='Measured by Artificial Analysis'>61.4<sup>β </sup></a><small>max</small></td><td class=''><a href='https://artificialanalysis.ai/evaluations/humanitys-last-exam' target='_blank' rel='noopener' title='Measured by Artificial Analysis'>59.1<sup>β </sup></a><small>max</small></td><td class=''><a href='https://artificialanalysis.ai/models/claude-sonnet-5-5' target='_blank' rel='noopener' title='Measured by Artificial Analysis'>55.0<sup>β </sup></a><small>max</small></td><td class=''><a href='https://artificialanalysis.ai/models/claude-4-5-haiku-reasoning' target='_blank' rel='noopener' title='Measured by Artificial Analysis'>10.4<sup>β </sup></a><small></small></td><td class=''><a href='https://artificialanalysis.ai/models/gpt-6-astra' target='_blank' rel='noopener' title='Measured by Artificial Analysis'>54.7<sup>β </sup></a><small>max</small></td><td class=''><a href='https://artificialanalysis.ai/models/gpt-6-1-sol' target='_blank' rel='noopener' title='Measured by Artificial Analysis'>52.9<sup>β </sup></a><small>max</small></td><td class=''><a href='https://artificialanalysis.ai/models/gpt-6-luna' target='_blank' rel='noopener' title='Measured by Artificial Analysis'>38.5<sup>β </sup></a><small>max</small></td><td class=''><a href='https://artificialanalysis.ai/models/gpt-oss-120b' target='_blank' rel='noopener' title='Measured by Artificial Analysis'>19.6<sup>β </sup></a><small>high</small></td></tr><tr><th scope='row'>MMLU-Pro<span>Broad expert knowledge</span></th><td class='t'><span class='tv'>88.1</span><small>sample</small></td><td class='na' aria-label='not published'>β</td><td class='best'><a href='https://www.vals.ai/benchmarks/mmlu_pro' target='_blank' rel='noopener' title='Measured by Vals AI'>92.4<sup>β </sup></a><small>max</small></td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://www.vals.ai/benchmarks/mmlu_pro' target='_blank' rel='noopener' title='Measured by Vals AI'>78.7<sup>β </sup></a><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://www.vals.ai/benchmarks/mmlu_pro' target='_blank' rel='noopener' title='Measured by Vals AI'>79.2<sup>β </sup></a><small></small></td></tr><tr><th scope='row'>AIME 2025<span>Competition maths, 2025</span></th><td class='t best'><span class='tv'>96.3</span><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://www.anthropic.com/news/claude-haiku-4-5' target='_blank' rel='noopener' title='Published by the model's maker'>80.7</a><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf' target='_blank' rel='noopener' title='Published by the model's maker'>92.5</a><small>high</small></td></tr><tr><th scope='row'>MATH-500<span>Competition maths</span></th><td class='t best'><span class='tv'>99.4</span><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://www.vals.ai/benchmarks/math500' target='_blank' rel='noopener' title='Measured by Vals AI'>94.8<sup>β </sup></a><small></small></td></tr><tr><th scope='row'>LiveCodeBench v6<span>Contest coding, 2025 problems</span></th><td class='t'><span class='tv'>77.1</span><small></small></td><td class='na' aria-label='not published'>β</td><td class='best'><a href='https://www.vals.ai/benchmarks/lcb' target='_blank' rel='noopener' title='Measured by Vals AI'>90.5<sup>β </sup></a><small>max</small></td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://artificialanalysis.ai/models/claude-4-5-haiku-reasoning' target='_blank' rel='noopener' title='Measured by Artificial Analysis'>61.5<sup>β </sup></a><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://www.vals.ai/benchmarks/lcb' target='_blank' rel='noopener' title='Measured by Vals AI'>83.2<sup>β </sup></a><small></small></td></tr><tr><th scope='row'>IFBench<span>Unseen instructions</span></th><td class='t best'><span class='tv'>73.3</span><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://artificialanalysis.ai/models/claude-4-5-haiku-reasoning' target='_blank' rel='noopener' title='Measured by Artificial Analysis'>54.3<sup>β </sup></a><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://artificialanalysis.ai/models/gpt-oss-120b' target='_blank' rel='noopener' title='Measured by Artificial Analysis'>69.0<sup>β </sup></a><small>high</small></td></tr><tr><th scope='row'>MMMU<span>College questions about images</span></th><td class='t best'><span class='tv'>84.7</span><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://www.anthropic.com/news/claude-haiku-4-5' target='_blank' rel='noopener' title='Published by the model's maker'>73.2</a><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td></tr><tr><th scope='row'>MILU<span>11 Indian languages</span></th><td class='t'><span class='tv'>85.1</span><small>sample</small></td><td class='best'><a href='https://www.anthropic.com/claude-opus-5-5-system-card' target='_blank' rel='noopener' title='Published by the model's maker'>93.1</a><small>max</small></td><td class=''><a href='https://www.anthropic.com/claude-fable-5-1-mythos-5-1-system-card' target='_blank' rel='noopener' title='Published by the model's maker'>93.0</a><small>max</small></td><td class=''><a href='https://www.anthropic.com/claude-sonnet-5-5-system-card' target='_blank' rel='noopener' title='Published by the model's maker'>91.6</a><small>max</small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td></tr><tr><th scope='row'>MGSM<span>Maths in 11 languages</span></th><td class='t'><span class='tv'>86.0</span><small>see note</small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='best'><a href='https://www.vals.ai/benchmarks/mgsm' target='_blank' rel='noopener' title='Measured by Vals AI'>92.2<sup>β </sup></a><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://www.vals.ai/benchmarks/mgsm' target='_blank' rel='noopener' title='Measured by Vals AI'>92.0<sup>β </sup></a><small></small></td></tr></tbody></table></div><ol class='notes'><li>Bold marks the best score in each row, whoever holds it. A dash means no comparable number is published.</li><li>β Independent measurement, used where the model's maker publishes none: Artificial Analysis, Epoch AI, Vals AI.</li><li>Small text under a number is its reasoning setting; Torki 2 runs at its default thinking depth.</li><li>Torki 2 samples: Humanity's Last Exam (first 400 text-only questions, graded with the official prompt), MMLU-Pro and MILU. Torki 2's MGSM is 93.4 if Telugu and Bengali spelling variants of 'Answer' are accepted.</li><li>Claude numbers, from Anthropic and from independent testers, include Anthropic's safeguard fallback to another Claude model, as Anthropic's system cards describe.</li></ol></section><section class='sec' id='google-meta-xai'><h2>Against Google, Meta and xAI</h2><p class='intro'>Gemini, Gemma, Meta's Muse and Llama, and Grok. Google and Meta publish most of their own numbers; Grok's come from independent testing.</p><div class='tw'><table class='cmp'><thead><tr class='labs'><th class='bh'></th><th></th><th colspan='4'>Google</th><th colspan='3'>Meta</th><th colspan='2'>xAI</th></tr><tr><th class='bh'>Benchmark</th><th class='t'>Torki 2</th><th>Gemini<br>3.1 Pro</th><th>Gemini<br>3 Flash</th><th>Gemini<br>3.1 Flash-Lite</th><th>Gemma<br>4 31B</th><th>Muse<br>Spark 1.3</th><th>Muse<br>Glimmer-30B</th><th>Llama<br>4 Maverick</th><th>Grok 4.7</th><th>Grok 4.3</th></tr></thead><tbody><tr><th scope='row'>GPQA Diamond<span>Graduate-level science</span></th><td class='t'><span class='tv'>89.7</span><small></small></td><td class='best'><a href='https://deepmind.google/models/gemini/pro/' target='_blank' rel='noopener' title='Published by the model's maker'>94.3</a><small>high</small></td><td class=''><a href='https://storage.googleapis.com/deepmind-media/gemini/gemini_3_flash_model_evaluation.pdf' target='_blank' rel='noopener' title='Published by the model's maker'>90.4</a><small></small></td><td class=''><a href='https://storage.googleapis.com/deepmind-media/gemini/gemini_3-1_flash-lite_model_evaluation.pdf' target='_blank' rel='noopener' title='Published by the model's maker'>86.9</a><small>high</small></td><td class=''><a href='https://ai.google.dev/gemma/docs/core/model_card_4' target='_blank' rel='noopener' title='Published by the model's maker'>84.3</a><small></small></td><td class=''><a href='https://artificialanalysis.ai/models/muse-spark-1-3' target='_blank' rel='noopener' title='Measured by Artificial Analysis'>93.5<sup>β </sup></a><small>max</small></td><td class=''><a href='https://huggingface.co/meta-models/Muse-Glimmer-30B/raw/main/README.md' target='_blank' rel='noopener' title='Published by the model's maker'>83.5</a><small>high</small></td><td class=''><a href='https://raw.githubusercontent.com/meta-llama/llama-models/main/models/llama4/MODEL_CARD.md' target='_blank' rel='noopener' title='Published by the model's maker'>69.8</a><small>no thinking</small></td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://artificialanalysis.ai/models/grok-4-3' target='_blank' rel='noopener' title='Measured by Artificial Analysis'>90.1<sup>β </sup></a><small>high</small></td></tr><tr><th scope='row'>Humanity's Last Exam (text-only)<span>Expert questions, text only</span></th><td class='t'><span class='tv'>38.0</span><small>sample</small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='best'><a href='https://artificialanalysis.ai/models/muse-spark-1-3' target='_blank' rel='noopener' title='Measured by Artificial Analysis'>48.7<sup>β </sup></a><small>max</small></td><td class=''><a href='https://huggingface.co/meta-models/Muse-Glimmer-30B/raw/main/README.md' target='_blank' rel='noopener' title='Published by the model's maker'>22.0</a><small>high</small></td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://artificialanalysis.ai/models/grok-4-7' target='_blank' rel='noopener' title='Measured by Artificial Analysis'>43.1<sup>β </sup></a><small>xhigh</small></td><td class=''><a href='https://artificialanalysis.ai/models/grok-4-3' target='_blank' rel='noopener' title='Measured by Artificial Analysis'>37.2<sup>β </sup></a><small>high</small></td></tr><tr><th scope='row'>MMLU-Pro<span>Broad expert knowledge</span></th><td class='t best'><span class='tv'>88.1</span><small>sample</small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://ai.google.dev/gemma/docs/core/model_card_4' target='_blank' rel='noopener' title='Published by the model's maker'>85.2</a><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://raw.githubusercontent.com/meta-llama/llama-models/main/models/llama4/MODEL_CARD.md' target='_blank' rel='noopener' title='Published by the model's maker'>80.5</a><small>no thinking</small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td></tr><tr><th scope='row'>AIME 2026<span>Competition maths, 2026</span></th><td class='t best'><span class='tv'>95.0</span><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://ai.google.dev/gemma/docs/core/model_card_4' target='_blank' rel='noopener' title='Published by the model's maker'>89.2</a><small></small></td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://huggingface.co/meta-models/Muse-Glimmer-30B/raw/main/README.md' target='_blank' rel='noopener' title='Published by the model's maker'>94.7</a><small>high</small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td></tr><tr><th scope='row'>AIME 2025<span>Competition maths, 2025</span></th><td class='t best'><span class='tv'>96.3</span><small></small></td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://storage.googleapis.com/deepmind-media/gemini/gemini_3_flash_model_evaluation.pdf' target='_blank' rel='noopener' title='Published by the model's maker'>95.2</a><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td></tr><tr><th scope='row'>LiveCodeBench v6<span>Contest coding, 2025 problems</span></th><td class='t'><span class='tv'>77.1</span><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://storage.googleapis.com/deepmind-media/gemini/gemini_3-1_flash-lite_model_evaluation.pdf' target='_blank' rel='noopener' title='Published by the model's maker'>72.0</a><small>high</small></td><td class='best'><a href='https://ai.google.dev/gemma/docs/core/model_card_4' target='_blank' rel='noopener' title='Published by the model's maker'>80.0</a><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td></tr><tr><th scope='row'>IFBench<span>Unseen instructions</span></th><td class='t'><span class='tv'>73.3</span><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://huggingface.co/meta-models/Muse-Glimmer-30B/raw/main/README.md' target='_blank' rel='noopener' title='Published by the model's maker'>77.0</a><small>high</small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='best'><a href='https://artificialanalysis.ai/models/grok-4-3' target='_blank' rel='noopener' title='Measured by Artificial Analysis'>81.3<sup>β </sup></a><small>high</small></td></tr><tr><th scope='row'>MMMU<span>College questions about images</span></th><td class='t best'><span class='tv'>84.7</span><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://raw.githubusercontent.com/meta-llama/llama-models/main/models/llama4/MODEL_CARD.md' target='_blank' rel='noopener' title='Published by the model's maker'>73.4</a><small>no thinking</small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td></tr><tr><th scope='row'>MathVista<span>Maths in images</span></th><td class='t best'><span class='tv'>89.1</span><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://raw.githubusercontent.com/meta-llama/llama-models/main/models/llama4/MODEL_CARD.md' target='_blank' rel='noopener' title='Published by the model's maker'>73.7</a><small>no thinking</small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td></tr><tr><th scope='row'>DocVQA<span>Reading documents</span></th><td class='t best'><span class='tv'>94.6</span><small>thinking off</small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://raw.githubusercontent.com/meta-llama/llama-models/main/models/llama4/MODEL_CARD.md' target='_blank' rel='noopener' title='Published by the model's maker'>94.4</a><small>no thinking</small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td></tr><tr><th scope='row'>ChartQA<span>Reading charts</span></th><td class='t'><span class='tv'>85.2</span><small>thinking off</small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='best'><a href='https://raw.githubusercontent.com/meta-llama/llama-models/main/models/llama4/MODEL_CARD.md' target='_blank' rel='noopener' title='Published by the model's maker'>90.0</a><small>no thinking</small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td></tr><tr><th scope='row'>MGSM<span>Maths in 11 languages</span></th><td class='t'><span class='tv'>86.0</span><small>see note</small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='best'><a href='https://raw.githubusercontent.com/meta-llama/llama-models/main/models/llama4/MODEL_CARD.md' target='_blank' rel='noopener' title='Published by the model's maker'>92.3</a><small>no thinking</small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td></tr></tbody></table></div><ol class='notes'><li>Bold marks the best score in each row, whoever holds it. A dash means no comparable number is published.</li><li>β Independent measurement, used where the model's maker publishes none: Artificial Analysis.</li><li>Small text under a number is its reasoning setting; Torki 2 runs at its default thinking depth.</li><li>Torki 2 samples: Humanity's Last Exam (first 400 text-only questions, graded with the official prompt), MMLU-Pro and MILU. Torki 2's MGSM is 93.4 if Telugu and Bengali spelling variants of 'Answer' are accepted.</li><li>Llama 4 Maverick has no thinking mode. Gemini 3.1 Flash-Lite's LiveCodeBench window is January to May 2025; Torki 2's is February to April 2025.</li></ol></section><section class='sec' id='china-india'><h2>Against China and India</h2><p class='intro'>The current flagships from DeepSeek, Moonshot (Kimi), Zhipu (GLM), MiniMax, ByteDance (Seed), Tencent (Hunyuan), Baidu (ERNIE) and StepFun, and Sarvam from India. Most numbers are the labs' own.</p><div class='tw'><table class='cmp'><thead><tr class='labs'><th class='bh'></th><th></th><th colspan='3'>DeepSeek</th><th colspan='1'>Moonshot AI</th><th colspan='1'>Zhipu AI</th><th colspan='1'>MiniMax</th><th colspan='1'>ByteDance</th><th colspan='1'>Tencent</th><th colspan='1'>Baidu</th><th colspan='1'>StepFun</th><th colspan='2'>Sarvam AI</th></tr><tr><th class='bh'>Benchmark</th><th class='t'>Torki 2</th><th>DeepSeek-V4-Pro</th><th>DeepSeek-V4.1-Flash</th><th>DeepSeek-V4-Pro<br>(preview)</th><th>Kimi K3</th><th>GLM-5.3</th><th>MiniMax-M3</th><th>Seed<br>2.0 Pro</th><th>Hunyuan<br>Hy4 (preview)</th><th>ERNIE 5.1</th><th>Step<br>5 (preview)</th><th>Sarvam-105B</th><th>Sarvam-30B</th></tr></thead><tbody><tr><th scope='row'>GPQA Diamond<span>Graduate-level science</span></th><td class='t'><span class='tv'>89.7</span><small></small></td><td class=''><a href='https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/blob/main/DeepSeek_V41_Tech_Report.pdf' target='_blank' rel='noopener' title='Published by the model's maker'>92.4</a><small>max</small></td><td class=''><a href='https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash' target='_blank' rel='noopener' title='Published by the model's maker'>90.9</a><small>max</small></td><td class='na' aria-label='not published'>β</td><td class='best'><a href='https://huggingface.co/moonshotai/Kimi-K3/raw/main/README.md' target='_blank' rel='noopener' title='Published by the model's maker'>93.5</a><small>max</small></td><td class=''><a href='https://artificialanalysis.ai/models/glm-5-3' target='_blank' rel='noopener' title='Measured by Artificial Analysis'>91.7<sup>β </sup></a><small>max</small></td><td class=''><a href='https://artificialanalysis.ai/models/minimax-m3' target='_blank' rel='noopener' title='Measured by Artificial Analysis'>92.9<sup>β </sup></a><small></small></td><td class=''><a href='https://lf3-static.bytednsdoc.com/obj/eden-cn/lapzild-tss/ljhwZthlaukjlkulzlp/seed2/0214/Seed2.0%20Model%20Card.pdf' target='_blank' rel='noopener' title='Published by the model's maker'>88.9</a><small></small></td><td class=''><a href='https://huggingface.co/tencent/Hy4-preview/raw/main/README.md' target='_blank' rel='noopener' title='Published by the model's maker'>92.3</a><small></small></td><td class=''><a href='https://ernie.baidu.com/blog/posts/ernie-5.1-0508-release/' target='_blank' rel='noopener' title='Published by the model's maker'>91.0</a><small></small></td><td class='best'><a href='https://www.stepfun.com/step-5-preview' target='_blank' rel='noopener' title='Published by the model's maker'>93.5</a><small>high</small></td><td class=''><a href='https://www.sarvam.ai/blogs/sarvam-30b-105b' target='_blank' rel='noopener' title='Published by the model's maker'>78.7</a><small></small></td><td class=''><a href='https://www.sarvam.ai/blogs/sarvam-30b-105b' target='_blank' rel='noopener' title='Published by the model's maker'>66.5</a><small></small></td></tr><tr><th scope='row'>Humanity's Last Exam (text-only)<span>Expert questions, text only</span></th><td class='t'><span class='tv'>38.0</span><small>sample</small></td><td class=''><a href='https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813' target='_blank' rel='noopener' title='Published by the model's maker'>42.7</a><small>max</small></td><td class=''><a href='https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash' target='_blank' rel='noopener' title='Published by the model's maker'>39.1</a><small>max</small></td><td class='na' aria-label='not published'>β</td><td class='best'><a href='https://artificialanalysis.ai/models/kimi-k3' target='_blank' rel='noopener' title='Measured by Artificial Analysis'>46.9<sup>β </sup></a><small>max</small></td><td class=''><a href='https://artificialanalysis.ai/models/glm-5-3' target='_blank' rel='noopener' title='Measured by Artificial Analysis'>42.3<sup>β </sup></a><small>max</small></td><td class=''><a href='https://artificialanalysis.ai/models/minimax-m3' target='_blank' rel='noopener' title='Measured by Artificial Analysis'>39.0<sup>β </sup></a><small></small></td><td class=''><a href='https://lf3-static.bytednsdoc.com/obj/eden-cn/lapzild-tss/ljhwZthlaukjlkulzlp/seed2/0214/Seed2.0%20Model%20Card.pdf' target='_blank' rel='noopener' title='Published by the model's maker'>32.4</a><small></small></td><td class=''><a href='https://huggingface.co/tencent/Hy4-preview/raw/main/README.md' target='_blank' rel='noopener' title='Published by the model's maker'>43.4</a><small></small></td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://artificialanalysis.ai/models/step-5' target='_blank' rel='noopener' title='Measured by Artificial Analysis'>46.5<sup>β </sup></a><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td></tr><tr><th scope='row'>MMLU-Pro<span>Broad expert knowledge</span></th><td class='t best'><span class='tv'>88.1</span><small>sample</small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro' target='_blank' rel='noopener' title='Published by the model's maker'>87.5</a><small>max</small></td><td class=''><a href='https://www.vals.ai/benchmarks/mmlu_pro' target='_blank' rel='noopener' title='Measured by Vals AI'>88.0<sup>β </sup></a><small>max</small></td><td class=''><a href='https://www.vals.ai/benchmarks/mmlu_pro' target='_blank' rel='noopener' title='Measured by Vals AI'>86.8<sup>β </sup></a><small>max</small></td><td class=''><a href='https://www.vals.ai/benchmarks/mmlu_pro' target='_blank' rel='noopener' title='Measured by Vals AI'>84.2<sup>β </sup></a><small></small></td><td class=''><a href='https://lf3-static.bytednsdoc.com/obj/eden-cn/lapzild-tss/ljhwZthlaukjlkulzlp/seed2/0214/Seed2.0%20Model%20Card.pdf' target='_blank' rel='noopener' title='Published by the model's maker'>87.0</a><small></small></td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://ernie.baidu.com/blog/posts/ernie-5.1-0508-release/' target='_blank' rel='noopener' title='Published by the model's maker'>84.3</a><small></small></td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://www.sarvam.ai/blogs/sarvam-30b-105b' target='_blank' rel='noopener' title='Published by the model's maker'>81.7</a><small></small></td><td class=''><a href='https://www.sarvam.ai/blogs/sarvam-30b-105b' target='_blank' rel='noopener' title='Published by the model's maker'>80.0</a><small></small></td></tr><tr><th scope='row'>AIME 2026<span>Competition maths, 2026</span></th><td class='t'><span class='tv'>95.0</span><small></small></td><td class='na' aria-label='not published'>β</td><td class='best'><a href='https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/resolve/main/DeepSeek_V41_Tech_Report.pdf' target='_blank' rel='noopener' title='Published by the model's maker'>100.0</a><small>max</small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://lf3-static.bytednsdoc.com/obj/eden-cn/lapzild-tss/ljhwZthlaukjlkulzlp/seed2/0214/Seed2.0%20Model%20Card.pdf' target='_blank' rel='noopener' title='Published by the model's maker'>94.2</a><small></small></td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://ernie.baidu.com/blog/posts/ernie-5.1-0508-release/' target='_blank' rel='noopener' title='Published by the model's maker'>95.0</a><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td></tr><tr><th scope='row'>AIME 2025<span>Competition maths, 2025</span></th><td class='t'><span class='tv'>96.3</span><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='best'><a href='https://lf3-static.bytednsdoc.com/obj/eden-cn/lapzild-tss/ljhwZthlaukjlkulzlp/seed2/0214/Seed2.0%20Model%20Card.pdf' target='_blank' rel='noopener' title='Published by the model's maker'>98.3</a><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://www.sarvam.ai/blogs/sarvam-30b-105b' target='_blank' rel='noopener' title='Published by the model's maker'>88.3</a><small></small></td><td class=''><a href='https://www.sarvam.ai/blogs/sarvam-30b-105b' target='_blank' rel='noopener' title='Published by the model's maker'>80.0</a><small></small></td></tr><tr><th scope='row'>MATH-500<span>Competition maths</span></th><td class='t best'><span class='tv'>99.4</span><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://www.sarvam.ai/blogs/sarvam-30b-105b' target='_blank' rel='noopener' title='Published by the model's maker'>98.6</a><small></small></td><td class=''><a href='https://www.sarvam.ai/blogs/sarvam-30b-105b' target='_blank' rel='noopener' title='Published by the model's maker'>97.0</a><small></small></td></tr><tr><th scope='row'>LiveCodeBench v6<span>Contest coding, 2025 problems</span></th><td class='t'><span class='tv'>77.1</span><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='best'><a href='https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro/blob/89d501aed9/DeepSeek_V4.pdf' target='_blank' rel='noopener' title='Published by the model's maker'>93.5</a><small>max</small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://lf3-static.bytednsdoc.com/obj/eden-cn/lapzild-tss/ljhwZthlaukjlkulzlp/seed2/0214/Seed2.0%20Model%20Card.pdf' target='_blank' rel='noopener' title='Published by the model's maker'>87.8</a><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://www.sarvam.ai/blogs/sarvam-30b-105b' target='_blank' rel='noopener' title='Published by the model's maker'>71.7</a><small></small></td><td class=''><a href='https://www.sarvam.ai/blogs/sarvam-30b-105b' target='_blank' rel='noopener' title='Published by the model's maker'>70.0</a><small></small></td></tr><tr><th scope='row'>IFEval<span>Precise formatting instructions</span></th><td class='t best'><span class='tv'>90.9</span><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://www.sarvam.ai/blogs/sarvam-30b-105b' target='_blank' rel='noopener' title='Published by the model's maker'>84.8</a><small></small></td><td class='na' aria-label='not published'>β</td></tr><tr><th scope='row'>IFBench<span>Unseen instructions</span></th><td class='t'><span class='tv'>73.3</span><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='best'><a href='https://artificialanalysis.ai/models/minimax-m3' target='_blank' rel='noopener' title='Measured by Artificial Analysis'>82.9<sup>β </sup></a><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td></tr><tr><th scope='row'>MMMU<span>College questions about images</span></th><td class='t'><span class='tv'>84.7</span><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='best'><a href='https://lf3-static.bytednsdoc.com/obj/eden-cn/lapzild-tss/ljhwZthlaukjlkulzlp/seed2/0214/Seed2.0%20Model%20Card.pdf' target='_blank' rel='noopener' title='Published by the model's maker'>85.4</a><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td></tr><tr><th scope='row'>MathVista<span>Maths in images</span></th><td class='t'><span class='tv'>89.1</span><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='best'><a href='https://lf3-static.bytednsdoc.com/obj/eden-cn/lapzild-tss/ljhwZthlaukjlkulzlp/seed2/0214/Seed2.0%20Model%20Card.pdf' target='_blank' rel='noopener' title='Published by the model's maker'>89.8</a><small></small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td></tr><tr><th scope='row'>MILU<span>11 Indian languages</span></th><td class='t best'><span class='tv'>85.1</span><small>sample</small></td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class='na' aria-label='not published'>β</td><td class=''><a href='https://huggingface.co/sarvamai/sarvam-30b' target='_blank' rel='noopener' title='Published by the model's maker'>76.8</a><small></small></td></tr></tbody></table></div><ol class='notes'><li>Bold marks the best score in each row, whoever holds it. A dash means no comparable number is published.</li><li>β Independent measurement, used where the model's maker publishes none: Artificial Analysis, Vals AI.</li><li>Small text under a number is its reasoning setting; Torki 2 runs at its default thinking depth.</li><li>Torki 2 samples: Humanity's Last Exam (first 400 text-only questions, graded with the official prompt), MMLU-Pro and MILU. Torki 2's MGSM is 93.4 if Telugu and Bengali spelling variants of 'Answer' are accepted.</li><li>DeepSeek-V4-Pro (preview) numbers use DeepSeek's maximum-effort setup with a special system prompt. Seed 2.0 Pro is ByteDance's latest model with published numbers on these benchmarks; Seed 2.1 reports few of them. Hy4 and Step 5 are previews. Baidu labels ERNIE 5.1's science score "GPQA" without saying it is the Diamond set.</li></ol></section> | |
| <section class="sec" id="results"><h2>Every result</h2> | |
| <p class="intro">All 22 benchmarks, thinking on unless marked. Unanswered questions, including answers that ran out of room, count as wrong. A sample is a fixed subset; where subjects were sampled unevenly the score is weighted back to the full subject mix.</p> | |
| <div class='cats'><section class='cat'><h3>Reasoning and knowledge</h3><ul><li><div><strong>GPQA Diamond</strong><span>accuracy, average of 4 runs</span></div><b>89.7</b></li><li><div><strong>Humanity's Last Exam (text-only)</strong><em class='flag'>sample</em><span>accuracy, weighted to full subject mix</span></div><b>38.0</b></li><li><div><strong>MMLU-Pro</strong><em class='flag'>sample</em><span>accuracy, weighted to full subject mix</span></div><b>88.1</b></li></ul></section><section class='cat'><h3>Mathematics</h3><ul><li><div><strong>AIME 2026</strong><span>accuracy, average of 4 attempts</span></div><b>95.0</b></li><li><div><strong>AIME 2025</strong><span>accuracy, average of 8 attempts</span></div><b>96.3</b></li><li><div><strong>AIME 2024</strong><span>accuracy, average of 8 attempts</span></div><b>95.8</b></li><li><div><strong>MATH-500</strong><span>accuracy</span></div><b>99.4</b></li></ul></section><section class='cat'><h3>Coding</h3><ul><li><div><strong>LiveCodeBench</strong><span>pass@1, v6 window 2025-02..2025-04</span></div><b>77.1</b></li><li><div><strong>LiveCodeBench</strong><span>pass@1, v5 window 2024-08..2025-01</span></div><b>79.6</b></li><li><div><strong>HumanEval+</strong><span>pass@1 (base + extended tests)</span></div><b>94.5</b></li><li><div><strong>MBPP+</strong><span>pass@1 (base + extended tests)</span></div><b>83.9</b></li></ul></section><section class='cat'><h3>Instruction following</h3><ul><li><div><strong>IFEval</strong><span>prompt-level strict accuracy</span></div><b>90.9</b></li><li><div><strong>IFBench</strong><span>prompt-level strict accuracy</span></div><b>73.3</b></li></ul></section><section class='cat'><h3>Vision</h3><ul><li><div><strong>MMMU</strong><span>accuracy</span></div><b>84.7</b></li><li><div><strong>MathVista</strong><span>accuracy</span></div><b>89.1</b></li><li><div><strong>DocVQA</strong><em class='flag'>thinking off</em><span>ANLS</span></div><b>94.6</b></li><li><div><strong>ChartQA</strong><em class='flag'>thinking off</em><span>relaxed accuracy</span></div><b>85.2</b></li></ul></section><section class='cat'><h3>Languages</h3><ul><li><div><strong>MILU (11 Indian languages)</strong><em class='flag'>sample</em><span>accuracy</span></div><b>85.1</b></li><li><div><strong>MGSM</strong><span>accuracy, reference scorer</span></div><b>86.0</b></li><li><div><strong>MMMLU (14 languages)</strong><em class='flag'>sample</em><em class='flag'>thinking off</em><span>accuracy</span></div><b>75.3</b></li></ul></section><section class='cat'><h3>Long context</h3><ul><li><div><strong>RULER (32K-128K)</strong><em class='flag'>sample</em><em class='flag'>thinking off</em><span>average of 13 tasks x 3 lengths</span></div><b>94.1</b></li></ul></section><section class='cat'><h3>Tool use</h3><ul><li><div><strong>BFCL v4</strong><span>accuracy, non-live, official AST average</span></div><b>85.2</b></li><li><div><strong>BFCL v4</strong><span>accuracy, live (real-world calls)</span></div><b>75.4</b></li><li><div><strong>BFCL v4</strong><span>accuracy, multi-turn</span></div><b>50.8</b></li><li><div><strong>BFCL v4</strong><span>accuracy, irrelevance detection</span></div><b>74.6</b></li></ul></section></div></section> | |
| <section class="sec" id="built"><h2>Built by Torki 2</h2> | |
| <p class="intro">Each project was written by Torki 2 from the prompt shown, as a single file with no libraries, and not edited by hand. Play with them here or open them full screen.</p> | |
| <article class='build feature'><div class='frame'><iframe src='showcase/chess.html' title='Chess against a computer opponent' loading='lazy'></iframe></div><div class='bm'><h3>Chess against a computer opponent</h3><p>The only complete attempt of two; the other ran out of room. In our test it answered 1.e4 with 1β¦d5 after searching 12,498 positions.</p><details><summary>Read the prompt</summary><p class='prompt'>Build a complete chess game I can play in the browser against a computer opponent. Implement all the rules: legal move generation, check, checkmate, stalemate, castling, en passant and promotion. The computer should search a few moves ahead with minimax and alpha-beta pruning. Show captured pieces, a move list in algebraic notation, highlight legal moves when I pick a piece, and offer undo and new game. | |
| Return the complete program in ONE html code block. Everything must be in that single file: no external libraries, CDNs, fonts, images or network requests of any kind. It must work by opening the file in a modern desktop browser, and it should also be usable on a phone.</p></details><a class='open' href='showcase/chess.html' target='_blank' rel='noopener'>Open full screen</a></div></article><div class='builds'><article class='build'><div class='frame'><iframe src='showcase/shooter.html' title='Neon Void: an arcade space shooter' loading='lazy'></iframe></div><div class='bm'><h3>Neon Void: an arcade space shooter</h3><p>Better of two attempts. Sound is synthesised in the browser; press Start, then move with the arrow keys or drag.</p><details><summary>Read the prompt</summary><p class='prompt'>Build a polished neon arcade space shooter. Waves of enemies with different movement patterns, power-ups, a boss every fifth wave, particle explosions, screen shake, a score and a high score saved in localStorage, sound effects synthesised with the Web Audio API, keyboard and touch controls, pause, and a game-over screen with restart. | |
| Return the complete program in ONE html code block. Everything must be in that single file: no external libraries, CDNs, fonts, images or network requests of any kind. It must work by opening the file in a modern desktop browser, and it should also be usable on a phone.</p></details><a class='open' href='showcase/shooter.html' target='_blank' rel='noopener'>Open full screen</a></div></article><article class='build'><div class='frame'><iframe src='showcase/solar.html' title='Solar system simulator' loading='lazy'></iframe></div><div class='bm'><h3>Solar system simulator</h3><p>Better of two attempts. Real relative orbital periods on a compressed distance scale; tap a planet for its facts.</p><details><summary>Read the prompt</summary><p class='prompt'>Build an interactive solar system simulator on a canvas. The eight planets orbit the Sun with their real relative orbital periods and roughly correct relative distances (use a compressed scale and say so on screen). Include a time-speed slider, pause, zoom and pan, orbit trails, and clicking a planet shows a panel with real facts about it (diameter, distance from the Sun, orbital period, number of moons). | |
| Return the complete program in ONE html code block. Everything must be in that single file: no external libraries, CDNs, fonts, images or network requests of any kind. It must work by opening the file in a modern desktop browser, and it should also be usable on a phone.</p></details><a class='open' href='showcase/solar.html' target='_blank' rel='noopener'>Open full screen</a></div></article><article class='build'><div class='frame'><iframe src='showcase/drums.html' title='16-step drum machine and bass synth' loading='lazy'></iframe></div><div class='bm'><h3>16-step drum machine and bass synth</h3><p>Better of two attempts. Every sound is synthesised with the Web Audio API; tap Play.</p><details><summary>Read the prompt</summary><p class='prompt'>Build a 16-step drum machine and bass synth using only the Web Audio API, with every sound synthesised (no samples): kick, snare, closed hat, open hat, clap and a bass line with a selectable note per step. Tempo and swing controls, per-track volume and mute, a playhead that lights the current step, three preset patterns, and save and load of patterns in localStorage. | |
| Return the complete program in ONE html code block. Everything must be in that single file: no external libraries, CDNs, fonts, images or network requests of any kind. It must work by opening the file in a modern desktop browser, and it should also be usable on a phone.</p></details><a class='open' href='showcase/drums.html' target='_blank' rel='noopener'>Open full screen</a></div></article><article class='build'><div class='frame'><iframe src='showcase/finance.html' title='Personal finance tracker in rupees' loading='lazy'></iframe></div><div class='bm'><h3>Personal finance tracker in rupees</h3><p>The working attempt of two; the other had a script error. Data stays in your browser.</p><details><summary>Read the prompt</summary><p class='prompt'>Build a personal finance tracker app. Add income and expenses with amount, category, date and note; set a monthly budget per category; show this month's totals, budget progress bars, a spending-by-category donut chart and a 6-month income-versus-expense bar chart, all drawn by hand on canvas. Search and filter the transaction list, edit and delete entries, save everything in localStorage, and import and export CSV. Use Indian rupee formatting. | |
| Return the complete program in ONE html code block. Everything must be in that single file: no external libraries, CDNs, fonts, images or network requests of any kind. It must work by opening the file in a modern desktop browser, and it should also be usable on a phone.</p></details><a class='open' href='showcase/finance.html' target='_blank' rel='noopener'>Open full screen</a></div></article><article class='build'><div class='frame'><iframe src='showcase/cloth.html' title='Cloth simulation you can tear' loading='lazy'></iframe></div><div class='bm'><h3>Cloth simulation you can tear</h3><p>Better of two attempts. Drag to pull the cloth; hold Shift, or switch on Tear, to cut it.</p><details><summary>Read the prompt</summary><p class='prompt'>Build an interactive cloth simulation on a canvas using Verlet integration: a grid of points joined by springy constraints, pinned along the top edge, swaying under gravity and a gentle wind. Dragging the mouse or a finger pulls the cloth; holding a modifier key (or a toggle button on touch screens) tears it. Add sliders for gravity, wind and stiffness, and a reset button. Keep it smooth at 60 frames per second. | |
| Return the complete program in ONE html code block. Everything must be in that single file: no external libraries, CDNs, fonts, images or network requests of any kind. It must work by opening the file in a modern desktop browser, and it should also be usable on a phone.</p></details><a class='open' href='showcase/cloth.html' target='_blank' rel='noopener'>Open full screen</a></div></article></div></section> | |
| <section class="sec" id="method"><h2>How we measured</h2> | |
| <div class="method"><p class="intro" style="margin:0">Torki 2 was tested on the same production model that serves Torki, between 30 September and 1 October 2026. Every score was checked by hand from the raw responses before publishing.</p> | |
| <dl> | |
| <dt>Settings</dt><dd>Thinking on at the depth Torki's Stellar and Cosmos modes use, temperature 0.7, answers up to 65,536 tokens. No web search, code execution or other tools on any benchmark.</dd> | |
| <dt>Prompts and scoring</dt><dd>Each benchmark's public reference prompts and scorers. GPQA Diamond is averaged over 4 runs with reshuffled options; AIME over 8 attempts per problem (4 for AIME 2026).</dd> | |
| <dt>Samples</dt><dd>Humanity's Last Exam (first 400 text-only questions), MMLU-Pro (788 questions), MILU (50 per language), MMMLU (200 per language) and RULER (50 per task and length) are fixed samples. Humanity's Last Exam and MMLU-Pro are weighted to the full subject mix.</dd> | |
| <dt>Grading Humanity's Last Exam</dt><dd>The reference grader is a commercial model from another lab, so we ran the official grading prompt on Torki 2; 16 of 16 decisions checked by hand matched the official answers. A strict rule-based reading gives 27.0%, the floor.</dd> | |
| <dt>Other models</dt><dd>Their makers' published numbers, or an independent measurement (Artificial Analysis, Epoch AI, Vals AI) where the maker publishes none, at the highest stated reasoning setting and without tools.</dd> | |
| <dt>Data</dt><dd>Per-question results and every comparison source: <a href="https://huggingface.co/datasets/TorkeHub/torki-2-evals">TorkeHub/torki-2-evals</a>. Launch page: <a href="https://torkehub.com/torki-2">torkehub.com/torki-2</a>.</dd> | |
| </dl></div></section> | |
| </main> | |
| <footer><div class="w"><span>Β© 2026 TorkeHub Inc.</span><span><a href="https://torkehub.com">torkehub.com</a> <a href="https://torkiai.com">Try Torki</a></span></div></footer> | |
| </body></html> |