Scientific Reasoning on GPQA Diamond (Acc, Tok, CR, Cost)
65.25AccuracyFull
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| FullModel=Qwen3-14B, KV Cache Budget=100%2026.06 | 65.25 | — | — | — | |
| VanillaModel=Qwen3-14B2026.04 | 65.2 | 8,447 | 100 | 8,447 | |
| RCPDModel=Qwen3-14B2026.04 | 65.2 | 8,098 | 95.9 | 8,433 | |
| SeerAttention-RModel=Qwen3-14B, KV Cache Budget=~25%2026.06 | 61.68 | — | — | — | |
| R-KVModel=Qwen3-14B, KV Cache Budget=~25%2026.06 | 59.09 | — | — | — | |
| EATModel=Qwen3-14B2026.04 | 58.3 | 6,355 | 75.2 | 6,533 | |
| VASE-ATTNVModel=Qwen3-14B, KV Cache Budget=~25%2026.06 | 57.39 | — | — | — | |
| VASE-DKVModel=Qwen3-14B, KV Cache Budget=~25%2026.06 | 56.76 | — | — | — | |
| FullModel=Qwen3-4B, KV Cache Budget=100%2026.06 | 56.19 | — | — | — | |
| DEERModel=Qwen3-14B2026.04 | 56 | 3,763 | 44.5 | 3,799 | |
| VanillaModel=Qwen3-4B2026.04 | 55.8 | 9,536 | 100 | 9,536 | |
| RCPDModel=Qwen3-4B2026.04 | 55.8 | 9,424 | 98.8 | 9,825 | |
| Think or NotModel=Qwen3-4B2026.04 | 54.1 | 6,865 | 72 | 10,371 | |
| CoDE-StopModel=Qwen3-14B2026.04 | 53.8 | 3,128 | 37 | 3,159 | |
| VanillaModel=Nemotron-8B2026.04 | 53.2 | 7,318 | 100 | 7,318 | |
| RCPDModel=Nemotron-8B2026.04 | 53.2 | 7,243 | 99 | 7,558 | |
| DEERModel=Qwen3-4B2026.04 | 53 | 6,412 | 67.2 | 6,496 | |
| CoDE-StopModel=Qwen3-4B2026.04 | 52.8 | 6,064 | 63.6 | 6,144 | |
| Think or NotModel=Nemotron-8B2026.04 | 52.8 | 6,804 | 93 | 11,142 | |
| DEERModel=Nemotron-8B2026.04 | 51.7 | 5,883 | 80.4 | 5,967 | |
| CoDE-StopModel=Nemotron-8B2026.04 | 50.9 | 5,634 | 77 | 5,715 | |
| SeerAttention-RModel=Qwen3-4B, KV Cache Budget=~25%2026.06 | 49.94 | — | — | — | |
| R-KVModel=Qwen3-4B, KV Cache Budget=~25%2026.06 | 49.05 | — | — | — | |
| EATModel=Qwen3-4B2026.04 | 49 | 6,459 | 67.7 | 6,670 | |
| EATModel=Nemotron-8B2026.04 | 49 | 6,334 | 86.6 | 6,535 | |
| CurDKVModel=Qwen3-14B, KV Cache Budget=~25%2026.06 | 48.48 | — | — | — | |
| VASE-ATTNVModel=Qwen3-4B, KV Cache Budget=~25%2026.06 | 47.98 | — | — | — | |
| Think or NotModel=DeepSeek-R12026.04 | 47.5 | 9,056 | 98.7 | 18,243 | |
| VanillaModel=DeepSeek-R12026.04 | 47.4 | 9,177 | 100 | 9,177 | |
| RCPDModel=DeepSeek-R12026.04 | 47.4 | 9,178 | 100 | 9,527 | |
| ExpThinkBackbone=DeepSeek-R1-Distill-Qwen-7B2026.05 | 47.2 | 2,870.9 | — | — | |
| DEERModel=DeepSeek-R12026.04 | 47 | 8,032 | 87.5 | 8,468 | |
| CoDE-StopModel=DeepSeek-R12026.04 | 46.9 | 7,898 | 86.1 | 8,318 | |
| EATModel=DeepSeek-R12026.04 | 45.7 | 7,948 | 86.6 | 8,375 | |
| Answer ConvergenceModel=Qwen3-14B2026.04 | 44.7 | 830 | 9.8 | 1,060 | |
| LaserBackbone=DeepSeek-R1-Distill-Qwen-7B2026.05 | 44.1 | 3,492.9 | — | — | |
| VASE-DKVModel=Qwen3-4B, KV Cache Budget=~25%2026.06 | 43.94 | — | — | — | |
| LC-R1Backbone=DeepSeek-R1-Distill-Qwen-7B2026.05 | 40.8 | 4,295.6 | — | — | |
| JETBackbone=DeepSeek-R1-Distill-Qwen-7B2026.05 | 40.3 | 6,491.1 | — | — | |
| Answer ConvergenceModel=Qwen3-4B2026.04 | 39.1 | 947 | 9.9 | 1,210 | |
| AdaptThinkBackbone=DeepSeek-R1-Distill-Qwen-7B2026.05 | 38.3 | 5,225 | — | — | |
| CurDKVModel=Qwen3-4B, KV Cache Budget=~25%2026.06 | 37.94 | — | — | — | |
| AutoThinkBackbone=DeepSeek-R1-Distill-Qwen-7B2026.05 | 37.9 | 5,431.5 | — | — | |
| SnapKVModel=Qwen3-4B, KV Cache Budget=~25%2026.06 | 36.87 | — | — | — | |
| VanillaBackbone=DeepSeek-R1-Distill-Qwen-7B2026.05 | 35.2 | 7,288.7 | — | — | |
| Answer ConvergenceModel=Nemotron-8B2026.04 | 33.9 | 1,231 | 16.8 | 1,656 | |
| Answer ConvergenceModel=DeepSeek-R12026.04 | 33.3 | 2,629 | 28.6 | 4,394 | |
| ExpThinkBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.05 | 25.4 | 3,259.4 | — | — | |
| AutoThinkBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.05 | 25 | 5,324 | — | — | |
| AdaptThinkBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.05 | 24.4 | 4,013.8 | — | — | |
| JETBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.05 | 23.9 | 6,566.8 | — | — | |
| LC-R1Backbone=DeepSeek-R1-Distill-Qwen-1.5B2026.05 | 23 | 5,285.2 | — | — | |
| LaserBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.05 | 21.8 | 5,136.9 | — | — | |
| VanillaBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.05 | 16 | 8,514.3 | — | — |