Mathematical Reasoning on Math 500 (Flex Match, Exact Match, Avg. Gen. Length)
91.3Flex MatchQUOKA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| QUOKABase Model=GPT-OSS-20b, Budget=256, Reasoning generation length limit=8192 tokens2026.02 | 91.3 | 71.1 | 2,539.11 | |
| QUOKABase Model=GPT-OSS-20b, Budget=128, Reasoning generation length limit=8192 tokens2026.02 | 91.1 | 70.7 | 2,199.93 | |
| LokiBase Model=GPT-OSS-20b, Budget=128, Reasoning generation length limit=8192 tokens2026.02 | 90.5 | 71.1 | 2,276 | |
| LokiBase Model=GPT-OSS-20b, Budget=256, Reasoning generation length limit=8192 tokens2026.02 | 90.3 | 71.4 | 2,181.49 | |
| GPT-OSS-20bBudget=Dense, Reasoning generation length limit=8192 tokens2026.02 | 89.3 | 69.4 | 2,558.262 | |
| LessIsMoreBase Model=GPT-OSS-20b, Budget=256, Reasoning generation length limit=8192 tokens2026.02 | 85 | 68.1 | 2,830.49 | |
| SparQBase Model=GPT-OSS-20b, Budget=256, Reasoning generation length limit=8192 tokens2026.02 | 84.8 | 64.8 | 2,436.86 | |
| QUOKABase Model=Qwen3-4B, Budget=128, Reasoning generation length limit=8192 tokens2026.02 | 82.8 | 70.8 | 1,119.46 | |
| QUOKABase Model=Qwen3-4B, Budget=256, Reasoning generation length limit=8192 tokens2026.02 | 82.3 | 70.3 | 1,156.12 | |
| LokiBase Model=Qwen3-4B, Budget=256, Reasoning generation length limit=8192 tokens2026.02 | 82.1 | 70.2 | 1,143.51 | |
| Qwen3-4BBudget=Dense, Reasoning generation length limit=8192 tokens2026.02 | 82 | 70.1 | 1,179.78 | |
| LessIsMoreBase Model=Qwen3-4B, Budget=256, Reasoning generation length limit=8192 tokens2026.02 | 81.4 | 69.5 | 1,618 | |
| LokiBase Model=Qwen3-4B, Budget=128, Reasoning generation length limit=8192 tokens2026.02 | 78.5 | 66.8 | 1,345.88 | |
| LessIsMoreBase Model=GPT-OSS-20b, Budget=128, Reasoning generation length limit=8192 tokens2026.02 | 78.2 | 62 | 3,591.4 | |
| SparQBase Model=Qwen3-4B, Budget=256, Reasoning generation length limit=8192 tokens2026.02 | 77.7 | 65.8 | 1,518.82 | |
| SparQBase Model=GPT-OSS-20b, Budget=128, Reasoning generation length limit=8192 tokens2026.02 | 74.4 | 56.9 | 3,023.71 | |
| LessIsMoreBase Model=Qwen3-4B, Budget=128, Reasoning generation length limit=8192 tokens2026.02 | 72.9 | 62.1 | 2,674.25 | |
| SparQBase Model=Qwen3-4B, Budget=128, Reasoning generation length limit=8192 tokens2026.02 | 72.7 | 61.5 | 2,044.902 | |
| SparQBase Model=Smollm3, Budget=256, Reasoning generation length limit=8192 tokens2026.02 | 66.5 | 53.2 | 634.34 | |
| QUOKABase Model=Smollm3, Budget=256, Reasoning generation length limit=8192 tokens2026.02 | 66.1 | 52.7 | 614.73 | |
| QUOKABase Model=Smollm3, Budget=128, Reasoning generation length limit=8192 tokens2026.02 | 66 | 52.1 | 578.1 | |
| Smollm3Budget=Dense, Reasoning generation length limit=8192 tokens2026.02 | 65.9 | 52.2 | 629.04 | |
| LessIsMoreBase Model=Smollm3, Budget=256, Reasoning generation length limit=8192 tokens2026.02 | 65.7 | 52.7 | 598.907 | |
| LokiBase Model=Smollm3, Budget=256, Reasoning generation length limit=8192 tokens2026.02 | 65.4 | 52 | 618.47 | |
| SparQBase Model=Smollm3, Budget=128, Reasoning generation length limit=8192 tokens2026.02 | 64.7 | 51.4 | 715.18 | |
| LessIsMoreBase Model=Smollm3, Budget=128, Reasoning generation length limit=8192 tokens2026.02 | 62.3 | 49.5 | 642.26 | |
| LokiBase Model=Smollm3, Budget=128, Reasoning generation length limit=8192 tokens2026.02 | 61.5 | 49.4 | 621.02 |