Mathematics on AIME25 (Accuracy)
93.33Accuracygpt-oss-puzzle-88B
Evaluation Results
| Method | Links | |
|---|---|---|
| gpt-oss-puzzle-88BReasoning effort=High, KV cache precision=FP82026.02 | 93.33 | |
| gpt-oss-puzzle-88BReasoning effort=High, KV cache precision=BF162026.02 | 92.92 | |
| Regex2026.04 | 91.8 | |
| gpt-oss-120BReasoning effort=High, KV cache precision=BF162026.02 | 91.46 | |
| BERT-Judgeclean_name=BERT-as-a-Judge2026.04 | 91.4 | |
| Lowest CentroidModel=GPT-OSS-120B, Number of Samples=642026.04 | 90 | |
| gpt-oss-120BReasoning effort=High, KV cache precision=FP82026.02 | 89.58 | |
| BF16Precision=BF16, Backbone=Nemotron 3 Nano2026.01 | 89.1 | |
| NVFP4 QADPrecision=NVFP4, Type=QAD, Backbone=Nemotron 3 Nano2026.01 | 87.9 | |
| gpt-oss-puzzle-88BReasoning effort=Medium, KV cache precision=BF162026.02 | 86.88 | |
| Lowest CentroidModel=Olmo-3.1-Think, Number of Samples=642026.04 | 86.7 | |
| gpt-oss-puzzle-88BReasoning effort=Medium, KV cache precision=FP82026.02 | 86.67 | |
| NVFP4 PTQPrecision=NVFP4, Type=PTQ, Backbone=Nemotron 3 Nano2026.01 | 85 | |
| NVFP4 QATPrecision=NVFP4, Type=QAT, Backbone=Nemotron 3 Nano2026.01 | 83.3 | |
| Lowest CentroidModel=QWQ-32B, Number of Samples=642026.04 | 83.3 | |
| LLM-Judgebackbone=Qwen-3 0.6B2026.04 | 83 | |
| Bottom WindowModel=Olmo-3.1-Think, Number of Samples=642026.04 | 80 | |
| DSMoEModel=GPT-OSS-20B2026.04 | 78.6 | |
| gpt-oss-120BReasoning effort=Medium, KV cache precision=FP82026.02 | 77.92 | |
| gpt-oss-120BReasoning effort=Medium, KV cache precision=BF162026.02 | 76.88 | |
| Self-CertaintyModel=Olmo-3.1-Think, Number of Samples=642026.04 | 76.7 | |
| Lowest CentroidModel=Qwen3-14B, Number of Samples=642026.04 | 76.7 | |
| Tail ConfidenceModel=QWQ-32B, Number of Samples=642026.04 | 76.7 | |
| Greedy DecodingModel=GPT-OSS-120B, Number of Samples=642026.04 | 76.7 | |
| Pass@1Model=GPT-OSS-120B, Number of Samples=642026.04 | 76.5 | |
| Pass@1Model=Olmo-3.1-Think, Number of Samples=642026.04 | 75.7 | |
| Tail ConfidenceModel=Olmo-3.1-Think, Number of Samples=642026.04 | 73.3 | |
| Self-CertaintyModel=Qwen3-14B, Number of Samples=642026.04 | 73.3 | |
| Bottom WindowModel=Qwen3-14B, Number of Samples=642026.04 | 73.3 | |
| Bottom WindowModel=GPT-OSS-120B, Number of Samples=642026.04 | 73.3 | |
| Pass@1Model=Qwen3-14B, Number of Samples=642026.04 | 71.4 | |
| Greedy DecodingModel=Olmo-3.1-Think, Number of Samples=642026.04 | 70 | |
| Greedy DecodingModel=Qwen3-14B, Number of Samples=642026.04 | 70 | |
| Tail ConfidenceModel=Qwen3-14B, Number of Samples=642026.04 | 70 | |
| Self-CertaintyModel=QWQ-32B, Number of Samples=642026.04 | 70 | |
| Bottom WindowModel=QWQ-32B, Number of Samples=642026.04 | 70 | |
| Self-CertaintyModel=GPT-OSS-120B, Number of Samples=642026.04 | 70 | |
| Tail ConfidenceModel=GPT-OSS-120B, Number of Samples=642026.04 | 70 | |
| Pass@1Model=QWQ-32B, Number of Samples=642026.04 | 68.5 | |
| Orig.Model=GPT-OSS-20B2026.04 | 66.3 | |
| gpt-oss-puzzle-88BReasoning effort=Low, KV cache precision=BF162026.02 | 66.25 | |
| LobsterBase Model=GPT-OSS:120B, Framework Category=Commercial Framework (multi-agent mode)2026.04 | 63.33 | |
| Agent Q-MixBase Model=GPT-OSS:120B2026.04 | 63.33 | |
| gpt-oss-puzzle-88BReasoning effort=Low, KV cache precision=FP82026.02 | 62.89 | |
| Qwen3-8BInference Mode=think2026.03 | 60 | |
| DSMoEModel=Qwen3-30B-Instruct2026.04 | 60 | |
| SFTModel=GPT-OSS-20B2026.04 | 60 | |
| Greedy DecodingModel=QWQ-32B, Number of Samples=642026.04 | 56.7 | |
| SSA-LLM-8BInference Mode=think2026.03 | 56.67 | |
| AutoGenBase Model=GPT-OSS:120B, Framework Category=Commercial Framework (multi-agent mode)2026.04 | 53.33 | |
| Agent FrameworkBase Model=GPT-OSS:120B, Framework Category=Commercial Framework (multi-agent mode)2026.04 | 53.33 | |
| gpt-oss-120BReasoning effort=Low, KV cache precision=FP82026.02 | 51.88 | |
| gpt-oss-120BReasoning effort=Low, KV cache precision=BF162026.02 | 50 | |
| SFTModel=Qwen3-30B-Instruct2026.04 | 50 | |
| Orig.Model=Qwen3-30B-Instruct2026.04 | 46.7 | |
| RICEModel=Qwen3-30B-Instruct2026.04 | 46.7 | |
| RICEModel=GPT-OSS-20B2026.04 | 46.7 | |
| Tail ConfidenceModel=Ministral-3-14B-Instruct, Number of Samples=642026.04 | 46.7 | |
| LangGraphBase Model=GPT-OSS:120B, Framework Category=Commercial Framework (multi-agent mode)2026.04 | 46.67 | |
| LangGraphBase Model=Gemini-3.1-Flash-Lite, Framework Category=Commercial Framework (multi-agent mode)2026.04 | 46.67 | |
| Self-CertaintyModel=Ministral-3-14B-Instruct, Number of Samples=642026.04 | 43.3 | |
| G-DesignerBase Model=GPT-OSS:120B, Framework Category=Adaptive topology methods2026.04 | 40 | |
| GTDBase Model=Gemini-3.1-Flash-Lite, Framework Category=Adaptive topology methods2026.04 | 40 | |
| TopoDIMBase Model=Gemini-3.1-Flash-Lite, Framework Category=Adaptive topology methods2026.04 | 40 | |
| AutoGenBase Model=Gemini-3.1-Flash-Lite, Framework Category=Commercial Framework (multi-agent mode)2026.04 | 40 | |
| Agent FrameworkBase Model=Gemini-3.1-Flash-Lite, Framework Category=Commercial Framework (multi-agent mode)2026.04 | 40 | |
| Agent Q-MixBase Model=Gemini-3.1-Flash-Lite2026.04 | 40 | |
| Lowest CentroidModel=Ministral-3-14B-Instruct, Number of Samples=642026.04 | 40 | |
| Bottom WindowModel=Ministral-3-14B-Instruct, Number of Samples=642026.04 | 36.7 | |
| GPTSwarmBase Model=GPT-OSS:120B, Framework Category=Adaptive topology methods2026.04 | 36.67 | |
| MaASBase Model=GPT-OSS:120B, Framework Category=Adaptive topology methods2026.04 | 36.67 | |
| TopoDIMBase Model=GPT-OSS:120B, Framework Category=Adaptive topology methods2026.04 | 36.67 | |
| GPTSwarmBase Model=Gemini-3.1-Flash-Lite, Framework Category=Adaptive topology methods2026.04 | 36.67 | |
| MaASBase Model=Gemini-3.1-Flash-Lite, Framework Category=Adaptive topology methods2026.04 | 36.67 | |
| LobsterBase Model=Gemini-3.1-Flash-Lite, Framework Category=Commercial Framework (multi-agent mode)2026.04 | 36.67 | |
| AgentDropoutBase Model=GPT-OSS:120B, Framework Category=Adaptive topology methods2026.04 | 33.33 | |
| GTDBase Model=GPT-OSS:120B, Framework Category=Adaptive topology methods2026.04 | 33.33 | |
| LLM-DebateBase Model=Gemini-3.1-Flash-Lite, Framework Category=Static multi-agent2026.04 | 33.33 | |
| G-DesignerBase Model=Gemini-3.1-Flash-Lite, Framework Category=Adaptive topology methods2026.04 | 33.33 | |
| LLM-DebateBase Model=GPT-OSS:120B, Framework Category=Static multi-agent2026.04 | 30 | |
| Base (direct)Base Model=Gemini-3.1-Flash-Lite, Framework Category=Single-agent baseline2026.04 | 30 | |
| Greedy DecodingModel=Ministral-3-14B-Instruct, Number of Samples=642026.04 | 30 | |
| Pass@1Model=Ministral-3-14B-Instruct, Number of Samples=642026.04 | 27.5 | |
| AgentDropoutBase Model=Gemini-3.1-Flash-Lite, Framework Category=Adaptive topology methods2026.04 | 26.67 | |
| LLaDA2.1-minigeneration length=optimal, block length=optimal, denoising steps=optimal2026.04 | 26.67 | |
| SSA-LLM-8BInference Mode=no-think2026.03 | 23.33 | |
| Base (direct)Base Model=GPT-OSS:120B, Framework Category=Single-agent baseline2026.04 | 23.33 | |
| LLaDA2.0-minigeneration length=optimal, block length=optimal, denoising steps=optimal2026.04 | 23.33 | |
| TIESGroup=Qwen3-4B-Base, Category=Merged Models2026.06 | 20 | |
| DARE + TIESGroup=Qwen3-4B-Base, Category=Merged Models2026.06 | 20 | |
| RESMERGEGroup=Qwen3-4B-Base, Category=Merged Models2026.06 | 20 | |
| Qwen3-8BInference Mode=no-think2026.03 | 16.67 | |
| SDAR-8B-Chatgeneration length=optimal, block length=optimal, denoising steps=optimal2026.04 | 16.67 | |
| GRPOGroup=Qwen3-4B-Base, Category=Base and Expert Models2026.06 | 16.67 | |
| General-ReasonerGroup=Qwen3-4B-Base, Category=Base and Expert Models2026.06 | 16.67 | |
| SpiralGroup=Qwen3-4B-Base, Category=Base and Expert Models2026.06 | 16.67 | |
| TSV-MergeGroup=Qwen3-4B-Base, Category=Merged Models2026.06 | 16.67 | |
| RAMGroup=Qwen3-4B-Base, Category=Merged Models2026.06 | 16.67 | |
| TAGroup=Qwen3-4B-Base, Category=Merged Models2026.06 | 13.33 | |
| SDAR-30B-A3Bgeneration length=optimal, block length=optimal, denoising steps=optimal2026.04 | 6.67 |