Question Answering on GPQA Diamond
84.2AccuracyDSMoE
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DSMoEModel=GPT-OSS-20B2026.04 | 84.2 | — | |
| SFTModel=GPT-OSS-120B2026.04 | 73.7 | — | |
| DSMoEModel=Qwen3-30B-Instruct2026.04 | 73.7 | — | |
| DSMoEModel=GPT-OSS-120B2026.04 | 68.4 | — | |
| Orig.Model=Qwen3-30B-Instruct2026.04 | 68.4 | — | |
| Primitives-based MASModel=DeepSeek-R1-Distill Llama-70B2026.02 | 66.7 | — | |
| TextMASModel=DeepSeek-R1-Distill Llama-70B2026.02 | 65.7 | — | |
| VotingModel=DeepSeek-R1-Distill Llama-70B2026.02 | 65.7 | — | |
| SingleModel=DeepSeek-R1-Distill Llama-70B2026.02 | 65.2 | — | |
| ReviewModel=DeepSeek-R1-Distill Llama-70B2026.02 | 65.2 | — | |
| PlanningModel=DeepSeek-R1-Distill Llama-70B2026.02 | 65.2 | — | |
| Primitives-based MASModel=DeepSeek-R1-Distill Qwen-32B2026.02 | 64.6 | — | |
| LatentMASModel=DeepSeek-R1-Distill Qwen-32B2026.02 | 63.6 | — | |
| RICEModel=GPT-OSS-120B2026.04 | 63.2 | — | |
| SFTModel=Qwen3-30B-Instruct2026.04 | 63.2 | — | |
| SFTModel=GPT-OSS-20B2026.04 | 63.2 | — | |
| VotingModel=DeepSeek-R1-Distill Qwen-32B2026.02 | 63.1 | — | |
| TextMASModel=DeepSeek-R1-Distill Qwen-32B2026.02 | 62.6 | — | |
| PlanningModel=DeepSeek-R1-Distill Qwen-32B2026.02 | 62.6 | — | |
| SingleModel=DeepSeek-R1-Distill Qwen-32B2026.02 | 62.1 | — | |
| ReviewModel=DeepSeek-R1-Distill Qwen-32B2026.02 | 62.1 | — | |
| Deliberative SearchApproach=Ours2026.04 | 61.62 | — | |
| Primitives-based MASModel=Qwen3-8B2026.02 | 59.6 | — | |
| Orig.Model=GPT-OSS-120B2026.04 | 58 | — | |
| RICEModel=GPT-OSS-20B2026.04 | 57.9 | — | |
| DSMoEModel=Qwen3-30B-Thinking2026.04 | 57.9 | — | |
| VotingModel=Qwen3-8B2026.02 | 55 | — | |
| Claude-3-7-Sonnet2026.04 | 54.04 | — | |
| Claude-3-7-Sonnet-ThinkingReasoning Mode=Thinking2026.04 | 54.04 | — | |
| Primitives-based MASBackbone=Qwen3-14B2026.02 | 53.5 | 5 | |
| AdaptOrchLat.=1.5×, Tok(K)=15.92026.02 | 53.1 | — | |
| Orig.Model=Qwen3-30B-Thinking2026.04 | 52.6 | — | |
| Self-MoA (matched)Lat.=1.4×, Tok(K)=16.82026.02 | 52.3 | — | |
| Gemini-2.5-Pro-Exp2026.04 | 52.02 | — | |
| LatentMASBackbone=Qwen3-14B2026.02 | 52 | 3.5 | |
| ReviewBackbone=Qwen3-14B2026.02 | 52 | 3.5 | |
| VotingBackbone=Qwen3-14B2026.02 | 52 | 3.5 | |
| TextMASBackbone=Qwen3-14B2026.02 | 51.5 | 3 | |
| PlanningBackbone=Qwen3-14B2026.02 | 51.5 | 3 | |
| PlanningModel=Qwen3-8B2026.02 | 51 | — | |
| Primitives-based MASBackbone=Qwen3-4B2026.02 | 50.5 | 14.2 | |
| Static-SequentialLat.=2.4×, Tok(K)=16.42026.02 | 50.3 | — | |
| R1-CI-14B2025.05 | 50.2 | — | |
| MoA-3LLat.=2.8×, Tok(K)=31.22026.02 | 49.8 | — | |
| Qwen-VL2.5-72BParameters=72B2026.04 | 48.99 | — | |
| ReviewModel=Qwen3-8B2026.02 | 48.9 | — | |
| SingleBackbone=Qwen3-14B2026.02 | 48.5 | — | |
| VotingBackbone=Qwen3-4B2026.02 | 47.7 | 11.4 | |
| LLM-BlenderLat.=1.6×, Tok(K)=22.32026.02 | 47.7 | — | |
| RICEModel=Qwen3-30B-Instruct2026.04 | 47.4 | — | |
| Orig.Model=GPT-OSS-20B2026.04 | 47.4 | — | |
| Single BestLat.=1.0×, Tok(K)=4.12026.02 | 46.2 | — | |
| ReviewBackbone=Qwen3-4B2026.02 | 45.5 | 9.2 | |
| LatentMASModel=Qwen3-8B2026.02 | 45.5 | — | |
| Static-ParallelLat.=1.3×, Tok(K)=18.72026.02 | 44.1 | — | |
| TextMASModel=Qwen3-8B2026.02 | 43.4 | — | |
| PlanningBackbone=Qwen3-4B2026.02 | 42.4 | 6.1 | |
| RICEModel=Qwen3-30B-Thinking2026.04 | 42.1 | — | |
| LatentMASBackbone=Qwen3-4B2026.02 | 41.9 | 5.6 | |
| LatentMASModel=DeepSeek-R1-Distill Llama-70B2026.02 | 41.9 | — | |
| Qwen-2.5-14BInference Mode=CodeSteer2025.05 | 41.2 | — | |
| Gemini-2.0-Flash-ThinkingReasoning Mode=Thinking2026.04 | 40.91 | — | |
| TextMASBackbone=Qwen3-4B2026.02 | 40.4 | 4.1 | |
| Qwen-2.5-14BInference Mode=All Text2025.05 | 40.1 | — | |
| SingleModel=Qwen3-8B2026.02 | 39.9 | — | |
| R1-CI-7B2025.05 | 39 | — | |
| SingleBackbone=Qwen3-4B2026.02 | 36.3 | — | |
| GPT-4o2026.04 | 33.48 | — | |
| Qwen-2.5-7BInference Mode=CodeSteer2025.05 | 32.9 | — | |
| Gemini-2.0-Flash2026.04 | 32.39 | — | |
| Qwen-2.5-7BInference Mode=All Text2025.05 | 31.2 | — | |
| Base ModelBackbone=Meta-Llama-3-8B-Instruct, Zero-shot=true2026.02 | 30.81 | — | |
| LLaMA-3-70B-Instruct2025.06 | 30.3 | — | |
| RCOBase Model=LLaMA-3-8B-Instruct2025.06 | 30.06 | — | |
| WIM Fixed JudgeBackbone=Meta-Llama-3-8B-Instruct, Judge Strategy=Fixed, Zero-shot=true2026.02 | 29.8 | — | |
| Self-refinement2025.06 | 29.6 | — | |
| RCOBase Model=LLaMA-2-7B-Chat2025.06 | 28.83 | — | |
| DPCOBase Model=LLaMA-2-7B-Chat2025.06 | 28.79 | — | |
| RCOBase Model=LLaMA-2-13B-Chat2025.06 | 28.35 | — | |
| NumericalBackbone=Meta-Llama-3-8B-Instruct, Zero-shot=true2026.02 | 28.28 | — | |
| WIM Changing JudgeBackbone=Meta-Llama-3-8B-Instruct, Judge Strategy=Changing, Zero-shot=true2026.02 | 28.28 | — | |
| RCOBase Model=Auto-J-13B2025.06 | 28.04 | — | |
| RCOBase Model=UltraCM-13B2025.06 | 27.83 | — | |
| RandomBackbone=Meta-Llama-3-8B-Instruct, Zero-shot=true2026.02 | 27.78 | — | |
| Initial Answer2025.06 | 27.65 | — | |
| LLaMA-2-7B-Chat2025.06 | 27.47 | — | |
| DPCOBase Model=LLaMA-2-13B-Chat2025.06 | 27.17 | — | |
| LLAMA-2-70B-Chat2025.06 | 26.77 | — | |
| DPCOBase Model=Auto-J-13B2025.06 | 26.61 | — | |
| LLaMA-2-13B-Chat2025.06 | 26.49 | — | |
| DPCOBase Model=UltraCM-13B2025.06 | 26.35 | — | |
| UltraCM-13B2025.06 | 25.96 | — | |
| DPCOBase Model=LLaMA-3-8B-Instruct2025.06 | 25.76 | — | |
| LLaMA-3-8B-Instruct2025.06 | 25.47 | — | |
| Aligner2025.06 | 24.15 | — | |
| Auto-J-13B2025.06 | 23.56 | — | |
| SFTModel=Qwen3-30B-Thinking2026.04 | 21.1 | — |