Reasoning on GPQA Diamond
91.9AccuracyGemini-3.0 Pro
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Gemini-3.0 ProVariant=Pro2026.01 | 91.9 | — | — | — | — | — | — | — | — | — | — | — | |
| GPT-5 HighVariant=High2026.01 | 85.7 | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen3.5-27BArchitecture=Dense, # Total Params=27B, # Activated Params=27B, Reasoning Mode=REASONING2026.04 | 85.5 | — | — | — | — | — | — | — | — | — | — | — | |
| RSAModel 2=GPT-5 mini, Pairing Type=Heterogeneous2026.04 | 85 | — | — | — | — | — | — | — | — | — | 0.52 | 1 | |
| Kimi-K2 ThinkingThinking Mode=true2026.01 | 84.5 | — | — | — | — | — | — | — | — | — | — | — | |
| MiMo-V2-FlashVariant=Flash2026.01 | 84.3 | — | — | — | — | — | — | — | — | — | — | — | |
| SQUEEZE EVOLVEModel 1=Qwen3-30B-A3B-I, Model 2=GPT-5 mini, p-value=20, Pairing Type=Heterogeneous2026.04 | 83.6 | — | — | — | — | — | — | — | — | — | 0.35 | 1.5 | |
| Claude Sonnet 4.5Variant=Sonnet 4.52026.01 | 83.4 | — | — | — | — | — | — | — | — | — | — | — | |
| DeepSeek-V3.2 ThinkingThinking Mode=true2026.01 | 82.4 | — | — | — | — | — | — | — | — | — | — | — | |
| PETS-On.LLM=GPT-120B, Confidence-Weighted=false2026.02 | 82.4 | — | — | — | — | — | — | — | 4,172 | 100 | — | — | |
| GPT-5 miniReasoning Mode=REASONING: HIGH2026.04 | 82.3 | — | — | — | — | — | — | — | — | — | — | — | |
| UniformLLM=GPT-120B, Confidence-Weighted=false2026.02 | 81.8 | — | — | — | — | — | — | — | 8,109 | 98 | — | — | |
| PETS-On. (conf)LLM=GPT-120B, Confidence-Weighted=true2026.02 | 81.6 | — | — | — | — | — | — | — | 5,094 | 99 | — | — | |
| Uniform (conf)LLM=GPT-120B, Confidence-Weighted=true2026.02 | 81.4 | — | — | — | — | — | — | — | 9,593 | 97 | — | — | |
| EXAONE 4.5 33BArchitecture=Dense, # Total Params=33B, # Activated Params=32B, Reasoning Mode=REASONING2026.04 | 80.5 | — | — | — | — | — | — | — | — | — | — | — | |
| K-EXAONE-236B-A23BArchitecture=MoE, # Total Params=236B, # Activated Params=23B, Reasoning Mode=REASONING2026.04 | 79.1 | — | — | — | — | — | — | — | — | — | — | — | |
| DeepSeek-R1-0528Quantization=FP8-Block-Wise2026.02 | 78.28 | — | — | — | — | — | — | — | — | — | — | — | |
| DeepSeek-R1-0528Quantization=W4A8-FP82026.02 | 77.37 | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen3-VL-235B-A22BArchitecture=MoE, # Total Params=236B, # Activated Params=22B, Reasoning Mode=Thinking2026.04 | 77.1 | — | — | — | — | — | — | — | — | — | — | — | |
| PETS-On. (conf)LLM=Qwen-Long, Confidence-Weighted=true2026.02 | 76.9 | — | — | — | — | — | — | — | 4,146 | 100 | — | — | |
| Uniform (conf)LLM=Qwen-Long, Confidence-Weighted=true2026.02 | 76.4 | — | — | — | — | — | — | — | 9,526 | 97 | — | — | |
| PETS-On.LLM=Qwen-Long, Confidence-Weighted=false2026.02 | 76.3 | — | — | — | — | — | — | — | 4,852 | 100 | — | — | |
| Uniform (conf)LLM=GPT-20B, Confidence-Weighted=true2026.02 | 76.3 | — | — | — | — | — | — | — | 9,895 | 98 | — | — | |
| PETS-On. (conf)LLM=GPT-20B, Confidence-Weighted=true2026.02 | 76.1 | — | — | — | — | — | — | — | 7,269 | 99 | — | — | |
| UniformLLM=GPT-20B, Confidence-Weighted=false2026.02 | 76.1 | — | — | — | — | — | — | — | 9,699 | 98 | — | — | |
| UniformLLM=Qwen-Long, Confidence-Weighted=false2026.02 | 76 | — | — | — | — | — | — | — | 9,195 | 97 | — | — | |
| PETS-On.LLM=GPT-20B, Confidence-Weighted=false2026.02 | 75.9 | — | — | — | — | — | — | — | 6,826 | 100 | — | — | |
| SQUEEZE EVOLVEModel 1=Qwen3-30B-A3B-I, Model 2=Qwen3-30B-A3B-T, p-value=0, Pairing Type=Homogeneous2026.04 | 75.9 | — | — | — | — | — | — | — | — | — | 0.32 | 1.8 | |
| RSAModel 2=Qwen3-30B-A3B-T, Pairing Type=Homogeneous2026.04 | 74 | — | — | — | — | — | — | — | — | — | 0.57 | 1 | |
| Uniform (conf)LLM=Qwen3-30B, Confidence-Weighted=true2026.02 | 72 | — | — | — | — | — | — | — | 9,257 | 96 | — | — | |
| PETS-On. (conf)LLM=Qwen3-30B, Confidence-Weighted=true2026.02 | 71.8 | — | — | — | — | — | — | — | 4,357 | 99 | — | — | |
| PETS-On.LLM=Qwen3-30B, Confidence-Weighted=false2026.02 | 71.6 | — | — | — | — | — | — | — | 3,826 | 100 | — | — | |
| UniformLLM=Qwen3-30B, Confidence-Weighted=false2026.02 | 71.2 | — | — | — | — | — | — | — | 8,456 | 97 | — | — | |
| PETS-On. (conf)LLM=Qwen3-4B, Confidence-Weighted=true2026.02 | 69.6 | — | — | — | — | — | — | — | 4,888 | 100 | — | — | |
| Uniform (conf)LLM=Qwen3-4B, Confidence-Weighted=true2026.02 | 69.4 | — | — | — | — | — | — | — | 9,856 | 97 | — | — | |
| PETS-On.LLM=Qwen3-4B, Confidence-Weighted=false2026.02 | 68.8 | — | — | — | — | — | — | — | 4,662 | 100 | — | — | |
| PIRBackbone=DeepSeek-R1-32B2026.02 | 68.6 | — | — | — | — | — | — | 3,048.82 | — | — | — | — | |
| UniformLLM=Qwen3-4B, Confidence-Weighted=false2026.02 | 68.1 | — | — | — | — | — | — | — | 9,520 | 97 | — | — | |
| GPT-4.1Access=Close-source2025.07 | 67.17 | — | — | — | — | — | — | — | — | — | — | — | |
| GPT-o3-miniAccess=Close-source2025.07 | 66.67 | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen3-32BAccess=Open-source, Parameters=32B2025.07 | 65.15 | — | — | — | — | — | — | — | — | — | — | — | |
| Open-source Upper BoundCategory=Baselines2025.07 | 65.15 | — | — | — | — | — | — | — | — | — | — | — | |
| SMCSCategory=Ours2025.07 | 65.15 | — | — | — | — | — | — | — | — | — | — | — | |
| Self-MoACategory=Other Methods2025.07 | 64.41 | — | — | — | — | — | — | — | — | — | — | — | |
| QTIPBackbone=Qwen3-14B, Type=vector, Zero-shot=true2025.11 | 64 | — | — | — | — | — | — | — | — | — | — | — | |
| Claude-3.7-SonnetAccess=Close-source2025.07 | 63.64 | — | — | — | — | — | — | — | — | — | — | — | |
| Self Consistency (Best on Validation)Category=Other Methods2025.07 | 63.64 | — | — | — | — | — | — | — | — | — | — | — | |
| ParoQuantBackbone=Qwen3-14B, Type=linear, Zero-shot=true2025.11 | 63.5 | — | — | — | — | — | — | — | — | — | — | — | |
| Symbolic-MoE*Category=Other Methods2025.07 | 62.63 | — | — | — | — | — | — | — | — | — | — | — | |
| FP16Backbone=Qwen3-14B, Zero-shot=true2025.11 | 62.5 | — | — | — | — | — | — | — | — | — | — | — | |
| AWQBackbone=Qwen3-14B, Type=linear, Zero-shot=true2025.11 | 62 | — | — | — | — | — | — | — | — | — | — | — | |
| Claude-3.5-SonnetAccess=Close-source2025.07 | 61.62 | — | — | — | — | — | — | — | — | — | — | — | |
| FP16Backbone=Qwen3-8B, Zero-shot=true2025.11 | 60.3 | — | — | — | — | — | — | — | — | — | — | — | |
| AWQBackbone=Qwen3-8B, Type=linear, Zero-shot=true2025.11 | 60.2 | — | — | — | — | — | — | — | — | — | — | — | |
| DeepSeek-R1-Distill-Llama-70BAccess=Open-source, Parameters=70B2025.07 | 60.1 | — | — | — | — | — | — | — | — | — | — | — | |
| DeepSeek-R1-Distill-Qwen-32BAccess=Open-source, Parameters=32B2025.07 | 60.1 | — | — | — | — | — | — | — | — | — | — | — | |
| Simple RouterCategory=Other Methods2025.07 | 60.1 | — | — | — | — | — | — | — | — | — | — | — | |
| GLM-Z1-32B-0414Access=Open-source, Parameters=32B2025.07 | 59.6 | — | — | — | — | — | — | — | — | — | — | — | |
| QTIPBackbone=Qwen3-8B, Type=vector, Zero-shot=true2025.11 | 59.3 | — | — | — | — | — | — | — | — | — | — | — | |
| MoACategory=Other Methods2025.07 | 58.8 | — | — | — | — | — | — | — | — | — | — | — | |
| EXAONE-Deep-32BAccess=Open-source, Parameters=32B2025.07 | 58.59 | — | — | — | — | — | — | — | — | — | — | — | |
| E-QATBackbone=Qwen3-14B, Type=linear, Zero-shot=true2025.11 | 58.4 | — | — | — | — | — | — | — | — | — | — | — | |
| ParoQuantBackbone=Qwen3-8B, Type=linear, Zero-shot=true2025.11 | 57.7 | — | — | — | — | — | — | — | — | — | — | — | |
| QwQ-32BAccess=Open-source, Parameters=32B2025.07 | 57.07 | — | — | — | — | — | — | — | — | — | — | — | |
| ASPO-Math-8BBackbone=Qwen3-8B2025.10 | 56.3 | — | — | — | — | — | — | — | — | — | — | — | |
| E-QATBackbone=Qwen3-8B, Type=linear, Zero-shot=true2025.11 | 56.1 | — | — | — | — | — | — | — | — | — | — | — | |
| DeepSeek-R1-32BBackbone=DeepSeek-R1-32B2026.02 | 55.81 | — | — | — | — | — | — | 5,307.51 | — | — | — | — | |
| PIRBackbone=QwQ-32B2026.02 | 55.56 | — | — | — | — | — | — | 4,033.46 | — | — | — | — | |
| QTIPBackbone=Qwen3-4B, Type=vector, Zero-shot=true2025.11 | 55.2 | — | — | — | — | — | — | — | — | — | — | — | |
| ParoQuantBackbone=Qwen3-4B, Type=linear, Zero-shot=true2025.11 | 53.7 | — | — | — | — | — | — | — | — | — | — | — | |
| High Conf.Backbone=DS-R1-Distill-Llama-8B2025.10 | 53 | — | — | — | — | — | — | — | — | — | — | — | |
| GPT-4oAccess=Close-source2025.07 | 52.53 | — | — | — | — | — | — | — | — | — | — | — | |
| AWQBackbone=Qwen3-4B, Type=linear, Zero-shot=true2025.11 | 52.2 | — | — | — | — | — | — | — | — | — | — | — | |
| Loc. uniBackbone=DS-R1-Distill-Llama-8B2025.10 | 52 | — | — | — | — | — | — | — | — | — | — | — | |
| E-QATBackbone=Qwen3-4B, Type=linear, Zero-shot=true2025.11 | 51 | — | — | — | — | — | — | — | — | — | — | — | |
| Self-Cert.Backbone=DS-R1-Distill-Llama-8B2025.10 | 51 | — | — | — | — | — | — | — | — | — | — | — | |
| Self-Cert.Backbone=DS-R1-Distill-Qwen-7B2025.10 | 51 | — | — | — | — | — | — | — | — | — | — | — | |
| Loc. uniBackbone=DS-R1-Distill-Qwen-7B2025.10 | 51 | — | — | — | — | — | — | — | — | — | — | — | |
| Gemma-3-27b-itAccess=Open-source, Parameters=27B2025.07 | 50.51 | — | — | — | — | — | — | — | — | — | — | — | |
| FP16Backbone=Qwen3-4B, Zero-shot=true2025.11 | 50 | — | — | — | — | — | — | — | — | — | — | — | |
| Glob. non-uniBackbone=DS-R1-Distill-Qwen-7B2025.10 | 50 | — | — | — | — | — | — | — | — | — | — | — | |
| HuatuoGPT-o1-72BAccess=Open-source, Parameters=72B2025.07 | 50 | — | — | — | — | — | — | — | — | — | — | — | |
| Low Ent.Backbone=DS-R1-Distill-Llama-8B2025.10 | 49 | — | — | — | — | — | — | — | — | — | — | — | |
| QwQ-32BBackbone=QwQ-32B2026.02 | 48.84 | — | — | — | — | — | — | 6,132.83 | — | — | — | — | |
| Llama-3.3-Nemotron-Super-49B-v1Access=Open-source, Parameters=49B2025.07 | 48.48 | — | — | — | — | — | — | — | — | — | — | — | |
| Mean Acc.Backbone=DS-R1-Distill-Llama-8B2025.10 | 48 | — | — | — | — | — | — | — | — | — | — | — | |
| Mean Acc.Backbone=DS-R1-Distill-Qwen-7B2025.10 | 48 | — | — | — | — | — | — | — | — | — | — | — | |
| Low Ent.Backbone=DS-R1-Distill-Qwen-7B2025.10 | 48 | — | — | — | — | — | — | — | — | — | — | — | |
| ParoQuantBackbone=R1-Distill-Llama-8B, Type=linear, Zero-shot=true2025.11 | 47.5 | — | — | — | — | — | — | — | — | — | — | — | |
| ReASC (online)Model=GEMMA-3-27B2026.01 | 47.21 | — | — | — | 161.68 | 0.29 | — | — | — | — | — | — | |
| Glob. non-uniBackbone=DS-R1-Distill-Llama-8B2025.10 | 47 | — | — | — | — | — | — | — | — | — | — | — | |
| Llama-3.3-70B-InstructAccess=Open-source, Parameters=70B2025.07 | 46.97 | — | — | — | — | — | — | — | — | — | — | — | |
| FP16Backbone=R1-Distill-Llama-8B, Zero-shot=true2025.11 | 46.6 | — | — | — | — | — | — | — | — | — | — | — | |
| PreSafebase_model=DeepSeek-R1-Distill-Qwen-7B2026.03 | 46.3 | — | — | — | — | — | — | — | — | — | — | — | |
| SafeChainbase_model=DeepSeek-R1-Distill-Qwen-7B2026.03 | 46.1 | — | — | — | — | — | — | — | — | — | — | — | |
| High Conf.Backbone=DS-R1-Distill-Qwen-7B2025.10 | 46 | — | — | — | — | — | — | — | — | — | — | — | |
| Glob. uniBackbone=DS-R1-Distill-Qwen-7B2025.10 | 46 | — | — | — | — | — | — | — | — | — | — | — | |
| PIRBackbone=DeepSeek-R1-14B2026.02 | 45.83 | — | — | — | — | — | — | 3,576.72 | — | — | — | — | |
| SCModel=GEMMA-3-27B, k=162026.01 | 45.69 | — | — | — | 368.97 | 0.12 | — | — | — | — | — | — | |
| ESCModel=GEMMA-3-27B, w=42026.01 | 45.69 | — | — | — | 210.78 | 0.22 | — | — | — | — | — | — |