List your company
ResearchFormerly WizWand

Scientific Reasoning on GPQA Main

67.97AccuracyLLaMA-70B
2.387619.413836.4453.4662Feb 5, 2026Feb 24, 2026Mar 16, 2026Apr 5, 2026Apr 25, 2026May 15, 2026Jun 4, 2026
Updated 4mo ago

Evaluation results

MethodLinks
LLaMA-70BSetting=Steer2026.0667.97
LLaMA-70BSetting=Base2026.0667.19
Qwen3-32BSetting=Steer2026.0665.96
Qwen3-32BSetting=Base2026.0664.4
Qwen3-14BSetting=Steer2026.0660.49
Qwen3-14BSetting=Base2026.0658.71
Qwen3-8BSetting=Steer2026.0655.28
LLaMA-8BSetting=Steer2026.0655.13
Qwen3-8BSetting=Base2026.0654.69
Qwen3-4BSetting=Steer2026.0651.33
LLaMA-8BSetting=Base2026.0650.45
Qwen3-4BSetting=Base2026.0650.22
Llama 70B2026.0243.08
Qwen3-1.7BSetting=Steer2026.0641.07
Mistral 24B2026.0239.95
FiMI Instruct2026.0237.95
Gemma 2 27B2026.0237.72
Qwen3-1.7BSetting=Base2026.0637.05
Official Instruct ModelEvaluation Category=Official Instruct Model2026.0535.49
NeMo 12B IT2026.0234.59
Mistral 7B2026.0233.92
Official Instruct ModelProtocol=Official Instruct, Backbone=SmolLM3-3B2026.0533.26
Min-kModel=LLaMA3-8B-Instruct, Temperature=2.02026.0430.8
Top-nσModel=LLaMA3-8B-Instruct, Temperature=2.02026.0429.69
Min-kModel=LLaMA3-8B-Instruct, Temperature=10.02026.0429.69
Opus-4.7 (xHigh)Evaluation Category=Agent-Post-Trained, Base Model=Qwen3-1.7B2026.0529.32
Min-kModel=LLaMA3-8B-Instruct, Temperature=5.02026.0429.24
Min-pModel=LLaMA3-8B-Instruct, Temperature=1.02026.0429.02
Top-nσModel=LLaMA3-8B-Instruct, Temperature=3.02026.0429.02
Min-kModel=LLaMA3-8B-Instruct, Temperature=1.02026.0429.02
GPT-5.4 (High)Protocol=Agent-Post-Trained, Backbone=SmolLM3-3B2026.0529.02
Llama 3.1 8B2026.0228.57
Top-pModel=LLaMA3-8B-Instruct, Temperature=1.02026.0428.57
Top-kModel=LLaMA3-8B-Instruct, Temperature=1.02026.0428.35
Top-nσModel=LLaMA3-8B-Instruct, Temperature=10.02026.0428.35
Opus-4.7 (xHigh)Protocol=Agent-Post-Trained, Backbone=SmolLM3-3B2026.0527.83
Top-nσModel=LLaMA3-8B-Instruct, Temperature=5.02026.0427.46
Min-kModel=LLaMA3-8B-Instruct, Temperature=4.02026.0427.46
GLM-4.7 & ANDESEvaluation Category=Proposed Method Comparison, Base Model=Qwen3-1.7B2026.0527.03
Top-kModel=LLaMA3-8B-Instruct, Temperature=2.02026.0426.79
Top-nσModel=LLaMA3-8B-Instruct, Temperature=4.02026.0426.79
Top-nσModel=LLaMA3-8B-Instruct, Temperature=1.02026.0426.56
Min-kModel=LLaMA3-8B-Instruct, Temperature=3.02026.0426.56
Opus-4.6 (1M)Protocol=Agent-Post-Trained, Backbone=SmolLM3-3B2026.0526.41
Min-pModel=LLaMA3-8B-Instruct, Temperature=2.02026.0426.34
GPT-5.2Protocol=Agent-Post-Trained, Backbone=SmolLM3-3B2026.0525.82
GLM-4.7 & ANDES (Ours)Protocol=Proposed Method, Backbone=SmolLM3-3B2026.0525.8
Opus-4.6Protocol=Agent-Post-Trained, Backbone=SmolLM3-3B2026.0525.52
GPT-5.1-Codex-MaxEvaluation Category=Agent-Post-Trained, Base Model=Qwen3-1.7B2026.0524.78
GPT-5.4 (High)Evaluation Category=Agent-Post-Trained, Base Model=Qwen3-1.7B2026.0524.33
Opus-4.6 (1M)Evaluation Category=Agent-Post-Trained, Base Model=Qwen3-1.7B2026.0522.99
Opus-4.6Evaluation Category=Agent-Post-Trained, Base Model=Qwen3-1.7B2026.0522.92
GPT-5.1-Codex-MaxProtocol=Agent-Post-Trained, Backbone=SmolLM3-3B2026.0521.65
Top-pModel=LLaMA3-8B-Instruct, Temperature=2.02026.0421.21
Min-pModel=LLaMA3-8B-Instruct, Temperature=3.02026.0421.21
GLM-4.7 (Scaffold-only)Evaluation Category=Proposed Method Comparison, Base Model=Qwen3-1.7B2026.0520.13
GLM-4.7 (Scaffold-only)Protocol=Proposed Method, Backbone=SmolLM3-3B2026.0519.31
Min-pModel=Qwen3-4B-Instruct, Temperature=1.02026.0419.2
Min-kModel=Qwen3-4B-Instruct, Temperature=4.02026.0419.2
Min-kModel=Qwen3-4B-Instruct, Temperature=1.02026.0418.97
Min-kModel=Qwen3-4B-Instruct, Temperature=2.02026.0418.97
Min-kModel=Qwen3-4B-Instruct, Temperature=10.02026.0418.97
Top-kModel=Qwen3-4B-Instruct, Temperature=2.02026.0418.75
Top-nσModel=Qwen3-4B-Instruct, Temperature=2.02026.0418.75
Min-kModel=Qwen3-4B-Instruct, Temperature=5.02026.0418.53
Gemini-3.1-ProProtocol=Agent-Post-Trained, Backbone=SmolLM3-3B2026.0518.01
Top-nσModel=Qwen3-4B-Instruct, Temperature=3.02026.0417.86
Top-nσModel=Qwen3-4B-Instruct, Temperature=1.02026.0417.41
Top-nσModel=Qwen3-4B-Instruct, Temperature=4.02026.0417.41
Top-nσModel=Qwen3-4B-Instruct, Temperature=5.02026.0417.41
Min-kModel=Qwen3-4B-Instruct, Temperature=3.02026.0417.41
GPT-5.2Evaluation Category=Agent-Post-Trained, Base Model=Qwen3-1.7B2026.0517.41
Top-kModel=Qwen3-4B-Instruct, Temperature=1.02026.0417.19
Top-pModel=Qwen3-4B-Instruct, Temperature=1.02026.0417.19
MiniMax-M2.5Protocol=Agent-Post-Trained, Backbone=SmolLM3-3B2026.0517.19
Min-pModel=Qwen3-4B-Instruct, Temperature=2.02026.0416.96
Sonnet-4.5Evaluation Category=Agent-Post-Trained, Base Model=Qwen3-1.7B2026.0516.74
Gemini-3.1-ProEvaluation Category=Agent-Post-Trained, Base Model=Qwen3-1.7B2026.0516.15
Top-pModel=Qwen3-4B-Instruct, Temperature=2.02026.0416.07
Top-nσModel=Qwen3-4B-Instruct, Temperature=10.02026.0416.07
Top-kModel=LLaMA3-8B-Instruct, Temperature=3.02026.0415.18
Min-pModel=Qwen3-4B-Instruct, Temperature=3.02026.0414.73
MiniMax-M2.1Evaluation Category=Agent-Post-Trained, Base Model=Qwen3-1.7B2026.0514.51
Base Model (Qwen3-1.7B)Evaluation Category=Zero-Shot2026.0514.06
MiniMax-M2.5Evaluation Category=Agent-Post-Trained, Base Model=Qwen3-1.7B2026.0514.06
Qwen3-MaxEvaluation Category=Agent-Post-Trained, Base Model=Qwen3-1.7B2026.0514.06
Kimi-K2-ThinkingEvaluation Category=Agent-Post-Trained, Base Model=Qwen3-1.7B2026.0514.06
GLM-4.7 (OpenCode)Evaluation Category=Proposed Method Comparison, Base Model=Qwen3-1.7B2026.0514.06
Top-kModel=Qwen3-4B-Instruct, Temperature=3.02026.0412.28
MiniMax-M2.1Protocol=Agent-Post-Trained, Backbone=SmolLM3-3B2026.0512.05
Top-pModel=Qwen3-4B-Instruct, Temperature=3.02026.0410.71
Min-pModel=Qwen3-4B-Instruct, Temperature=4.02026.0410.04
Top-kModel=Qwen3-4B-Instruct, Temperature=4.02026.049.6
Top-pModel=LLaMA3-8B-Instruct, Temperature=3.02026.047.59
Min-pModel=LLaMA3-8B-Instruct, Temperature=4.02026.046.92
Top-kModel=LLaMA3-8B-Instruct, Temperature=4.02026.045.58
Min-pModel=Qwen3-4B-Instruct, Temperature=5.02026.045.58
Top-pModel=Qwen3-4B-Instruct, Temperature=4.02026.045.36
Base Model (SmolLM3-3B)Protocol=Zero-Shot, Backbone=SmolLM3-3B2026.054.91
Qwen3-MaxProtocol=Agent-Post-Trained, Backbone=SmolLM3-3B2026.054.91