List your company
ResearchFormerly WizWand

General Reasoning on GPQA Diamond (pass@1 accuracy)

86.4Pass@1 Accuracygemini-2.5-pro
12.35231.57650.870.024Sep 29, 2025Oct 25, 2025Nov 21, 2025Dec 18, 2025Jan 14, 2026Feb 10, 2026Mar 9, 2026
Updated 4mo ago

Evaluation results

MethodLinks
gemini-2.5-pro# Params=–, Scale Category=Large Scale (> 70B)2026.0386.4
o4-mini (high)# Params=–, Scale Category=Large Scale (> 70B)2026.0381.4
Qwen3-235B-A22B-Thinking-2507# Params=235B, Scale Category=Large Scale (> 70B)2026.0381.1
DeepSeek-R1-0528# Params=671B, Scale Category=Large Scale (> 70B)2026.0381
o3-mini (medium)# Params=–, Scale Category=Large Scale (> 70B)2026.0376.8
DeepSeek-R1# Params=671B, Scale Category=Large Scale (> 70B)2026.0371.5
Qwen3-235B-A22B# Params=235B, Scale Category=Large Scale (> 70B)2026.0371.1
Qwen3-4B-Thinking-2507 + CHIMERA# Params=4B, Scale Category=Small to Medium Scale (≤ 70B)2026.0370.1
Qwen3-32B# Params=32B, Scale Category=Small to Medium Scale (≤ 70B)2026.0368.4
Qwen3-4B-Thinking-2507# Params=4B, Scale Category=Small to Medium Scale (≤ 70B)2026.0365.8
DeepSeek-R1-Distill-Llama-70B# Params=70B, Scale Category=Small to Medium Scale (≤ 70B)2026.0365.2
DeepSeek-R1-0528-Qwen3-8B# Params=8B, Scale Category=Small to Medium Scale (≤ 70B)2026.0361.1
CLPOOptimization Policy=Restructuring-Aware RL, Base Model=Qwen3-8B2025.0956.06
Qwen3-4B-Thinking-2507 + OpenScience# Params=4B, Scale Category=Small to Medium Scale (≤ 70B)2026.0353.5
ACTTraining Data=ALFWorld agentic data2026.0353.37
RLTraining Data=ALFWorld agentic data2026.0352.36
Early Experience (Self-Reflection)Training Data=ALFWorld agentic data with self-reflection2026.0351.85
Prompt w/ CoT thinkingReasoning Mode=Chain of Thought2026.0351.52
Critique-GRPO (CoT)Optimization Policy=Critique-Driven RL, Base Model=Qwen3-8B2025.0950.5
LUFFYOptimization Policy=Off-Policy Imitation, Base Model=Qwen3-8B2025.0949.49
Critique-GRPO (Simple)Optimization Policy=Critique-Driven RL, Base Model=Qwen3-8B2025.0948.63
DAPOOptimization Policy=Dynamic Sampling, Base Model=Qwen3-8B2025.0948.48
GRPOOptimization Policy=Group-Based RL, Base Model=Qwen3-8B2025.0947.8
Imitation LearningTraining Data=ALFWorld agentic data2026.0344.61
Prompt w/o CoT thinkingReasoning Mode=Direct Answer2026.0342.93
TTSRBackbone Model=Qwen3-8B-Base2026.0242.6
R-ZeroBackbone Model=Qwen3-8B-Base2026.0240.5
AEROBackbone Model=Qwen3-8B-Base, Evolutionary Round=R42026.0238.4
TTRLBackbone Model=Qwen3-8B-Base2026.0238.4
AEROBackbone Model=Qwen3-8B-Base, Evolutionary Round=R52026.0236.9
Absolute ZeroBackbone Model=Qwen3-8B-Base2026.0236.8
RAFTOptimization Policy=Ranking-Based Imitation, Base Model=Qwen3-8B2025.0936.76
R-ZeroBackbone Model=Qwen3-8B-Base2026.0236.7
R-ZeroBackbone Model=Qwen3-4B-Base2026.0236.4
CITL-FTOptimization Policy=Mixed-Data SFT, Base Model=Qwen3-8B2025.0936.36
AEROBackbone Model=Qwen3-8B-Base, Evolutionary Round=R22026.0235.9
Absolute ZeroBackbone Model=Qwen3-4B-Base2026.0235.3
Critique-FTOptimization Policy=Learning to Critique, Base Model=Qwen3-8B2025.0934.84
Refinement-FTOptimization Policy=Guided Refinement, Base Model=Qwen3-8B2025.0934.47
AEROBackbone Model=Qwen3-4B-Base, Evolutionary Round=R42026.0234.3
AEROBackbone Model=Qwen3-4B-Base, Evolutionary Round=R52026.0234.3
AEROBackbone Model=Qwen3-8B-Base, Evolutionary Round=R32026.0234.3
TTSRBackbone Model=Qwen3-4B-Base2026.0234.2
AEROBackbone Model=Qwen3-8B-Base, Evolutionary Round=R12026.0233.8
AEROBackbone Model=Qwen3-4B-Base, Evolutionary Round=R32026.0233.3
Qwen3-8B-BaseBackbone Model=Qwen3-8B-Base2026.0233.3
Base ModelBackbone Model=Qwen3-8B-Base2026.0233.1
AEROBackbone Model=Qwen3-4B-Base, Evolutionary Round=R22026.0232.3
TTRLBackbone Model=Qwen3-4B-Base2026.0229.1
R-ZeroBackbone Model=Qwen3-4B-Base2026.0228.4
TTSRBackbone Model=OctoThinker-8B-Hybrid-Base2026.0227.9
Qwen3-4B-BaseBackbone Model=Qwen3-4B-Base2026.0226.3
AEROBackbone Model=Qwen3-4B-Base, Evolutionary Round=R12026.0226.3
Base ModelBackbone Model=Qwen3-4B-Base2026.0225.7
TTRLBackbone Model=OctoThinker-8B-Hybrid-Base2026.0221.5
R-ZeroBackbone Model=OctoThinker-8B-Hybrid-Base2026.0219.7
Base ModelBackbone Model=OctoThinker-8B-Hybrid-Base2026.0215.2