List your company
ResearchFormerly WizWand

Reasoning on GPQA Diamond (Accuracy)

58.08AccuracySequential MAS + MASPO
8.180821.135434.0947.0446Apr 21, 2026Apr 27, 2026May 4, 2026May 10, 2026May 17, 2026May 23, 2026May 30, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Sequential MAS + MASPOPrompt Opt.=true, Joint Opt.=true2026.0558.08
Hierarchical MAS + MASPOPrompt Opt.=true, Joint Opt.=true2026.0554.04
Hierarchical MAS + SPOPrompt Opt.=true, Joint Opt.=false2026.0551.01
Hierarchical MASPrompt Opt.=false, Joint Opt.=false2026.0550.63
Sequential MAS + SPOPrompt Opt.=true, Joint Opt.=false2026.0549.52
Hierarchical MAS + TPEPrompt Opt.=true, Joint Opt.=false2026.0549.49
Sequential MAS + TPEPrompt Opt.=true, Joint Opt.=false2026.0548.04
AgentDropoutPrompt Opt.=false, Joint Opt.=false2026.0547.98
Self-RefinePrompt Opt.=false, Joint Opt.=false2026.0547.73
Sequential MASPrompt Opt.=false, Joint Opt.=false2026.0547.73
SC (CoT)Prompt Opt.=false, Joint Opt.=false2026.0547.52
CoTPrompt Opt.=false, Joint Opt.=false2026.0546.72
VanillaPrompt Opt.=false, Joint Opt.=false2026.0545.96
LRSModel=OPEN-REASONER-7B, Prompting=0-shot2026.0539.4
few-shotModel=OPEN-REASONER-7B2026.0538.4
BaseModel=Qwen-2.5-7B-Inst, Training Strategy=Base, Evaluation Protocol=0-shot2026.0436.36
CoTModel=OPEN-REASONER-7B2026.0535.9
POPModel=Qwen-2.5-7B-Inst, Training Strategy=POP, Evaluation Protocol=0-shot2026.0434.85
BaseModel=Qwen-2.5-7B, Training Strategy=Base, Evaluation Protocol=0-shot2026.0433.84
Train on DModel=Qwen-2.5-7B-Inst, Training Strategy=Naive pretraining on D, Evaluation Protocol=0-shot2026.0432.83
Rubric-grounded GRPOCheckpoint=Best by held-out rubric reward, Backbone=Llama-3.1-8B-Instruct2026.0532.32
BaseModel=OPEN-REASONER-7B, Prompting=0-shot2026.0532.3
Train on DModel=Qwen-2.5-7B, Training Strategy=Naive pretraining on D, Evaluation Protocol=0-shot2026.0431.82
POPModel=Qwen-2.5-7B, Training Strategy=POP, Evaluation Protocol=0-shot2026.0430.81
LRS BASICModel=OPEN-REASONER-7B, Prompting=0-shot2026.0530.8
Nautile-370MTraining tokens=∼0.8T, Evaluation Protocol=0-shot2026.0427.3
Granite 350MTraining tokens=10–12T, Evaluation Protocol=0-shot2026.0426.3
LFM2.5 350MTraining tokens=28T, Evaluation Protocol=0-shot2026.0424.8
Llama-3.1-8B-InstructCheckpoint=Base2026.0524.24
SmolLM2 360MTraining tokens=4T, Evaluation Protocol=0-shot2026.0423.2
LRSModel=OPEN-REASONER-1.5B, Prompting=0-shot2026.0522.8
BaseModel=OPEN-REASONER-1.5B, Prompting=0-shot2026.0518.2
CoTModel=OPEN-REASONER-1.5B2026.0517.2
few-shotModel=OPEN-REASONER-1.5B2026.0517.2
LRS BASICModel=OPEN-REASONER-1.5B, Prompting=0-shot2026.0515.7
Qwen2.5 0.5BTraining tokens=18T, Evaluation Protocol=0-shot2026.0410.1