List your company
ResearchFormerly WizWand

Question Answering on GPQA Diamond (Accuracy)

84AccuracyGemini 2.5 Pro
15.56833.33451.168.866Apr 15, 2026Apr 23, 2026May 1, 2026May 10, 2026May 18, 2026May 26, 2026Jun 4, 2026
Updated 4mo ago

Evaluation results

MethodLinks
Gemini 2.5 ProParameters=Proprietary, Conditions=pass@1 [37]2026.0484
BF16 (Teacher)sampling=8 samples per problem2026.0673
QAD (KL-only)sampling=8 samples per problem2026.0672.7
CKA-QADsampling=8 samples per problem2026.0672.7
NVFP4 PTQsampling=8 samples per problem2026.0671.6
PhD domain expertsParameters=—, Conditions=[36]2026.0469.7
Gemini 2.0 ProParameters=Proprietary, Conditions=pass@1 [35]2026.0464.7
SST (staged compute imax=4)Parameters=27B, Conditions=0-shot, greedy2026.0461.11
Gemini 2.0 FlashParameters=Proprietary, Conditions=pass@1 [34]2026.0460.1
DeepSeek V3Parameters=671B, Conditions=pass@1 [33]2026.0459.1
GPT-4oParameters=Proprietary, Conditions=pass@1 [32]2026.0456.1
DCLSize=70B, Selection=Best-of-1002026.0456.06
GMMSize=70B, Selection=Best-of-1002026.0454.13
FOREVERModel Scale=14B2026.0554
DSRL (Ours)Backbone=Qwen3-8B2026.0453.48
DSSize=70B, Selection=Best-of-1002026.0452.4
GRPOBackbone=Qwen3-8B2026.0450.96
Llama 3.3 70B InstructParameters=70B, Conditions=0-shot CoT [31]2026.0450.5
Qwen 2.5 72B InstructParameters=72B, Conditions=Self-reported [28]2026.0449
VanillaBackbone=Qwen3-8B2026.0445.86
Seq. SFTModel Scale=14B2026.0543.4
EWCModel Scale=14B2026.0543.4
DSRL (Ours)Backbone=Qwen3-4B2026.0443.18
Gemma 3 27B-ITParameters=27B, Conditions=5-shot CoT [29]2026.0442.4
VanillaBackbone=Qwen3-4B2026.0442.12
GMMSize=8B, Selection=Best-of-1002026.0441.84
GCWMModel Scale=14B2026.0539.9
GRPOBackbone=Qwen3-4B2026.0439.39
AIMMergingModel Scale=14B2026.0538.8
L&SModel Scale=14B2026.0538.4
POPBackbone=Qwen-2.5-7B-Inst, Evaluation Protocol=0-shot2026.0438.38
OPCMModel Scale=14B2026.0538
BaseBackbone=Qwen-2.5-7B-Inst, Evaluation Protocol=0-shot2026.0436.36
BaseBackbone=Qwen-2.5-7B, Evaluation Protocol=0-shot2026.0433.84
Teacher-OPPOBackbone=Phi-4-mini, Training Framework=DeepScaleR2026.0533.6
MTLModel Scale=14B2026.0533.3
DAPO + OPPOBackbone=Phi-4-mini, Training Framework=DeepScaleR2026.0533.2
DSSize=8B, Selection=Best-of-1002026.0432.95
Train on DBackbone=Qwen-2.5-7B-Inst, Evaluation Protocol=0-shot2026.0432.83
DCLSize=8B, Selection=Best-of-1002026.0432.79
Self-OPPOBackbone=Phi-4-mini, Training Framework=DeepScaleR2026.0532.4
SCALESEARCHATTENTIONModel=Llama 3.1 8B Instruct2026.0532.32
Instruction-Queryable LoRABackbone=Qwen0.5B, Type=Test Accuracy2026.0532.3
Dr.GRPO + OPPOBackbone=Phi-4-mini, Training Framework=DeepScaleR2026.0532
FULLPRECModel=Llama 3.1 8B Instruct2026.0531.81
SDPOBackbone=Phi-4-mini, Training Framework=DeepScaleR2026.0530.8
DAPOBackbone=Phi-4-mini, Training Framework=DeepScaleR2026.0530.4
Dr.GRPOBackbone=Phi-4-mini, Training Framework=DeepScaleR2026.0529.6
GRPOBackbone=Phi-4-mini, Training Framework=DeepScaleR2026.0529.4
Queryable LoRABackbone=Qwen0.5B, Type=Test Accuracy2026.0529.3
Train on DBackbone=Qwen-2.5-7B, Evaluation Protocol=0-shot2026.0427.78
POPBackbone=Qwen-2.5-7B, Evaluation Protocol=0-shot2026.0427.78
DoRABackbone=Qwen0.5B, Type=Test Accuracy2026.0527.3
RepLoRABackbone=Qwen0.5B, Type=Test Accuracy2026.0527.3
FOREVERModel Scale=1.7B2026.0527.3
MTLModel Scale=1.7B2026.0526.7
GCWMModel Scale=1.7B2026.0526.3
SA3Model=Llama 3.1 8B Instruct2026.0526.26
Base modelBackbone=Phi-4-mini, Training Framework=DeepScaleR2026.0525.6
LoRABackbone=Qwen0.5B, Type=Test Accuracy2026.0525.3
EWCModel Scale=1.7B2026.0524.8
OPCMModel Scale=1.7B2026.0523.2
AIMMergingModel Scale=1.7B2026.0521.7
L&SModel Scale=1.7B2026.0521.3
Seq. SFTModel Scale=1.7B2026.0518.2