List your company
ResearchFormerly WizWand

Expert-level Reasoning on GPQA Diamond (Pass@1)

84.3Pass@1 ScoreGemma 4 31B
-3.37219.38942.1564.911Aug 12, 2025Oct 5, 2025Nov 28, 2025Jan 21, 2026Mar 16, 2026May 9, 2026Jul 2, 2026
Updated 3mo ago

Evaluation results

MethodLinks
Gemma 4 31BModel size=31B, Thinking mode=true2026.0784.3
Gemma 4 26B-A4BModel size=26B-A4B, Thinking mode=true2026.0782.3
Gemma 4 12BModel size=12B, Thinking mode=true2026.0778.8
Gemma 4 E4BModel size=E4B, Thinking mode=true2026.0758.6
openPangu-Embedded KD# Total Params=1B, Training Stage=KD (NPD)2026.0550.51
openPangu-Embedded RL# Total Params=1B, Training Stage=RL2026.0548.48
openPangu-Embedded SFT# Total Params=1B, Training Stage=SFT2026.0543.43
Gemma 4 E2BModel size=E2B, Thinking mode=true2026.0743.4
Gemma 3 27BModel size=27B, Thinking mode=false2026.0742.4
Full Attention2025.0838.9
Questrelative budget (b)=0.152025.0833.6
RetroAttentionrelative budget (b)=0.15, retrospective window size (w)=22025.0833.6
Qwen2.5# Total Params=1.5B2026.0529.8
Llama3.2# Total Params=1B2026.0529.29
Qwen3# Total Params=1.7B2026.0528.6
MiniCPM4# Total Params=0.5B2026.0528.28
Qwen3# Total Params=0.6B2026.0522.9
StreamingLLM2025.0819.7
Gemma3# Total Params=1B2026.0519.2
TOVA2025.080