Expert-level Reasoning on GPQA Diamond (Pass@1)
84.3Pass@1 ScoreGemma 4 31B
Evaluation Results
| Method | Links | |
|---|---|---|
| Gemma 4 31BModel size=31B, Thinking mode=true2026.07 | 84.3 | |
| Gemma 4 26B-A4BModel size=26B-A4B, Thinking mode=true2026.07 | 82.3 | |
| Gemma 4 12BModel size=12B, Thinking mode=true2026.07 | 78.8 | |
| Gemma 4 E4BModel size=E4B, Thinking mode=true2026.07 | 58.6 | |
| openPangu-Embedded KD# Total Params=1B, Training Stage=KD (NPD)2026.05 | 50.51 | |
| openPangu-Embedded RL# Total Params=1B, Training Stage=RL2026.05 | 48.48 | |
| openPangu-Embedded SFT# Total Params=1B, Training Stage=SFT2026.05 | 43.43 | |
| Gemma 4 E2BModel size=E2B, Thinking mode=true2026.07 | 43.4 | |
| Gemma 3 27BModel size=27B, Thinking mode=false2026.07 | 42.4 | |
| Full Attention2025.08 | 38.9 | |
| Questrelative budget (b)=0.152025.08 | 33.6 | |
| RetroAttentionrelative budget (b)=0.15, retrospective window size (w)=22025.08 | 33.6 | |
| Qwen2.5# Total Params=1.5B2026.05 | 29.8 | |
| Llama3.2# Total Params=1B2026.05 | 29.29 | |
| Qwen3# Total Params=1.7B2026.05 | 28.6 | |
| MiniCPM4# Total Params=0.5B2026.05 | 28.28 | |
| Qwen3# Total Params=0.6B2026.05 | 22.9 | |
| StreamingLLM2025.08 | 19.7 | |
| Gemma3# Total Params=1B2026.05 | 19.2 | |
| TOVA2025.08 | 0 |