Scientific Reasoning on GPQA Diamond (pass@k metrics)
98pass@kEAGER
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| EAGERModel=GPT-Oss 20B2025.10 | 98 | 68 | 66 | |
| EAGER-INITModel=GPT-Oss 20B2025.10 | 97 | 71 | 66 | |
| FULL PARALLELModel=GPT-Oss 20B2025.10 | 96 | 68 | 65 | |
| FULL PARALLELModel=DeepSeek 8B2025.10 | 95 | 15 | 18 | |
| EAGERModel=DeepSeek 8B2025.10 | 94 | 26 | 33 | |
| EAGER-INITModel=DeepSeek 8B2025.10 | 93 | 25 | 24 | |
| EAGERModel=SmolLM 3B2025.10 | 85 | 12 | 18 | |
| EAGERModel=Qwen3 4B2025.10 | 81 | 59 | 54 | |
| EAGER-INITModel=Qwen3 4B2025.10 | 79 | 51 | 43 | |
| FULL PARALLELModel=Qwen3 4B2025.10 | 75 | 51 | 43 | |
| EAGER-INITModel=SmolLM 3B2025.10 | 59 | 10 | 15 | |
| FULL PARALLELModel=SmolLM 3B2025.10 | 49 | 0 | 3 |