Graduate-level Science QA on GPQA Main (Score)
73.2AccuracyRaw CoT
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Raw CoTLLM=DEEPSEEK-R1-0528-QWEN3-8B2026.06 | 73.2 | — | 5,380 | |
| CoDLLM=DEEPSEEK-R1-0528-QWEN3-8B2026.06 | 73.2 | — | 1,892 | |
| CLSRLLM=DEEPSEEK-R1-0528-QWEN3-8B2026.06 | 73.1 | — | 1,326 | |
| SoTLLM=DEEPSEEK-R1-0528-QWEN3-8B2026.06 | 71.4 | — | 1,536 | |
| CCoTLLM=DEEPSEEK-R1-0528-QWEN3-8B2026.06 | 70.4 | — | 1,605 | |
| Raw CoTLLM=QWEN3-32B2026.06 | 54.2 | — | 983 | |
| CLSRLLM=QWEN3-32B2026.06 | 54 | — | 209 | |
| SoTLLM=QWEN3-32B2026.06 | 51.4 | — | 247 | |
| CCoTLLM=QWEN3-32B2026.06 | 50.8 | — | 312 | |
| CoDLLM=QWEN3-32B2026.06 | 49.2 | — | 276 | |
| Raw CoTLLM=QWEN3-8B2026.06 | 49.1 | — | 1,085 | |
| CLSRLLM=QWEN3-8B2026.06 | 47.7 | — | 228 | |
| SoTLLM=QWEN3-8B2026.06 | 45.2 | — | 228 | |
| CoDLLM=QWEN3-8B2026.06 | 42.7 | — | 282 | |
| CCoTLLM=QWEN3-8B2026.06 | 41.3 | — | 245 | |
| CLSRLLM=LLaMA3-8B2026.06 | 30.8 | — | 362 | |
| Raw CoTLLM=LLaMA3-8B2026.06 | 30.4 | — | 1,209 | |
| CCoTLLM=LLaMA3-8B2026.06 | 29.3 | — | 450 | |
| SoTLLM=LLaMA3-8B2026.06 | 28.2 | — | 398 | |
| CoDLLM=LLaMA3-8B2026.06 | 26.4 | — | 386 | |
| daVinciModel Type=Fully-open, Number of Parameters=3B2026.03 | — | 32.37 | — | |
| LLaMa-3.2Model Type=Open-weight, Number of Parameters=3B2026.03 | — | 29.46 | — | |
| OLMO-2Model Type=Fully-open, Number of Parameters=7B2026.03 | — | 30.8 | — | |
| OLMO-3Model Type=Fully-open, Number of Parameters=7B2026.03 | — | 37.05 | — | |
| Qwen-2.5Model Type=Open-weight, Number of Parameters=3B2026.03 | — | 31.47 | — | |
| Qwen-3Model Type=Open-weight, Number of Parameters=4B2026.03 | — | 38.17 | — | |
| Qwen-3.5Model Type=Open-weight, Number of Parameters=4B2026.03 | — | 41.07 | — | |
| YulanModel Type=Fully-open, Number of Parameters=2.4B2026.03 | — | 29.91 | — |