Science Question Answering on GPQA Diamond (Acc, throughput)
63.1AccuracyDense
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DenseModel=DeepSeek R1 Distill Qwen 32B2025.07 | 63.1 | 112.4 | |
| DenseModel=QwQ 32B2025.07 | 61.1 | 79.06 | |
| ReasonCacheModel=DeepSeek R1 Distill Qwen 32B2025.07 | 60.7 | 176.21 | |
| ReasonCacheModel=QwQ 32B2025.07 | 58.1 | 125.67 | |
| DenseModel=Phi 4 reasoning plus2025.07 | 52 | 175.04 | |
| ReasonCacheModel=Phi 4 reasoning plus2025.07 | 50.8 | 243.22 | |
| RandomModel=DeepSeek R1 Distill Qwen 32B2025.07 | 32.3 | 172.91 | |
| RandomModel=QwQ 32B2025.07 | 27.8 | 123.24 | |
| RandomModel=Phi 4 reasoning plus2025.07 | 20.3 | 241.81 |