Question Answering on GPQA (Accuracy, Latency)
34.5AccuracySPECS
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SPECSn (per-step beam width)=162025.06 | 34.5 | 22.42 | |
| Beam Search (Big Model)n (per-step beam width)=4, Model Configuration=Big Model2025.06 | 34.2 | 18.6 | |
| Beam Search (Big Model)n (per-step beam width)=16, Model Configuration=Big Model2025.06 | 33.3 | 28.3 | |
| Beam Search (Big Model)n (per-step beam width)=8, Model Configuration=Big Model2025.06 | 32.5 | 23.1 | |
| SPECSn (per-step beam width)=82025.06 | 32.2 | 21.83 | |
| SPECSn (per-step beam width)=42025.06 | 30.8 | 9 | |
| Beam Search (Small Model)n (per-step beam width)=16, Model Configuration=Small Model2025.06 | 24.2 | 15.08 | |
| Beam Search (Small Model)n (per-step beam width)=8, Model Configuration=Small Model2025.06 | 22.6 | 10.31 | |
| Beam Search (Small Model)n (per-step beam width)=4, Model Configuration=Small Model2025.06 | 17.8 | 7.77 |