Question Answering on GPQA Diamond n=198 (Acc, HC-Cov, HC-Prec, Gap)
87.4AccuracySC
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| SCk=8, type=extended thinking (Epoch AI benchmark)2026.05 | 87.4 | 81.3 | 96.3 | 47.6 | 0.001 | |
| USCk=82026.05 | 85.4 | 81.3 | 96.3 | 58.4 | 0.001 | |
| Debateagents=3, rounds=22026.05 | 83.3 | 95.5 | 85.7 | 52.4 | 0.001 | |
| Reflexionmaximum trials=52026.05 | 82.3 | 92.9 | 82.6 | 4 | 0.717 | |
| PVDRun=self-play2026.05 | 80.3 | 63.1 | 94.4 | 38.2 | 0.001 | |
| PVD+retryRun=retry2026.05 | 79.8 | 79.8 | 88.6 | 43.6 | 0.001 | |
| PVDRun=min-12026.05 | 76.3 | 51 | 91.1 | 30.3 | 0.001 | |
| Single-call PVDRun=single-call2026.05 | 75.8 | 59.6 | 83.1 | 18.1 | 0.004 | |
| PVDRun=standard2026.05 | 74.2 | 54.5 | 93.5 | 42.4 | 0.001 |