Question Answering on GPQA Diamond (Acc, HC-Cov, HC-Prec, Gap)
94.9AccuracySC* (k=8)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SC* (k=8)Prover=GPT-5.4, HC Signal=Full consensus, Calls=82026.05 | 94.9 | 90 | 97.2 | 23.5 | |
| PVDProver=Gemini 3.1 Pro, Verifier=Flash-Lite, HC Signal=ANC, Calls=∼42026.05 | 94.4 | 57 | 97.3 | 6.6 | |
| SC* (k=8)Prover=Gemini 3.1 Pro, HC Signal=Full consensus, Calls=82026.05 | 93.9 | 94 | 97.3 | 60.9 | |
| PVD+retryProver=Gemini 3.1 Pro, Verifier=GPT-5.5-pro, HC Signal=ANC, Calls=∼292026.05 | 92.9 | 75 | 97.3 | 17.3 | |
| SC (k=8)Prover=Sonnet 4.6, HC Signal=Full consensus, Calls=82026.05 | 83.3 | 72 | 91.5 | 29 | |
| Debate (3×2)Prover=Sonnet 4.6, Verifier=Sonnet 4.6, HC Signal=Agent consensus, Calls=92026.05 | 83.3 | 95 | 85.7 | 52.4 | |
| ReflexionProver=Sonnet 4.6, HC Signal=Stable, unchanged, Calls=≤52026.05 | 82.3 | 93 | 82.6 | 4 | |
| USC (k=8)Prover=Sonnet 4.6, Verifier=Sonnet 4.6, HC Signal=Full consensus, Calls=92026.05 | 81.8 | 72 | 91.5 | 29 | |
| PVD†Prover=Sonnet 4.6, Verifier=Haiku 4.5, HC Signal=ANC, Calls=∼62026.05 | 79.8 | 65 | 89.9 | 29.1 | |
| Single-call PVDProver=Sonnet 4.6, Verifier=Self, HC Signal=ANC, Calls=12026.05 | 78.8 | 63 | 83.9 | 13.6 | |
| PVDProver=GPT-5.4, Verifier=GPT-5.4-mini, HC Signal=ANC, Calls=∼32026.05 | 77.8 | 43 | 97.6 | 34.8 | |
| PVDProver=Sonnet 4.6, Verifier=Haiku 4.5, HC Signal=ANC, Calls=∼32026.05 | 76.8 | 77 | 84.2 | 32 | |
| DirectProver=GPT-5.4, Calls=12026.05 | 72.7 | — | — | — |