Commonsense Reasoning on CommonsenseQA (Accuracy, Extra Calls, Token Usage, Flips)
85.18AccuracySC sampled-rollout oracle
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SC sampled-rollout oraclePolicy=SC sampled-rollout oracle2026.06 | 85.18 | — | — | — | |
| Sampled-rollout oracle, SC samplesPolicy=Sampled-rollout oracle, SC samples2026.06 | 85.18 | — | — | — | |
| Expected-action oracle, SC samplesPolicy=Expected-action oracle, SC samples2026.06 | 80.98 | — | — | — | |
| Self-Consistency@5Policy=Self-Consistency@52026.06 | 78.38 | 5 | 11,343 | 1.56 | |
| Self-Consistency@5Policy=Self-Consistency@52026.06 | 78.38 | — | 11,343 | 1.56 | |
| Expected-action oracle, active verifyPolicy=Expected-action oracle, active verify2026.06 | 78.26 | — | — | — | |
| Base, 4,096 limitPolicy=Base, 4,096 limit2026.06 | 76.49 | 1 | 2,234 | 0 | |
| Accept basePolicy=Accept base2026.06 | 76.49 | — | 0 | 0 | |
| Always active verifyPolicy=Always active verify2026.06 | 72.32 | 2 | 4,794 | 5.94 | |
| Always active verifyPolicy=Always active verify2026.06 | 72.32 | — | 2,561 | 5.94 |