Selective Prediction on BBH, GPQA, and MMLU-Pro Pooled (test)
1,384N0/N CountPooled (21 runs)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Pooled (21 runs)Trace Format=All2026.04 | 1,384 | 0.254 | 0.961 | 0.949 | |
| Claude Sonnet 4.6Trace Format=Thought summaries2026.04 | 425 | 0.533 | 0.981 | 0.963 | |
| Grok 4.1 FastTrace Format=Thought summaries2026.04 | 406 | 0.509 | 0.948 | 0.922 | |
| GPT OSS 20BTrace Format=Full traces2026.04 | 233 | 0.302 | 0.961 | 0.928 | |
| GPT OSS 120BTrace Format=Full traces2026.04 | 205 | 0.257 | 0.966 | 0.931 | |
| Qwen 3 14BTrace Format=Full traces2026.04 | 77 | 0.097 | 0.961 | 0.892 | |
| Qwen 3 4BTrace Format=Full traces2026.04 | 31 | 0.043 | 0.903 | 0.751 | |
| Gemini 2.5 FlashTrace Format=Thought summaries2026.04 | 7 | 0.009 | 0.571 | 0.251 |