Abstention Metacognition on AbstentionBench Normal Prompt
67F1 ScoreSFT-80K+RL
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SFT-80K+RLScale=4B2026.05 | 67 | 89.4 | 53.6 | 28 | |
| CPT + RLScale=8B2026.05 | 66.8 | 90.8 | 52.9 | 27.2 | |
| CPT + RLScale=14B2026.05 | 66.4 | 92.5 | 51.8 | 26.1 | |
| SFT-80K+RLScale=8B2026.05 | 66.3 | 89.8 | 52.6 | 27.3 | |
| CPTScale=32B, Backbone=Olmo-3 32B Base, RL=No RL2026.05 | 66 | 91.2 | 51.6 | 26.4 | |
| SFT (no RL)Scale=32B, Backbone=Olmo-3 32B Base2026.05 | 64.9 | 90.3 | 50.7 | 26.2 | |
| CPT + RLScale=4B2026.05 | 64.5 | 91 | 49.9 | 25.6 | |
| SFT-80K+RLScale=14B2026.05 | 64.4 | 92.1 | 49.8 | 25.2 | |
| Abs-RLScale=8B2026.05 | 64 | 87.4 | 50.5 | 27 | |
| DPO+RLScale=14B2026.05 | 63.6 | 92.4 | 48.5 | 24.5 | |
| Abs-RLScale=14B2026.05 | 62.5 | 92 | 47.2 | 23.9 | |
| Abs-RLScale=4B2026.05 | 62.3 | 88.5 | 48.2 | 25.3 | |
| SFT-80K (no RL)Scale=32B, Backbone=Olmo-3 32B Base2026.05 | 62.2 | 92 | 46.9 | 23.7 | |
| SFT+RLScale=8B2026.05 | 61.7 | 88.8 | 47.3 | 24.8 | |
| DPO (no RL)Scale=32B, Backbone=Olmo-3 32B Base2026.05 | 61.6 | 90.5 | 46.7 | 24.1 | |
| SFT+RLScale=14B2026.05 | 60.8 | 92.8 | 45.2 | 22.7 | |
| DPO+RLScale=8B2026.05 | 59.2 | 87.9 | 44.6 | 23.7 | |
| SFT+RLScale=4B2026.05 | 58.6 | 91.2 | 43.2 | 22.1 | |
| SFT-80K+RLScale=3B, Backbone=LLaMA-3.2-3B-Instruct2026.05 | 57.4 | 85.6 | 43.2 | 23.5 | |
| DPO+RLScale=4B2026.05 | 56.6 | 91.1 | 41 | 21 | |
| CPT + RLScale=3B, Backbone=LLaMA-3.2-3B-Instruct2026.05 | 56.5 | 83.8 | 42.6 | 23.7 | |
| Abs-RLScale=3B, Backbone=LLaMA-3.2-3B-Instruct2026.05 | 56 | 84.6 | 41.8 | 23.1 | |
| BaseScale=3B, Backbone=LLaMA-3.2-3B-Instruct2026.05 | 54.3 | 84.5 | 40.1 | 22.1 | |
| BaseScale=14B2026.05 | 54.2 | 90.6 | 38.7 | 19.9 | |
| DPO+RLScale=3B, Backbone=LLaMA-3.2-3B-Instruct2026.05 | 52.6 | 85.3 | 38.1 | 20.8 | |
| BaseScale=4B2026.05 | 52.1 | 88.4 | 36.9 | 19.4 | |
| BaseScale=8B2026.05 | 48.3 | 87.6 | 33.3 | 17.7 | |
| SFT+RLScale=3B, Backbone=LLaMA-3.2-3B-Instruct2026.05 | 45.4 | 86.7 | 30.8 | 16.6 |