Abstention Metacognition on AbstentionBench Abstention Prompt
74.9F1 ScoreSFT-80K+RL
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SFT-80K+RLScale=8B2026.05 | 74.9 | 85.3 | 66.7 | 36.5 | |
| Abs-RLScale=4B2026.05 | 74.9 | 78 | 72 | 43.1 | |
| Abs-RLScale=8B2026.05 | 74.3 | 83.1 | 67.2 | 37.7 | |
| CPTScale=32B, Backbone=Olmo-3 32B Base, RL=No RL2026.05 | 73.9 | 88 | 63.7 | 33.7 | |
| CPT + RLScale=14B2026.05 | 73.4 | 88.9 | 62.5 | 32.8 | |
| SFT-80K+RLScale=4B2026.05 | 73.3 | 86.5 | 63.6 | 34.3 | |
| SFT+RLScale=4B2026.05 | 72.5 | 84.7 | 63.4 | 34.9 | |
| SFT-80K+RLScale=14B2026.05 | 72.4 | 89.1 | 61 | 31.9 | |
| CPT + RLScale=8B2026.05 | 72.2 | 88.5 | 60.9 | 32.1 | |
| DPO+RLScale=4B2026.05 | 72.1 | 83.1 | 63.7 | 35.8 | |
| CPT + RLScale=4B2026.05 | 72.1 | 86.3 | 61.9 | 33.5 | |
| SFT+RLScale=8B2026.05 | 71.8 | 86.6 | 61.3 | 33 | |
| SFT-80K (no RL)Scale=32B, Backbone=Olmo-3 32B Base2026.05 | 70.8 | 89.2 | 58.8 | 30.7 | |
| BaseScale=14B2026.05 | 70.2 | 85.6 | 59.5 | 32.5 | |
| DPO+RLScale=8B2026.05 | 69.2 | 85.5 | 58.1 | 31.7 | |
| SFT (no RL)Scale=32B, Backbone=Olmo-3 32B Base2026.05 | 68.8 | 89.2 | 56 | 29.3 | |
| BaseScale=4B2026.05 | 68.6 | 79.6 | 60.3 | 35.3 | |
| Abs-RLScale=14B2026.05 | 68.2 | 91.1 | 54.5 | 27.9 | |
| DPO (no RL)Scale=32B, Backbone=Olmo-3 32B Base2026.05 | 67.8 | 89.6 | 54.5 | 28.4 | |
| DPO+RLScale=14B2026.05 | 66.9 | 92 | 52.5 | 26.7 | |
| Abs-RLScale=3B, Backbone=LLaMA-3.2-3B-Instruct2026.05 | 66.3 | 77.1 | 58.1 | 35.2 | |
| BaseScale=8B2026.05 | 65.6 | 82 | 54.5 | 31 | |
| SFT+RLScale=14B2026.05 | 64.7 | 92.3 | 49.9 | 25.2 | |
| CPT + RLScale=3B, Backbone=LLaMA-3.2-3B-Instruct2026.05 | 64.5 | 80.8 | 53.6 | 31 | |
| BaseScale=3B, Backbone=LLaMA-3.2-3B-Instruct2026.05 | 64 | 78.1 | 54.2 | 32.3 | |
| SFT-80K+RLScale=3B, Backbone=LLaMA-3.2-3B-Instruct2026.05 | 63.5 | 80.3 | 52.5 | 31.2 | |
| DPO+RLScale=3B, Backbone=LLaMA-3.2-3B-Instruct2026.05 | 60.7 | 80.5 | 48.8 | 28.3 | |
| SFT+RLScale=3B, Backbone=LLaMA-3.2-3B-Instruct2026.05 | 55.8 | 82.8 | 42 | 23.7 |