Confidence Estimation on GRACE
48.27AccuracyVANILLA-VERB
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| VANILLA-VERBBackbone=Llama3.1-70b2026.01 | 48.27 | 39.06 | 0.4713 | |
| COT-VERBBackbone=Llama3.1-70b2026.01 | 48.27 | 96.04 | 0.613 | |
| SCBackbone=Llama3.1-70b2026.01 | 48.27 | 15.91 | 0.4136 | |
| P(TRUE)Backbone=Llama3.1-70b2026.01 | 48.27 | 37.04 | 0.5894 | |
| P(SUFFICIENT)Backbone=Llama3.1-70b2026.01 | 48.27 | 11.52 | 0.6686 | |
| VANILLA-VERBBackbone=Qwen2.5-72b2026.01 | 47.49 | 43.18 | 0.7259 | |
| COT-VERBBackbone=Qwen2.5-72b2026.01 | 47.49 | 46.49 | 0.7304 | |
| SCBackbone=Qwen2.5-72b2026.01 | 47.49 | 32.38 | 0.4917 | |
| P(TRUE)Backbone=Qwen2.5-72b2026.01 | 47.49 | 31.86 | 0.6402 | |
| P(SUFFICIENT)Backbone=Qwen2.5-72b2026.01 | 47.49 | 32.93 | 0.6604 | |
| VANILLA-VERBBackbone=Llama3.1-8b2026.01 | 35.73 | 51 | 0.5221 | |
| COT-VERBBackbone=Llama3.1-8b2026.01 | 35.73 | 45.21 | 0.4325 | |
| SCBackbone=Llama3.1-8b2026.01 | 35.73 | 10.57 | 0.524 | |
| P(TRUE)Backbone=Llama3.1-8b2026.01 | 35.73 | 50.43 | 0.4848 | |
| P(SUFFICIENT)Backbone=Llama3.1-8b2026.01 | 35.73 | 23.77 | 0.5394 | |
| VANILLA-VERBBackbone=Qwen2.5-7b2026.01 | 27.34 | 50.4 | 0.5555 | |
| COT-VERBBackbone=Qwen2.5-7b2026.01 | 27.34 | 66.2 | 0.5038 | |
| SCBackbone=Qwen2.5-7b2026.01 | 27.34 | 32.78 | 0.4016 | |
| P(TRUE)Backbone=Qwen2.5-7b2026.01 | 27.34 | 35.15 | 0.4404 | |
| P(SUFFICIENT)Backbone=Qwen2.5-7b2026.01 | 27.34 | 28.67 | 0.4779 |