Confidence Estimation on GUESS
0.1858AccuracyVANILLA-VERB
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| VANILLA-VERBBackbone=Llama3.1-70b2026.01 | 0.1858 | 0.6552 | 16.92 | |
| COT-VERBBackbone=Llama3.1-70b2026.01 | 0.1858 | 0.7016 | 18.24 | |
| SCBackbone=Llama3.1-70b2026.01 | 0.1858 | 0.5688 | 2.59 | |
| P(TRUE)Backbone=Llama3.1-70b2026.01 | 0.1858 | 0.7997 | 3.29 | |
| P(SUFFICIENT)Backbone=Llama3.1-70b2026.01 | 0.1858 | 0.0527 | 81.51 | |
| VANILLA-VERBBackbone=Qwen2.5-72b2026.01 | 0.1612 | 0.6704 | 52.81 | |
| COT-VERBBackbone=Qwen2.5-72b2026.01 | 0.1612 | 0.6463 | 79 | |
| SCBackbone=Qwen2.5-72b2026.01 | 0.1612 | 0.6893 | 12.52 | |
| P(TRUE)Backbone=Qwen2.5-72b2026.01 | 0.1612 | 0.5756 | 54.87 | |
| P(SUFFICIENT)Backbone=Qwen2.5-72b2026.01 | 0.1612 | 0.2832 | 83.76 | |
| VANILLA-VERBBackbone=Llama3.1-8b2026.01 | 0.1452 | 0.7489 | -6.58 | |
| COT-VERBBackbone=Llama3.1-8b2026.01 | 0.1452 | 0.7028 | 37.84 | |
| SCBackbone=Llama3.1-8b2026.01 | 0.1452 | 0.3814 | 9.43 | |
| P(TRUE)Backbone=Llama3.1-8b2026.01 | 0.1452 | 0.6708 | 19.91 | |
| P(SUFFICIENT)Backbone=Llama3.1-8b2026.01 | 0.1452 | 0.3517 | 68.51 | |
| VANILLA-VERBBackbone=Qwen2.5-7b2026.01 | 0.1292 | 0.4868 | 26.99 | |
| COT-VERBBackbone=Qwen2.5-7b2026.01 | 0.1292 | 0.7184 | 56.01 | |
| SCBackbone=Qwen2.5-7b2026.01 | 0.1292 | 0.5053 | 36.1 | |
| P(TRUE)Backbone=Qwen2.5-7b2026.01 | 0.1292 | 0.3786 | 22.04 | |
| P(SUFFICIENT)Backbone=Qwen2.5-7b2026.01 | 0.1292 | 0.2663 | 51.44 |