Confidence Estimation on 20Q
33.87AccuracyVANILLA-VERB
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| VANILLA-VERBBackbone=Llama3.1-70b2026.01 | 33.87 | 59.63 | 17.6 | |
| COT-VERBBackbone=Llama3.1-70b2026.01 | 33.87 | 58.39 | 34.49 | |
| SCBackbone=Llama3.1-70b2026.01 | 33.87 | 32.99 | 28.98 | |
| P(TRUE)Backbone=Llama3.1-70b2026.01 | 33.87 | 67.82 | 40.82 | |
| P(SUFFICIENT)Backbone=Llama3.1-70b2026.01 | 33.87 | 13.05 | 48.43 | |
| VANILLA-VERBBackbone=Qwen2.5-72b2026.01 | 32.36 | 47.92 | 67.97 | |
| COT-VERBBackbone=Qwen2.5-72b2026.01 | 32.36 | 51.43 | 72.33 | |
| SCBackbone=Qwen2.5-72b2026.01 | 32.36 | 45.69 | 28.9 | |
| P(TRUE)Backbone=Qwen2.5-72b2026.01 | 32.36 | 42.12 | 68.88 | |
| P(SUFFICIENT)Backbone=Qwen2.5-72b2026.01 | 32.36 | 45.86 | 66.81 | |
| VANILLA-VERBBackbone=Qwen2.5-7b2026.01 | 25.22 | 58.05 | 61.13 | |
| COT-VERBBackbone=Qwen2.5-7b2026.01 | 25.22 | 64.93 | 52.6 | |
| SCBackbone=Qwen2.5-7b2026.01 | 25.22 | 45.44 | 13.85 | |
| P(TRUE)Backbone=Qwen2.5-7b2026.01 | 25.22 | 46.68 | 47.16 | |
| P(SUFFICIENT)Backbone=Qwen2.5-7b2026.01 | 25.22 | 36.64 | 55.24 | |
| VANILLA-VERBBackbone=Llama3.1-8b2026.01 | 24.95 | 67.82 | -6.36 | |
| COT-VERBBackbone=Llama3.1-8b2026.01 | 24.95 | 63.75 | 46.97 | |
| SCBackbone=Llama3.1-8b2026.01 | 24.95 | 18.05 | 36.73 | |
| P(TRUE)Backbone=Llama3.1-8b2026.01 | 24.95 | 69.02 | 42.1 | |
| P(SUFFICIENT)Backbone=Llama3.1-8b2026.01 | 24.95 | 41.08 | 38.57 |