Confidence Estimation on TRICKME
53.88AccuracyVANILLA-VERB
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| VANILLA-VERBBackbone=Qwen2.5-72b2026.01 | 53.88 | 41.62 | 71.32 | |
| COT-VERBBackbone=Qwen2.5-72b2026.01 | 53.88 | 43.5 | 70.35 | |
| SCBackbone=Qwen2.5-72b2026.01 | 53.88 | 36.5 | 48.52 | |
| P(TRUE)Backbone=Qwen2.5-72b2026.01 | 53.88 | 32.87 | 69.28 | |
| P(SUFFICIENT)Backbone=Qwen2.5-72b2026.01 | 53.88 | 32.41 | 71.24 | |
| VANILLA-VERBBackbone=Llama3.1-70b2026.01 | 53.75 | 47.47 | 44.49 | |
| COT-VERBBackbone=Llama3.1-70b2026.01 | 53.75 | 80.97 | 57.27 | |
| SCBackbone=Llama3.1-70b2026.01 | 53.75 | 19.9 | 38.26 | |
| P(TRUE)Backbone=Llama3.1-70b2026.01 | 53.75 | 35.62 | 64.25 | |
| P(SUFFICIENT)Backbone=Llama3.1-70b2026.01 | 53.75 | 23.16 | 71.38 | |
| VANILLA-VERBBackbone=Llama3.1-8b2026.01 | 41.91 | 59.26 | 54.55 | |
| COT-VERBBackbone=Llama3.1-8b2026.01 | 41.91 | 87.7 | 48.63 | |
| SCBackbone=Llama3.1-8b2026.01 | 41.91 | 18.97 | 55.37 | |
| P(TRUE)Backbone=Llama3.1-8b2026.01 | 41.91 | 55.61 | 52.37 | |
| P(SUFFICIENT)Backbone=Llama3.1-8b2026.01 | 41.91 | 33.74 | 58.34 | |
| VANILLA-VERBBackbone=Qwen2.5-7b2026.01 | 34.06 | 54.62 | 56.39 | |
| COT-VERBBackbone=Qwen2.5-7b2026.01 | 34.06 | 65.37 | 49.89 | |
| SCBackbone=Qwen2.5-7b2026.01 | 34.06 | 33.28 | 42.54 | |
| P(TRUE)Backbone=Qwen2.5-7b2026.01 | 34.06 | 39.45 | 51 | |
| P(SUFFICIENT)Backbone=Qwen2.5-7b2026.01 | 34.06 | 35.26 | 52.11 |