Confidence Alignment on TruthfulQA
0.0228ECECONQORD
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| CONQORDFoundation Model=Mistral 7B2024.04 | 0.0228 | 0.1545 | 0 | 0.1509 | 0 | 0.3293 | |
| CONQORDFoundation Model=Zephyr 7B2024.04 | 0.1471 | 0.3992 | 0 | 0.41 | 0 | 0.3696 | |
| CONQORDFoundation Model=LLAMA-2 7B2024.04 | 0.1856 | 0.1086 | 0.0019 | 0.1096 | 0.0017 | 0.2387 | |
| VanillaFoundation Model=Zephyr 7B2024.04 | 0.2132 | 0.3494 | 0 | 0.3814 | 0 | 0.4213 | |
| CoT+AggFoundation Model=Zephyr 7B2024.04 | 0.2271 | 0.3952 | 0 | 0.4174 | 0 | 0.5006 | |
| Top-kFoundation Model=Zephyr 7B2024.04 | 0.2469 | 0.2571 | 0 | 0.2455 | 0 | 0.4419 | |
| Top-kFoundation Model=Mistral 7B2024.04 | 0.2741 | 0.1531 | 0 | 0.1422 | 0 | 0.2558 | |
| VanillaFoundation Model=Mistral 7B2024.04 | 0.3379 | 0.0096 | 0.78 | 0.0333 | 0.34 | 0.3244 | |
| CoT+AggFoundation Model=LLAMA-2 13B2024.04 | 0.3696 | 0.0683 | 0.13 | 0.0683 | 0.13 | 0.51 | |
| CoT+AggFoundation Model=LLAMA-2 7B2024.04 | 0.4086 | -0.0275 | 0.54 | -0.0275 | 0.54 | 0.3488 | |
| CONQORDFoundation Model=LLAMA-2 13B2024.04 | 0.4942 | 0.0998 | 0.0043 | 0.1789 | 0 | 0.3011 | |
| Top-kFoundation Model=LLAMA-2 13B2024.04 | 0.495 | -0.0296 | 0.4 | 0.0055 | 0.88 | 0.4002 | |
| Top-kFoundation Model=LLAMA-2 7B2024.04 | 0.5339 | -0.0524 | 0.13 | -0.0577 | 0.099 | 0.3611 | |
| VanillaFoundation Model=LLAMA-2 13B2024.04 | 0.5887 | 0.0578 | 0.099 | 0.0616 | 0.079 | 0.3048 | |
| CoT+AggFoundation Model=Mistral 7B2024.04 | 0.6021 | 0.0465 | 0.18 | 0.0411 | 0.24 | 0.257 | |
| VanillaFoundation Model=LLAMA-2 7B2024.04 | 0.6327 | 0.0154 | 0.66 | 0.0159 | 0.65 | 0.2387 |