LLM Calibration on MACE
8.6ECEConsistency
Evaluation results
| Method | Links | |
|---|---|---|
| ConsistencyBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Answer-level2026.02 | 8.6 | |
| Sem EntropyBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Question-level2026.02 | 9.6 | |
| ConsistencyBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Answer-level2026.02 | 10.4 | |
| Consis-VerbBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Answer-level2026.02 | 11.7 | |
| Consis-VerbBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Answer-level2026.02 | 12.3 | |
| Consis-Verb-TopkBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Answer-level2026.02 | 12.4 | |
| Sem EntropyBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Question-level2026.02 | 12.5 | |
| Consis-Verb-TopkBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Answer-level2026.02 | 14.3 | |
| Sem EntropyBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Question-level2026.02 | 16.6 | |
| VerbTopkBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Answer-level2026.02 | 19.3 | |
| Prob EntropyBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Question-level2026.02 | 19.4 | |
| PerplexityBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Answer-level2026.02 | 19.6 | |
| Prob EntropyBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Question-level2026.02 | 20.8 | |
| PerplexityBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Answer-level2026.02 | 20.9 | |
| Sem EntropyBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Question-level2026.02 | 21.3 | |
| Consis-VerbBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Answer-level2026.02 | 21.5 | |
| ConsistencyBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Answer-level2026.02 | 21.6 | |
| Prob EntropyBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Question-level2026.02 | 21.7 | |
| VerbTopkBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Answer-level2026.02 | 21.9 | |
| VerbTopkBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Answer-level2026.02 | 22.5 | |
| SelfAsk-CandBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Double-turn2026.02 | 22.8 | |
| PerplexityBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Answer-level2026.02 | 23.4 | |
| SelfAsk-CandBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Double-turn2026.02 | 23.5 | |
| Consis-Verb-TopkBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Answer-level2026.02 | 23.6 | |
| SelfAsk-CandBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Double-turn2026.02 | 25.6 | |
| Prob EntropyBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Question-level2026.02 | 26.2 | |
| SelfAsk-CandBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Double-turn2026.02 | 26.3 | |
| ConsistencyBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Answer-level2026.02 | 27 | |
| Consis-VerbBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Answer-level2026.02 | 27.1 | |
| Consis-Verb-TopkBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Answer-level2026.02 | 29.1 | |
| PTrue-Prob-CandBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Double-turn2026.02 | 30 | |
| PTrue-Prob-CandBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Double-turn2026.02 | 30.2 | |
| N-Prob EntropyBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Question-level2026.02 | 30.3 | |
| PTrue-Consis-CandBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Double-turn2026.02 | 30.4 | |
| PerplexityBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Answer-level2026.02 | 30.6 | |
| PTrue-Consis-CandBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Double-turn2026.02 | 30.6 | |
| PTrue-ProbBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Double-turn2026.02 | 31.4 | |
| PTrue-ConsisBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Double-turn2026.02 | 31.5 | |
| N-Prob EntropyBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Question-level2026.02 | 31.7 | |
| PTrue-ProbBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Double-turn2026.02 | 32.7 | |
| PTrue-ConsisBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Double-turn2026.02 | 33 | |
| VerbBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Answer-level2026.02 | 33.1 | |
| PTrue-ProbBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Double-turn2026.02 | 33.7 | |
| PTrue-ConsisBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Double-turn2026.02 | 34.1 | |
| VerbBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Answer-level2026.02 | 34.6 | |
| PTrue-ProbBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Double-turn2026.02 | 35.1 | |
| PTrue-ConsisBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Double-turn2026.02 | 35.3 | |
| PTrue-Prob-CandBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Double-turn2026.02 | 35.4 | |
| N-Prob EntropyBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Question-level2026.02 | 35.5 | |
| PTrue-Consis-CandBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Double-turn2026.02 | 35.6 | |
| VerbTopkBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Answer-level2026.02 | 35.8 | |
| SelfAskBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Double-turn2026.02 | 36.6 | |
| VerbBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Answer-level2026.02 | 39.5 | |
| SelfAskBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Double-turn2026.02 | 40.9 | |
| PTrue-Prob-CandBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Double-turn2026.02 | 41.6 | |
| PTrue-Consis-CandBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Double-turn2026.02 | 41.7 | |
| N-Prob EntropyBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Question-level2026.02 | 43.5 | |
| SelfAskBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Double-turn2026.02 | 45.8 | |
| SelfAskBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Double-turn2026.02 | 46.7 | |
| VerbBackbone Model=LLaMA-3.1-70B-Instruct, Turn Type=Single-turn, Evaluation Level=Answer-level2026.02 | 46.8 |