Model Calibration on MACE
82.4AUROCProb Entropy
Evaluation results
| Method | Links | |
|---|---|---|
| Prob EntropyBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Question-level, Answer Mixture=[1]2026.02 | 82.4 | |
| PerplexityBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Answer-level, Answer Mixture=[1]2026.02 | 82 | |
| SCABackbone Model=Qwen-2.5-72B-Instruct, tau=0.3, Answer Mixture=[1]2026.02 | 82 | |
| SCABackbone Model=Qwen-2.5-72B-Instruct, tau=0, Answer Mixture=[1]2026.02 | 82 | |
| N-Prob EntropyBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Question-level, Answer Mixture=[1]2026.02 | 81.8 | |
| PTrue-Prob-CandBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Double-turn, Answer Mixture=[1]2026.02 | 81.5 | |
| PTrue-ProbBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Double-turn, Answer Mixture=[1]2026.02 | 81.3 | |
| SCABackbone Model=Qwen-2.5-72B-Instruct, tau=0, Answer Mixture=[1,2]2026.02 | 80.4 | |
| SCABackbone Model=Qwen-2.5-72B-Instruct, tau=0.3, Answer Mixture=[1,2]2026.02 | 80.3 | |
| PerplexityBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Answer-level, Answer Mixture=[1,2]2026.02 | 79.9 | |
| SCABackbone Model=Qwen-2.5-72B-Instruct, tau=0.3, Answer Mixture=[1,2,4]2026.02 | 79.5 | |
| SCABackbone Model=Qwen-2.5-72B-Instruct, tau=0, Answer Mixture=[1,2,4]2026.02 | 79.5 | |
| Prob EntropyBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Question-level, Answer Mixture=[1,2]2026.02 | 79.1 | |
| PerplexityBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Answer-level, Answer Mixture=[1,2,4]2026.02 | 78.5 | |
| Prob EntropyBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Question-level, Answer Mixture=[1,2,4]2026.02 | 78.4 | |
| N-Prob EntropyBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Question-level, Answer Mixture=[1,2]2026.02 | 78.4 | |
| SCABackbone Model=Qwen-2.5-72B-Instruct, tau=0.3, Answer Mixture=[1,2,4,6]2026.02 | 78.4 | |
| SCABackbone Model=Qwen-2.5-72B-Instruct, tau=0, Answer Mixture=[1,2,4,6]2026.02 | 78.4 | |
| ConsisBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Answer-level, Answer Mixture=[1]2026.02 | 78.2 | |
| Prob EntropyBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Question-level, Answer Mixture=[1,2,4,6]2026.02 | 77.5 | |
| Consis-VerbBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Answer-level, Answer Mixture=[1]2026.02 | 77.5 | |
| PTrue-Prob-CandBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Double-turn, Answer Mixture=[1,2]2026.02 | 77.5 | |
| PerplexityBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Answer-level, Answer Mixture=[1,2,4,6]2026.02 | 77.4 | |
| Sem EntropyBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Question-level, Answer Mixture=[1]2026.02 | 77.3 | |
| PTrue-ProbBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Double-turn, Answer Mixture=[1,2]2026.02 | 77.2 | |
| ConsisBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Answer-level, Answer Mixture=[1,2]2026.02 | 77.1 | |
| N-Prob EntropyBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Question-level, Answer Mixture=[1,2,4]2026.02 | 77 | |
| ConsisBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Answer-level, Answer Mixture=[1,2,4]2026.02 | 76.7 | |
| SNCABackbone Model=Qwen-2.5-72B-Instruct, tau=0.2, Answer Mixture=[1]2026.02 | 76.5 | |
| SFCABackbone Model=Qwen-2.5-72B-Instruct, tau=0.2, Answer Mixture=[1]2026.02 | 76.4 | |
| PTrue-ProbBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Double-turn, Answer Mixture=[1,2,4,6]2026.02 | 76.3 | |
| N-Prob EntropyBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Question-level, Answer Mixture=[1,2,4,6]2026.02 | 76.2 | |
| PTrue-ProbBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Double-turn, Answer Mixture=[1,2,4]2026.02 | 76.2 | |
| ConsisBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Answer-level, Answer Mixture=[1,2,4,6]2026.02 | 75.8 | |
| Consis-VerbBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Answer-level, Answer Mixture=[1,2]2026.02 | 75.8 | |
| PTrue-Prob-CandBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Double-turn, Answer Mixture=[1,2,4]2026.02 | 75.8 | |
| Sem EntropyBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Question-level, Answer Mixture=[1,2]2026.02 | 75.6 | |
| PTrue-Prob-CandBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Double-turn, Answer Mixture=[1,2,4,6]2026.02 | 75.6 | |
| Consis-VerbBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Answer-level, Answer Mixture=[1,2,4]2026.02 | 75.4 | |
| SFCABackbone Model=Qwen-2.5-72B-Instruct, tau=0.2, Answer Mixture=[1,2]2026.02 | 75.2 | |
| Sem EntropyBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Question-level, Answer Mixture=[1,2,4]2026.02 | 75.1 | |
| Consis-Verb-TopkBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Answer-level, Answer Mixture=[1]2026.02 | 75.1 | |
| SNCABackbone Model=Qwen-2.5-72B-Instruct, tau=0.2, Answer Mixture=[1,2]2026.02 | 74.9 | |
| Self-Ask-CandBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Double-turn, Answer Mixture=[1]2026.02 | 74.4 | |
| SFCABackbone Model=Qwen-2.5-72B-Instruct, tau=0.2, Answer Mixture=[1,2,4]2026.02 | 74.2 | |
| Consis-VerbBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Answer-level, Answer Mixture=[1,2,4,6]2026.02 | 74.1 | |
| Sem EntropyBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Question-level, Answer Mixture=[1,2,4,6]2026.02 | 73.9 | |
| PTrue-ConsisBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Double-turn, Answer Mixture=[1]2026.02 | 73.7 | |
| SNCABackbone Model=Qwen-2.5-72B-Instruct, tau=0.2, Answer Mixture=[1,2,4]2026.02 | 73.6 | |
| SFCABackbone Model=Qwen-2.5-72B-Instruct, tau=0.2, Answer Mixture=[1,2,4,6]2026.02 | 73.4 | |
| Consis-Verb-TopkBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Answer-level, Answer Mixture=[1,2]2026.02 | 73.3 | |
| Consis-Verb-TopkBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Answer-level, Answer Mixture=[1,2,4]2026.02 | 72.3 | |
| SNCABackbone Model=Qwen-2.5-72B-Instruct, tau=0.2, Answer Mixture=[1,2,4,6]2026.02 | 72 | |
| PTrue-Consis-CandBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Double-turn, Answer Mixture=[1]2026.02 | 71.1 | |
| Consis-Verb-TopkBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Answer-level, Answer Mixture=[1,2,4,6]2026.02 | 71 | |
| Self-Ask-CandBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Double-turn, Answer Mixture=[1,2]2026.02 | 70.6 | |
| PTrue-ConsisBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Double-turn, Answer Mixture=[1,2]2026.02 | 70 | |
| PTrue-ConsisBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Double-turn, Answer Mixture=[1,2,4,6]2026.02 | 69.6 | |
| PTrue-ConsisBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Double-turn, Answer Mixture=[1,2,4]2026.02 | 69.4 | |
| PTrue-Consis-CandBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Double-turn, Answer Mixture=[1,2]2026.02 | 69.2 | |
| PTrue-Consis-CandBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Double-turn, Answer Mixture=[1,2,4]2026.02 | 68.3 | |
| PTrue-Consis-CandBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Double-turn, Answer Mixture=[1,2,4,6]2026.02 | 68.2 | |
| Verb-TopkBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Answer-level, Answer Mixture=[1]2026.02 | 67.7 | |
| Self-Ask-CandBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Double-turn, Answer Mixture=[1,2,4]2026.02 | 67.2 | |
| Self-AskBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Double-turn, Answer Mixture=[1]2026.02 | 67.1 | |
| Self-Ask-CandBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Double-turn, Answer Mixture=[1,2,4,6]2026.02 | 66 | |
| VerbBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Answer-level, Answer Mixture=[1]2026.02 | 64.3 | |
| Verb-TopkBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Answer-level, Answer Mixture=[1,2]2026.02 | 64.1 | |
| Self-AskBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Double-turn, Answer Mixture=[1,2]2026.02 | 64.1 | |
| Self-AskBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Double-turn, Answer Mixture=[1,2,4]2026.02 | 62.1 | |
| Self-AskBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Double-turn, Answer Mixture=[1,2,4,6]2026.02 | 61.6 | |
| VerbBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Answer-level, Answer Mixture=[1,2]2026.02 | 60.6 | |
| Verb-TopkBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Answer-level, Answer Mixture=[1,2,4]2026.02 | 59.9 | |
| VerbBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Answer-level, Answer Mixture=[1,2,4]2026.02 | 58.5 | |
| Verb-TopkBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Answer-level, Answer Mixture=[1,2,4,6]2026.02 | 57.7 | |
| VerbBackbone Model=Qwen-2.5-72B-Instruct, Turn Type=Single-turn, Calibration Level=Answer-level, Answer Mixture=[1,2,4,6]2026.02 | 57.4 | |
| SNCABackbone Model=Qwen-2.5-72B-Instruct, tau=0, Answer Mixture=[1]2026.02 | 50 | |
| SNCABackbone Model=Qwen-2.5-72B-Instruct, tau=0, Answer Mixture=[1,2]2026.02 | 50 | |
| SNCABackbone Model=Qwen-2.5-72B-Instruct, tau=0, Answer Mixture=[1,2,4]2026.02 | 50 | |
| SNCABackbone Model=Qwen-2.5-72B-Instruct, tau=0, Answer Mixture=[1,2,4,6]2026.02 | 50 | |
| SFCABackbone Model=Qwen-2.5-72B-Instruct, tau=0, Answer Mixture=[1]2026.02 | 50 | |
| SFCABackbone Model=Qwen-2.5-72B-Instruct, tau=0, Answer Mixture=[1,2]2026.02 | 50 | |
| SFCABackbone Model=Qwen-2.5-72B-Instruct, tau=0, Answer Mixture=[1,2,4]2026.02 | 50 | |
| SFCABackbone Model=Qwen-2.5-72B-Instruct, tau=0, Answer Mixture=[1,2,4,6]2026.02 | 50 |