Confidence Calibration on MACE (test)
81.2AUROCSCA
Evaluation results
| Method | Links | |
|---|---|---|
| SCABackbone=LLaMA-3.1-70B-Instruct, Filtering Threshold (τ)=0.052026.02 | 81.2 | |
| SCABackbone=LLaMA-3.1-70B-Instruct, Filtering Threshold (τ)=02026.02 | 81.2 | |
| PerplexityBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Answer-level2026.02 | 80.9 | |
| SCABackbone=LLaMA-3.1-70B-Instruct, Filtering Threshold (τ)=0.052026.02 | 80.6 | |
| SCABackbone=LLaMA-3.1-70B-Instruct, Filtering Threshold (τ)=02026.02 | 80.5 | |
| ConsisBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Answer-level2026.02 | 80.3 | |
| ConsisBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Answer-level2026.02 | 80.2 | |
| Consis-Verb-TopkBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Answer-level2026.02 | 80 | |
| Consis-VerbBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Answer-level2026.02 | 79.7 | |
| PerplexityBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Answer-level2026.02 | 79.4 | |
| SCABackbone=LLaMA-3.1-70B-Instruct, Filtering Threshold (τ)=0.052026.02 | 79.2 | |
| Consis-VerbBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Answer-level2026.02 | 79.1 | |
| SCABackbone=LLaMA-3.1-70B-Instruct, Filtering Threshold (τ)=02026.02 | 79.1 | |
| Consis-Verb-TopkBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Answer-level2026.02 | 79 | |
| Sem EntropyBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Question-level2026.02 | 78.9 | |
| Sem EntropyBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Question-level2026.02 | 78.6 | |
| ConsisBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Answer-level2026.02 | 78.1 | |
| SNCABackbone=LLaMA-3.1-70B-Instruct, Filtering Threshold (τ)=0.52026.02 | 77.9 | |
| SNCABackbone=LLaMA-3.1-70B-Instruct, Filtering Threshold (τ)=0.52026.02 | 77.3 | |
| SFCABackbone=LLaMA-3.1-70B-Instruct, Filtering Threshold (τ)=0.252026.02 | 77.2 | |
| SFCABackbone=LLaMA-3.1-70B-Instruct, Filtering Threshold (τ)=0.252026.02 | 77.1 | |
| Consis-Verb-TopkBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Answer-level2026.02 | 77 | |
| SCABackbone=LLaMA-3.1-70B-Instruct, Filtering Threshold (τ)=0.052026.02 | 76.7 | |
| SCABackbone=LLaMA-3.1-70B-Instruct, Filtering Threshold (τ)=02026.02 | 76.7 | |
| Consis-VerbBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Answer-level2026.02 | 76.6 | |
| PerplexityBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Answer-level2026.02 | 76.5 | |
| PTrue-Prob-CandBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Double-turn2026.02 | 76 | |
| Sem EntropyBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Question-level2026.02 | 75.8 | |
| ConsisBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Answer-level2026.02 | 75.3 | |
| SFCABackbone=LLaMA-3.1-70B-Instruct, Filtering Threshold (τ)=0.252026.02 | 75.2 | |
| Prob EntropyBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Question-level2026.02 | 74.5 | |
| Consis-Verb-TopkBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Answer-level2026.02 | 74.3 | |
| SNCABackbone=LLaMA-3.1-70B-Instruct, Filtering Threshold (τ)=0.52026.02 | 74.2 | |
| N-Prob EntropyBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Question-level2026.02 | 73.8 | |
| Consis-VerbBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Answer-level2026.02 | 73.8 | |
| PerplexityBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Answer-level2026.02 | 73.8 | |
| PTrue-Prob-CandBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Double-turn2026.02 | 73.5 | |
| SFCABackbone=LLaMA-3.1-70B-Instruct, Filtering Threshold (τ)=0.252026.02 | 73 | |
| PTrue-ProbBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Double-turn2026.02 | 72.9 | |
| Prob EntropyBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Question-level2026.02 | 72.6 | |
| PTrue-Prob-CandBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Double-turn2026.02 | 72.5 | |
| Sem EntropyBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Question-level2026.02 | 72.5 | |
| PTrue-Prob-CandBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Double-turn2026.02 | 71.1 | |
| SNCABackbone=LLaMA-3.1-70B-Instruct, Filtering Threshold (τ)=0.52026.02 | 70.5 | |
| N-Prob EntropyBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Question-level2026.02 | 70.4 | |
| PTrue-ProbBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Double-turn2026.02 | 70.1 | |
| Prob EntropyBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Question-level2026.02 | 69 | |
| PTrue-ConsisBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Double-turn2026.02 | 68.1 | |
| PTrue-ProbBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Double-turn2026.02 | 67.6 | |
| Self-Ask-CandBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Double-turn2026.02 | 66.9 | |
| Self-Ask-CandBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Double-turn2026.02 | 66.8 | |
| PTrue-ProbBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Double-turn2026.02 | 66.7 | |
| Self-AskBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Double-turn2026.02 | 66.5 | |
| Self-Ask-CandBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Double-turn2026.02 | 66.5 | |
| PTrue-ConsisBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Double-turn2026.02 | 66.4 | |
| N-Prob EntropyBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Question-level2026.02 | 66.2 | |
| Self-AskBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Double-turn2026.02 | 66.1 | |
| Prob EntropyBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Question-level2026.02 | 66.1 | |
| Self-Ask-CandBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Double-turn2026.02 | 66.1 | |
| Self-AskBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Double-turn2026.02 | 65 | |
| PTrue-ConsisBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Double-turn2026.02 | 64.1 | |
| Self-AskBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Double-turn2026.02 | 63.9 | |
| PTrue-ConsisBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Double-turn2026.02 | 63.5 | |
| N-Prob EntropyBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Question-level2026.02 | 63.3 | |
| PTrue-Consis-CandBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Double-turn2026.02 | 62.9 | |
| PTrue-Consis-CandBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Double-turn2026.02 | 62.7 | |
| PTrue-Consis-CandBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Double-turn2026.02 | 62.6 | |
| PTrue-Consis-CandBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Double-turn2026.02 | 62.5 | |
| Verb-TopkBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Answer-level2026.02 | 60.7 | |
| VerbBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Answer-level2026.02 | 58.4 | |
| VerbBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Answer-level2026.02 | 58.4 | |
| VerbBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Answer-level2026.02 | 57.2 | |
| Verb-TopkBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Answer-level2026.02 | 56.2 | |
| VerbBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Answer-level2026.02 | 55.8 | |
| Verb-TopkBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Answer-level2026.02 | 51.2 | |
| SNCABackbone=LLaMA-3.1-70B-Instruct, Filtering Threshold (τ)=02026.02 | 50 | |
| SFCABackbone=LLaMA-3.1-70B-Instruct, Filtering Threshold (τ)=02026.02 | 50 | |
| SNCABackbone=LLaMA-3.1-70B-Instruct, Filtering Threshold (τ)=02026.02 | 50 | |
| SFCABackbone=LLaMA-3.1-70B-Instruct, Filtering Threshold (τ)=02026.02 | 50 | |
| SNCABackbone=LLaMA-3.1-70B-Instruct, Filtering Threshold (τ)=02026.02 | 50 | |
| SFCABackbone=LLaMA-3.1-70B-Instruct, Filtering Threshold (τ)=02026.02 | 50 | |
| SNCABackbone=LLaMA-3.1-70B-Instruct, Filtering Threshold (τ)=02026.02 | 50 | |
| SFCABackbone=LLaMA-3.1-70B-Instruct, Filtering Threshold (τ)=02026.02 | 50 | |
| Verb-TopkBackbone=LLaMA-3.1-70B-Instruct, Interaction Turn=Single-turn, Calibration Granularity=Answer-level2026.02 | 48.4 |