Question Answering Calibration on NQ, TQ, HQ, 2Wiki, Pararel
0.05Calibration Error (NQ)EliCal(1k)
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| EliCal(1k)Backbone=Qwen2.5-7B-Instruct, LoRA rank=8, LoRA alpha=16, Batch size=162025.10 | 0.05 | 0.06 | 0.03 | 0.05 | 0.08 | 0.05 | |
| ThermometerBackbone=Qwen2.5-7B-Instruct2025.10 | 0.06 | 0.05 | 0.04 | 0.05 | 0.04 | 0.06 | |
| Consis-LexBackbone=Qwen2.5-7B-Instruct2025.10 | 0.07 | 0.07 | 0.16 | 0.17 | 0.03 | 0.12 | |
| DACABackbone=Qwen2.5-7B-Instruct2025.10 | 0.09 | 0.05 | 0.1 | 0.14 | 0.1 | 0.1 | |
| ElicitationBackbone=Qwen2.5-7B-Instruct, LoRA rank=8, LoRA alpha=16, Batch size=162025.10 | 0.09 | 0.03 | 0.06 | 0.24 | 0.21 | 0.13 | |
| Cal-Only(1k)Backbone=Qwen2.5-7B-Instruct, LoRA rank=8, LoRA alpha=16, Batch size=162025.10 | 0.12 | 0.16 | 0.18 | 0.13 | 0.13 | 0.15 | |
| Consis-SemBackbone=Qwen2.5-7B-Instruct2025.10 | 0.13 | 0.04 | 0.11 | 0.28 | 0.25 | 0.16 | |
| Verbal-10Backbone=Qwen2.5-7B-Instruct2025.10 | 0.27 | 0.23 | 0.26 | 0.15 | 0.3 | 0.22 | |
| ProbBackbone=Qwen2.5-7B-Instruct2025.10 | 0.39 | 0.53 | 0.32 | 0.31 | 0.46 | 0.4 | |
| N-ProbBackbone=Qwen2.5-7B-Instruct2025.10 | 0.39 | 0.24 | 0.47 | 0.5 | 0.32 | 0.39 | |
| Verbal-0Backbone=Qwen2.5-7B-Instruct2025.10 | 0.41 | 0.28 | 0.44 | 0.41 | 0.28 | 0.37 |