Question Answering Calibration on HotpotQA
81AUROCVEROIC
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| VEROICsetting=CMIS, inference_path=LLaMA-3.1-8B -> LLaMA-3.1-70B2026.04 | 81 | 43 | 16.6 | 0.513 | 6.4 | |
| RISK PREDICTORsetting=CMIS, inference_path=LLaMA-3.1-8B -> LLaMA-3.1-70B2026.04 | 80 | 41 | 17.2 | 0.52 | 7.2 | |
| CONFIDENCE-THRESHOLDsetting=CMIS, inference_path=LLaMA-3.1-8B -> LLaMA-3.1-70B2026.04 | 78 | 39 | 18.3 | 0.535 | 8.2 | |
| OUTPUT-TRIGGERsetting=CMIS, inference_path=LLaMA-3.1-8B -> LLaMA-3.1-70B2026.04 | 77 | 38 | 18 | 0.528 | 7.9 | |
| PERIODICsetting=CMIS, inference_path=LLaMA-3.1-8B -> LLaMA-3.1-70B2026.04 | 72 | 31 | 18.5 | 0.548 | 8.5 | |
| RANDOMsetting=CMIS, inference_path=LLaMA-3.1-8B -> LLaMA-3.1-70B2026.04 | 69 | 28 | 19.2 | 0.563 | 9.3 | |
| FIXEDsetting=CMIS, inference_path=LLaMA-3.1-8B -> LLaMA-3.1-70B2026.04 | 64 | 23 | 21.8 | 0.615 | 11.4 |