Question Answering on TriviaQA (BLEU, ROUGE-L, METEOR, BS(%))
92.42BS (%)KA2L (10k Unknown)
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| KA2L (10k Unknown)Model=Mistral-7B-Instruct-v0.12026.03 | 92.42 | 32.77 | 63.64 | 47.43 | |
| KA2L (10k Unknown)Model=Llama-3.1-8B-Instruct2026.03 | 92.39 | 30.84 | 63.38 | 47.05 | |
| KA2L (10k Unknown)Model=Mistral-7B-Instruct-v0.32026.03 | 92.21 | 36.84 | 62.41 | 46.66 | |
| KA2L (10k Unknown)Model=Llama-2-7b-chat-hf2026.03 | 91.58 | 33.25 | 58.91 | 44.77 | |
| KA2L (10k Unknown)Model=glm4-9b-chat2026.03 | 91 | 11.68 | 55.8 | 42.43 | |
| SFT (10k Augmented)Model=Llama-3.1-8B-Instruct2026.03 | 90.12 | 22.05 | 49.56 | 36.23 | |
| KA2L (10k Unknown)Model=Qwen1.5-7B-Chat2026.03 | 90.03 | 25.91 | 48.38 | 35.8 | |
| KA2L (10k Unknown)Model=Phi-3.5-mini-instruct2026.03 | 89.73 | 25.72 | 47.15 | 34.19 | |
| SFT (10k Combine)Model=Llama-3.1-8B-Instruct2026.03 | 89.65 | 23.59 | 49.78 | 36.37 | |
| KA2L (5k Unknown)Model=Llama-3.1-8B-Instruct2026.03 | 89.61 | 18.96 | 49.23 | 35.95 | |
| SFT (10k Combine)Model=Llama-2-7b-chat-hf2026.03 | 89.37 | 26.36 | 49.16 | 36.84 | |
| KA2L (5k Unknown)Model=Llama-2-7b-chat-hf2026.03 | 89.33 | 24.57 | 48.9 | 36.69 | |
| SFT (10k Augmented)Model=Llama-2-7b-chat-hf2026.03 | 89.14 | 21.97 | 47.57 | 35.75 | |
| SFT (10k Combine)Model=Mistral-7B-Instruct-v0.32026.03 | 89.12 | 22.75 | 47.65 | 34.81 | |
| SFT (10k Combine)Model=Mistral-7B-Instruct-v0.12026.03 | 89.08 | 18.41 | 46.51 | 33.61 | |
| KA2L (5k Unknown)Model=Mistral-7B-Instruct-v0.32026.03 | 89.02 | 21.21 | 47.21 | 34.42 | |
| SFT (10k Augmented)Model=Mistral-7B-Instruct-v0.12026.03 | 89.01 | 15.67 | 45.86 | 33.21 | |
| SFT (10k Combine)Model=Phi-3.5-mini-instruct2026.03 | 89.01 | 22.93 | 43.39 | 31.6 | |
| SFT (10k Augmented)Model=Mistral-7B-Instruct-v0.32026.03 | 88.99 | 16.63 | 47.13 | 34.27 | |
| SFT (10k Augmented)Model=Phi-3.5-mini-instruct2026.03 | 88.99 | 19.26 | 42.98 | 31.27 | |
| KA2L (5k Unknown)Model=Phi-3.5-mini-instruct2026.03 | 88.98 | 23.26 | 42.94 | 31.24 | |
| KA2L (5k Unknown)Model=Mistral-7B-Instruct-v0.12026.03 | 88.92 | 15.93 | 45.52 | 32.98 | |
| SFT (5k Known)Model=Llama-3.1-8B-Instruct2026.03 | 88.5 | 14.81 | 43.14 | 31.54 | |
| SFT (5k Known)Model=Qwen1.5-7B-Chat2026.03 | 88.4 | 20.04 | 39.05 | 28.53 | |
| KA2L (5k Unknown)Model=Qwen1.5-7B-Chat2026.03 | 88.4 | 19.89 | 39.02 | 28.54 | |
| SFT (10k Combine)Model=Qwen1.5-7B-Chat2026.03 | 88.39 | 20.78 | 38.85 | 28.16 | |
| SFT (5k Known)Model=Phi-3.5-mini-instruct2026.03 | 88.29 | 18.13 | 39.1 | 28.31 | |
| SFT (10k Augmented)Model=Qwen1.5-7B-Chat2026.03 | 88.24 | 13.85 | 38.06 | 27.56 | |
| SFT (10k Combine)Model=glm4-9b-chat2026.03 | 88.18 | 21.51 | 41.26 | 30.71 | |
| SFT (10k Augmented)Model=glm4-9b-chat2026.03 | 88.09 | 15.21 | 40.47 | 30.07 | |
| SFT (5k Known)Model=Llama-2-7b-chat-hf2026.03 | 88.06 | 19.67 | 41.86 | 31.33 | |
| KA2L (5k Unknown)Model=glm4-9b-chat2026.03 | 87.97 | 19.62 | 40.28 | 29.85 | |
| SFT (5k Known)Model=Mistral-7B-Instruct-v0.32026.03 | 87.94 | 16.87 | 40.87 | 29.36 | |
| SFT (5k Known)Model=Mistral-7B-Instruct-v0.12026.03 | 87.88 | 13.2 | 39.35 | 28.41 | |
| KA2L (10k Unknown)Model=DeepSeek-R1-Distill-Qwen-7B2026.03 | 87.56 | 14.2 | 27 | 18.86 | |
| SFT (10k Augmented)Model=DeepSeek-R1-Distill-Qwen-7B2026.03 | 87.16 | 6.68 | 24.46 | 17.11 | |
| SFT (10k Combine)Model=DeepSeek-R1-Distill-Qwen-7B2026.03 | 87.05 | 9.98 | 23.65 | 16.38 | |
| KA2L (5k Unknown)Model=DeepSeek-R1-Distill-Qwen-7B2026.03 | 86.85 | 9.41 | 22.22 | 15.39 | |
| KA2L (10k Unknown)Model=Qwen2.5-7B-Instruct2026.03 | 86.49 | 14.49 | 35.88 | 27.07 | |
| SFT (5k Known)Model=glm4-9b-chat2026.03 | 86.34 | 14.68 | 31.65 | 23.03 | |
| SFT (5k Known)Model=DeepSeek-R1-Distill-Qwen-7B2026.03 | 86.32 | 7.79 | 18.77 | 12.82 | |
| SFT (10k Augmented)Model=Qwen2.5-7B-Instruct2026.03 | 85.8 | 5.82 | 31.99 | 24.24 | |
| SFT (10k Combine)Model=Qwen2.5-7B-Instruct2026.03 | 85.65 | 11.37 | 32.52 | 24.36 | |
| SFT (5k Known)Model=Qwen2.5-7B-Instruct2026.03 | 85.63 | 11.72 | 32.24 | 24.18 | |
| KA2L (5k Unknown)Model=Qwen2.5-7B-Instruct2026.03 | 85.63 | 11.72 | 32.3 | 24.2 | |
| None (Zero-shot)Model=Mistral-7B-Instruct-v0.12026.03 | 81.85 | 0.32 | 15.73 | 17.64 | |
| None (Zero-shot)Model=Qwen1.5-7B-Chat2026.03 | 79.28 | 0.14 | 3.84 | 8.18 | |
| None (Zero-shot)Model=Llama-3.1-8B-Instruct2026.03 | 79.26 | 0.12 | 5.67 | 11.11 | |
| None (Zero-shot)Model=glm4-9b-chat2026.03 | 79.13 | 0.15 | 3.62 | 8.16 | |
| None (Zero-shot)Model=Mistral-7B-Instruct-v0.32026.03 | 79.07 | 0.16 | 3.14 | 7.11 | |
| None (Zero-shot)Model=Qwen2.5-7B-Instruct2026.03 | 78.92 | 0.15 | 4.07 | 8.79 | |
| None (Zero-shot)Model=Phi-3.5-mini-instruct2026.03 | 78.86 | 0.11 | 2.41 | 5.85 | |
| None (Zero-shot)Model=Llama-2-7b-chat-hf2026.03 | 78.23 | 0.09 | 2.3 | 5.24 | |
| None (Zero-shot)Model=DeepSeek-R1-Distill-Qwen-7B2026.03 | 76.13 | 0.02 | 0.49 | 1.24 | |
| + Scaling-bin.Calibration category=Post-hoc plurality calibrators2026.05 | 12.29 | — | — | — | |
| GraphCalCalibration category=Trained calibrators2026.05 | 12.29 | — | — | — | |
| Plurality shareCalibration category=Post-hoc plurality calibrators2026.05 | 12.12 | — | — | — | |
| Scalar + GBTCalibration category=Trained calibrators2026.05 | 11.43 | — | — | — | |
| + PlattCalibration category=Post-hoc plurality calibrators2026.05 | 10.91 | — | — | — | |
| + IsotonicCalibration category=Post-hoc plurality calibrators2026.05 | 10.89 | — | — | — | |
| LLM-Cal (+topo)Calibration category=LLM-elicited confidence estimators2026.05 | 10.75 | — | — | — | |
| DiscoUQ-LLMCalibration category=Trained calibrators2026.05 | 10.47 | — | — | — | |
| Collab. Cal.Calibration category=LLM-elicited confidence estimators2026.05 | 10.32 | — | — | — | |
| LLM-Cal (no topo)Calibration category=LLM-elicited confidence estimators2026.05 | 10.11 | — | — | — | |
| CAGE-CALCalibration category=Trained calibrators2026.05 | 9.55 | — | — | — |