Confidence Calibration in Retrieval-Augmented Generation on NQ k=5 OOD (test)
0.248ECENAACL
Evaluation results
| Method | Links | ||
|---|---|---|---|
| NAACLBackbone=Qwen2.5-7B-Instruct, Number of retrieved passages (k)=52026.01 | 0.248 | 0.75 | |
| NAACLBackbone=Llama-3.1-8B-Instruct, Number of retrieved passages (k)=52026.01 | 0.265 | 0.674 | |
| Label-only SFTBackbone=Llama-3.1-8B-Instruct, Number of retrieved passages (k)=52026.01 | 0.273 | 0.653 | |
| NAACLBackbone=DeepSeek-R1-Distill-Llama-8B, Number of retrieved passages (k)=52026.01 | 0.276 | 0.628 | |
| Noise-awareBackbone=Llama-3.1-8B-Instruct, Number of retrieved passages (k)=52026.01 | 0.289 | 0.667 | |
| Noise-awareBackbone=DeepSeek-R1-Distill-Llama-8B, Number of retrieved passages (k)=52026.01 | 0.29 | 0.605 | |
| Noise-awareBackbone=Qwen2.5-7B-Instruct, Number of retrieved passages (k)=52026.01 | 0.304 | 0.641 | |
| CoTBackbone=Qwen2.5-7B-Instruct, Number of retrieved passages (k)=52026.01 | 0.313 | 0.696 | |
| VanillaBackbone=Qwen2.5-7B-Instruct, Number of retrieved passages (k)=52026.01 | 0.322 | 0.706 | |
| EnsembleBackbone=Qwen2.5-7B-Instruct, Number of retrieved passages (k)=52026.01 | 0.325 | 0.694 | |
| Label-only SFTBackbone=DeepSeek-R1-Distill-Llama-8B, Number of retrieved passages (k)=52026.01 | 0.329 | 0.604 | |
| EnsembleBackbone=Llama-3.1-8B-Instruct, Number of retrieved passages (k)=52026.01 | 0.334 | 0.693 | |
| Label-only SFTBackbone=Qwen2.5-7B-Instruct, Number of retrieved passages (k)=52026.01 | 0.335 | 0.667 | |
| NAACLBackbone=DeepSeek-R1-Distill-Qwen-7B, Number of retrieved passages (k)=52026.01 | 0.335 | 0.64 | |
| EnsembleBackbone=DeepSeek-R1-Distill-Llama-8B, Number of retrieved passages (k)=52026.01 | 0.351 | 0.59 | |
| CoTBackbone=Llama-3.1-8B-Instruct, Number of retrieved passages (k)=52026.01 | 0.352 | 0.67 | |
| VanillaBackbone=Llama-3.1-8B-Instruct, Number of retrieved passages (k)=52026.01 | 0.371 | 0.645 | |
| CoTBackbone=DeepSeek-R1-Distill-Llama-8B, Number of retrieved passages (k)=52026.01 | 0.373 | 0.621 | |
| VanillaBackbone=DeepSeek-R1-Distill-Llama-8B, Number of retrieved passages (k)=52026.01 | 0.376 | 0.625 | |
| Noise-awareBackbone=DeepSeek-R1-Distill-Qwen-7B, Number of retrieved passages (k)=52026.01 | 0.376 | 0.477 | |
| EnsembleBackbone=DeepSeek-R1-Distill-Qwen-7B, Number of retrieved passages (k)=52026.01 | 0.398 | 0.597 | |
| Label-only SFTBackbone=DeepSeek-R1-Distill-Qwen-7B, Number of retrieved passages (k)=52026.01 | 0.402 | 0.686 | |
| VanillaBackbone=DeepSeek-R1-Distill-Qwen-7B, Number of retrieved passages (k)=52026.01 | 0.435 | 0.608 | |
| CoTBackbone=DeepSeek-R1-Distill-Qwen-7B, Number of retrieved passages (k)=52026.01 | 0.454 | 0.627 |