Question Answering on HotpotQA (ECE and AUROC)
0.45ECEBM25 (CoT)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| BM25 (CoT)Base Model=Qwen2.5-7B-Instruct, Retrieval Method=BM25, Prompting Strategy=CoT2026.01 | 0.45 | 0.726 | |
| BM25 (CoT)Base Model=DeepSeek-R1-Distill-Llama-8B, Retrieval Method=BM25, Prompting Strategy=CoT2026.01 | 0.459 | 0.628 | |
| Contriever (CoT)Base Model=DeepSeek-R1-Distill-Llama-8B, Retrieval Method=Contriever, Prompting Strategy=CoT2026.01 | 0.527 | 0.615 | |
| BM25 (CoT)Base Model=Llama-3.1-8B-Instruct, Retrieval Method=BM25, Prompting Strategy=CoT2026.01 | 0.538 | 0.548 | |
| Contriever (CoT)Base Model=Qwen2.5-7B-Instruct, Retrieval Method=Contriever, Prompting Strategy=CoT2026.01 | 0.57 | 0.625 | |
| BM25 (CoT)Base Model=DeepSeek-R1-Distill-Qwen-7B, Retrieval Method=BM25, Prompting Strategy=CoT2026.01 | 0.583 | 0.591 | |
| Contriever (CoT)Base Model=DeepSeek-R1-Distill-Qwen-7B, Retrieval Method=Contriever, Prompting Strategy=CoT2026.01 | 0.644 | 0.455 | |
| Contriever (CoT)Base Model=Llama-3.1-8B-Instruct, Retrieval Method=Contriever, Prompting Strategy=CoT2026.01 | 0.663 | 0.417 |