Question Answering on Bamboogle (ECE, AUROC)
0.521ECEBM25 (CoT)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| BM25 (CoT)Base Model=DeepSeek-R1-Distill-Llama-8B, Retrieval Method=BM25, Prompting Strategy=CoT2026.01 | 0.521 | 0.754 | |
| Contriever (CoT)Base Model=DeepSeek-R1-Distill-Llama-8B, Retrieval Method=Contriever, Prompting Strategy=CoT2026.01 | 0.557 | 0.714 | |
| BM25 (CoT)Base Model=Llama-3.1-8B-Instruct, Retrieval Method=BM25, Prompting Strategy=CoT2026.01 | 0.613 | 0.452 | |
| BM25 (CoT)Base Model=Qwen2.5-7B-Instruct, Retrieval Method=BM25, Prompting Strategy=CoT2026.01 | 0.628 | 0.599 | |
| Contriever (CoT)Base Model=Llama-3.1-8B-Instruct, Retrieval Method=Contriever, Prompting Strategy=CoT2026.01 | 0.629 | 0.445 | |
| Contriever (CoT)Base Model=Qwen2.5-7B-Instruct, Retrieval Method=Contriever, Prompting Strategy=CoT2026.01 | 0.632 | 0.696 | |
| BM25 (CoT)Base Model=DeepSeek-R1-Distill-Qwen-7B, Retrieval Method=BM25, Prompting Strategy=CoT2026.01 | 0.756 | 0.584 | |
| Contriever (CoT)Base Model=DeepSeek-R1-Distill-Qwen-7B, Retrieval Method=Contriever, Prompting Strategy=CoT2026.01 | 0.773 | 0.632 |