Question Answering on NQ (ECE, AUROC)
0.416ECEBM25 (CoT)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| BM25 (CoT)Base Model=Llama-3.1-8B-Instruct, Retrieval Method=BM25, Prompting Strategy=CoT2026.01 | 0.416 | 0.648 | |
| BM25 (CoT)Base Model=Qwen2.5-7B-Instruct, Retrieval Method=BM25, Prompting Strategy=CoT2026.01 | 0.439 | 0.742 | |
| Contriever (CoT)Base Model=Llama-3.1-8B-Instruct, Retrieval Method=Contriever, Prompting Strategy=CoT2026.01 | 0.446 | 0.558 | |
| BM25 (CoT)Base Model=DeepSeek-R1-Distill-Llama-8B, Retrieval Method=BM25, Prompting Strategy=CoT2026.01 | 0.461 | 0.707 | |
| Contriever (CoT)Base Model=DeepSeek-R1-Distill-Llama-8B, Retrieval Method=Contriever, Prompting Strategy=CoT2026.01 | 0.477 | 0.754 | |
| Contriever (CoT)Base Model=Qwen2.5-7B-Instruct, Retrieval Method=Contriever, Prompting Strategy=CoT2026.01 | 0.553 | 0.699 | |
| BM25 (CoT)Base Model=DeepSeek-R1-Distill-Qwen-7B, Retrieval Method=BM25, Prompting Strategy=CoT2026.01 | 0.56 | 0.686 | |
| Contriever (CoT)Base Model=DeepSeek-R1-Distill-Qwen-7B, Retrieval Method=Contriever, Prompting Strategy=CoT2026.01 | 0.6 | 0.723 |