Question Answering on StrategyQA (ECE & AUROC)
0.217ECEBM25 (CoT)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| BM25 (CoT)Base Model=Llama-3.1-8B-Instruct, Retrieval Method=BM25, Prompting Strategy=CoT2026.01 | 0.217 | 0.48 | |
| Contriever (CoT)Base Model=Llama-3.1-8B-Instruct, Retrieval Method=Contriever, Prompting Strategy=CoT2026.01 | 0.228 | 0.455 | |
| Contriever (CoT)Base Model=Qwen2.5-7B-Instruct, Retrieval Method=Contriever, Prompting Strategy=CoT2026.01 | 0.231 | 0.564 | |
| Contriever (CoT)Base Model=DeepSeek-R1-Distill-Llama-8B, Retrieval Method=Contriever, Prompting Strategy=CoT2026.01 | 0.235 | 0.572 | |
| BM25 (CoT)Base Model=Qwen2.5-7B-Instruct, Retrieval Method=BM25, Prompting Strategy=CoT2026.01 | 0.243 | 0.562 | |
| BM25 (CoT)Base Model=DeepSeek-R1-Distill-Llama-8B, Retrieval Method=BM25, Prompting Strategy=CoT2026.01 | 0.246 | 0.574 | |
| BM25 (CoT)Base Model=DeepSeek-R1-Distill-Qwen-7B, Retrieval Method=BM25, Prompting Strategy=CoT2026.01 | 0.292 | 0.539 | |
| Contriever (CoT)Base Model=DeepSeek-R1-Distill-Qwen-7B, Retrieval Method=Contriever, Prompting Strategy=CoT2026.01 | 0.292 | 0.539 |