Question Answering on HotpotQA (EM, F1, Precision, Recall)
89.5RecallAGDLRPc
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| AGDLRPcModel=Qwen 2.5 (7B), Setting=Open-book2025.09 | 89.5 | — | — | — | |
| AGDLRPhModel=Llama 3.1 (8B), Setting=Open-book2025.09 | 87.9 | — | — | — | |
| AGDLRPcModel=Llama 3.1 (8B), Setting=Open-book2025.09 | 87.9 | — | — | — | |
| AGDLRPhModel=Qwen 2.5 (7B), Setting=Open-book2025.09 | 86.4 | — | — | — | |
| AGDI×GhModel=Qwen 2.5 (7B), Setting=Open-book2025.09 | 85.5 | — | — | — | |
| AGDI×GcModel=Qwen 2.5 (7B), Setting=Open-book2025.09 | 85.1 | — | — | — | |
| NucleusModel=Qwen 2.5 (7B), Setting=Open-book2025.09 | 85 | — | — | — | |
| CADModel=Qwen 2.5 (7B), Setting=Open-book2025.09 | 85 | — | — | — | |
| GreedyModel=Qwen 2.5 (7B), Setting=Open-book2025.09 | 84.5 | — | — | — | |
| AGDI×GcModel=Llama 3.1 (8B), Setting=Open-book2025.09 | 83.9 | — | — | — | |
| CADModel=Llama 3.1 (8B), Setting=Open-book2025.09 | 83.4 | — | — | — | |
| AGDI×GhModel=Llama 3.1 (8B), Setting=Open-book2025.09 | 81.6 | — | — | — | |
| GreedyModel=Llama 3.1 (8B), Setting=Open-book2025.09 | 81.3 | — | — | — | |
| NucleusModel=Llama 3.1 (8B), Setting=Open-book2025.09 | 80.7 | — | — | — | |
| AGDLRPcModel=Gemma 3 (4B), Setting=Open-book2025.09 | 70.9 | — | — | — | |
| AGDLRPhModel=Gemma 3 (4B), Setting=Open-book2025.09 | 70.6 | — | — | — | |
| AGDI×GhModel=Gemma 3 (4B), Setting=Open-book2025.09 | 70.4 | — | — | — | |
| GreedyModel=Gemma 3 (4B), Setting=Open-book2025.09 | 70.1 | — | — | — | |
| AGDI×GcModel=Gemma 3 (4B), Setting=Open-book2025.09 | 70.1 | — | — | — | |
| NucleusModel=Gemma 3 (4B), Setting=Open-book2025.09 | 69.8 | — | — | — | |
| CADModel=Gemma 3 (4B), Setting=Open-book2025.09 | 64.1 | — | — | — | |
| AGDLRPhModel=Llama 3.1 (8B), Setting=Closed-book2025.09 | 39.6 | — | — | — | |
| DioRRetrieval Strategy=BM25, Backbone=LLaMA2-7B-CHAT2025.04 | 38.1 | 27.4 | 37.9 | 39.9 | |
| AGDI×GhModel=Llama 3.1 (8B), Setting=Closed-book2025.09 | 37.5 | — | — | — | |
| GreedyModel=Llama 3.1 (8B), Setting=Closed-book2025.09 | 34.6 | — | — | — | |
| DoLAModel=Llama 3.1 (8B), Setting=Closed-book2025.09 | 34.3 | — | — | — | |
| AGDLRPhModel=Qwen 2.5 (7B), Setting=Closed-book2025.09 | 34.3 | — | — | — | |
| GreedyModel=Qwen 2.5 (7B), Setting=Closed-book2025.09 | 33.8 | — | — | — | |
| AGDI×GhModel=Qwen 2.5 (7B), Setting=Closed-book2025.09 | 33.6 | — | — | — | |
| NucleusModel=Qwen 2.5 (7B), Setting=Closed-book2025.09 | 32.5 | — | — | — | |
| DoLAModel=Qwen 2.5 (7B), Setting=Closed-book2025.09 | 32.4 | — | — | — | |
| NucleusModel=Llama 3.1 (8B), Setting=Closed-book2025.09 | 31.9 | — | — | — | |
| DraginRetrieval Strategy=BM25, Backbone=LLaMA2-7B-CHAT2025.04 | 31.6 | 21.9 | 31.4 | 33.1 | |
| DioRRetrieval Strategy=SGPT, Backbone=LLaMA2-7B-CHAT2025.04 | 31.2 | 21.2 | 30.7 | 33.3 | |
| DraginRetrieval Strategy=SGPT, Backbone=LLaMA2-7B-CHAT2025.04 | 30.5 | 20.2 | 30.1 | 31.9 | |
| DioRRetrieval Strategy=SBERT, Backbone=LLaMA2-7B-CHAT2025.04 | 30.3 | 20.5 | 30.3 | 32.4 | |
| AGDI×GhModel=Gemma 3 (4B), Setting=Closed-book2025.09 | 28 | — | — | — | |
| AGDLRPhModel=Gemma 3 (4B), Setting=Closed-book2025.09 | 28 | — | — | — | |
| GreedyModel=Gemma 3 (4B), Setting=Closed-book2025.09 | 27.7 | — | — | — | |
| NucleusModel=Gemma 3 (4B), Setting=Closed-book2025.09 | 27.7 | — | — | — | |
| DoLAModel=Gemma 3 (4B), Setting=Closed-book2025.09 | 27.3 | — | — | — | |
| DraginRetrieval Strategy=SBERT, Backbone=LLaMA2-7B-CHAT2025.04 | 25.5 | 16.5 | 24.4 | 25.6 |