Question Answering on NQ (recall)
90.6NQ Recall (%)AGDLRPc
Evaluation Results
| Method | Links | |
|---|---|---|
| AGDLRPcModel=Qwen 2.5 (7B), Setting=Open-book2025.09 | 90.6 | |
| CADModel=Qwen 2.5 (7B), Setting=Open-book2025.09 | 90 | |
| AGDI×GhModel=Qwen 2.5 (7B), Setting=Open-book2025.09 | 90 | |
| AGDLRPhModel=Qwen 2.5 (7B), Setting=Open-book2025.09 | 89.7 | |
| GreedyModel=Qwen 2.5 (7B), Setting=Open-book2025.09 | 89 | |
| NucleusModel=Qwen 2.5 (7B), Setting=Open-book2025.09 | 88.7 | |
| AGDI×GcModel=Qwen 2.5 (7B), Setting=Open-book2025.09 | 88.6 | |
| AGDLRPcModel=Llama 3.1 (8B), Setting=Open-book2025.09 | 87.9 | |
| AGDLRPhModel=Llama 3.1 (8B), Setting=Open-book2025.09 | 87 | |
| AGDI×GhModel=Llama 3.1 (8B), Setting=Open-book2025.09 | 85.7 | |
| CADModel=Llama 3.1 (8B), Setting=Open-book2025.09 | 84.6 | |
| GreedyModel=Llama 3.1 (8B), Setting=Open-book2025.09 | 83.5 | |
| AGDI×GcModel=Llama 3.1 (8B), Setting=Open-book2025.09 | 83.5 | |
| NucleusModel=Llama 3.1 (8B), Setting=Open-book2025.09 | 83.3 | |
| AGDLRPhModel=Gemma 3 (4B), Setting=Open-book2025.09 | 83 | |
| AGDLRPcModel=Gemma 3 (4B), Setting=Open-book2025.09 | 83 | |
| AGDI×GhModel=Gemma 3 (4B), Setting=Open-book2025.09 | 82.5 | |
| AGDI×GcModel=Gemma 3 (4B), Setting=Open-book2025.09 | 82.3 | |
| GreedyModel=Gemma 3 (4B), Setting=Open-book2025.09 | 82 | |
| NucleusModel=Gemma 3 (4B), Setting=Open-book2025.09 | 82 | |
| CADModel=Gemma 3 (4B), Setting=Open-book2025.09 | 75.7 | |
| DoLAModel=Llama 3.1 (8B), Setting=Closed-book2025.09 | 63.8 | |
| GreedyModel=Llama 3.1 (8B), Setting=Closed-book2025.09 | 63.6 | |
| AGDI×GhModel=Llama 3.1 (8B), Setting=Closed-book2025.09 | 63 | |
| AGDLRPhModel=Llama 3.1 (8B), Setting=Closed-book2025.09 | 63 | |
| NucleusModel=Llama 3.1 (8B), Setting=Closed-book2025.09 | 59.9 | |
| GreedyModel=Qwen 2.5 (7B), Setting=Closed-book2025.09 | 47.8 | |
| AGDI×GhModel=Qwen 2.5 (7B), Setting=Closed-book2025.09 | 47.1 | |
| AGDLRPhModel=Qwen 2.5 (7B), Setting=Closed-book2025.09 | 46.9 | |
| NucleusModel=Qwen 2.5 (7B), Setting=Closed-book2025.09 | 45.4 | |
| DoLAModel=Qwen 2.5 (7B), Setting=Closed-book2025.09 | 44.3 | |
| AGDI×GhModel=Gemma 3 (4B), Setting=Closed-book2025.09 | 42.1 | |
| AGDLRPhModel=Gemma 3 (4B), Setting=Closed-book2025.09 | 41.9 | |
| GreedyModel=Gemma 3 (4B), Setting=Closed-book2025.09 | 41.6 | |
| NucleusModel=Gemma 3 (4B), Setting=Closed-book2025.09 | 41.2 | |
| DoLAModel=Gemma 3 (4B), Setting=Closed-book2025.09 | 40.8 |