Question Answering on PopQA (Accuracy)
87.12AccuracyNWCAD
Evaluation results
| Method | Links | |
|---|---|---|
| NWCADModel=Llama-3.1-70B2026.04 | 87.12 | |
| NWCADModel=Ministral-3-8B2026.04 | 85.69 | |
| AdaCADModel=Llama-3.1-70B2026.04 | 85.62 | |
| With-contextModel=Ministral-3-8B2026.04 | 85.12 | |
| AdaCADModel=Ministral-3-8B2026.04 | 84.94 | |
| NWCADModel=Llama-3.1-8B2026.04 | 84.12 | |
| CoCoAModel=Llama-3.1-70B2026.04 | 84.12 | |
| With-contextModel=Llama-3.1-8B2026.04 | 83.81 | |
| AdaCADModel=Llama-3.1-8B2026.04 | 82.69 | |
| With-contextModel=Llama-3.1-70B2026.04 | 82.38 | |
| CoCoAModel=Llama-3.1-8B2026.04 | 80.94 | |
| KARLAModel=Qwen 4B, Setup=KARLA2026.06 | 80.91 | |
| KARLAModel=Qwen 8B, Setup=KARLA2026.06 | 80.63 | |
| CADModel=Ministral-3-8B2026.04 | 80.31 | |
| KARLAModel=Qwen 1.7B, Setup=KARLA2026.06 | 78.98 | |
| KARLAModel=Qwen 0.6B, Setup=KARLA2026.06 | 78.56 | |
| CADModel=Llama-3.1-8B2026.04 | 78.44 | |
| NWCADModel=Llama-3.1-70B2026.04 | 77.88 | |
| NWCADModel=Ministral-3-8B2026.04 | 75.29 | |
| AdaCADModel=Llama-3.1-70B2026.04 | 74.62 | |
| Neuro-RITRanking Strategy=Reranker2026.04 | 74.29 | |
| CoCoAModel=Ministral-3-8B2026.04 | 74 | |
| AdaCADModel=Ministral-3-8B2026.04 | 73.4 | |
| NWCADModel=Llama-3.1-8B2026.04 | 72.8 | |
| CoCoAModel=Ministral-3-8B2026.04 | 72.8 | |
| CADModel=Llama-3.1-70B2026.04 | 72.5 | |
| CoCoAModel=Llama-3.1-70B2026.04 | 72.38 | |
| AdaCADModel=Llama-3.1-8B2026.04 | 71.8 | |
| PA-RAGRanking Strategy=Reranker2026.04 | 71.08 | |
| Neuro-RIT + RerankerGenerator=Mistral-7B-Instruct-v0.22026.04 | 70.65 | |
| No compressionBackbone=Mistral-7B2025.03 | 70 | |
| No compressionBackbone=Mistral-24B2025.03 | 70 | |
| OSCAR-llamaBackbone=Mistral-24B2025.03 | 70 | |
| RALM + RerankerGenerator=Mistral-7B-Instruct-v0.22026.04 | 69.05 | |
| ProvenceBackbone=Mistral-7B2025.03 | 69 | |
| With-contextModel=Llama-3.1-70B2026.04 | 68.88 | |
| CoCoAModel=Llama-3.1-8B2026.04 | 68.8 | |
| OSCAR-llamaBackbone=Mistral-7B2025.03 | 68 | |
| OSCAR-8-LayersBackbone=Mistral-7B2025.03 | 68 | |
| OSCAR-llamaBackbone=Qwen-7B2025.03 | 68 | |
| Neuro-RITRanking Strategy=None2026.04 | 67.37 | |
| RECOMPBackbone=Mistral-7B2025.03 | 67 | |
| OSCAR-8-LayersBackbone=Qwen-7B2025.03 | 67 | |
| KARLA-no-descModel=Qwen 4B, Setup=KARLA-no-desc2026.06 | 66.98 | |
| KARLA-no-descModel=Qwen 8B, Setup=KARLA-no-desc2026.06 | 66.38 | |
| OSCAR-5-LayersBackbone=Mistral-7B2025.03 | 66 | |
| KARLA-no-descModel=Qwen 1.7B, Setup=KARLA-no-desc2026.06 | 65.76 | |
| PA-RAGRanking Strategy=RankCoT2026.04 | 65.68 | |
| CADModel=Ministral-3-8B2026.04 | 65.6 | |
| With-contextModel=Ministral-3-8B2026.04 | 65.2 | |
| InstructRAGRanking Strategy=Reranker2026.04 | 64.6 | |
| PA-RAGRanking Strategy=None2026.04 | 64.37 | |
| Neuro-RITGenerator=Mistral-7B-Instruct-v0.22026.04 | 64.15 | |
| PISCOBackbone=Mistral-7B2025.03 | 64 | |
| No compressionBackbone=Qwen-7B2025.03 | 64 | |
| CADModel=Llama-3.1-8B2026.04 | 63.6 | |
| Neuro-RITRanking Strategy=RankCoT2026.04 | 63.28 | |
| Neuro-RIT + RankCoTGenerator=Mistral-7B-Instruct-v0.22026.04 | 63.19 | |
| RALMGenerator=Mistral-7B-Instruct-v0.22026.04 | 62.78 | |
| With-contextModel=Llama-3.1-8B2026.04 | 62.4 | |
| InstructRAGRanking Strategy=RankCoT2026.04 | 62.24 | |
| RALMRanking Strategy=None2026.04 | 60.85 | |
| RALM + RankCoTGenerator=Mistral-7B-Instruct-v0.22026.04 | 60.83 | |
| RALMRanking Strategy=Reranker2026.04 | 60.37 | |
| InstructRAGRanking Strategy=None2026.04 | 60.25 | |
| RALMRanking Strategy=RankCoT2026.04 | 59.8 | |
| No compressionBackbone=Llama-1B2025.03 | 59 | |
| OSCAR-5-LayersBackbone=Llama-1B2025.03 | 59 | |
| CADModel=Llama-3.1-70B2026.04 | 59 | |
| 1-hop graph RAGModel=Qwen 8B, Setup=1-hop graph RAG2026.06 | 58.68 | |
| 1-hop graph RAGModel=Qwen 4B, Setup=1-hop graph RAG2026.06 | 56.17 | |
| KARLA-no-descModel=Qwen 0.6B, Setup=KARLA-no-desc2026.06 | 55.97 | |
| 1-hop graph RAGModel=Qwen 1.7B, Setup=1-hop graph RAG2026.06 | 55.02 | |
| RetRobustRanking Strategy=Reranker2026.04 | 54.92 | |
| 1-hop graph RAGModel=Qwen 0.6B, Setup=1-hop graph RAG2026.06 | 54.45 | |
| RetRobustRanking Strategy=None2026.04 | 53.24 | |
| RetRobustRanking Strategy=RankCoT2026.04 | 53.18 | |
| LMLMModel=LLAMA2-382M, Setup=LMLM2026.06 | 52 | |
| LMLMModel=GPT2-774M, Setup=LMLM2026.06 | 50.8 | |
| State-SFT & RL‡Backbone=Youtu-LLM-2B-Base, Framework=WM-AMT, Optimization Stage=RL stage2026.06 | 42.8 | |
| FE-SFT & FC-RLBackbone=Youtu-LLM-2B-Base, Framework=WM-AMT, Optimization Stage=RL stage2026.06 | 42.6 | |
| Tool-schema promptModel=Qwen 4B, Setup=Tool-schema prompt2026.06 | 41.74 | |
| RAGBackbone=Qwen2.5-14B-Instruct2026.02 | 41.3 | |
| State-SFT & RL†Backbone=Youtu-LLM-2B-Base, Framework=Standard, Optimization Stage=RL stage2026.06 | 40.8 | |
| FE-SFT & RLBackbone=Youtu-LLM-2B-Base, Framework=WM-AMT, Optimization Stage=RL stage2026.06 | 40.8 | |
| RAGBackbone=Qwen2.5-3B-Instruct2026.02 | 39.57 | |
| IWM & RLBackbone=Youtu-LLM-2B-Base, Framework=Standard, Optimization Stage=RL stage2026.06 | 38.8 | |
| FE-SFT & RL*Backbone=Youtu-LLM-2B-Base, Framework=Standard, Optimization Stage=RL stage2026.06 | 38.2 | |
| Tool-schema promptModel=Qwen 8B, Setup=Tool-schema prompt2026.06 | 35.67 | |
| SFT & RLBackbone=Youtu-LLM-2B-Base, Framework=WM-AMT, Optimization Stage=RL stage2026.06 | 35.6 | |
| CDKCBackbone=Qwen2.5-14B-Instruct2026.02 | 35.2 | |
| SFT & RL*Backbone=Youtu-LLM-2B-Base, Framework=Standard, Optimization Stage=RL stage2026.06 | 35.2 | |
| GRPOBackbone=Qwen2.5-14B-Instruct2026.02 | 34.23 | |
| FE-SFTBackbone=Youtu-LLM-2B-Base, Framework=WM-AMT, Optimization Stage=SFT stage2026.06 | 33.8 | |
| SFTBackbone=Youtu-LLM-2B-Base, Framework=WM-AMT, Optimization Stage=SFT stage2026.06 | 32.6 | |
| IWMBackbone=Youtu-LLM-2B-Base, Framework=Standard, Optimization Stage=SFT stage2026.06 | 31.8 | |
| RLBackbone=Llama-3.1-8B2026.05 | 31.44 | |
| SFT*Backbone=Youtu-LLM-2B-Base, Framework=Standard, Optimization Stage=SFT stage2026.06 | 31 | |
| FE-SFT*Backbone=Youtu-LLM-2B-Base, Framework=Standard, Optimization Stage=SFT stage2026.06 | 30.6 | |
| SFTBackbone=Llama-3.1-8B2026.05 | 29.52 |