Question Answering on TriviaQA (EM)
86.1EMPaLM 2-L
Evaluation results
| Method | Links | |
|---|---|---|
| PaLM 2-Lprompting=1-shot2023.05 | 86.1 | |
| CoRectModel=LLaMA3-8b-instruct2026.02 | 83 | |
| PTRModel=Claude Sonnet 4.62026.04 | 82 | |
| PaLM 2-Mprompting=1-shot2023.05 | 81.7 | |
| PaLMprompting=1-shot2023.05 | 81.4 | |
| CADModel=LLaMA3-8b-instruct2026.02 | 81.2 | |
| COIECDModel=LLaMA3-8b-instruct2026.02 | 80.95 | |
| COIECDModel=Qwen2.5-7b-instruct2026.02 | 80.75 | |
| GreedyModel=LLaMA3-8b-instruct2026.02 | 80.3 | |
| AdaCADModel=LLaMA3-8b-instruct2026.02 | 80.3 | |
| CoRectModel=Qwen2.5-7b-instruct2026.02 | 79.6 | |
| CoRectModel=Mistral-7b2026.02 | 78.6 | |
| PTRModel=GPT-5.42026.04 | 78 | |
| AdaCADModel=Qwen2.5-7b-instruct2026.02 | 76.9 | |
| GreedyModel=Qwen2.5-7b-instruct2026.02 | 76.85 | |
| CADModel=Qwen2.5-7b-instruct2026.02 | 76.75 | |
| COIECDModel=Mistral-7b2026.02 | 76.35 | |
| PTRModel=Claude Haiku 4.52026.04 | 76 | |
| PaLM 2-Sprompting=1-shot2023.05 | 75.2 | |
| CADModel=Mistral-7b2026.02 | 72.5 | |
| GreedyModel=Mistral-7b2026.02 | 72.1 | |
| AdaCADModel=Mistral-7b2026.02 | 72 | |
| R_BayesBackbone=Gemma-3-4B2026.01 | 71.9 | |
| KLEBackbone=Gemma-3-4B2026.01 | 70.4 | |
| SEBackbone=Gemma-3-4B2026.01 | 70.2 | |
| ReActModel=GPT-5.42026.04 | 70 | |
| CoRectModel=LLaMA2-13b-chat2026.02 | 68.15 | |
| PTRModel=GPT-4o-Mini2026.04 | 66 | |
| R_BayesBackbone=Qwen3-4B2026.01 | 65.8 | |
| SLATEBackbone=Qwen2.5-7B-Base2026.02 | 65.2 | |
| DeepSeekMoE 16B# Shot=5-shot, # Total Params=16.4B, # Activated Params=2.8B, FLOPs per 4K Tokens=74.4T, # Training Tokens=2T2024.01 | 64.8 | |
| DeepSeekMoE 16B# Shot=5-shot, # Total Params=16.4B, # Activated Params=2.8B, FLOPs per 4K Tokens=74.4T, # Training Tokens=2T2024.01 | 64.8 | |
| Self-ConsistencyModel=Gemma-3-4B, Decoding Strategy=Self-Consistency2026.01 | 64.2 | |
| Self-ConsistencyModel=Qwen3-4B, Decoding Strategy=Self-Consistency2026.01 | 64 | |
| LLaMA2 7B# Shot=5-shot, # Total Params=6.7B, # Activated Params=6.7B, FLOPs per 4K Tokens=187.9T, # Training Tokens=2T2024.01 | 63.8 | |
| SEARCH-R1Backbone=Qwen2.5-7B-Base2026.02 | 63.8 | |
| CADModel=LLaMA2-13b-chat2026.02 | 63.55 | |
| DeepSeekMoE Chat 16B# Shot=5-shot, Total Params=16.4B, Activated Params=2.8B, FLOPs per 4K Tokens=74.4T2024.01 | 63.3 | |
| ℓSampleModel=Gemma-3-4B, Decoding Strategy=ℓSample2026.01 | 63.3 | |
| Bayes (ours)Model=Gemma-3-4B, Decoding Strategy=Bayes (ours)2026.01 | 63.3 | |
| BeamModel=Qwen3-4B, Decoding Strategy=Beam2026.01 | 63.2 | |
| ℓSampleModel=Qwen3-4B, Decoding Strategy=ℓSample2026.01 | 63.2 | |
| Bayes (ours)Model=Qwen3-4B, Decoding Strategy=Bayes (ours)2026.01 | 63.2 | |
| 13B (Multi-Task Tuning)Tuning=Multi-task2024.06 | 62.79 | |
| BeamModel=Gemma-3-4B, Decoding Strategy=Beam2026.01 | 62.7 | |
| LoRA TuningModel Size=13B, Transfer Source=N/A2024.06 | 61.89 | |
| DeepSeekMoE 145B# Shot=5-shot2024.01 | 61.1 | |
| COIECDModel=LLaMA2-13b-chat2026.02 | 60.35 | |
| SLATEBackbone=Qwen2.5-3B-Base, Training Data=NQ+HotpotQA2026.02 | 60.3 | |
| LLaMA2 SFT 7B# Shot=5-shot, Total Params=6.7B, Activated Params=6.7B, FLOPs per 4K Tokens=187.9T2024.01 | 60.1 | |
| ZeroSearchBackbone=Qwen2.5-7B-Base2026.02 | 60.1 | |
| DeepSeekMoE 142B (Half Activated)# Shot=5-shot2024.01 | 59.8 | |
| DeepSeek 7B (Dense)# Shot=5-shot, # Total Params=6.9B, # Activated Params=6.9B, FLOPs per 4K Tokens=183.5T, # Training Tokens=2T2024.01 | 59.7 | |
| DeepSeek Chat 7B# Shot=5-shot, Total Params=6.9B, Activated Params=6.9B, FLOPs per 4K Tokens=183.5T2024.01 | 59.5 | |
| COCOABackbone=Gemma-3-4B2026.01 | 59.4 | |
| ReSearchBackbone=Qwen2.5-7B-Base2026.02 | 59.4 | |
| KLEBackbone=Qwen3-4B2026.01 | 59.3 | |
| SEARCH-R1Backbone=Qwen2.5-3B-Base, Training Data=NQ+HotpotQA2026.02 | 58.7 | |
| SARBackbone=Gemma-3-12B2026.01 | 58.5 | |
| RAGBackbone=Qwen2.5-7B-Base2026.02 | 58.2 | |
| KLEBackbone=Gemma-3-12B2026.01 | 58 | |
| ReActModel=Claude Haiku 4.52026.04 | 58 | |
| SARBackbone=Gemma-3-4B2026.01 | 57.9 | |
| SEBackbone=Qwen3-4B2026.01 | 57.7 | |
| DeepSeek 67B (Dense)# Shot=5-shot2024.01 | 57.2 | |
| R_BayesBackbone=Gemma-3-12B2026.01 | 57.2 | |
| OursModel Size=13B, Transfer Source=7B2024.06 | 57.11 | |
| SARBackbone=Qwen3-4B2026.01 | 57.1 | |
| SFTBackbone=Qwen2.5-7B-Base2026.02 | 56.9 | |
| COCOABackbone=Gemma-3-12B2026.01 | 56.6 | |
| COCOABackbone=Qwen3-4B2026.01 | 56.4 | |
| Full Fine-tuningModel Size=13B, Transfer Source=N/A2024.06 | 56.36 | |
| SEBackbone=Gemma-3-12B2026.01 | 56.1 | |
| IRCoTBackbone=Qwen2.5-7B-Base2026.02 | 56.1 | |
| MSPBackbone=Qwen3-4B2026.01 | 55.7 | |
| GreedyModel=LLaMA2-13b-chat2026.02 | 55.6 | |
| Document FilterBackbone=Qwen-2.5-7B-Instruct2025.10 | 55.1 | |
| AdaCADModel=LLaMA2-13b-chat2026.02 | 55.05 | |
| COCOABackbone=Qwen3-30B-A3B2026.01 | 54.9 | |
| PPO Utility-Only Ft.Backbone=Qwen-2.5-7B-Instruct2025.10 | 54.9 | |
| PPO SafeSearchBackbone=Qwen-2.5-7B-Instruct2025.10 | 54.9 | |
| Search-o1Backbone=Qwen2.5-7B-Base2026.02 | 54.8 | |
| PPO SafeSearch w/o qr.Backbone=Qwen-2.5-7B-Instruct2025.10 | 54.7 | |
| GRPO Utility-Only Ft.Backbone=Qwen-2.5-7B-Instruct2025.10 | 54.7 | |
| Search BlockBackbone=Qwen-2.5-7B-Instruct2025.10 | 54.6 | |
| PPO SafeSearch w/o qr. & hf.Backbone=Qwen-2.5-7B-Instruct2025.10 | 54.6 | |
| PPO SafeSearch w/o hf.Backbone=Qwen-2.5-7B-Instruct2025.10 | 54.5 | |
| R1 (no search)Backbone=Qwen2.5-7B-Base, search=disabled2026.02 | 54.1 | |
| OursModel Size=13B, Transfer Source=1.1B2024.06 | 53.81 | |
| MSPBackbone=Gemma-3-12B2026.01 | 53.7 | |
| GRPO SafeSearchBackbone=Qwen-2.5-7B-Instruct2025.10 | 53.7 | |
| MSPBackbone=Gemma-3-4B2026.01 | 53.3 | |
| ZeroSearchBackbone=Qwen2.5-3B-Base2026.02 | 53.1 | |
| CoTBackbone=Qwen2.5-7B-Base2026.02 | 52.9 | |
| Proxy TuningModel Size=13B, Transfer Source=7B2024.06 | 52.81 | |
| KLEBackbone=Qwen3-30B-A3B2026.01 | 52.7 | |
| MSPBackbone=Qwen3-30B-A3B2026.01 | 52.7 | |
| R_BayesBackbone=Qwen3-30B-A3B2026.01 | 52.7 | |
| SARBackbone=Qwen3-30B-A3B2026.01 | 52.6 | |
| GShard 137B# Shot=5-shot2024.01 | 52.5 |