Question Answering on Natural Questions (Accuracy)
46.39AccuracyRL
Evaluation Results
| Method | Links | |
|---|---|---|
| RLBackbone=Llama-3.1-8B2026.05 | 46.39 | |
| InternLM2-20BModel size=13 ~ 20B, shot=5-shot2024.03 | 44.6 | |
| InternLM2-7BModel size=< 7B, shot=5-shot2024.03 | 42.2 | |
| Mixtral-8x7B-v0.1Model size=13 ~ 20B, shot=5-shot2024.03 | 39.6 | |
| ChatGLM3-6B-BaseModel size=< 7B, shot=5-shot2024.03 | 38.7 | |
| Qwen-14BModel size=13 ~ 20B, shot=5-shot2024.03 | 37.1 | |
| RLBackbone=OLMo-2-7B2026.05 | 36.91 | |
| RFTBackbone=Llama-3.1-8B2026.05 | 36.7 | |
| Llama2-13BModel size=13 ~ 20B, shot=5-shot2024.03 | 34.5 | |
| RLBackbone=Qwen2.5-7B2026.05 | 34.12 | |
| LLAMA2Size=13B, Tokens=2T, Context=4K2024.04 | 31.2 | |
| Mistral-7B-v0.1Model size=< 7B, shot=5-shot2024.03 | 31.1 | |
| DPOBackbone=Llama-3.1-8B2026.05 | 30.48 | |
| InternLM2-20B-BaseModel size=13 ~ 20B, shot=5-shot2024.03 | 30.3 | |
| BaseBackbone=Llama-3.1-8B2026.05 | 30.15 | |
| SFTBackbone=Llama-3.1-8B2026.05 | 29.45 | |
| Llama2-7BModel size=< 7B, shot=5-shot2024.03 | 28.8 | |
| InternLM2-7B-BaseModel size=< 7B, shot=5-shot2024.03 | 28.4 | |
| Baichuan2-13B-BaseModel size=13 ~ 20B, shot=5-shot2024.03 | 27.5 | |
| RFTBackbone=Qwen2.5-7B2026.05 | 27.03 | |
| LLAMA2Size=7B, Tokens=2T, Context=4K2024.04 | 25.7 | |
| MEGALODONSize=7B, Tokens=2T, Context=32K2024.04 | 25.7 | |
| SFTBackbone=OLMo-2-7B2026.05 | 25.39 | |
| RFTBackbone=OLMo-2-7B2026.05 | 25.33 | |
| Ours (SCD)decoding=SCD2025.02 | 24.3 | |
| Baichuan2-7B-BaseModel size=< 7B, shot=5-shot2024.03 | 24 | |
| SFTBackbone=Qwen2.5-7B2026.05 | 23.67 | |
| MistralSize=7B, Context=16K2024.04 | 23.2 | |
| DPOBackbone=OLMo-2-7B2026.05 | 23.06 | |
| GemmaSize=8B, Tokens=6T, Context=8K2024.04 | 23 | |
| DPOBackbone=Qwen2.5-7B2026.05 | 22.94 | |
| BaseBackbone=OLMo-2-7B2026.05 | 22.79 | |
| Qwen-7BModel size=< 7B, shot=5-shot2024.03 | 22.7 | |
| Greedydecoding=Greedy2025.02 | 22.1 | |
| BaseBackbone=Qwen2.5-7B2026.05 | 21.42 | |
| MPTSize=7B, Tokens=1T, Context=4K2024.04 | 20.8 |