Multi-hop Question Answering on FRAMES (Accuracy)
86AccuracyCoverage Anchor
Evaluation results
| Method | Links | |
|---|---|---|
| Coverage Anchorgeneration_type=reasoning, generator=GPT-o32026.03 | 86 | |
| PAR2-RAGgeneration_type=reasoning, generator=GPT-o32026.03 | 85.8 | |
| Qwen3-30B-A3B-Thinking-2507 + Data Recipe for Long-Context RLBase Model=Qwen3-30B-A3B-Thinking-2507, Training Recipe=Data Recipe for Long-Context RL2026.06 | 85.8 | |
| Qwen3-30B-A3B-Thinking-2507 + KeyChain-15KBase Model=Qwen3-30B-A3B-Thinking-2507, Training Recipe=KeyChain-15K2026.06 | 85.07 | |
| Iterative Chaingeneration_type=reasoning, generator=GPT-o32026.03 | 83.4 | |
| ReActgeneration_type=reasoning, generator=GPT-o32026.03 | 83.3 | |
| Qwen3-4B-Thinking-2507 + Data Recipe for Long-Context RLBase Model=Qwen3-4B-Thinking-2507, Training Recipe=Data Recipe for Long-Context RL2026.06 | 82.52 | |
| Qwen3-4B-Thinking-2507 + KeyChain-15KBase Model=Qwen3-4B-Thinking-2507, Training Recipe=KeyChain-15K2026.06 | 82.4 | |
| Qwen3-30B-A3B-Thinking-2507 + DocQA-RL-1.6KBase Model=Qwen3-30B-A3B-Thinking-2507, Training Recipe=DocQA-RL-1.6K2026.06 | 81.55 | |
| PAR2-RAGGeneration Setting=Non-reasoning2026.03 | 81.1 | |
| Qwen3-8B-128K + KeyChain-15KBase Model=Qwen3-8B-128K, Training Recipe=KeyChain-15K2026.06 | 80.22 | |
| Qwen3-30B-A3B-Thinking-2507Base Model=Qwen3-30B-A3B-Thinking-2507, Training Recipe=Base2026.06 | 79.37 | |
| Iterative ChainGeneration Setting=Non-reasoning2026.03 | 78.8 | |
| Qwen3-4B-Thinking-2507 + DocQA-RL-1.6KBase Model=Qwen3-4B-Thinking-2507, Training Recipe=DocQA-RL-1.6K2026.06 | 78.28 | |
| IdleSpecBackbone=Gemini-2.5-Flash, Use idle?=true2026.05 | 78 | |
| Qwen3-8B-128K + Data Recipe for Long-Context RLBase Model=Qwen3-8B-128K, Training Recipe=Data Recipe for Long-Context RL2026.06 | 77.67 | |
| Qwen3-4B-Thinking-2507Base Model=Qwen3-4B-Thinking-2507, Training Recipe=Base2026.06 | 77.55 | |
| Coverage AnchorGeneration Setting=Non-reasoning2026.03 | 77.4 | |
| Sequential RevisionBackbone=Gemini-2.5-Flash, Use idle?=false2026.05 | 76.7 | |
| IRCoTgeneration_type=reasoning, generator=GPT-o32026.03 | 75.4 | |
| Qwen3-8B-128KBase Model=Qwen3-8B-128K, Training Recipe=Base2026.06 | 74.76 | |
| ReActGeneration Setting=Non-reasoning2026.03 | 74.5 | |
| CoTgeneration_type=reasoning, generator=GPT-o32026.03 | 74.5 | |
| VanillaBackbone=Gemini-2.5-Flash, Use idle?=false2026.05 | 74 | |
| Direct Inferencegeneration_type=reasoning, generator=GPT-o32026.03 | 73.9 | |
| Sleep-time ComputeBackbone=Gemini-2.5-Flash, Use idle?=true2026.05 | 72.7 | |
| IRCoTGeneration Setting=Non-reasoning2026.03 | 72.6 | |
| Qwen3-8B-128K + DocQA-RL-1.6KBase Model=Qwen3-8B-128K, Training Recipe=DocQA-RL-1.6K2026.06 | 71.36 | |
| IdleSpecBackbone=Qwen3.5-4B, Use idle?=true2026.05 | 65.3 | |
| CoTGeneration Setting=Non-reasoning2026.03 | 63.1 | |
| VanillaBackbone=Qwen3.5-4B, Use idle?=false2026.05 | 57.3 | |
| IdleSpecBackbone=Gemma4-E4B, Use idle?=true2026.05 | 56 | |
| Sequential RevisionBackbone=Qwen3.5-4B, Use idle?=false2026.05 | 56 | |
| Sequential RevisionBackbone=Gemma4-E4B, Use idle?=false2026.05 | 53.3 | |
| Sleep-time ComputeBackbone=Qwen3.5-4B, Use idle?=true2026.05 | 50.7 | |
| gemini-2.5-flashTraining Data=-, Backbone Model=gemini-2.5-flash2026.01 | 50 | |
| VanillaBackbone=Gemma4-E4B, Use idle?=false2026.05 | 50 | |
| Sleep-time ComputeBackbone=Gemma4-E4B, Use idle?=true2026.05 | 50 | |
| gemini-2.0-flashTraining Data=-, Backbone Model=gemini-2.0-flash2026.01 | 45.9 | |
| SAGETraining Data=SAGE, Backbone Model=QWEN-7B2026.01 | 32.3 | |
| Direct InferenceGeneration Setting=Non-reasoning2026.03 | 29.4 | |
| Search-R1Training Data=NQ + HotpotQA, Backbone Model=QWEN-7B2026.01 | 26.2 | |
| Musique-trained AgentTraining Data=Musique, Backbone Model=QWEN-7B2026.01 | 25 | |
| SAGETraining Data=SAGE, Backbone Model=QWEN-3B2026.01 | 23.8 | |
| Musique-trained AgentTraining Data=Musique, Backbone Model=QWEN-3B2026.01 | 21.5 | |
| Search-R1Training Data=NQ + HotpotQA, Backbone Model=QWEN-3B2026.01 | 13.3 |