Multi-hop Factual Reasoning on FRAMES
82.3AccuracyROMA
Evaluation results
| Method | Links | |
|---|---|---|
| ROMASearch=true, Backbone=GLM-4.6, Source_Type=Open-source2026.02 | 82.3 | |
| Kimi-ResearcherSearch=true, Source_Type=Open-source2026.02 | 78.8 | |
| Open Deep Search (DeepSeek-R1)Search=true, Source_Type=Open-source2026.02 | 75.3 | |
| GLM-4.6Search=true, Source_Type=Open-source2026.02 | 71.2 | |
| GPT-4o Search PreviewSearch=true, Source_Type=Closed-source2026.02 | 65.6 | |
| GPT-4oSearch=false, Source_Type=Closed-source2026.02 | 50.5 | |
| Perplexity Sonar Reasoning ProSearch=true, Source_Type=Closed-source2026.02 | 44.4 | |
| PerplexitySearch=true, Source_Type=Closed-source2026.02 | 42.4 | |
| Llama-3.1-70BSearch=false, Source_Type=Open-source2026.02 | 34.3 | |
| DeepSeek-R1Search=false, Source_Type=Open-source2026.02 | 30.1 |