Multi-hop Reasoning on MuSiQue
53EMGPT-4o-0806
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GPT-4o-0806Version=08062025.02 | 53 | — | |
| DeepSeek-V3Version=V32025.02 | 52.5 | — | |
| Qwen2.5-OpAmp-72BParameters=72B, Adaptation=OpAmp2025.02 | 48 | — | |
| Llama3.3-70B-instParameters=70B, Type=Instruction-tuned2025.02 | 47.5 | — | |
| Qwen2.5-72B-instParameters=72B, Type=Instruction-tuned2025.02 | 44 | — | |
| GHS-TDABackbone=DeepSeek-V32026.02 | 40.1 | — | |
| GHS-TDABackbone=GPT-4o-mini2026.02 | 39.8 | — | |
| GHS-TDABackbone=Qwen-Turbo2026.02 | 39.6 | — | |
| AoTBackbone=DeepSeek-V3, Reasoning Paradigm=atomic reasoning2026.02 | 39.6 | — | |
| ToTBackbone=DeepSeek-V3, Reasoning Paradigm=tree-based2026.02 | 39.4 | — | |
| AoTBackbone=Qwen-Turbo, Reasoning Paradigm=atomic reasoning2026.02 | 39.2 | — | |
| ToTBackbone=GPT-4o-mini, Reasoning Paradigm=tree-based2026.02 | 39.1 | — | |
| Llama3-ChatQA2-70BParameters=70B, Version=ChatQA22025.02 | 39 | — | |
| AoTBackbone=GPT-4o-mini, Reasoning Paradigm=atomic reasoning2026.02 | 38.4 | — | |
| ToTBackbone=Qwen-Turbo, Reasoning Paradigm=tree-based2026.02 | 38.4 | — | |
| AFlowBackbone=DeepSeek-V3, Reasoning Paradigm=search-based2026.02 | 38.2 | — | |
| AFlowBackbone=GPT-4o-mini, Reasoning Paradigm=search-based2026.02 | 38.1 | — | |
| AFlowBackbone=Qwen-Turbo, Reasoning Paradigm=search-based2026.02 | 37.8 | — | |
| GoTBackbone=DeepSeek-V3, Reasoning Paradigm=graph-based2026.02 | 36.7 | — | |
| GoTBackbone=GPT-4o-mini, Reasoning Paradigm=graph-based2026.02 | 36.5 | — | |
| GoTBackbone=Qwen-Turbo, Reasoning Paradigm=graph-based2026.02 | 36.4 | — | |
| FoTBackbone=DeepSeek-V3, Reasoning Paradigm=forest-based, n=82026.02 | 36.2 | — | |
| FoTBackbone=GPT-4o-mini, Reasoning Paradigm=forest-based, n=82026.02 | 35.8 | — | |
| FoTBackbone=Qwen-Turbo, Reasoning Paradigm=forest-based, n=82026.02 | 35.8 | — | |
| Self-RefineBackbone=GPT-4o-mini, Reasoning Paradigm=chain-based2026.02 | 35.1 | — | |
| Self-RefineBackbone=DeepSeek-V3, Reasoning Paradigm=chain-based2026.02 | 35.1 | — | |
| Llama3.1-OpAmp-8BParameters=8B2025.02 | 35 | — | |
| Self-RefineBackbone=Qwen-Turbo, Reasoning Paradigm=chain-based2026.02 | 35 | — | |
| CoTBackbone=DeepSeek-V3, Reasoning Paradigm=chain-based2026.02 | 34.2 | — | |
| CoTBackbone=GPT-4o-mini, Reasoning Paradigm=chain-based2026.02 | 34.1 | — | |
| CoT-SCBackbone=DeepSeek-V3, Reasoning Paradigm=chain-based, n=52026.02 | 34 | — | |
| CoT-SCBackbone=GPT-4o-mini, Reasoning Paradigm=chain-based, n=52026.02 | 33.8 | — | |
| CoTBackbone=Qwen-Turbo, Reasoning Paradigm=chain-based2026.02 | 33.6 | — | |
| CoT-SCBackbone=Qwen-Turbo, Reasoning Paradigm=chain-based, n=52026.02 | 33.5 | — | |
| Analogical PromptingBackbone=DeepSeek-V3, Reasoning Paradigm=chain-based2026.02 | 33.1 | — | |
| Analogical PromptingBackbone=GPT-4o-mini, Reasoning Paradigm=chain-based2026.02 | 32.8 | — | |
| Analogical PromptingBackbone=Qwen-Turbo, Reasoning Paradigm=chain-based2026.02 | 32.6 | — | |
| Qwen2.5-7B-instParameters=7B2025.02 | 31.5 | — | |
| Llama3.1-8B-instParameters=8B2025.02 | 29.5 | — | |
| Mistral-7B-inst-v0.3Parameters=7B2025.02 | 28.5 | — | |
| Llama3-ChatQA2-8BParameters=8B2025.02 | 23 | — | |
| AoTBase Model=GPT-4o2026.02 | — | 42.1 | |
| AoTBase Model=Claude 3.5 Sonnet2026.02 | — | 43.2 | |
| CoTBase Model=GPT-4o2026.02 | — | 36.2 | |
| CoTBase Model=Claude 3.5 Sonnet2026.02 | — | 37.1 | |
| GHG-TDABase Model=GPT-4o2026.02 | — | 43.6 | |
| GHG-TDABase Model=Claude 3.5 Sonnet2026.02 | — | 44.6 | |
| GoTBase Model=GPT-4o2026.02 | — | 39.6 | |
| GoTBase Model=Claude 3.5 Sonnet2026.02 | — | 40.6 | |
| ToTBase Model=GPT-4o2026.02 | — | 40.5 | |
| ToTBase Model=Claude 3.5 Sonnet2026.02 | — | 41.5 |