Multi-hop Reasoning on MuSiQue (Accuracy)
51Accuracygemma-3-27b-it
Evaluation results
| Method | Links | |
|---|---|---|
| gemma-3-27b-itthinking mode=false2026.05 | 51 | |
| Qwen3-32Bthinking mode=false2026.05 | 49.7 | |
| Qwen3-32Bthinking mode=true2026.05 | 49.3 | |
| Qwen3-14Bthinking mode=true2026.05 | 45.6 | |
| gemma-3-12b-itthinking mode=false2026.05 | 44.6 | |
| Qwen3-8Bthinking mode=true2026.05 | 43.9 | |
| Qwen3-4Bthinking mode=true2026.05 | 41.5 | |
| AoTBackbone=DeepSeek-V32026.03 | 39.6 | |
| ToTBackbone=DeepSeek-V32026.03 | 39.4 | |
| Qwen3-8Bthinking mode=false2026.05 | 39.3 | |
| AoTBackbone=Qwen-Turbo2026.03 | 39.2 | |
| ToTBackbone=GPT-4o-mini2026.03 | 39.1 | |
| Qwen3-14Bthinking mode=false2026.05 | 38.7 | |
| AoTBackbone=GPT-4o-mini2026.03 | 38.4 | |
| ToTBackbone=Qwen-Turbo2026.03 | 38.4 | |
| AFlowBackbone=DeepSeek-V32026.03 | 38.2 | |
| OCC-RAG-1.7Bthinking mode=false2026.05 | 38.2 | |
| AFlowBackbone=GPT-4o-mini2026.03 | 38.1 | |
| TDA-RCBackbone=DeepSeek-V32026.03 | 37.9 | |
| TDA-RCBackbone=GPT-4o-mini2026.03 | 37.8 | |
| AFlowBackbone=Qwen-Turbo2026.03 | 37.8 | |
| TDA-RCBackbone=Qwen-Turbo2026.03 | 37 | |
| GoTBackbone=DeepSeek-V32026.03 | 36.7 | |
| OCC-RAG-0.6Bthinking mode=false2026.05 | 36.6 | |
| GoTBackbone=GPT-4o-mini2026.03 | 36.5 | |
| GoTBackbone=Qwen-Turbo2026.03 | 36.4 | |
| FoT (n=8)Backbone=DeepSeek-V32026.03 | 36.2 | |
| FoT (n=8)Backbone=GPT-4o-mini2026.03 | 35.8 | |
| FoT (n=8)Backbone=Qwen-Turbo2026.03 | 35.8 | |
| Self-RefineBackbone=GPT-4o-mini2026.03 | 35.1 | |
| Self-RefineBackbone=DeepSeek-V32026.03 | 35.1 | |
| Self-RefineBackbone=Qwen-Turbo2026.03 | 35 | |
| CoTBackbone=DeepSeek-V32026.03 | 34.2 | |
| CoTBackbone=GPT-4o-mini2026.03 | 34.1 | |
| CoT-SCBackbone=DeepSeek-V32026.03 | 34 | |
| CoT-SC (n=5)Backbone=GPT-4o-mini2026.03 | 33.8 | |
| CoTBackbone=Qwen-Turbo2026.03 | 33.6 | |
| CoT-SC (n=5)Backbone=Qwen-Turbo2026.03 | 33.5 | |
| Qwen3-4Bthinking mode=false2026.05 | 33.1 | |
| Qwen3-1.7Bthinking mode=true2026.05 | 30.7 | |
| gemma-3-4b-itthinking mode=false2026.05 | 30.1 | |
| SmolLM3-3Bthinking mode=true2026.05 | 29.4 | |
| SmolLM3-3Bthinking mode=false2026.05 | 21.5 | |
| Qwen3-1.7Bthinking mode=false2026.05 | 20.1 | |
| Qwen3-0.6Bthinking mode=true2026.05 | 17.2 | |
| Pleias-RAG-1.2Bthinking mode=false2026.05 | 15 | |
| Qwen3-0.6Bthinking mode=false2026.05 | 13.2 | |
| gemma-3-1b-itthinking mode=false2026.05 | 12.8 |