Multi-hop Question Answering on HotpotQA (EM, F1, Faithfulness)
75.97F1ACPS
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ACPSModel=GPT-3.5-turbo2026.01 | 75.97 | 61.31 | — | |
| CPModel=GPT-3.5-turbo2026.01 | 72.51 | 57.05 | — | |
| ACPSModel=LLaMA-32026.01 | 70.22 | 56.67 | — | |
| CPModel=LLaMA-32026.01 | 69.98 | 55 | — | |
| CoT-SCModel=GPT-3.5-turbo2026.01 | 69.03 | 54.74 | — | |
| DeCoTModel=GPT-3.5-turbo2026.01 | 68.35 | 53.91 | — | |
| CADModel=GPT-3.5-turbo2026.01 | 67.45 | 52.66 | — | |
| ACPSModel=Ministral-3B2026.01 | 66.67 | 50 | — | |
| RT-RAGModels=Qwen2.5-14B2026.01 | 66.24 | 51 | — | |
| RT-RAGModels=GPT-4o-mini2026.01 | 65.26 | 52.5 | — | |
| LongRAGModels=GPT-4o-mini2026.01 | 64.74 | 51 | — | |
| ChainRAG (CxtInt)Models=GPT-4o-mini2026.01 | 64.59 | 50 | — | |
| DeCoTModel=LLaMA-32026.01 | 64.52 | 54.63 | — | |
| CADModel=LLaMA-32026.01 | 63.95 | 53.08 | — | |
| ICLModel=GPT-3.5-turbo2026.01 | 63.72 | 46 | — | |
| LongRAGModels=Qwen2.5-14B2026.01 | 62.64 | 49.5 | — | |
| ChainRAG (AnsInt)Models=GPT-4o-mini2026.01 | 60.73 | 46 | — | |
| NaiveRAG w/ QDModels=GPT-4o-mini2026.01 | 60 | 43.5 | — | |
| IRCoTModels=GPT-4o-mini2026.01 | 59.92 | 45 | — | |
| SoTModel=GPT-3.5-turbo2026.01 | 59.04 | 40.83 | — | |
| ItER-RETGENModels=Qwen2.5-14B2026.01 | 58.63 | 44.5 | — | |
| CoTModel=GPT-3.5-turbo2026.01 | 58.6 | 40.54 | — | |
| NaiveRAGModels=Qwen2.5-14B2026.01 | 57.96 | 43.5 | — | |
| ItER-RETGENModels=GPT-4o-mini2026.01 | 57.77 | 42 | — | |
| NaiveRAGModels=GPT-4o-mini2026.01 | 56.92 | 42 | — | |
| HippoRAG w/ IRCoTModels=GPT-4o-mini2026.01 | 56.12 | 40 | — | |
| CoT-SCModel=LLaMA-32026.01 | 55.94 | 40.88 | — | |
| SoTModel=LLaMA-32026.01 | 54.44 | 39.17 | — | |
| SuReModels=GPT-4o-mini2026.01 | 52.8 | 37.5 | — | |
| NaiveRAG w/ QDModels=Qwen2.5-14B2026.01 | 50.95 | 38.5 | — | |
| Self-AskModels=GPT-4o-mini2026.01 | 50.64 | 38 | — | |
| DeCoTModel=Ministral-3B2026.01 | 50.21 | 35.65 | — | |
| CoTModel=LLaMA-32026.01 | 50.2 | 39.16 | — | |
| CoT-SCModel=Ministral-3B2026.01 | 49.16 | 33.33 | — | |
| CADModel=Ministral-3B2026.01 | 49.01 | 35.23 | — | |
| CPModel=Ministral-3B2026.01 | 48.98 | 38.03 | — | |
| IRCoTModels=Qwen2.5-14B2026.01 | 48.79 | 36.5 | — | |
| SuReModels=Qwen2.5-14B2026.01 | 48.21 | 33 | — | |
| CoTModel=Ministral-3B2026.01 | 47.2 | 31.58 | — | |
| HippoRAG w/ IRCoTModels=Qwen2.5-14B2026.01 | 47.11 | 35.5 | — | |
| SoTModel=Ministral-3B2026.01 | 46.37 | 31.19 | — | |
| Self-AskModels=Qwen2.5-14B2026.01 | 45.12 | 35 | — | |
| ICLModel=LLaMA-32026.01 | 43.18 | 32.55 | — | |
| ICLModel=Ministral-3B2026.01 | 42.06 | 26.92 | — | |
| CoTModels=GPT-4o-mini2026.01 | 39.74 | 28 | — | |
| DirectModels=GPT-4o-mini2026.01 | 37.85 | 27.5 | — | |
| CoTModels=Qwen2.5-14B2026.01 | 32.03 | 21.5 | — | |
| DirectModels=Qwen2.5-14B2026.01 | 30.52 | 20.5 | — |