Multi-hop Question Answering on HotpotQA (Ans, SF)
78.6Answer Accuracy (Ans)SABA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SABANormalized inference cost (T)=9.22026.04 | 78.6 | 73.5 | |
| GoTNormalized inference cost (T)=35.72026.04 | 78 | 73.2 | |
| SELF-DISC.Normalized inference cost (T)=5.02026.04 | 75.6 | 68.3 | |
| CRITICNormalized inference cost (T)=29.32026.04 | 74 | 68.7 | |
| S^2RNormalized inference cost (T)=18.42026.04 | 73.4 | 68.7 | |
| SC(k=5)Normalized inference cost (T)=12.0, k=52026.04 | 72 | 62.4 | |
| Self-RefineNormalized inference cost (T)=6.32026.04 | 68.6 | 59 | |
| CoTNormalized inference cost (T)=2.52026.04 | 68.4 | 60.3 | |
| DirectNormalized inference cost (T)=1.02026.04 | 62.4 | 52.3 |