Multi-hop Question Answering on HotpotQA subset of 100 questions
71EMLATS
Evaluation results
| Method | Links | |
|---|---|---|
| LATSLLM=GPT-3.5, Prompt Method=CoT + ReAct, Prompt Strategy=Acting-based, Number of nodes (n)=5, Number of trajectories (k)=502023.10 | 71 | |
| LATSLLM=GPT-3.5, Prompt Strategy=Acting-based, Number of nodes (n)=10, Number of trajectories (k)=502023.10 | 65 | |
| LATSLLM=GPT-3.5, Prompt Method=ReAct, Prompt Strategy=Acting-based, Number of nodes (n)=5, Number of trajectories (k)=502023.10 | 63 | |
| LATSBackbone=GPT-3.5, Prompt Design=CoT, n=5, k=502023.10 | 62 | |
| RAPBackbone=GPT-3.5, k=502023.10 | 60 | |
| RAPBackbone=GPT-3.5, n=102023.10 | 60 | |
| LATSLLM=GPT-3.5, Prompt Strategy=Acting-based, Number of nodes (n)=3, Number of trajectories (k)=502023.10 | 58 | |
| ToTBackbone=GPT-3.5, Search Algorithm=DFS, k=502023.10 | 55 | |
| RAPLLM=GPT-3.5, Prompt Method=ReAct, Prompt Strategy=Acting-based2023.10 | 54 | |
| ReflexionLLM=GPT-3.5, Prompt Strategy=Acting-based2023.10 | 51 | |
| ToTLLM=GPT-3.5, Prompt Method=ReAct, Prompt Strategy=Acting-based2023.10 | 39 | |
| CoT-SCBackbone=GPT-3.52023.10 | 38 | |
| ReActLLM=GPT-3.5, Prompt Strategy=Acting-based, selection=best of k, Number of trajectories (k)=502023.10 | 38 | |
| CoTBackbone=GPT-3.52023.10 | 34 | |
| Base LMBackbone=GPT-3.52023.10 | 32 | |
| ReActLLM=GPT-3.5, Prompt Strategy=Acting-based2023.10 | 32 |