Question Answering on HotpotQA v1.1 (test)
56.94Easy ScoreAUTOACT
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| AUTOACTBackbone=Llama-2 70B-chat, Agent Learning Type=Fine-tuning-based, Agent Architecture=Multi-agent2024.01 | 56.94 | 50.12 | 38.35 | 48.47 | |
| FireActBackbone=Llama-2 70B-chat, Agent Learning Type=Fine-tuning-based, Agent Architecture=Single-agent2024.01 | 50.82 | 41.43 | 35.86 | 42.7 | |
| Zero-Shot Plan*Backbone=GPT-3.5 Turbo, Agent Learning Type=Prompt-based, Agent Architecture=Single-agent2024.01 | 50.71 | 45.17 | 38.23 | 44.7 | |
| AUTOACTBackbone=Mistral-7B Instruct v0.2, Agent Learning Type=Fine-tuning-based, Agent Architecture=Multi-agent2024.01 | 48.69 | 36.65 | 31.37 | 38.89 | |
| CoTBackbone=GPT-3.5 Turbo, Agent Learning Type=Prompt-based, Agent Architecture=Single-agent2024.01 | 48.21 | 44.52 | 34.22 | 42.32 | |
| ReflexionBackbone=Llama-2 70B-chat, Agent Learning Type=Fine-tuning-based, Agent Architecture=Single-agent2024.01 | 48.01 | 46.35 | 35.64 | 43.33 | |
| AUTOACTBackbone=Llama-2 13B-chat, Agent Learning Type=Fine-tuning-based, Agent Architecture=Multi-agent2024.01 | 47.29 | 41.27 | 32.92 | 40.49 | |
| ChameleonBackbone=Llama-2 70B-chat, Agent Learning Type=Prompt-based, Agent Architecture=Single-agent2024.01 | 46.86 | 38.79 | 34.43 | 40.03 | |
| BOLAABackbone=Llama-2 70B-chat, Agent Learning Type=Prompt-based, Agent Architecture=Multi-agent2024.01 | 46.44 | 37.29 | 33.49 | 39.07 | |
| FireActBackbone=Llama-2 13B-chat, Agent Learning Type=Fine-tuning-based, Agent Architecture=Single-agent2024.01 | 45.83 | 38.94 | 26.06 | 36.94 | |
| FireActBackbone=Mistral-7B Instruct v0.2, Agent Learning Type=Fine-tuning-based, Agent Architecture=Single-agent2024.01 | 45.52 | 32.02 | 30.17 | 35.9 | |
| CoTBackbone=Llama-2 70B-chat, Agent Learning Type=Prompt-based, Agent Architecture=Single-agent2024.01 | 45.37 | 36.33 | 32.27 | 37.99 | |
| ReflexionBackbone=Llama-2 13B-chat, Agent Learning Type=Fine-tuning-based, Agent Architecture=Single-agent2024.01 | 44.43 | 37.5 | 28.17 | 36.7 | |
| ReWOOBackbone=Llama-2 70B-chat, Agent Learning Type=Prompt-based, Agent Architecture=Single-agent2024.01 | 42 | 39.58 | 35.32 | 38.96 | |
| BOLAABackbone=Mistral-7B Instruct v0.2, Agent Learning Type=Prompt-based, Agent Architecture=Multi-agent2024.01 | 40.86 | 32.11 | 22.36 | 31.78 | |
| ReflexionBackbone=Mistral-7B Instruct v0.2, Agent Learning Type=Prompt-based, Agent Architecture=Single-agent2024.01 | 40.78 | 35.02 | 28.36 | 34.72 | |
| ChameleonBackbone=Llama-2 13B-chat, Agent Learning Type=Prompt-based, Agent Architecture=Single-agent2024.01 | 40.01 | 25.39 | 22.82 | 29.41 | |
| ReActBackbone=Llama-2 70B-chat, Agent Learning Type=Prompt-based, Agent Architecture=Single-agent2024.01 | 39.7 | 37.19 | 33.62 | 36.83 | |
| ReWOOBackbone=Mistral-7B Instruct v0.2, Agent Learning Type=Prompt-based, Agent Architecture=Single-agent2024.01 | 38.42 | 31.89 | 25.98 | 32.1 | |
| ReActBackbone=Mistral-7B Instruct v0.2, Agent Learning Type=Prompt-based, Agent Architecture=Single-agent2024.01 | 38.09 | 27.57 | 22.05 | 29.24 | |
| CoTBackbone=Llama-2 13B-chat, Agent Learning Type=Prompt-based, Agent Architecture=Single-agent2024.01 | 37.9 | 25.28 | 21.64 | 28.27 | |
| ChameleonBackbone=Mistral-7B Instruct v0.2, Agent Learning Type=Prompt-based, Agent Architecture=Single-agent2024.01 | 37.07 | 26.67 | 19.2 | 27.65 | |
| CoTBackbone=Mistral-7B Instruct v0.2, Agent Learning Type=Prompt-based, Agent Architecture=Single-agent2024.01 | 33.7 | 22.38 | 22.14 | 26.07 | |
| BOLAABackbone=Llama-2 13B-chat, Agent Learning Type=Prompt-based, Agent Architecture=Multi-agent2024.01 | 33.23 | 25.46 | 25.23 | 27.97 | |
| ReWOOBackbone=Llama-2 13B-chat, Agent Learning Type=Prompt-based, Agent Architecture=Single-agent2024.01 | 30.09 | 24.01 | 21.13 | 25.08 | |
| ReActBackbone=Llama-2 13B-chat, Agent Learning Type=Prompt-based, Agent Architecture=Single-agent2024.01 | 28.68 | 22.15 | 21.69 | 24.17 |