List your company
ResearchFormerly WizWand

Question Answering on HotpotQA (LLM Accuracy and EM)

36.3Exact Match (EM)LUMOS-IQA
21.7425.5229.333.08Nov 9, 2023
Updated 8mo ago

Evaluation results

MethodLinks
LUMOS-IQAAgent Model=LLAMA-2-13B, QA Tool=GPT-4, fine-tuning=true2023.1136.357.4
LUMOS-IQAAgent Model=LLAMA-2-7B, QA Tool=GPT-4, fine-tuning=true2023.113656.8
ReAcTAgent Model=GPT-3.5-turbo, QA Tool=GPT-3.5-turbo, fine-tuning=false2023.1132.440.8
LUMOS-IQAAgent Model=LLAMA-2-13B, QA Tool=GPT-3.5-turbo, fine-tuning=true2023.1131.450.2
ReWOOAgent Model=GPT-3.5-turbo, QA Tool=GPT-3.5-turbo, fine-tuning=false2023.1130.442.4
LUMOS-IQAAgent Model=LLAMA-2-7B, QA Tool=GPT-3.5-turbo, fine-tuning=true2023.1129.445.9
FiReActAgent Model=CodeLLAMA-34B, QA Tool=CodeLLAMA-34B, fine-tuning=true2023.1127.8—
FiReActAgent Model=LLAMA-2-7B, QA Tool=LLAMA-2-7B, fine-tuning=true2023.1126.2—
LUMOS-OQAAgent Model=LLAMA-2-7B, QA Tool=GPT-3.5-turbo, fine-tuning=true2023.1124.939.2
LUMOS-IQAAgent Model=LLAMA-2-7B, QA Tool=LLAMA-2-7B, fine-tuning=true2023.1123.537.3
GPT-3.5-CoTAgent Model=GPT-3.5-turbo, QA Tool=GPT-3.5-turbo, fine-tuning=false2023.1122.437.8
AgentLMAgent Model=LLAMA-2-7B, QA Tool=LLAMA-2-7B, fine-tuning=false2023.1122.3—
ReWOO-openAgent Model=LLAMA-7B, QA Tool=GPT-3.5-turbo, fine-tuning=true2023.11—37