List your company

ResearchFormerly WizWand

HotpotQA

Benchmarks

TaskDatasetSOTA resultTrendResults

Factual Reconstruction

HotpotQA (test)

98.2F1

5

Question Answering

HotpotQA (test)

48.71Accuracy

5

Knowledge Graph Construction

HotpotQA

124,460Triplets Count

5

HotpotQA

0.33Avg Running Time (s)

5

Question Answering

HotpotQA (test)

60Exact Match (EM)

5

Question Answering

HotpotQA (test)

0.77LLM Judge Score

5

Multi-hop Question Answering

HotpotQA

29.1F1 Score

5

Multi-hop Numeric Reasoning QA

HotpotQA Dynamic Numeric Reasoning suite (randomly sampled 200 multi-hop numeric reasoning questions)

2,252Tokens (No-adv)

5

HotpotQA

60.6Accuracy

5

Multi-step Question Answering

HotpotQA (test)

66.7EM

5

Question Answering

HotpotQA (test)

68.75EM

5

Question Answering

HotpotQA

52EM

5

Question Answering

HotpotQA L-M pair

40.58F1 Score

5

Question Answering

HotpotQA L-Q pair

43.16F1 Score

5

Open-domain Question Answering

HotpotQA distractor (val)

21EM

5

Question Answering Calibration

HotpotQA (test)

65.7Accuracy

5

Multi-hop Reading Comprehension

HotpotQA distractor setting (dev)

70.57All Score

5

Malicious Behavior Defense in Multi-Agent Systems

HotPotQA MA

5ASR (Attack Success Rate)

4

Question Answering

HotPotQA-Vanilla In-Domain

0.54AURC

4

Question Answering

HotpotQA

6.17Average Response Latency (s)

4

Question Answering

HotpotQA single-predictor (test)

60.4Exact Match

4

Question Answering

HotpotQA single-predictor (val)

64Exact Match

4

HotpotQA

78.6Recall@2

4

Selective Prediction

HotpotQA

86.6Sel-AUC

4

Multi-hop Question Answering

HotpotQA (val)

80F1 Score

4