AMBIGQA
Benchmarks
TaskDatasetSOTA resultTrendResults
AmbigQA (val)
69.8PRR
90
AmbigQA
69.8Accuracy (One Intent)
36
AmbigQA
79.5F1 Score
29
Non-AmbigQA
79.32AUROC
20
AmbigQA
60Cover EM
18
AmbigQA
78.5AUROC
16
AmbigQA (test)
54.43Accuracy
13
AmbigQA
61.3EM
11
AmbigQA
0.113Personalization Bias
9
AmbigQA Unambiguous queries (dev)
42.05Reward
8
AmbigQA Ambiguous queries (dev)
15.81Reward
8
AmbigQA (test)
65.8Correctness (%)
7
AmbigQA
59.8Accuracy
7
AmbigQA Nq=300
0.473Acc
6
AmbigQA
4.96Helpfulness
5
AmbigQA High Aleatoric Uncertainty Superset (top 20% examples)
43.87Clarification Rate
4
AmbigQA (sampled)
65.5Accuracy
4
AMBIGQA (test)
46.2F1 (All Questions)
3
AMBIGQA (dev)
52.1F1 (all questions)
3