NQ
Benchmarks
TaskDatasetSOTA resultTrendResults
NQ
0.889AUC
199
NQ (test)
66.4EM Accuracy
143
NQ
70Accuracy
123
NQ
87Accuracy
113
NQ
74.2Exact Match
101
NQ
85.1AUROC
98
NQ 16 additional samples (test)
85.6AUROC
98
NQ 8 additional samples
0.853AUROC
98
NQ 4 additional samples (test)
85.3AUROC
98
NQ (test)
95.2AUC ROC
91
NQ (test)
83AUROC
90
NQ
0.064Absolute Execution Time Overhead (s)
90
NQ
0.65PRR
90
NQ-Open
0.793QE5 Score
80
NQ
61.6Accuracy
74
NQ
0.496ACE Score
70
NQ
99.65ASR
70
NQ
67EM
69
NQ
79EM
69
NQ
78.8F1 Score (NQ)
64
NQ-Table
80.1F1 Score
63
NQ
51.7Exact Match (EM)
60
NQ (test)
55.1Exact Match (EM)
59
NQ
44.11F1 Score
56
NQ
0.046ECE
55