NQ
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
NQ (test)
3.79Inference Time (h)
5
NQ
5.98Average response latency (s)
4
NQ-Swap
77.46Exact Match
4
NQ
47Mean Score
4
NQ
89.33Precision
4
NQ
29.23R@1
4
NQ
1.14Latency (s/query)
4
NQ (test)
55.7ASR
4
NQ
30.9EM
4
NQ (sampled)
69.5Accuracy
4
NQ-Swap
44.91EM
4
NQ
0.512Exact Match (EM)
4
NQ
73.15Flans
4
NQ
0.426EM
4
NQ-Swap
29.4Accuracy (entity)
4
NQ (dev)
52.2Span F1
4
NQ (in-domain)
77.1MRR
4
NQ (test)
68.5Always-RAG Accuracy
3
NQ
99.42Accuracy
3
NQ
29.21nDCG@10
3
NQ
47.1EM
3
NQ Search-o1 framework (50% held-out test)
44.8Accuracy
3
NQ MultiTurn
57ASR
3
NQ
44.55Success Rate
2
NQ
1.96H2D Latency (ms)
2