LoTTE
Benchmarks
TaskDatasetSOTA resultTrendResults
LoTTE pooled (test)
1.3Retrieval Time (ms)
41
LoTTE
11Latency (ms)
26
LoTTE Average across Sessions 0-9
48.6Success@5
14
LoTTE Session 9
52.2Success@5
14
LoTTE Session 8
66.7Success@5
14
LoTTE Session 7
48.1Success@5
14
LoTTE Session 6
45.9Success@5
14
LoTTE Session 5
44.8Success@5
14
LoTTE Session 4
58.5Success@5
14
LoTTE Session 3
0.578Success@5
14
LoTTE Session 2
47.8Success@5
14
LoTTE Session 1
47.4Success@5
14
LoTTE Session 0
37.2Success@5
14
LoTTE
0.682Success@5
14
LoTTE Forum (test)
79.7IR Score (Lifestyle)
9
LoTTE Search (test)
87.6Lifestyle Score
9
LoTTE 5 search sets mean (test)
72Success@5
8
LoTTE Search 1.0 (dev)
81.7Success@5 (Writing)
6
LoTTE Forum v1.0 (test)
76.3Success@5 (Writing)
6
LoTTE Search v1.0 (test)
80.1Success@5 (Writing)
6
LoTTE Forum
0.66Mean Success@5
4
LoTTE Search
74.7Mean Success@5
4
LoTTE
73.5Search (pooled) Score
3
LoTTE Forum zero-shot
83.4Writing Score
2
LoTTE Search zero-shot
83.3Writing Score
2