Locomo
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
LoCoMo Temporal
59.76F1
12
LoCoMo Multi-Hop
48.15F1 Score
12
LoCoMo-Plus (n=401)
93.3Accuracy (%)
12
LoCoMo v1 (full)
35.7F1 (Multi-Hop)
12
LoCoMo (test)
48.58Multi Hop (Q)
12
LoCoMo
97.15Accuracy (Single-Hop)
11
LoCoMo w/o conv-26
87.9Accuracy (Non-Adversarial)
11
LoCoMo
48.844o-J
11
LoCoMo Overall
39.81BLEU-1
11
LoCoMo
30Accuracy
10
LOCOMO 1,542 QA (full)
22.7ROUGE-L
10
LoCoMo 1.0 (test)
37.5Single-hop F1
10
LoCoMo 10K context input token limit
78.1Single-hop L-J Score
10
LoCoMo
66.5Accuracy (LoCoMo QA)
10
LoCoMo All convs.
88.25Accuracy (Non-Adv)
10
LoCoMo
91.08Single-Hop J-Score
10
LoCoMo 0.6.0
94.48Accuracy
10
LoCoMo
49.8Multi-hop R-1 Score
10
LoCoMo
87.4Single-hop Accuracy
9
LOCOMO
26.6Token F1
9
LoCoMo
54.9LLM-judge Accuracy
9
LoCoMo
50.6F1 Score
9
LoCoMo
89.6Accuracy
9
LoCoMo 1.0 (test)
47F1 (Multi-Hop)
9
LoCoMo
88.53Single-hop Accuracy
9