LongMemEval
Benchmarks
TaskDatasetSOTA resultTrendResults
LongMemEval
92.9R@5
4
LongMemEval-S Full-500 official protocol
47.7Mean Accuracy
4
LongMemEval-S cross-benchmark replication Full-500
50.4Jaccard Score (S-4.5, 3x)
4
LongMemEval-S cross-benchmark replication Full-500
0.683Precision@5
4
LongMemEval 90K-token dialogue history + 300-turn conversation
46.2QA Accuracy
3
LongMemEval-S (subset 12)
75P@1
3
LongMemEval
68.2Memory Precision
3
LongMemEval-10
0.767Overall Score
2
LongMemEval knowledge-update question type n=45 questions
29Correct Count
2
LongMemEval stratified
41.7Mean Score
2
LongMemEval-S (Full 500)
80.2P@1
2
LongMemEval
76.4Accuracy
2
LongMemEval-S (seed 101)
0.01Delta
1
LongMemEval-S (seed 5)
0.1Delta
1
LongMemEval-S (seed 23)
0.05Delta Score
1
LongMemEval-S (Seed 11)
0.06Delta
1
LongMemEval
72.3Score
1
LongMemEval-S
—F1
0
LongMemEval S
—F1 Score
0
LongMemEval S
—F1
0
LongMemEval S
—F1 Score
0