LongMemEval
Benchmarks
TaskDatasetSOTA resultTrendResults
LongMemEval 500 samples 24k (test)
73Accuracy
6
LongMemEval-S challenge subset n=150
59.4Recall@1
6
LongMemEval-S (full run)
99.5Recall@60
6
LongMemEval
71.45Accuracy
6
LongMemEval-s oracle variant HuggingFace xiaowu0162
100Single-session Accuracy (User)
6
LongMemEval_s Dynamic Agent Environment (Cognitive Asymmetry)
0.97Score
6
LongMemEval_s Static Environment (Ideal Symmetric Prefix)
0.33Score
6
LONGMEMEVAL S (noisy)
94.64SS-A
6
LONGMEMEVAL Oracle (clean)
98.21SS-A
6
LongMemEval-S
1,091.51Memory Footprint (tokens)
6
LongMemEval
65Accuracy
6
LongMemEval Clean500
98.35Recall@10
5
LongMemEval
75.2Score
5
LongMemEval M-tier (475 Sessions)
78.2KU Score
5
LongMemEval
76.2Accuracy
5
LongMemEval failures 8B
76.2Accuracy
5
LongMemEval
0.896Hit@5
5
LongMemEval (test)
94.36F1 Score
5
LongMemEval 1M
60NDCG@1
5
LongMemEval 128K
67.33NDCG@1
5
LongMemEval
11.6Accuracy
4
LongMemEval M 500K
40.2Score
4
LongMemEval S 125K
82LongMemEval Score
4
LongMemEval 115K-token S
12.8LLM-as-Judge Accuracy
4
LongMemEval Stress1000 seed 23
81.95Recall@10
4