Categories Companies Research

Loading the SOTA2 catalog…

Explore

Categories Companies Research

Contribute

Join SOTA2 Contact

SOTA2

© 2026 SOTA2

Build in Bellevue, USA

WizWand is now SOTA2 Research

Papers Benchmarks Datasets Tasks

LongMemEval dataset and benchmarks – Page 2 · SOTA2 Research

Research Datasets

LongMemEval

Benchmarks

Task NameDataset NameSOTA MethodSort

Task NameDataset NameSOTA ResultTrendResults

Long-term Agent Memory Evaluation

LongMemEval

95.7SS-U

15

Long-term conversational question answering

LongMemEval

97.14SS User Accuracy

14

Gold-evidence retention

LongMemEval-S (test)

99.3Gold-evidence Retention

14

Long-term memory assessment

LONGMEMEVAL

86.46LLM-Judge Multi-session

14

Long-term memory performance

LongMemEval S (test)

86.4Accuracy

13

Long-horizon conversation utility evaluation

LongMemEval

77.8Accuracy

12

Question Answering

LongMemEval 500 questions

61.4QA Accuracy

12

LongMemEval (500 questions)

97Fact Recall

12

Long-context memory evaluation

LongMemEval-s

75Overall Score

12

Question Answering

LongMemEval s

60.24o-J Score

11

Information Retrieval

LongMemEval

93Recall@10

10

Long Multi-session Question Answering

LongMemEval 1.0 (test)

34.6KU F1

10

Long-Context Memory Retrieval and Reasoning

LongMemEval 50K context input token limit

89.7KU L-J Score

10

Memory-augmented Question Answering

LongMemEval

92.9SSU Score

10

Long-term memory retrieval

LongMemEval

96.15Knowledge Update

10

LongMemEval-M

77.4Recall@5

10

Long-context Retrieval-Augmented Generation

LongMemEval

63.2Temperature Score

9

Question Answering

LongMemEval MAB

72LLM-judge Accuracy

9

Long-horizon Conversational Memory Answer-Quality

LongMemEval MAB

64.7Accuracy

9

Long-horizon memory Question Answering

LongMemEval

68.4Accuracy

9

Memory-intensive task

LongMemEval

57.41F1 Score

8

Memory-augmented generation

LongMemEvalS (test)

98.4User LLM (Single-Session)

8

Question Answering

LongMemEval (100-question sample)

68Accuracy

8

Memory Retrieval

LongMemEval n=500 (test)

80.43MAP

8

Question Answering

LongMemEval-m

46.64o-J Score

8

Page 2 of 5

Popular tasks

Question Answering Retrieval Long-context Memory Evaluation Long-term memory evaluation Information Retrieval

Follow for updates

Join SOTA2 Contact