# Agent Memory on AMA-Bench real-world

Source: [Agent Memory on AMA-Bench real-world](<https://www.sota2.com/research/sota/agent-memory-on-ama-bench-real-world>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-05T06:57:19.660Z

Evaluation of agent memory capabilities on the AMA-Bench real-world dataset, assessing performance across sub-tasks including Recall, Causal Inference, State Updating, and State Abstraction, measured using Accuracy and F1 scores.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Agent Memory |
| Dataset (including any stated split) | AMA-Bench real-world |
| Metric | Recall (Accuracy) |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 14 of 14 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Recall (Accuracy)](<https://www.sota2.com/research/sota/agent-memory-on-ama-bench-real-world.md?metric=Recall%20(Accuracy)>): Higher is better; unit: PERCENT.

- [Recall (F1)](<https://www.sota2.com/research/sota/agent-memory-on-ama-bench-real-world.md?metric=Recall%20(F1)>): Higher is better; unit: PERCENT.

- [Causal Inference (Accuracy)](<https://www.sota2.com/research/sota/agent-memory-on-ama-bench-real-world.md?metric=Causal%20Inference%20(Accuracy)>): Higher is better; unit: PERCENT.

- [Causal Inference (F1)](<https://www.sota2.com/research/sota/agent-memory-on-ama-bench-real-world.md?metric=Causal%20Inference%20(F1)>): Higher is better; unit: PERCENT.

- [State Updating (Accuracy)](<https://www.sota2.com/research/sota/agent-memory-on-ama-bench-real-world.md?metric=State%20Updating%20(Accuracy)>): Higher is better; unit: PERCENT.

- [State Updating (F1)](<https://www.sota2.com/research/sota/agent-memory-on-ama-bench-real-world.md?metric=State%20Updating%20(F1)>): Higher is better; unit: PERCENT.

- [State Abstraction (Accuracy)](<https://www.sota2.com/research/sota/agent-memory-on-ama-bench-real-world.md?metric=State%20Abstraction%20(Accuracy)>): Higher is better; unit: PERCENT.

- [State Abstraction (F1)](<https://www.sota2.com/research/sota/agent-memory-on-ama-bench-real-world.md?metric=State%20Abstraction%20(F1)>): Higher is better; unit: PERCENT.

- [Average (Accuracy)](<https://www.sota2.com/research/sota/agent-memory-on-ama-bench-real-world.md?metric=Average%20(Accuracy)>): Higher is better; unit: PERCENT.

- [Average (F1)](<https://www.sota2.com/research/sota/agent-memory-on-ama-bench-real-world.md?metric=Average%20(F1)>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Recall (Accuracy) | Result date | Report paper |
| --- | --- | --- | --- |
| AMA-Agent | 62.38 | Not recorded | 69a124ed2d603e6de3e3c351 |
| Qwen3-Emb-4B | 48.43 | Not recorded | 69a124ed2d603e6de3e3c351 |
| MemoRAG | 47.08 | Not recorded | 69a124ed2d603e6de3e3c351 |
| HippoRAG2 | 45.79 | Not recorded | 69a124ed2d603e6de3e3c351 |
| BM25 | 33.01 | Not recorded | 69a124ed2d603e6de3e3c351 |
| MemGPT | 32.89 | Not recorded | 69a124ed2d603e6de3e3c351 |
| MemoryBank | 32.31 | Not recorded | 69a124ed2d603e6de3e3c351 |
| Amem | 30.84 | Not recorded | 69a124ed2d603e6de3e3c351 |
| GraphRAG | 30.77 | Not recorded | 69a124ed2d603e6de3e3c351 |
| Mem-alpha | 28.76 | Not recorded | 69a124ed2d603e6de3e3c351 |
| MemAgent | 25.5 | Not recorded | 69a124ed2d603e6de3e3c351 |
| Simple mem | 20.12 | Not recorded | 69a124ed2d603e6de3e3c351 |
| Mem0 | 20.11 | Not recorded | 69a124ed2d603e6de3e3c351 |
| Mem1 | 11.8 | Not recorded | 69a124ed2d603e6de3e3c351 |

### AMA-Agent

Recall (Accuracy): 62.38. Result ID: 69a928f06dbb7075b951fadd.

[Reporting paper](<https://www.sota2.com/research/paper/69a124ed2d603e6de3e3c351.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a124ed2d603e6de3e3c351.md>)

Recorded evaluation configuration: {"Base Model":"Qwen-32B","Method Category":"Agent Memory Methods"}. This record does not assert independent reproduction.

### Qwen3-Emb-4B

Recall (Accuracy): 48.43. Result ID: 69a928f06dbb7075b951fad1.

[Reporting paper](<https://www.sota2.com/research/paper/69a124ed2d603e6de3e3c351.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79601e209131cea7d516.md>)

Recorded evaluation configuration: {"Base Model":"Qwen-32B","Method Category":"RAG"}. This record does not assert independent reproduction.

### MemoRAG

Recall (Accuracy): 47.08. Result ID: 69a928f06dbb7075b951fad8.

[Reporting paper](<https://www.sota2.com/research/paper/69a124ed2d603e6de3e3c351.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77dd1e209131cea75a45.md>)

Recorded evaluation configuration: {"Base Model":"Qwen-32B","Method Category":"Agent Memory Methods"}. This record does not assert independent reproduction.

### HippoRAG2

Recall (Accuracy): 45.79. Result ID: 69a928f06dbb7075b951fad3.

[Reporting paper](<https://www.sota2.com/research/paper/69a124ed2d603e6de3e3c351.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79821e209131cea7de8a.md>)

Recorded evaluation configuration: {"Base Model":"Qwen-32B","Method Category":"RAG"}. This record does not assert independent reproduction.

### BM25

Recall (Accuracy): 33.01. Result ID: 69a928f06dbb7075b951fad0.

[Reporting paper](<https://www.sota2.com/research/paper/69a124ed2d603e6de3e3c351.md>)

Recorded evaluation configuration: {"Base Model":"Qwen-32B","Method Category":"RAG"}. This record does not assert independent reproduction.

### MemGPT

Recall (Accuracy): 32.89. Result ID: 69a928f06dbb7075b951fad9.

[Reporting paper](<https://www.sota2.com/research/paper/69a124ed2d603e6de3e3c351.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa722a1e209131cea54e5c.md>)

Recorded evaluation configuration: {"Base Model":"Qwen-32B","Method Category":"Agent Memory Methods"}. This record does not assert independent reproduction.

### MemoryBank

Recall (Accuracy): 32.31. Result ID: 69a928f06dbb7075b951fadb.

[Reporting paper](<https://www.sota2.com/research/paper/69a124ed2d603e6de3e3c351.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f731e209131cea45ab8.md>)

Recorded evaluation configuration: {"Base Model":"Qwen-32B","Method Category":"Agent Memory Methods"}. This record does not assert independent reproduction.

### Amem

Recall (Accuracy): 30.84. Result ID: 69a928f06dbb7075b951fad6.

[Reporting paper](<https://www.sota2.com/research/paper/69a124ed2d603e6de3e3c351.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7bf31e209131cea8b869.md>)

Recorded evaluation configuration: {"Base Model":"Qwen-32B","Method Category":"Agent Memory Methods"}. This record does not assert independent reproduction.

### GraphRAG

Recall (Accuracy): 30.77. Result ID: 69a928f06dbb7075b951fad2.

[Reporting paper](<https://www.sota2.com/research/paper/69a124ed2d603e6de3e3c351.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa770d1e209131cea70c5a.md>)

Recorded evaluation configuration: {"Base Model":"Qwen-32B","Method Category":"RAG"}. This record does not assert independent reproduction.

### Mem-alpha

Recall (Accuracy): 28.76. Result ID: 69a928f06dbb7075b951fada.

[Reporting paper](<https://www.sota2.com/research/paper/69a124ed2d603e6de3e3c351.md>)

Recorded evaluation configuration: {"Base Model":"Qwen-32B","Method Category":"Agent Memory Methods"}. This record does not assert independent reproduction.

### MemAgent

Recall (Accuracy): 25.5. Result ID: 69a928f06dbb7075b951fad4.

[Reporting paper](<https://www.sota2.com/research/paper/69a124ed2d603e6de3e3c351.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79c71e209131cea7f569.md>)

Recorded evaluation configuration: {"Base Model":"Qwen-32B","Method Category":"Agent Memory Methods"}. This record does not assert independent reproduction.

### Simple mem

Recall (Accuracy): 20.12. Result ID: 69a928f06dbb7075b951fadc.

[Reporting paper](<https://www.sota2.com/research/paper/69a124ed2d603e6de3e3c351.md>)

[Method origin paper](<https://www.sota2.com/research/paper/695de87553fdd5c361743bee.md>)

[Implementation](<https://github.com/aiming-lab/SimpleMem>)

Recorded evaluation configuration: {"Base Model":"Qwen-32B","Method Category":"Agent Memory Methods"}. This record does not assert independent reproduction.

### Mem0

Recall (Accuracy): 20.11. Result ID: 69a928f06dbb7075b951fad7.

[Reporting paper](<https://www.sota2.com/research/paper/69a124ed2d603e6de3e3c351.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa784b1e209131cea77ad6.md>)

Recorded evaluation configuration: {"Base Model":"Qwen-32B","Method Category":"Agent Memory Methods"}. This record does not assert independent reproduction.

### Mem1

Recall (Accuracy): 11.8. Result ID: 69a928f06dbb7075b951fad5.

[Reporting paper](<https://www.sota2.com/research/paper/69a124ed2d603e6de3e3c351.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a091e209131cea80b6f.md>)

Recorded evaluation configuration: {"Base Model":"Qwen-32B","Method Category":"Agent Memory Methods"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
