Temporal Reasoning on LoCoMo
65.06F1 ScoreMembox
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| MemboxModel=GPT-4o2026.01 | 65.06 | 54.81 | — | |
| MemboxModel=GPT-4o-mini2026.01 | 58.03 | 45.17 | — | |
| Mem0Model=GPT-4o-mini2026.04 | 56.42 | — | 45.89 | |
| GAMModel=GPT-4o-mini2026.04 | 51.96 | — | 42.26 | |
| GAMModel=Qwen 2.5-14B2026.04 | 50.25 | — | 39.33 | |
| Mem0Model=Qwen 2.5-14B2026.04 | 50.2 | — | 39.85 | |
| GAMModel=Qwen 2.5-7B2026.04 | 48.97 | — | 39.68 | |
| Mem0Model=GPT-4o-mini2026.01 | 48.93 | 40.51 | — | |
| GAMModel=Llama 3.2-3B2026.04 | 46.12 | — | 37.24 | |
| A-MEMModel=GPT-4o-mini2026.01 | 45.85 | 36.67 | — | |
| A-MemModel=GPT-4o-mini2026.04 | 45.85 | — | 36.67 | |
| Mem0Model=GPT-4o, Local re-implementation=true, k=302026.01 | 44.55 | 32.6 | — | |
| Mem0Model=Qwen 2.5-7B2026.04 | 41.22 | — | 32.22 | |
| MemoryOSModel=GPT-4o-mini2026.04 | 41.15 | — | 30.76 | |
| A-MEMModel=GPT-4o, Local re-implementation=true, k=302026.01 | 41.11 | 34.72 | — | |
| A-MEMModel=GPT-4o2026.01 | 39.41 | 31.23 | — | |
| Mem0Model=GPT-4o-mini, Local re-implementation=true, k=302026.01 | 34.52 | 26.38 | — | |
| Mem0Model=Llama 3.2-3B2026.04 | 30.08 | — | 20.73 | |
| A-MEMModel=GPT-4o-mini, Local re-implementation=true, k=302026.01 | 29.14 | 24.08 | — | |
| MemoryOSModel=Qwen 2.5-7B2026.04 | 28.96 | — | 20.69 | |
| A-MemModel=Qwen 2.5-14B2026.04 | 28.3 | — | 23.02 | |
| A-MemModel=Qwen 2.5-7B2026.04 | 27.54 | — | 22.49 | |
| A-MemModel=Llama 3.2-3B2026.04 | 26.38 | — | 19.5 | |
| MEMGPTModel=GPT-4o-mini2026.01 | 25.52 | 19.44 | — | |
| MemgptModel=GPT-4o-mini2026.04 | 25.52 | — | 19.44 | |
| MemoryOSModel=Qwen 2.5-14B2026.04 | 23.38 | — | 16.8 | |
| MemoryOSModel=Llama 3.2-3B2026.04 | 19.38 | — | 13.73 | |
| LoCoMoModel=GPT-4o-mini2026.01 | 18.41 | 14.77 | — | |
| MEMGPTModel=GPT-4o2026.01 | 17.29 | 13.18 | — | |
| READAGENTModel=GPT-4o-mini2026.01 | 12.6 | 8.87 | — | |
| ReadAgentModel=GPT-4o-mini2026.04 | 12.6 | — | 8.87 | |
| MEMORYBANKModel=GPT-4o-mini2026.01 | 9.68 | 6.99 | — | |
| MemorybankModel=GPT-4o-mini2026.04 | 9.68 | — | 6.99 | |
| LoCoMoModel=GPT-4o2026.01 | 9.09 | 5.78 | — | |
| ReadAgentModel=Llama 3.2-3B2026.04 | 4.33 | — | 4.33 | |
| READAGENTModel=GPT-4o2026.01 | 4.16 | 3.19 | — | |
| MemgptModel=Qwen 2.5-14B2026.04 | 4.05 | — | 3.64 | |
| MemorybankModel=Llama 3.2-3B2026.04 | 3.9 | — | 3.41 | |
| MemgptModel=Llama 3.2-3B2026.04 | 3.25 | — | 3.15 | |
| ReadAgentModel=Qwen 2.5-7B2026.04 | 2.82 | — | 2.74 | |
| ReadAgentModel=Qwen 2.5-14B2026.04 | 2.66 | — | 2.71 | |
| MemorybankModel=Qwen 2.5-14B2026.04 | 2.59 | — | 2.33 | |
| MemorybankModel=Qwen 2.5-7B2026.04 | 2.58 | — | 2.27 | |
| MEMORYBANKModel=GPT-4o2026.01 | 2.47 | 2.43 | — | |
| MemgptModel=Qwen 2.5-7B2026.04 | 2.18 | — | 2.77 |