Information Retrieval on LoCoMo (R@3, R@5, R@10)
86.46Recall@10Nano-Memory
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Nano-MemoryTime=0.74s (↓71.97%), Generator=gpt-4o-mini-2024-07-18, Retriever=Contriever, top-k=32026.04 | 86.46 | 69.39 | 71.19 | 76.33 | 74.58 | 77.99 | |
| MemGASTime=2.64s, Generator=gpt-4o-mini-2024-07-18, Retriever=Contriever, top-k=32026.04 | 81.22 | 56.85 | 59.22 | 67.98 | 64.25 | 68.71 | |
| SeComTime=2.17s, Generator=gpt-4o-mini-2024-07-18, Retriever=Contriever, top-k=32026.04 | 78.25 | 52.97 | 54.03 | 64.05 | 59.37 | 64.28 | |
| HippoRAG 2Time=3.70s, Generator=gpt-4o-mini-2024-07-18, Retriever=Contriever, top-k=32026.04 | 78.05 | 56.6 | 58.37 | 65.06 | 62.5 | 66.79 | |
| EvoEmbedding-4BSize=4B, Dim=10242026.06 | 76.3 | — | — | — | — | 61.7 | |
| RecurSumTime=1.26s, Generator=gpt-4o-mini-2024-07-18, Retriever=Contriever, top-k=32026.04 | 74.97 | 47.23 | 48.99 | 59.01 | 54.58 | 60.07 | |
| EvoEmbedding-2BSize=2B, Dim=10242026.06 | 74.1 | — | — | — | — | 58.3 | |
| MPCTime=0.81s, Generator=gpt-4o-mini-2024-07-18, Retriever=Contriever, top-k=32026.04 | 71.85 | 49.5 | 51.47 | 57.45 | 55.53 | 60.47 | |
| ContrieverTime=1.24s, Generator=gpt-4o-mini-2024-07-18, Retriever=Contriever, top-k=32026.04 | 71.8 | 49.95 | 52.15 | 58.31 | 56.29 | 60.91 | |
| MPNetTime=1.42s, Generator=gpt-4o-mini-2024-07-18, Retriever=Contriever, top-k=32026.04 | 68.58 | 45.92 | 47.68 | 53.98 | 51.76 | 56.85 | |
| Multilingual-e5-largeSize=0.6B, Dim=10242026.06 | 65.6 | — | — | — | — | 50.1 | |
| EvoEmbedding-0.8BSize=0.8B, Dim=10242026.06 | 63 | — | — | — | — | 48.8 | |
| KaLM-Embedding-Gemma3Size=12B, Dim=38402026.06 | 58.4 | — | — | — | — | 42.8 | |
| BGE-M3Size=1.2B, Dim=10242026.06 | 55.7 | — | — | — | — | 41.8 | |
| Jina-v5-text-smallSize=0.7B, Dim=10242026.06 | 53.1 | — | — | — | — | 39.4 | |
| Qwen3-Embedding-8BSize=8B, Dim=40962026.06 | 49.6 | — | — | — | — | 36.4 | |
| Qwen3-Embedding-0.6BSize=0.6B, Dim=10242026.06 | 46.8 | — | — | — | — | 34.2 | |
| Qwen3-Embedding-4BSize=4B, Dim=25602026.06 | 41.3 | — | — | — | — | 29.9 |