Question Answering on LoCoMo (4o-J, ROUGE, BERTScore)
48.844o-JNano-Memory
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Nano-MemoryAvg. Tokens=1,4032026.04 | 48.84 | 22.66 | 5.33 | 22.92 | 12.04 | 21.68 | 85.94 | |
| HippoRAG 2Avg. Tokens=2,9912026.04 | 45.62 | 14.95 | 2.76 | 15.38 | 7.52 | 14.35 | 84.5 | |
| SeComAvg. Tokens=1,0212026.04 | 43.45 | 15.28 | 3.12 | 17.16 | 8.52 | 16.01 | 84.84 | |
| MemGASAvg. Tokens=2,8252026.04 | 41.07 | 17.66 | 3.61 | 18 | 8.93 | 16.99 | 85.13 | |
| A-MemAvg. Tokens=3,0422026.04 | 40.81 | 14.72 | 2.83 | 16.22 | 7.71 | 14.89 | 84.72 | |
| MPCAvg. Tokens=2,6832026.04 | 40.38 | 14.81 | 1.99 | 15.1 | 6.83 | 14.13 | 84.43 | |
| ContrieverAvg. Tokens=2,3482026.04 | 40.33 | 15.76 | 2.77 | 16.08 | 7.75 | 15.1 | 84.7 | |
| MPNetAvg. Tokens=2,4742026.04 | 38.07 | 14.52 | 2.36 | 14.97 | 6.82 | 13.84 | 84.46 | |
| Full HistoryAvg. Tokens=20,0782026.04 | 33.43 | 12.23 | 1.84 | 12.7 | 5.66 | 11.73 | 84.07 | |
| RAPTORAvg. Tokens=1,9312026.04 | 29.46 | 14.44 | 2.85 | 14.97 | 7.37 | 14.02 | 84.39 | |
| RecurSumAvg. Tokens=3,0742026.04 | 22.56 | 9.14 | 0.99 | 9.82 | 3.38 | 8.98 | 83.45 |