Question Answering on LongMemEval s
60.24o-J ScoreMemGAS
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| MemGASAvg. Tokens=8,8292026.04 | 60.2 | 20.74 | 4.48 | 21.36 | 10.47 | 19.8 | 85.29 | |
| HippoRAG 2Avg. Tokens=8,5302026.04 | 57.6 | 14.73 | 2.15 | 15.3 | 7.36 | 13.83 | 83.86 | |
| Nano-MemoryAvg. Tokens=4,3022026.04 | 57.2 | 21.06 | 4.89 | 21.8 | 10.77 | 20.16 | 85.54 | |
| SeComAvg. Tokens=2,7412026.04 | 56 | 12.95 | 2.25 | 13.8 | 6.09 | 11.93 | 83.51 | |
| A-MemAvg. Tokens=9,0182026.04 | 55.6 | 13.73 | 2.11 | 14.82 | 6.81 | 12.98 | 83.88 | |
| ContrieverAvg. Tokens=8,4712026.04 | 55.4 | 13.37 | 2.15 | 13.9 | 6.72 | 12.43 | 83.64 | |
| MPCAvg. Tokens=8,4572026.04 | 53.8 | 13.6 | 1.74 | 14.27 | 6.49 | 12.95 | 83.49 | |
| MPNetAvg. Tokens=8,1732026.04 | 53.2 | 13.96 | 2.21 | 14.49 | 6.78 | 12.93 | 83.72 | |
| Full HistoryAvg. Tokens=103,1372026.04 | 50.6 | 11.48 | 1.4 | 12.1 | 5.47 | 10.85 | 83.07 | |
| RecurSumAvg. Tokens=8,8532026.04 | 35.4 | 12.29 | 2.09 | 13.01 | 5.55 | 11.52 | 83.6 | |
| RAPTORAvg. Tokens=6,2542026.04 | 32.2 | 12.08 | 1.9 | 12.73 | 5.82 | 11.25 | 83.5 |