Question Answering on LongMTBench+
67.714o-J ScoreMemGAS
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| MemGASAvg. Tokens=12,8732026.04 | 67.71 | 41.49 | 15.78 | 43.69 | 24.47 | 34.64 | 88.9 | |
| Full HistoryAvg. Tokens=19,1942026.04 | 67.44 | 36.07 | 11.32 | 37.9 | 20.51 | 29.25 | 87.81 | |
| A-MemAvg. Tokens=13,7352026.04 | 65.73 | 36.82 | 11.36 | 38.92 | 20.88 | 29.14 | 87.92 | |
| SeComAvg. Tokens=4,7142026.04 | 64.58 | 38.89 | 13.4 | 40.87 | 22.81 | 31.9 | 88.45 | |
| Nano-MemoryAvg. Tokens=6,1742026.04 | 64.15 | 42.4 | 16.83 | 44.55 | 25.45 | 35.57 | 89.34 | |
| MPNetAvg. Tokens=12,1872026.04 | 63.89 | 35.32 | 11.1 | 37.32 | 20.45 | 28.57 | 87.73 | |
| ContrieverAvg. Tokens=12,0452026.04 | 63.54 | 31.82 | 7.51 | 33.42 | 18.91 | 26.53 | 87.01 | |
| HippoRAG 2Avg. Tokens=13,5832026.04 | 63.54 | 35.64 | 11.05 | 37.61 | 20.37 | 28.76 | 87.7 | |
| MPCAvg. Tokens=12,2892026.04 | 61.81 | 31.52 | 7.97 | 33.56 | 17.27 | 25.2 | 86.51 | |
| RAPTORAvg. Tokens=10,6312026.04 | 59.72 | 37.69 | 13.47 | 40.08 | 21.68 | 30.88 | 88.38 | |
| RecurSumAvg. Tokens=13,5272026.04 | 24.65 | 26.58 | 6.91 | 29.23 | 11.93 | 20.9 | 86.11 |