Long context memory management on LongMemEval
68.5PrecisionOSL-MR
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| OSL-MRBudget=10242026.06 | 68.5 | 59.5 | 53.6 | 33.7 | 1,957.9 | |
| BC-learnerBudget=10242026.06 | 61.6 | 57.2 | 51.5 | 37 | 1,772.8 | |
| OSL-MRBudget=2562026.06 | 52.9 | 68.1 | 51.8 | 71.1 | 1,626.8 | |
| OSL-MRBudget=5122026.06 | 31.5 | 76.9 | 38.5 | 85.1 | 1,585.8 | |
| BC-learnerBudget=2562026.06 | 20.3 | 38.1 | 23 | 90.4 | 1,119.9 | |
| BC-learnerBudget=5122026.06 | 17.3 | 60.1 | 23.5 | 94.6 | 1,187.9 | |
| Mixed-ScoreBudget=2562026.06 | 5.2 | 32.4 | 8.2 | 99.4 | 1,062.1 | |
| Mixed-ScoreBudget=5122026.06 | 4.7 | 47.7 | 7.8 | 99.7 | 1,062.8 | |
| Mixed-ScoreBudget=10242026.06 | 3.3 | 53.3 | 5.8 | 99.8 | 718.9 | |
| GA-heuristicBudget=10242026.06 | 1.4 | 23.2 | 2.4 | 99.6 | 117 | |
| GA-heuristicBudget=5122026.06 | 1.2 | 13.1 | 2 | 99.3 | 489.3 | |
| GA-LLMBudget=10242026.06 | 1 | 16.8 | 1.7 | 99.6 | 44.2 | |
| GA-LLMBudget=5122026.06 | 0.9 | 10.5 | 1.4 | 99.1 | 461.7 | |
| RecencyBudget=10242026.06 | 0.8 | 11.7 | 1.3 | 99.8 | 0 | |
| GA-heuristicBudget=2562026.06 | 0.7 | 2.9 | 0.8 | 98.9 | 653 | |
| RecencyBudget=5122026.06 | 0.6 | 5.7 | 0.8 | 99.5 | 419.7 | |
| RecencyBudget=2562026.06 | 0.3 | 0.9 | 0.3 | 99 | 629.9 | |
| GA-LLMBudget=2562026.06 | 0.3 | 1.6 | 0.4 | 98.4 | 638 |