Evidence-round recall on LongMemEval 100 question sample
100Evidence-round Recalllc-mmr
Evaluation results
| Method | Links | |
|---|---|---|
| lc-mmrBudget=70%, Redundancy level (R)=82026.06 | 100 | |
| PACMSBudget=70%, Redundancy level (R)=82026.06 | 100 | |
| lc-mmrBudget=70%, Redundancy level (R)=42026.06 | 99.9 | |
| PACMSBudget=70%, Redundancy level (R)=42026.06 | 99.9 | |
| lc-mmrBudget=45%, Redundancy level (R)=82026.06 | 99.4 | |
| lc-mmrBudget=70%, Redundancy level (R)=22026.06 | 99.3 | |
| PACMSBudget=70%, Redundancy level (R)=22026.06 | 99.1 | |
| top-kBudget=70%, Redundancy level (R)=42026.06 | 98.2 | |
| top-kBudget=70%, Redundancy level (R)=82026.06 | 98.2 | |
| top-kBudget=70%, Redundancy level (R)=22026.06 | 97.8 | |
| top-kBudget=70%, Redundancy level (R)=02026.06 | 97.4 | |
| PACMSBudget=45%, Redundancy level (R)=82026.06 | 97.2 | |
| lc-mmrBudget=45%, Redundancy level (R)=42026.06 | 96.6 | |
| PACMSBudget=45%, Redundancy level (R)=42026.06 | 95.2 | |
| top-kBudget=45%, Redundancy level (R)=82026.06 | 94 | |
| PACMSBudget=70%, Redundancy level (R)=02026.06 | 94 | |
| top-kBudget=45%, Redundancy level (R)=42026.06 | 93.8 | |
| top-kBudget=45%, Redundancy level (R)=22026.06 | 93.3 | |
| top-kBudget=45%, Redundancy level (R)=02026.06 | 92.8 | |
| lc-mmrBudget=70%, Redundancy level (R)=02026.06 | 92.2 | |
| PACMSBudget=45%, Redundancy level (R)=22026.06 | 90.9 | |
| lc-mmrBudget=45%, Redundancy level (R)=22026.06 | 89.5 | |
| PACMSBudget=45%, Redundancy level (R)=02026.06 | 86 | |
| lc-mmrBudget=20%, Redundancy level (R)=82026.06 | 83.6 | |
| top-kBudget=20%, Redundancy level (R)=82026.06 | 81.7 | |
| top-kBudget=20%, Redundancy level (R)=42026.06 | 80.8 | |
| PACMSBudget=20%, Redundancy level (R)=82026.06 | 80.4 | |
| top-kBudget=20%, Redundancy level (R)=22026.06 | 80 | |
| top-kBudget=20%, Redundancy level (R)=02026.06 | 79 | |
| PACMSBudget=20%, Redundancy level (R)=42026.06 | 76 | |
| last-kBudget=70%, Redundancy level (R)=02026.06 | 74.4 | |
| lc-mmrBudget=45%, Redundancy level (R)=02026.06 | 74.3 | |
| PACMSBudget=20%, Redundancy level (R)=22026.06 | 73.7 | |
| PACMSBudget=20%, Redundancy level (R)=02026.06 | 70.4 | |
| lc-mmrBudget=20%, Redundancy level (R)=4, Lambda (λ)=0.52026.06 | 66.8 | |
| last-kBudget=70%, Redundancy level (R)=22026.06 | 62 | |
| lc-mmrBudget=20%, Redundancy level (R)=2, Lambda (λ)=0.52026.06 | 55.2 | |
| last-kBudget=45%, Redundancy level (R)=02026.06 | 52.1 | |
| last-kBudget=70%, Redundancy level (R)=42026.06 | 48.9 | |
| lc-mmrBudget=20%, Redundancy level (R)=0, Lambda (λ)=0.52026.06 | 42.1 | |
| last-kBudget=20%, Redundancy level (R)=02026.06 | 23.8 | |
| last-kBudget=45%, Redundancy level (R)=22026.06 | 22.1 | |
| last-kBudget=70%, Redundancy level (R)=82026.06 | 13.9 | |
| last-kBudget=45%, Redundancy level (R)=42026.06 | 0.3 | |
| last-kBudget=45%, Redundancy level (R)=82026.06 | 0.3 | |
| last-kBudget=20%, Redundancy level (R)=82026.06 | 0.1 | |
| last-kBudget=20%, Redundancy level (R)=22026.06 | 0 | |
| last-kBudget=20%, Redundancy level (R)=42026.06 | 0 |