Accurate Retrieval (AR) suite
881.7Convo ScoreUMA
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| UMA2026.02 | 881.7 | 77.34 | 50.28 | 54.8 | 68 | 82.5 | 87.32 | 77.36 | |
| UMAPhase I (sequential memory maintenance)=false2026.02 | 866.1 | 71.09 | 47.43 | 51.4 | 28.8 | 69.75 | 83.71 | 63.81 | |
| UMARL training=false2026.02 | 854.9 | 66.41 | 47.99 | 50.8 | 66.2 | 62.5 | 80.71 | 61.38 | |
| UMAtraining stages=2 Stage2026.02 | 854.9 | 69.53 | 45.97 | 52.2 | 67.6 | 71.5 | 66.45 | 66.29 | |
| UMAtask grouping strategy=Global Group2026.02 | 837.1 | 76.56 | 61.98 | 44 | 75.4 | 83 | 80.92 | 73.58 | |
| UMARL training=false, Phase I (sequential memory maintenance)=false2026.02 | 799.1 | 62.5 | 54.88 | 41 | 38.2 | 65.5 | 82.23 | 45.26 | |
| MemAgent2026.02 | 602.7 | 51.56 | 64.15 | 59.4 | 53 | 71.5 | 78.06 | 50.09 | |
| RAGk=202026.02 | 513.4 | 77.34 | 57.35 | 61.2 | 55.8 | 85.25 | 25.56 | 54.09 | |
| MemAlpha2026.02 | 84.8 | 78.91 | 48.64 | 55.6 | 56.6 | 79.5 | 83.4 | 64.5 | |
| Concat2026.02 | 53.6 | 65.62 | 33.43 | 33 | 61.2 | 42.5 | 64.42 | 51.87 | |
| UMA2026.02 | 46.39 | 66.16 | 40.5 | 41.52 | 53.28 | 33.14 | 72.04 | 65.4 | |
| UMAIs Proposed Full Method=true2026.02 | 46.33 | 65.42 | 40.96 | 41.73 | 52.59 | 32.88 | 71.37 | 65.27 | |
| UMA (2 Stage)Processing Stages=22026.02 | 44.4 | 59.47 | 32.72 | 37.4 | 50.89 | 29.36 | 50.59 | 54.63 | |
| UMAStages=22026.02 | 44.36 | 58.73 | 33.03 | 37.53 | 49.98 | 28.86 | 49.78 | 54.43 | |
| UMAPhase I=false2026.02 | 44.21 | 59.53 | 34.57 | 36.66 | 21.08 | 28.22 | 63.18 | 52.28 | |
| UMA (w/o Phase I)Phase I=false2026.02 | 44.16 | 59.34 | 34.15 | 36.51 | 21.29 | 28.57 | 64.2 | 52.34 | |
| UMAGrouping=Global2026.02 | 44.01 | 56.67 | 40.18 | 27.25 | 52.04 | 30.12 | 50.82 | 56.33 | |
| UMA (Global Group)Grouping Strategy=Global2026.02 | 43.91 | 57.61 | 39.85 | 27.33 | 52.41 | 29.97 | 51.05 | 56.37 | |
| UMAReinforcement Learning=false2026.02 | 43.38 | 56.81 | 32.96 | 36.32 | 46.73 | 24.01 | 60.38 | 49.31 | |
| UMA (w/o RL)Reinforcement Learning=false2026.02 | 43.28 | 56.79 | 32.87 | 36.17 | 48.05 | 24.14 | 61.38 | 49.47 | |
| UMAReinforcement Learning=false, Phase I=false2026.02 | 40.94 | 52.73 | 36.7 | 28.82 | 28.36 | 26.01 | 62.08 | 31.56 | |
| UMA (w/o RL & Phase I)Reinforcement Learning=false, Phase I=false2026.02 | 40.86 | 52.78 | 36.34 | 28.76 | 29.18 | 26.24 | 63.03 | 31.68 | |
| RAGk=202026.02 | 38.66 | 15.57 | 16.9 | 17.62 | 23.33 | 13.92 | 5.29 | 17.9 | |
| RAGk=202026.02 | 36.55 | 15.47 | 16.55 | 16.3 | 23.17 | 13.92 | 5.4 | 17.6 | |
| MemAgent2026.02 | 33.55 | 26.34 | 21.96 | 27.11 | 25.11 | 8.42 | 14.88 | 26.29 | |
| MemAgent2026.02 | 32.31 | 25.71 | 21.51 | 26.59 | 25.17 | 8.35 | 15.15 | 26.1 | |
| MemAgent-woq2026.02 | 26.8 | 14.84 | 29.71 | 9.8 | 29.6 | 26.75 | 25.8 | 30.94 | |
| MemAlpha2026.02 | 16.18 | 8.51 | 5.5 | 9.33 | 9.58 | 4.37 | 5.18 | 4.75 | |
| Mem12026.02 | 15.6 | 6.25 | 22.41 | 8 | 9.6 | 7.75 | 18.43 | 10.03 | |
| MemAlpha2026.02 | 14.33 | 8.49 | 5.26 | 8.43 | 9.71 | 4.29 | 5.22 | 4.53 | |
| Mem12026.02 | 14.22 | 1.66 | 3.37 | 5.6 | 5.69 | 1.44 | 1.48 | 3.45 | |
| MemAgent-woqwithout question=true2026.02 | 12.13 | 2.71 | 2.96 | 5.62 | 8.77 | 2.22 | 3.38 | 19.41 | |
| Mem12026.02 | 11.63 | 1.77 | 3.28 | 4.7 | 5.36 | 1.54 | 1.54 | 3.17 | |
| MemAgent-woq2026.02 | 10.8 | 2.74 | 2.88 | 4.81 | 8.78 | 2.33 | 3.44 | 19.25 | |
| Concat2026.02 | 8.65 | 49.27 | 25.89 | 27.84 | 48.21 | 18 | 53.57 | 45.94 | |
| Concat2026.02 | 8.63 | 50.44 | 25.97 | 27.76 | 48.79 | 18.36 | 54 | 46.13 |