Accurate Retrieval (AR) suite benchmark leaderboard · SOTA2 Research

Accurate Retrieval (AR) suite

881.7Convo ScoreUMA

Evaluation Results

Method	Links
UMA2026.02		881.7	77.34	50.28	54.8	68	82.5	87.32	77.36
UMA2026.02		866.1	71.09	47.43	51.4	28.8	69.75	83.71	63.81
UMA2026.02		854.9	66.41	47.99	50.8	66.2	62.5	80.71	61.38
UMA2026.02		854.9	69.53	45.97	52.2	67.6	71.5	66.45	66.29
UMA2026.02		837.1	76.56	61.98	44	75.4	83	80.92	73.58
UMA2026.02		799.1	62.5	54.88	41	38.2	65.5	82.23	45.26
MemAgent2026.02		602.7	51.56	64.15	59.4	53	71.5	78.06	50.09
RAG2026.02		513.4	77.34	57.35	61.2	55.8	85.25	25.56	54.09
MemAlpha2026.02		84.8	78.91	48.64	55.6	56.6	79.5	83.4	64.5
Concat2026.02		53.6	65.62	33.43	33	61.2	42.5	64.42	51.87
UMA2026.02		46.39	66.16	40.5	41.52	53.28	33.14	72.04	65.4
UMA2026.02		46.33	65.42	40.96	41.73	52.59	32.88	71.37	65.27
UMA (2 Stage)2026.02		44.4	59.47	32.72	37.4	50.89	29.36	50.59	54.63
UMA2026.02		44.36	58.73	33.03	37.53	49.98	28.86	49.78	54.43
UMA2026.02		44.21	59.53	34.57	36.66	21.08	28.22	63.18	52.28
UMA (w/o Phase I)2026.02		44.16	59.34	34.15	36.51	21.29	28.57	64.2	52.34
UMA2026.02		44.01	56.67	40.18	27.25	52.04	30.12	50.82	56.33
UMA (Global Group)2026.02		43.91	57.61	39.85	27.33	52.41	29.97	51.05	56.37
UMA2026.02		43.38	56.81	32.96	36.32	46.73	24.01	60.38	49.31
UMA (w/o RL)2026.02		43.28	56.79	32.87	36.17	48.05	24.14	61.38	49.47
UMA2026.02		40.94	52.73	36.7	28.82	28.36	26.01	62.08	31.56
UMA (w/o RL & Phase I)2026.02		40.86	52.78	36.34	28.76	29.18	26.24	63.03	31.68
RAG2026.02		38.66	15.57	16.9	17.62	23.33	13.92	5.29	17.9
RAG2026.02		36.55	15.47	16.55	16.3	23.17	13.92	5.4	17.6
MemAgent2026.02		33.55	26.34	21.96	27.11	25.11	8.42	14.88	26.29
MemAgent2026.02		32.31	25.71	21.51	26.59	25.17	8.35	15.15	26.1
MemAgent-woq2026.02		26.8	14.84	29.71	9.8	29.6	26.75	25.8	30.94
MemAlpha2026.02		16.18	8.51	5.5	9.33	9.58	4.37	5.18	4.75
Mem12026.02		15.6	6.25	22.41	8	9.6	7.75	18.43	10.03
MemAlpha2026.02		14.33	8.49	5.26	8.43	9.71	4.29	5.22	4.53
Mem12026.02		14.22	1.66	3.37	5.6	5.69	1.44	1.48	3.45
MemAgent-woq2026.02		12.13	2.71	2.96	5.62	8.77	2.22	3.38	19.41
Mem12026.02		11.63	1.77	3.28	4.7	5.36	1.54	1.54	3.17
MemAgent-woq2026.02		10.8	2.74	2.88	4.81	8.78	2.33	3.44	19.25
Concat2026.02		8.65	49.27	25.89	27.84	48.21	18	53.57	45.94
Concat2026.02		8.63	50.44	25.97	27.76	48.79	18.36	54	46.13