Embodied AI Task Planning on EB-ALFRED

82Average ScoreA-Mem

Evaluation Results

Method
A-Mem2026.05	82	90	84	86	78	82	72
G-Mem2026.05	79.27	82	80	78	82	76	78
LangMem2026.05	78	82	78	84	76	76	72
Mem02026.05	77.67	82	82	80	80	74	68
Gemini-1.5-pro2026.03	75.3	88	80	68	80	72	64
GPT-4o2026.03	75	82	82	64	78	72	72
Claude-3.5-Sonnet2026.03	74.7	78	84	66	72	64	84
ELITE2026.03	70.8	78	78	68	68	62	—
Claude-3.7-Sonnet2026.04	67.7	68	70	68	68	62	70
Claude-3.7-Sonnet2026.03	67.7	68	70	68	68	62	70
RoboAgent2026.04	67	72	64	78	48	60	80
Claude-3.5-Sonnet2026.03	66.4	72	76	60	66	58	—
ERA (3B)2026.03	65.2	72	72	62	54	66	—
GPT-4.12025.10	64.67	70	70	62	64	62	60
Claude-3.5-Sonnet2026.03	64	72	76	60	66	58	52
Gemini-1.5-Pro2026.03	63.2	70	72	58	64	52	—
WAP2026.04	62.7	66	70	56	62	52	70
Gemini-1.5-Pro2026.04	62.3	70	72	58	64	52	58
Gemini-1.5-Pro2026.03	62.3	70	72	58	64	52	58
No Mem2026.05	59	70	60	68	56	56	44
GPT-4o2026.03	56.8	64	68	46	54	52	—
GPT-4o2026.04	56.3	64	68	46	54	52	54
GPT-4o2026.03	56.3	64	68	46	54	52	54
RoboGPT-R12025.10	55.33	62	64	50	56	50	50
RoboGPT-R12026.04	55.3	62	64	50	56	50	50
VAGEN (3B)2026.03	52.8	70	70	58	38	28	—
Gemini-2.0-flash2025.10	52.3	62	54	46	48	46	58
GPT-4o2025.10	51.67	54	58	52	46	52	48
Gemini-2.0-flash2026.03	51.2	62	54	46	48	46	—
RL4VLM (3B)2026.03	51.2	70	70	56	32	28	—
Vlaser2026.04	50	—	—	—	—	—	—
Qwen2.5-VL-72B-Ins2026.03	49.8	55	55	50	42	47	—
LangMem2026.05	46.67	74	62	46	56	38	4
A-Mem2026.05	46.33	52	48	34	54	42	48
G-Mem2026.05	45.33	46	60	42	58	42	24
Qwen2.5-VL-72B-Ins2025.10	43.67	62	48	40	36	44	32
Qwen-VL-Max2026.03	43.2	44	44	42	48	38	—
Mem02026.05	43.15	48	60	47	58	36	10
Reinforced Reasoner (7B)2026.03	41.6	54	46	28	42	38	—
Qwen-VL-Max2026.04	41.3	44	44	42	48	38	32
Qwen-VL-Max2026.03	41.3	44	44	42	48	38	32
Qwen-VL-Max2025.10	41.3	44	44	42	48	38	32
MemCompiler2026.05	40	52	48	38	54	38	10
Qwen2.5-VL-72B2026.04	39.7	50	42	36	42	34	34
Qwen2.5-VL-72B2026.03	39.7	50	42	36	42	34	34
InternVL3-78B2026.03	39.6	38	46	42	34	38	—
Gemini-1.5-flash2026.03	39.3	44	56	42	40	26	28
gemma-3-27b-it2026.03	39.2	42	48	30	40	36	—
InternVL3-78B2026.03	39	38	46	42	34	38	36
LoHo-Manip-4B2026.04	38	48	54	28	36	30	31
InternVL2_5-78B2026.03	37.7	38	42	34	34	36	42
InternVL2 5-78B2026.03	36.8	38	42	34	34	36	—
REBP2026.04	35.6	54	46	28	42	38	6
MemCompiler2026.05	35.33	42	44	38	50	34	4
Llama-3.2-90B-Vision2026.03	35.2	38	44	28	34	32	—
REBP2025.10	35	52	46	28	46	32	6
Qwen2-VL-72B-Ins2026.03	34.4	40	40	30	30	32	—
GPT-4o-mini2025.10	34	66	68	14	56	0	0
A-Mem2026.05	32.33	46	34	30	50	30	4
Llama-3.2-90B-Vision2026.03	32	38	44	28	34	32	16
No Mem2026.05	32	60	28	36	40	28	0
Llama-3.2-90B-Vision-Ins2025.10	32	38	44	28	34	32	16
Langmem2026.05	31.33	38	44	28	44	26	8
Mem02026.05	30.33	40	40	26	42	26	8
Ovis2-34B2026.03	29.6	34	38	28	30	18	—
Ovis2-34B2026.03	28.7	34	38	28	30	18	24
Gemma-3-12b-its2025.10	25.7	32	38	26	26	20	12
G-Mem2026.05	25.33	34	38	24	36	14	6
GPT-4o-mini2026.03	24	34	36	24	28	22	0
GPT-4o mini2026.04	24	34	36	24	28	22	0
Qwen3-VL-4B2026.04	19	18	26	14	14	14	26
No Mem2026.05	19	24	28	24	24	14	0
No Mem2026.05	18	22	24	12	26	22	2
Mem02026.05	14.66	20	26	12	18	10	2
Langmem2026.05	14.33	22	26	6	20	12	0
MemGen2026.05	14.33	14	16	6	10	8	0
MemGen2026.05	14.22	22	16	6	18	20	4
RoboBrain2.02026.04	14	—	—	—	—	—	—
Llama3.2-11B2026.04	14	24	16	22	8	6	6
InternVL3-8B2026.04	10	20	14	12	14	0	2
G-Mem2026.05	7.67	4	6	16	14	6	0
A-Mem2026.05	3	2	4	6	4	2	0
Qwen2.5-VL-7B-Ins2025.10	2.67	6	6	0	2	2	0
InternVL2.5-8B2025.10	2	4	2	0	6	0	0
Qwen2.5-VL-3B-Ins2025.10	1.33	2	0	0	2	4	0
RoboBrain2025.10	0.33	2	0	0	0	0	0
Tapa2025.10	0	0	0	0	0	0	0