Embodied Task Completion on ALFRED EB

92Avg ScoreClaude-4.6

Evaluation Results

Method
Claude-4.62026.05	92	96	96	94	90	92	86
Claude-3.72026.05	91	96	96	92	86	86	90
Claude-3.72026.05	82	86	82	80	76	86	80
GPT-52026.05	81	90	90	88	84	54	78
Claude-4.62026.05	81	86	76	80	80	88	78
Qwen3-32B2026.05	77	92	88	88	78	38	78
GPT-4o2026.05	76	90	90	88	74	36	76
Qwen3-8B2026.05	72	88	86	88	70	26	72
Qwen3.5-35B2026.05	70	86	86	88	74	12	72
Qwen2.5-32B2026.05	66	88	74	88	70	12	66
GPT-52026.05	62	70	60	66	68	52	56
Gemma-4B w/ CoT + Gemma-4B FT Verifier (VEGAS)2026.05	51	67	56	67	37	25	53
Qwen-3B w/ CoT + Qwen-3B FT Verifier (VEGAS)2026.05	49	67	46	62	43	34	41
Gemma-4B w/ CoT2026.05	48	62	50	56	34	28	56
Gemma-4B w/ CoT + Gemma-4B ZS Verifier2026.05	48	66	55	61	32	27	49
GPT-4o2026.05	48	58	52	52	50	36	42
Qwen-3B w/ CoT2026.05	44	62	40	58	34	22	48
Qwen-3B w/ CoT + Qwen-3B ZS Verifier2026.05	44	64	41	53	35	24	46
Qwen-2.5-VL-72B + Qwen-3B FT Verifier (VEGAS)2026.05	38	45	39	44	36	15	47
Qwen3-32B2026.05	37	44	32	44	30	38	36
InternVL-3.5-38B + Qwen-3B FT Verifier (VEGAS)2026.05	35	38	36	39	26	27	41
Gemma-3-12B-IT + µOffline Sup.2026.01	32.2	48	31	32	23	27	—
Qwen-2.5-VL-72B2026.05	30	28	38	34	38	12	32
Gemma-3-12B-IT + µSimple2026.01	28	41	27	34	16	22	—
Gemma-3-12B-IT + µOnline RL2026.01	27.8	38	29	41	21	26	—
Gemma-3-12B-IT2026.01	25.6	32	26	38	20	12	—
InternVL-3.5-38B2026.05	24	32	26	26	18	10	30
Qwen2.5-32B2026.05	24	34	24	30	24	12	20
Qwen3.5-35B2026.05	24	30	28	32	22	12	22
Gemma-3-27B + Qwen-3B FT Verifier (VEGAS)2026.05	23	30	29	27	23	2	25
Qwen3-8B2026.05	20	14	18	28	14	26	20
Gemma-3-27B2026.05	19	24	24	26	24	0	18
Qwen2.5-VL-7B-Ins + µOnline RL2026.01	14.2	10	13	21	3	24	—
Qwen2.5-VL-7B-Ins + µOffline Sup.2026.01	12.2	9	10	14	2	26	—
Qwen2.5-VL-7B-Ins + µSimple2026.01	9.6	15	10	12	2	14	—
Qwen2.5-VL-7B-Ins2026.01	4.7	10	8	6	0	2	—