List your company
ResearchFormerly WizWand

Embodied Task Completion on ALFRED EB

92Avg ScoreClaude-4.6
1.20824.77948.3571.921Jan 28, 2026Feb 16, 2026Mar 8, 2026Mar 28, 2026Apr 16, 2026May 6, 2026May 26, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Claude-4.6Tool Use=True2026.0592969694909286
Claude-3.7Tool Use=True2026.0591969692868690
Claude-3.7Tool Use=False2026.0582868280768680
GPT-5Tool Use=True2026.0581909088845478
Claude-4.6Tool Use=False2026.0581867680808878
Qwen3-32BTool Use=True2026.0577928888783878
GPT-4oTool Use=True2026.0576909088743676
Qwen3-8BTool Use=True2026.0572888688702672
Qwen3.5-35BTool Use=True2026.0570868688741272
Qwen2.5-32BTool Use=True2026.0566887488701266
GPT-5Tool Use=False2026.0562706066685256
Gemma-4B w/ CoT + Gemma-4B FT Verifier (VEGAS)Policy Type=Finetuned, Verifier Type=Finetuned (VEGAS), Base Model=Gemma-4B2026.0551675667372553
Qwen-3B w/ CoT + Qwen-3B FT Verifier (VEGAS)Policy Type=Finetuned, Verifier Type=Finetuned (VEGAS), Base Model=Qwen-3B2026.0549674662433441
Gemma-4B w/ CoTPolicy Type=Finetuned, Decoding Strategy=CoT, Base Model=Gemma-4B2026.0548625056342856
Gemma-4B w/ CoT + Gemma-4B ZS VerifierPolicy Type=Finetuned, Verifier Type=Zero-shot, Base Model=Gemma-4B2026.0548665561322749
GPT-4oTool Use=False2026.0548585252503642
Qwen-3B w/ CoTPolicy Type=Finetuned, Decoding Strategy=CoT, Base Model=Qwen-3B2026.0544624058342248
Qwen-3B w/ CoT + Qwen-3B ZS VerifierPolicy Type=Finetuned, Verifier Type=Zero-shot, Base Model=Qwen-3B2026.0544644153352446
Qwen-2.5-VL-72B + Qwen-3B FT Verifier (VEGAS)Policy Type=Zero-shot, Verifier Type=Finetuned (VEGAS), Base Model=Qwen-2.5-VL-72B2026.0538453944361547
Qwen3-32BTool Use=False2026.0537443244303836
InternVL-3.5-38B + Qwen-3B FT Verifier (VEGAS)Policy Type=Zero-shot, Verifier Type=Finetuned (VEGAS), Base Model=InternVL-3.5-38B2026.0535383639262741
Gemma-3-12B-IT + µOffline Sup.Model Backbone=Gemma-3-12B-IT, Memory Head Variation (MemCtrl)=µOffline Sup.2026.0132.24831322327—
Qwen-2.5-VL-72BPolicy Type=Zero-shot, Base Model=Qwen-2.5-VL-72B2026.0530283834381232
Gemma-3-12B-IT + µSimpleModel Backbone=Gemma-3-12B-IT, Memory Head Variation (MemCtrl)=µSimple2026.01284127341622—
Gemma-3-12B-IT + µOnline RLModel Backbone=Gemma-3-12B-IT, Memory Head Variation (MemCtrl)=µOnline RL2026.0127.83829412126—
Gemma-3-12B-ITModel Backbone=Gemma-3-12B-IT, Memory Head Variation (MemCtrl)=None2026.0125.63226382012—
InternVL-3.5-38BPolicy Type=Zero-shot, Base Model=InternVL-3.5-38B2026.0524322626181030
Qwen2.5-32BTool Use=False2026.0524342430241220
Qwen3.5-35BTool Use=False2026.0524302832221222
Gemma-3-27B + Qwen-3B FT Verifier (VEGAS)Policy Type=Zero-shot, Verifier Type=Finetuned (VEGAS), Base Model=Gemma-3-27B2026.052330292723225
Qwen3-8BTool Use=False2026.0520141828142620
Gemma-3-27BPolicy Type=Zero-shot, Base Model=Gemma-3-27B2026.051924242624018
Qwen2.5-VL-7B-Ins + µOnline RLModel Backbone=Qwen2.5-VL-7B-Ins, Memory Head Variation (MemCtrl)=µOnline RL2026.0114.2101321324—
Qwen2.5-VL-7B-Ins + µOffline Sup.Model Backbone=Qwen2.5-VL-7B-Ins, Memory Head Variation (MemCtrl)=µOffline Sup.2026.0112.291014226—
Qwen2.5-VL-7B-Ins + µSimpleModel Backbone=Qwen2.5-VL-7B-Ins, Memory Head Variation (MemCtrl)=µSimple2026.019.6151012214—
Qwen2.5-VL-7B-InsModel Backbone=Qwen2.5-VL-7B-Ins, Memory Head Variation (MemCtrl)=None2026.014.7108602—