List your company
ResearchFormerly WizWand

Sequential Decision Making on ALFWorld (test)

96.27Success RateSMaRT
40.349254.867169.38583.9029May 20, 2025Jun 14, 2025Jul 10, 2025Aug 4, 2025Aug 30, 2025Sep 24, 2025Oct 20, 2025
Updated 7mo ago

Evaluation results

MethodLinks
SMaRTLLM=GPT-4, Shot setup=3-shot2025.1096.27
LLM-as-a-JudgeLLM=GPT-4, Shot setup=3-shot2025.1091.79
CoTLLM=GPT-4, Shot setup=3-shot2025.1088.81
SMaRTLLM=Gemini-1.5, Shot setup=3-shot2025.1083.58
LLM-as-a-JudgeLLM=Gemini-1.5, Shot setup=3-shot2025.1080.59
DirectLLM=GPT-4, Shot setup=3-shot2025.1076.87
OPTRole=Teacher, Size=13B2025.0576.5
LLaMARole=Teacher, Size=13B2025.0575.3
CoTLLM=Gemini-1.5, Shot setup=3-shot2025.1070.89
Structured Agent DistillationTeacher=LLaMA-13B, Student Size=7B2025.0568
Structured Agent DistillationTeacher=OPT-13B, Student Size=6.7B2025.0567.1
Token-level baselineTeacher=LLaMA-13B, Student Size=7B2025.0564.2
Token-level baselineTeacher=OPT-13B, Student Size=6.7B2025.0562.8
SeqKDTeacher=LLaMA-13B, Student Size=7B2025.0562
KDTeacher=LLaMA-13B, Student Size=7B2025.0560.1
Structured Agent DistillationTeacher=OPT-13B, Student Size=2.7B2025.0559.2
SeqKDTeacher=OPT-13B, Student Size=6.7B2025.0559
KDTeacher=OPT-13B, Student Size=6.7B2025.0557.2
DirectLLM=Gemini-1.5, Shot setup=3-shot2025.1055.97
Token-level baselineTeacher=OPT-13B, Student Size=2.7B2025.0555.6
Structured Agent DistillationTeacher=OPT-13B, Student Size=1.3B2025.0552.3
SeqKDTeacher=OPT-13B, Student Size=2.7B2025.0551
KDTeacher=OPT-13B, Student Size=2.7B2025.0549.6
Token-level baselineTeacher=OPT-13B, Student Size=1.3B2025.0547.8
SeqKDTeacher=OPT-13B, Student Size=1.3B2025.0544
KDTeacher=OPT-13B, Student Size=1.3B2025.0542.5