Web-based Reasoning on WebShop
34.8Average Reasoning Length (tokens)Teacher (LLaMA-13B)
Evaluation results
| Method | Links | |
|---|---|---|
| Teacher (LLaMA-13B)Distillation Method=None, Model Family=LLaMA, Model Size=13B, Teacher Model=N/A2025.05 | 34.8 | |
| Ours (LLaMA-7B)Distillation Method=Structured Agent Distillation, Model Family=LLaMA, Model Size=7B, Teacher Model=LLaMA-13B2025.05 | 34.9 | |
| Ours (OPT-6.7B)Distillation Method=Structured Agent Distillation, Model Family=OPT, Model Size=6.7B, Teacher Model=OPT-13B2025.05 | 35.1 | |
| Teacher (OPT-13B)Distillation Method=None, Model Family=OPT, Model Size=13B, Teacher Model=N/A2025.05 | 35.9 | |
| Ours (OPT-2.7B)Distillation Method=Structured Agent Distillation, Model Family=OPT, Model Size=2.7B, Teacher Model=OPT-13B2025.05 | 36.2 | |
| Token-level-OPT-6.7BDistillation Method=Token-level, Model Family=OPT, Model Size=6.7B, Teacher Model=OPT-13B2025.05 | 37.2 | |
| Token-level-7BDistillation Method=Token-level, Model Family=LLaMA, Model Size=7B, Teacher Model=LLaMA-13B2025.05 | 37.5 | |
| SeqKD (7B)Distillation Method=SeqKD, Model Family=LLaMA, Model Size=7B, Teacher Model=LLaMA-13B2025.05 | 37.8 | |
| Ours (OPT-1.3B)Distillation Method=Structured Agent Distillation, Model Family=OPT, Model Size=1.3B, Teacher Model=OPT-13B2025.05 | 38 | |
| SeqKD (6.7B)Distillation Method=SeqKD, Model Family=OPT, Model Size=6.7B, Teacher Model=OPT-13B2025.05 | 38.2 | |
| KD (7B)Distillation Method=KD, Model Family=LLaMA, Model Size=7B, Teacher Model=LLaMA-13B2025.05 | 38.9 | |
| Token-level-2.7BDistillation Method=Token-level, Model Family=OPT, Model Size=2.7B, Teacher Model=OPT-13B2025.05 | 39 | |
| KD (6.7B)Distillation Method=KD, Model Family=OPT, Model Size=6.7B, Teacher Model=OPT-13B2025.05 | 39.1 | |
| SeqKD (2.7B)Distillation Method=SeqKD, Model Family=OPT, Model Size=2.7B, Teacher Model=OPT-13B2025.05 | 39.8 | |
| KD (2.7B)Distillation Method=KD, Model Family=OPT, Model Size=2.7B, Teacher Model=OPT-13B2025.05 | 40.7 | |
| Token-level-1.3BDistillation Method=Token-level, Model Family=OPT, Model Size=1.3B, Teacher Model=OPT-13B2025.05 | 41.8 | |
| SeqKD (1.3B)Distillation Method=SeqKD, Model Family=OPT, Model Size=1.3B, Teacher Model=OPT-13B2025.05 | 42.9 | |
| KD (1.3B)Distillation Method=KD, Model Family=OPT, Model Size=1.3B, Teacher Model=OPT-13B2025.05 | 44.1 |