Multimodal app-use reasoning on AppWorld
0.05CostStudent (ZS)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Student (ZS)Model=Llama-3.3-70B, Zero-shot=true2025.12 | 0.05 | 11 | |
| Student (ZS)Model=GPT-4.1-mini, Zero-shot=true2025.12 | 0.09 | 28 | |
| Student (IC)Model=Llama-3.3-70B, In-Context Distillation=true2025.12 | 0.1 | 32 | |
| Student (IC)Model=GPT-4.1-mini, In-Context Distillation=true2025.12 | 0.15 | 55 | |
| Student (IC+Cascade)Model=Llama-3.3-70B, In-Context Distillation=true, Self-Consistency Cascade=true2025.12 | 0.19 | 44 | |
| Student (IC+Cascade)Model=GPT-4.1-mini, In-Context Distillation=true, Self-Consistency Cascade=true2025.12 | 0.29 | 66 | |
| TeacherModel=Sonnet 4.52025.12 | 1 | 83 |