Function Selection on TaskBench HuggingFace
77.1Function Selection AccuracyRaw Demonstrations
Evaluation Results
| Method | Links | |
|---|---|---|
| Raw DemonstrationsBackbone=GPT-4o2025.12 | 77.1 | |
| Raw DemonstrationsBackbone=Qwen3 14B2025.12 | 76.7 | |
| No ICLBackbone=GPT-4o2025.12 | 74.8 | |
| No ICLBackbone=Qwen3 14B2025.12 | 74.4 | |
| Raw DemonstrationsBackbone=Qwen3 8B2025.12 | 72.5 | |
| DTDR-CBackbone=GPT-4o2025.12 | 71.9 | |
| DTDR-LBackbone=GPT-4o2025.12 | 71.8 | |
| DTDR-CBackbone=Qwen3 14B2025.12 | 71.5 | |
| DTDR-LBackbone=Qwen3 14B2025.12 | 71.2 | |
| No ICLBackbone=Qwen3 8B2025.12 | 70.5 | |
| Dependency RetrievalBackbone=GPT-4o2025.12 | 70 | |
| Dependency RetrievalBackbone=Qwen3 14B2025.12 | 69.2 | |
| Dependency RetrievalBackbone=Qwen3 8B2025.12 | 67.3 | |
| DTDR-CBackbone=Qwen3 8B2025.12 | 67.3 | |
| DTDR-LBackbone=Qwen3 8B2025.12 | 63.8 | |
| DTDR-LBackbone=Qwen3 4B2025.12 | 60.5 | |
| Learned RetrievalBackbone=Qwen3 4B2025.12 | 56.9 | |
| DTDR-CBackbone=Qwen3 4B2025.12 | 56 | |
| Learned RetrievalBackbone=GPT-4o2025.12 | 55.2 | |
| Learned RetrievalBackbone=Qwen3 14B2025.12 | 53.5 | |
| DTDR-CBackbone=Qwen3 1.7B2025.12 | 52.8 | |
| Dependency RetrievalBackbone=Qwen3 4B2025.12 | 52.8 | |
| Learned RetrievalBackbone=Qwen3 8B2025.12 | 52 | |
| Raw DemonstrationsBackbone=Qwen3 4B2025.12 | 49.8 | |
| DTDR-LBackbone=Qwen3 1.7B2025.12 | 49 | |
| No ICLBackbone=Qwen3 4B2025.12 | 45.8 | |
| Raw DemonstrationsBackbone=Qwen3 1.7B2025.12 | 43.1 | |
| Learned RetrievalBackbone=Qwen3 1.7B2025.12 | 42.8 | |
| Dependency RetrievalBackbone=Qwen3 1.7B2025.12 | 41.6 | |
| No ICLBackbone=Qwen3 1.7B2025.12 | 31.2 | |
| Dynamic LR (DTDR-L)LLM Backbone=Qwen 3 0.6B, ICL Method=hard masking2025.12 | 27.8 | |
| Dynamic DR (DTDR-C)LLM Backbone=Qwen 3 0.6B, ICL Method=hard masking2025.12 | 27.5 | |
| DTDR-CBackbone=Qwen3 0.6B2025.12 | 20.8 | |
| LR (Erdogan et al. 2024)LLM Backbone=Qwen 3 0.6B, ICL Method=hard masking2025.12 | 20.5 | |
| DTDR-LBackbone=Qwen3 0.6B2025.12 | 18.9 | |
| DR (Liu et al. 2024a)LLM Backbone=Qwen 3 0.6B, ICL Method=hard masking2025.12 | 17.1 | |
| QTS (Gao et al. 2025)LLM Backbone=Qwen 3 0.6B, ICL Method=hard masking2025.12 | 14.6 | |
| QTS (Paramanayakam et al. 2025)LLM Backbone=Qwen 3 0.6B, ICL Method=hard masking2025.12 | 12.9 | |
| Learned RetrievalBackbone=Qwen3 0.6B2025.12 | 10 | |
| BM-25LLM Backbone=Qwen 3 0.6B, ICL Method=hard masking2025.12 | 8.3 | |
| QTS (Vanilla)LLM Backbone=Qwen 3 0.6B, ICL Method=hard masking2025.12 | 7.9 | |
| Dependency RetrievalBackbone=Qwen3 0.6B2025.12 | 6.7 | |
| No ICLBackbone=Qwen3 0.6B2025.12 | 5.5 | |
| Raw DemonstrationsBackbone=Qwen3 0.6B2025.12 | 4.3 | |
| Random GuessLLM Backbone=Qwen 3 0.6B, ICL Method=hard masking2025.12 | 4.2 |