Tool-use Agent Tasks on TinyAgent (500 samples, evaluation)
66.8AccuracyBaseline (normal SFT)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Baseline (normal SFT)Model=Llama-3.2-3B2026.05 | 66.8 | 5 | — | |
| Baseline (normal SFT)Model=Qwen2.5-3B2026.05 | 65.6 | 4.1 | — | |
| AsyncIO (Async-SFT)Model=Llama-3.2-3B2026.05 | 65.2 | 2.5 | — | |
| AsyncIO (Async-SFT)Model=Qwen2.5-3B2026.05 | 62.1 | 2.5 | — | |
| BaselineModel=openai-realtime-1.52026.05 | 54.9 | 7.6 | — | |
| AsyncIOModel=openai-realtime-1.52026.05 | 53.2 | 4.4 | 1.7 | |
| AsyncIO (normal SFT)Model=Qwen2.5-3B2026.05 | 14.3 | — | — | |
| AsyncIO (normal SFT)Model=Llama-3.2-3B2026.05 | 10.8 | — | — | |
| Baseline (no SFT)Model=Qwen2.5-3B2026.05 | 3.2 | — | — | |
| AsyncIO (no SFT)Model=Qwen2.5-3B2026.05 | 2.3 | — | — | |
| AsyncIO (no SFT)Model=Llama-3.2-3B2026.05 | 2.1 | — | — | |
| Baseline (no SFT)Model=Llama-3.2-3B2026.05 | 1.5 | — | — |