Conversational Tool-use on τ²-Bench (Domain Scores)
75.5Airline Success Ratedeepseek-v4-flash
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| deepseek-v4-flashSystem=Tool Calling2026.05 | 75.5 | 80.9 | 100 | 90 | 95.4 | |
| deepseek-v4-flashSystem=LACUNA2026.05 | 70 | 58.8 | 88.6 | 76 | 83.3 | |
| gemini-3.1-flash-liteSystem=Tool Calling2026.05 | 56 | 52.6 | 15.8 | 33.2 | 21.1 | |
| EnvFactoryBackbone=Qwen3-8B, Env. (Data Scale)=85, Tasks (Data Scale)=2,5752026.05 | 44 | 43.86 | 13.16 | 33.67 | — | |
| EnvFactory (SFT)Backbone=Qwen3-8B, Env. (Data Scale)=85, Tasks (Data Scale)=1,6222026.05 | 42 | 43.86 | 12.28 | 32.71 | — | |
| EnvScalerBackbone=Qwen3-8B, Env. (Data Scale)=191, Tasks (Data Scale)=11,5722026.05 | 38 | 49.12 | 15.79 | 34.3 | — | |
| gemini-3.1-flash-liteSystem=LACUNA2026.05 | 38 | 43.9 | 14.9 | 29.1 | 24.6 | |
| EnvScalerBackbone=Qwen3-4B, Env. (Data Scale)=191, Tasks (Data Scale)=11,5722026.05 | 36 | 41.23 | 10.53 | 29.25 | — | |
| EnvFactoryBackbone=Qwen3-4B, Env. (Data Scale)=85, Tasks (Data Scale)=2,5752026.05 | 36 | 42.11 | 12.28 | 30.13 | — | |
| BaseBackbone=Qwen3-8B, Env. (Data Scale)=–, Tasks (Data Scale)=–2026.05 | 32 | 42.98 | 21.93 | 32.3 | — | |
| AWMBackbone=Qwen3-8B, Env. (Data Scale)=526, Tasks (Data Scale)=3,3152026.05 | 30 | 29.82 | 25.44 | 28.42 | — | |
| BaseBackbone=Qwen3-4B, Env. (Data Scale)=–, Tasks (Data Scale)=–2026.05 | 24 | 38.6 | 13.16 | 25.25 | — | |
| EnvFactory (SFT)Backbone=Qwen3-4B, Env. (Data Scale)=85, Tasks (Data Scale)=1,6222026.05 | 24 | 47.37 | 4.39 | 25.25 | — | |
| AWMBackbone=Qwen3-4B, Env. (Data Scale)=526, Tasks (Data Scale)=3,3152026.05 | 18 | 31.58 | 17.54 | 22.37 | — | |
| EnvFactory (SFT)Backbone=Qwen3-1.7B, Env. (Data Scale)=85, Tasks (Data Scale)=1,6222026.05 | 16 | 20.18 | 10.53 | 15.57 | — | |
| BaseBackbone=Qwen3-1.7B, Env. (Data Scale)=–, Tasks (Data Scale)=–2026.05 | 14 | 7.02 | 22.81 | 14.61 | — | |
| EnvScalerBackbone=Qwen3-1.7B, Env. (Data Scale)=191, Tasks (Data Scale)=11,5722026.05 | 12 | 18.42 | 10.53 | 13.65 | — | |
| EnvFactoryBackbone=Qwen3-1.7B, Env. (Data Scale)=85, Tasks (Data Scale)=2,5752026.05 | 12 | 16.67 | 16.67 | 15.11 | — |