Tool Call Repair on Tool-Reflection-Bench (test)
14.9Repair@1 (%)structured reflection
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| structured reflectionBackbone=Qwen3-4B-Instruct, Model Category=Open-Sourced Models2025.09 | 14.9 | 18.5 | 19.5 | |
| Qwen3-4B-InstructModel Category=Open-Sourced Models, Training=Base2025.09 | 9.6 | 10.6 | 10.6 | |
| structured reflectionBackbone=Qwen2.5-7B-Instruct, Model Category=Open-Sourced Models2025.09 | 9.3 | 10.3 | 11.4 | |
| GPT-4o-miniModel Category=Close-Sourced Models2025.09 | 6.1 | 8.7 | 9 | |
| structured reflectionBackbone=Llama-3.1-8B-Instruct, Model Category=Open-Sourced Models2025.09 | 4.7 | 20.5 | 26.4 | |
| GPT-4.1-miniModel Category=Close-Sourced Models2025.09 | 3.1 | 4.3 | 5.1 | |
| Qwen2.5-7B-InstructModel Category=Open-Sourced Models, Training=Base2025.09 | 2.4 | 6.1 | 8 | |
| LongCat-Lite-8K-ChatModel Category=Close-Sourced Models2025.09 | 2.3 | 3.4 | 4.9 | |
| Llama-3.1-8B-InstructModel Category=Open-Sourced Models, Training=Base2025.09 | 0.7 | 5.1 | 6.8 |