Agentic Task Performance on τ-Telecom
52Pass@1 Success RateFAMA
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| FAMABackbone=Qwen2.5-72B-Instruct2026.04 | 52 | 44 | 41.1 | 40 | 39 | |
| ReActBackbone=Qwen2.5-72B-Instruct2026.04 | 51.11 | 39.44 | 32.22 | 26.66 | 22.22 | |
| IRMABackbone=Qwen2.5-72B-Instruct2026.04 | 44.44 | 30 | 23.88 | 22.22 | 22.22 | |
| FCBackbone=Qwen2.5-72B-Instruct2026.04 | 44 | 35 | 31 | 28 | 27 | |
| FAMABackbone=Qwen3-14B2026.04 | 43 | 35.5 | 31 | 29 | 28 | |
| IRMABackbone=Qwen3-32B2026.04 | 40 | 28 | 24.44 | 23.33 | 22.2 | |
| FAMABackbone=Qwen3-4B-Instruct2026.04 | 39 | 28 | 22 | 18 | 17 | |
| IRMABackbone=Qwen3-4B-Instruct2026.04 | 38 | 26 | 21 | 18 | 16 | |
| FCBackbone=Qwen3-14B2026.04 | 38 | 33.33 | 30 | 28 | 27 | |
| FAMABackbone=Qwen3-32B2026.04 | 37.77 | 31.66 | 30 | 28.88 | 27.77 | |
| FCBackbone=Qwen3-4B-Instruct2026.04 | 37 | 31 | 27 | 24 | 21 | |
| IRMABackbone=Qwen3-14B2026.04 | 36.66 | 23.3 | 16.66 | 13.33 | 11.11 | |
| FCBackbone=Qwen3-32B2026.04 | 32 | 25 | 21 | 18 | 16 | |
| ReActBackbone=Qwen3-32B2026.04 | 24.44 | 20 | 18.88 | 17.77 | 16.66 | |
| ReActBackbone=Qwen3-4B-Instruct2026.04 | 15.55 | 8.33 | 6.11 | 5.55 | 5.55 | |
| ReActBackbone=Qwen3-14B2026.04 | 15.55 | 11.11 | 11.11 | 11.11 | 11.11 |