Tool Use on StableToolBench cost-augmented
76PRINTENT
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| INTENTbackbone=GPT 5 nano, constraint_type=Enforce2026.02 | 76 | 92.6 | 86.1 | 100 | 29.2 | 17.9 | 1.79 | 2.16 | 2.4 | |
| MCObackbone=GPT 5 nano, constraint_type=Enforce2026.02 | 71.4 | 87 | 82.3 | 100 | 31.9 | 18.1 | 1.87 | 2.23 | 2.28 | |
| INTENTbackbone=GPT 4.1 mini, constraint_type=Enforce2026.02 | 63.8 | 77.8 | 73.3 | 100 | 24.9 | 19.1 | 1.23 | 1.76 | 1.7 | |
| MCObackbone=GPT 4.1 mini, constraint_type=Enforce2026.02 | 58.9 | 71.8 | 72.5 | 100 | 27.1 | 18.7 | 1.9 | 2.05 | 2.15 | |
| BTPbackbone=GPT 5 nano, constraint_type=Enforce2026.02 | 57.7 | 70.3 | 73.2 | 100 | 36.2 | 18.8 | 1.22 | 1.67 | 0.87 | |
| DFSDTbackbone=GPT 5 nano, constraint_type=Enforce2026.02 | 57.2 | 69.7 | 71.9 | 100 | 35.6 | 18.6 | 1.1 | 0.91 | 1.08 | |
| BATSbackbone=GPT 4.1 mini, constraint_type=Enforce2026.02 | 53 | 64.6 | 68.1 | 100 | 35.6 | 20.8 | 1.96 | 3.55 | 4.13 | |
| BATSbackbone=GPT 5 nano, constraint_type=Enforce2026.02 | 52.8 | 64.3 | 71.7 | 100 | 32.6 | 17.3 | 7.67 | 10.1 | 5.76 | |
| Promptbackbone=GPT 5 nano, constraint_type=Soft2026.02 | 48.5 | 59.1 | 62 | 87.6 | 38.1 | 20.1 | 0.64 | 0.6 | 0.76 | |
| BTPbackbone=GPT 4.1 mini, constraint_type=Enforce2026.02 | 46.4 | 56.8 | 59 | 100 | 32.5 | 19.4 | 1.01 | 0.76 | 0.84 | |
| DFSDTbackbone=GPT 4.1 mini, constraint_type=Enforce2026.02 | 44.9 | 54.7 | 55.4 | 100 | 35.2 | 20.2 | 0.97 | 0.99 | 1.14 | |
| Promptbackbone=GPT 4.1 mini, constraint_type=Soft2026.02 | 30.9 | 37.7 | 41.4 | 67.2 | 43.4 | 24.2 | 0.65 | 0.55 | 0.56 | |
| Rawbackbone=GPT 4.1 mini, constraint_type=Soft2026.02 | 19.1 | 23.3 | 37.5 | 34.5 | 102.1 | 28.2 | 1 | 1 | 1 | |
| Rawbackbone=GPT 5 nano, constraint_type=Soft2026.02 | 18.1 | 22.1 | 44.7 | 24.7 | 145.8 | 27.8 | 1 | 1 | 1 |