Agent Tool Use on StableToolBench Held-In
50.4Pass RateSHAD+RFT
Evaluation results
| Method | Links | |
|---|---|---|
| SHAD+RFTModel=LLaMA3.1-8B2024.12 | 50.4 | |
| SHAD+RFTModel=LLaMA3-8B2024.12 | 50.1 | |
| SHAD+α-FTModel=LLaMA3.1-8B2024.12 | 49.2 | |
| SFTModel=LLaMA3.1-8B2024.12 | 48.5 | |
| RewardFTModel=LLaMA3.1-8B2024.12 | 48.2 | |
| SHAD+α-FTModel=LLaMA3-8B2024.12 | 47 | |
| Regex+RFTModel=LLaMA3.1-8B2024.12 | 46.7 | |
| RewardFTModel=LLaMA3-8B2024.12 | 44.4 | |
| SFTModel=LLaMA3-8B2024.12 | 43.1 | |
| RegexModel=LLaMA3.1-8B2024.12 | 42.3 | |
| Regex+RFTModel=LLaMA3-8B2024.12 | 41.2 | |
| RegexModel=LLaMA3-8B2024.12 | 36.2 | |
| Rho-1Model=LLaMA3.1-8B2024.12 | 30.6 | |
| Rho-1Model=LLaMA3-8B2024.12 | 24.5 |