Agent Tool Use on T-eval (Held-Out)
71.8AccuracySHAD+RFT
Evaluation Results
| Method | Links | |
|---|---|---|
| SHAD+RFTModel=LLaMA3-8B2024.12 | 71.8 | |
| SHAD+α-FTModel=LLaMA3-8B2024.12 | 68.8 | |
| Rho-1Model=LLaMA3-8B2024.12 | 68.4 | |
| SHAD+RFTModel=LLaMA3.1-8B2024.12 | 68.3 | |
| SFTModel=LLaMA3-8B2024.12 | 67 | |
| Rho-1Model=LLaMA3.1-8B2024.12 | 67 | |
| RewardFTModel=LLaMA3.1-8B2024.12 | 66.4 | |
| RewardFTModel=LLaMA3-8B2024.12 | 66.3 | |
| SFTModel=LLaMA3.1-8B2024.12 | 64.2 | |
| SHAD+α-FTModel=LLaMA3.1-8B2024.12 | 63.8 | |
| Regex+RFTModel=LLaMA3-8B2024.12 | 61.1 | |
| RegexModel=LLaMA3.1-8B2024.12 | 58.6 | |
| Regex+RFTModel=LLaMA3.1-8B2024.12 | 57.6 | |
| RegexModel=LLaMA3-8B2024.12 | 54.3 |