Agent Tool Use on Nexus (Held-Out)
32AccuracySHAD+RFT
Evaluation Results
| Method | Links | |
|---|---|---|
| SHAD+RFTModel=LLaMA3.1-8B2024.12 | 32 | |
| SHAD+α-FTModel=LLaMA3.1-8B2024.12 | 28.9 | |
| SHAD+α-FTModel=LLaMA3-8B2024.12 | 28.7 | |
| SHAD+RFTModel=LLaMA3-8B2024.12 | 27.8 | |
| Rho-1Model=LLaMA3.1-8B2024.12 | 26 | |
| SFTModel=LLaMA3.1-8B2024.12 | 19.5 | |
| RewardFTModel=LLaMA3.1-8B2024.12 | 19.1 | |
| Rho-1Model=LLaMA3-8B2024.12 | 19 | |
| Regex+RFTModel=LLaMA3.1-8B2024.12 | 16.2 | |
| RegexModel=LLaMA3.1-8B2024.12 | 14.3 | |
| SFTModel=LLaMA3-8B2024.12 | 14 | |
| Regex+RFTModel=LLaMA3-8B2024.12 | 12.4 | |
| RewardFTModel=LLaMA3-8B2024.12 | 8 | |
| RegexModel=LLaMA3-8B2024.12 | 6.45 |