Function Calling on BFCL V3 (test)
63.34Overall AccuracyCoALM 405B
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CoALM 405BTraining Status=one completed epoch2025.02 | 63.34 | 90.46 | 84.75 | 74.59 | 28.25 | 100 | 72.26 | — | — | — | — | — | — | — | |
| CoALM 70B2025.02 | 60.49 | 82.94 | 81.36 | 72.19 | 26.25 | 72.22 | 85.36 | — | — | — | — | — | — | — | |
| GPT-4oVersion Date=2024-08-062025.02 | 59.83 | 70.08 | 60.79 | 76.41 | 34.62 | 51.22 | 87.34 | — | — | — | — | — | — | — | |
| GPT-4o-miniVersion Date=2024-07-182025.02 | 59.4 | 86.52 | 85.05 | 73.26 | 19 | 78.05 | 76.97 | — | — | — | — | — | — | — | |
| Llama-3.1-405B-Instruct2025.02 | 56.38 | 89.71 | 84.7 | 70.77 | 11.75 | 88.89 | 70.86 | — | — | — | — | — | — | — | |
| CoALM 8B2025.02 | 54.11 | 85.17 | 78.61 | 72.59 | 7 | 77.78 | 83 | — | — | — | — | — | — | — | |
| ToolAce2025.02 | 52.55 | 82.19 | 86.98 | 71.08 | 0.88 | 70.73 | 87.29 | — | — | — | — | — | — | — | |
| Hammer2.0-7b2025.02 | 52.13 | 86.94 | 83.66 | 71.17 | 0.38 | 95.12 | 73.2 | — | — | — | — | — | — | — | |
| Llama-3.3-70B-Instruct2025.02 | 51.36 | 84.85 | 90.05 | 62.51 | 7.25 | 95.12 | 48.33 | — | — | — | — | — | — | — | |
| Llama-3.1-8B-Instruct2025.02 | 49.84 | 84.25 | 79.75 | 60.33 | 10.25 | 75.61 | 47.92 | — | — | — | — | — | — | — | |
| Mistral-7B-InstructVersion=v0.32025.02 | 38.35 | 56.33 | 63.77 | 57.31 | 0.25 | 77.78 | 41.84 | — | — | — | — | — | — | — | |
| HiSMEmode=evolving2026.05 | — | — | — | — | — | — | — | 78 | 85.11 | 90.4 | 81.36 | 68,012.2 | 1,038 | 69,050.2 | |
| HiSME-staticmode=static2026.05 | — | — | — | — | — | — | — | 69.39 | 81.43 | 86.7 | 77.75 | 68,813.4 | 1,052.3 | 69,865.7 | |
| No Skill2026.05 | — | — | — | — | — | — | — | 52 | 76.91 | 84.34 | 72.54 | 61,142.3 | 1,016.8 | 62,159.1 | |
| SkillX2026.05 | — | — | — | — | — | — | — | 66 | 76.89 | 88.37 | 70.26 | 73,514.6 | 1,150.3 | 74,664.9 |