Function Calling on API-Bank
79.17Level-1 ScoreGENESISFUNC-8B
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GENESISFUNC-8Bsetting=Zero-shot2026.04 | 79.17 | 64.09 | 51.11 | 64.79 | — | — | — | |
| GPT-4osetting=Zero-shot2026.04 | 76.19 | 42.96 | 35.21 | 51.45 | — | — | — | |
| ToolACE-8Bsetting=Zero-shot2026.04 | 75.94 | 47.41 | 45.27 | 56.21 | — | — | — | |
| Gemini-1.5-Prosetting=Zero-shot2026.04 | 75.43 | 43.26 | 41.51 | 53.4 | — | — | — | |
| GPT-4o-minisetting=Zero-shot2026.04 | 74.69 | 45.93 | 40.77 | 53.8 | — | — | — | |
| Qwen-ToolRL-8Bsetting=Zero-shot2026.04 | 73.07 | 60.81 | 47.21 | 60.36 | — | — | — | |
| Qwen3-8Bsetting=Zero-shot2026.04 | 71.68 | 52.24 | 42.75 | 55.56 | — | — | — | |
| LLaMA-3.1-8B-Instructsetting=Zero-shot2026.04 | 71.18 | 37.04 | 35.88 | 48.03 | — | — | — | |
| GD2PO-SNRModel=Llama3.1-8B Instruct, SNR threshold τ=0.82026.06 | 71.18 | 52.24 | 42.44 | 62.75 | 99.97 | 1 | 2.627 | |
| GD2PO-HardModel=Llama3.1-8B Instruct2026.06 | 68.77 | 53.13 | 43.05 | 61.37 | 99.9 | 1 | 2.613 | |
| GD2PO-HardModel=Qwen2.5-3B Instruct2026.06 | 67.72 | 54.33 | 43.05 | 60.8 | 99.9 | 1 | 2.607 | |
| GDPOModel=Llama3.1-8B Instruct2026.06 | 67.32 | 52.24 | 44.88 | 60.7 | 100 | 1 | 2.607 | |
| GD2PO-SNRModel=Qwen2.5-3B Instruct, SNR threshold τ=0.82026.06 | 67.17 | 50.75 | 47.33 | 60.97 | 99.8 | 1 | 2.608 | |
| GRPOModel=Qwen2.5-3B Instruct2026.06 | 66.92 | 57.91 | 41.68 | 60.37 | 98.86 | 1 | 2.592 | |
| GDPOModel=Qwen2.5-3B Instruct2026.06 | 66.22 | 52.54 | 47.18 | 60.5 | 99.9 | 1 | 2.604 | |
| GRPOModel=Llama3.1-8B Instruct2026.06 | 63.56 | 43.58 | 43.51 | 56.92 | 99.97 | 1 | 2.569 | |
| GD2PO-SNRModel=Qwen2.5-1.5B Instruct, SNR threshold τ=0.82026.06 | 61.61 | 51.34 | 41.37 | 56.01 | 91.53 | 1 | 2.475 | |
| GD2PO-HardModel=Qwen2.5-1.5B Instruct2026.06 | 61.2 | 48.66 | 36.95 | 54.47 | 94.51 | 0.99 | 2.48 | |
| GDPOModel=Qwen2.5-1.5B Instruct2026.06 | 60.9 | 50.45 | 40.91 | 55.34 | 86.87 | 1 | 2.422 | |
| GRPOModel=Qwen2.5-1.5B Instruct2026.06 | 53.78 | 38.51 | 34.2 | 47.77 | 85.83 | 0.99 | 2.326 |