API Use on API-Bank
77.19Success RateHarnessForge
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| HarnessForgeBackbone=Qwen3-4B2026.06 | 77.19 | 80.07 | 82.27 | |
| HarnessForgeBackbone=Qwen3-8B2026.06 | 74.56 | 78.22 | 78.01 | |
| RLOOBackbone=Qwen3-4B2026.06 | 72.81 | 78.82 | 76.6 | |
| GRPOBackbone=Qwen3-4B2026.06 | 71.93 | 76.46 | 75.18 | |
| SFTBackbone=Qwen3-4B2026.06 | 69.3 | 75.1 | 73.76 | |
| GRPOBackbone=Qwen3-8B2026.06 | 69.3 | 74.71 | 73.76 | |
| SFTBackbone=Qwen3-8B2026.06 | 68.42 | 73.8 | 73.05 | |
| RLOOBackbone=Qwen3-8B2026.06 | 66.67 | 73.76 | 71.63 | |
| MaASBackbone=Qwen3-8B2026.06 | 57.89 | 66.72 | 60.28 | |
| MermaidFlowBackbone=Qwen3-8B2026.06 | 57.02 | 67.46 | 62.41 | |
| MermaidFlowBackbone=Qwen3-4B2026.06 | 54.39 | 64.33 | 60.28 | |
| ADASBackbone=Qwen3-8B2026.06 | 54.39 | 60.14 | 59.57 | |
| MaASBackbone=Qwen3-4B2026.06 | 52.63 | 62.74 | 63.83 | |
| ADASBackbone=Qwen3-4B2026.06 | 51.75 | 57.31 | 60.99 | |
| AgentSquareBackbone=Qwen3-8B2026.06 | 45.61 | 54.37 | 53.9 | |
| AFlowBackbone=Qwen3-8B2026.06 | 43.86 | 49.74 | 47.52 | |
| AgentSquareBackbone=Qwen3-4B2026.06 | 39.47 | 47.82 | 46.1 | |
| AFlowBackbone=Qwen3-4B2026.06 | 37.72 | 45.72 | 41.13 |