Tool-calling on SupChain-Bench
75.51AccuracySupChain-ReAct
Evaluation results
| Method | Links | |
|---|---|---|
| SupChain-ReActBackbone=Claude-4-Sonnet2026.02 | 75.51 | |
| SupChain-ReActBackbone=Claude-4-Opus2026.02 | 75.51 | |
| SupChain-ReActBackbone=Gemini-2.5-Pro2026.02 | 72.44 | |
| SupChain-ReActBackbone=GPT-4o2026.02 | 40.81 | |
| SupChain-ReActBackbone=Claude-3.5-Sonnet2026.02 | 40.81 | |
| Claude-4-SonnetSOP guidance=Standard SOP2026.02 | 39.79 | |
| Gemini-2.5-ProSOP guidance=Standard SOP2026.02 | 37.56 | |
| SupChain-ReActBackbone=GPT-4.1-mini2026.02 | 35.7 | |
| Claude-4-OpusSOP guidance=Standard SOP2026.02 | 34.69 | |
| SupChain-ReActBackbone=Qwen3-Max2026.02 | 33.67 | |
| Claude-4-SonnetSOP guidance=None2026.02 | 31.63 | |
| Claude-3.5-SonnetSOP guidance=Standard SOP2026.02 | 29.59 | |
| GPT-4.1-miniSOP guidance=Standard SOP2026.02 | 28.57 | |
| Qwen3-MaxSOP guidance=Standard SOP2026.02 | 28.57 | |
| Claude-4-OpusSOP guidance=None2026.02 | 26.53 | |
| SupChain-ReActBackbone=Qwen3-30B-A3B2026.02 | 21.42 | |
| GPT-4oSOP guidance=None2026.02 | 20.4 | |
| SupChain-ReActBackbone=GPT-4.12026.02 | 19.38 | |
| GPT-4oSOP guidance=Standard SOP2026.02 | 16.32 | |
| GPT-4.1SOP guidance=Standard SOP2026.02 | 16.32 | |
| GPT-4.1SOP guidance=None2026.02 | 12.24 | |
| GPT-4.1-miniSOP guidance=None2026.02 | 11.22 | |
| Claude-3.5-SonnetSOP guidance=None2026.02 | 11.22 | |
| Gemini-2.5-ProSOP guidance=None2026.02 | 11.22 | |
| Qwen3-30B-A3BSOP guidance=Standard SOP2026.02 | 8.16 | |
| Qwen3-MaxSOP guidance=None2026.02 | 7.14 | |
| Qwen3-30B-A3BSOP guidance=None2026.02 | 5.1 |