Agent Execution on CRMArena (test)
49Execution AccuracyGemini-2.5 Pro
Evaluation results
| Method | Links | |
|---|---|---|
| Gemini-2.5 ProPrompting=2-shot2026.03 | 49 | |
| Qwen3-8B Agentic GRPOTraining Strategy=Agentic GRPO, Training Data=<1K2026.03 | 35 | |
| Claude-3.5-SonnetPrompting=2-shot2026.03 | 34 | |
| GPT-4oPrompting=2-shot2026.03 | 32 | |
| Qwen3-8B SFTTraining Strategy=SFT, Training Data=<1K2026.03 | 30 | |
| Qwen3-8B BasePrompting=2-shot2026.03 | 25 | |
| xLAM-2-70BTraining Data=60K-trained2026.03 | 12 | |
| ToolAceTraining Data=26K-trained2026.03 | 10 |