Database Querying Agent Task on DB
70.1Success RateClaude-Sonnet-4
Evaluation Results
| Method | Links | |
|---|---|---|
| Claude-Sonnet-4Category=API LLMs2026.07 | 70.1 | |
| Claude-Sonnet-4 ThinkingCategory=API LLMs, Variant=Thinking2026.07 | 68.4 | |
| o4-miniCategory=API LLMs2026.07 | 63.4 | |
| GPT-5Category=API LLMs2026.07 | 63.2 | |
| DCPOCategory=RL Training2026.07 | 61 | |
| EPPOCategory=RL Training2026.07 | 60.5 | |
| DeepSeek-R1Category=Open LLMs2026.07 | 60.4 | |
| DeepSeek-V3Category=Open LLMs2026.07 | 58.4 | |
| BAPOCategory=RL Training2026.07 | 58.3 | |
| o3-miniCategory=API LLMs2026.07 | 56.5 | |
| AgentRLCategory=RL Training2026.07 | 56.3 | |
| Qwen2.5-32B-InstructCategory=Open LLMs, Parameters=32B, Variant=Instruct2026.07 | 55.8 | |
| Qwen2.5-14B-InstructCategory=Open LLMs, Parameters=14B, Variant=Instruct2026.07 | 48.4 | |
| AgentLM-70BCategory=Open LLMs, Parameters=70B2026.07 | 37.7 | |
| AgentLM-13BCategory=Open LLMs, Parameters=13B2026.07 | 33.7 | |
| Hephaestus-8B-BaseCategory=Open LLMs, Parameters=8B, Variant=Base2026.07 | 32.3 | |
| AgentLM-7BCategory=Open LLMs, Parameters=7B2026.07 | 30.6 | |
| Hephaestus-8B-IFTCategory=Open LLMs, Parameters=8B, Variant=IFT2026.07 | 29.7 | |
| Qwen2.5-3B-InstructCategory=Open LLMs, Parameters=3B, Variant=Instruct2026.07 | 25.5 |