Knowledge Graph Interaction Agent Task on KG
64.4Success RateClaude-Sonnet-4 Thinking
Evaluation Results
| Method | Links | |
|---|---|---|
| Claude-Sonnet-4 ThinkingCategory=API LLMs, Variant=Thinking2026.07 | 64.4 | |
| GPT-5Category=API LLMs2026.07 | 64.1 | |
| Claude-Sonnet-4Category=API LLMs2026.07 | 63.4 | |
| o3-miniCategory=API LLMs2026.07 | 51.8 | |
| DeepSeek-R1Category=Open LLMs2026.07 | 50.2 | |
| AgentLM-70BCategory=Open LLMs, Parameters=70B2026.07 | 47 | |
| Qwen2.5-14B-InstructCategory=Open LLMs, Parameters=14B, Variant=Instruct2026.07 | 35.3 | |
| Qwen2.5-32B-InstructCategory=Open LLMs, Parameters=32B, Variant=Instruct2026.07 | 33.8 | |
| o4-miniCategory=API LLMs2026.07 | 32.4 | |
| AgentRLCategory=RL Training2026.07 | 30 | |
| EPPOCategory=RL Training2026.07 | 29.6 | |
| AgentLM-13BCategory=Open LLMs, Parameters=13B2026.07 | 26.8 | |
| Hephaestus-8B-IFTCategory=Open LLMs, Parameters=8B, Variant=IFT2026.07 | 21.2 | |
| AgentLM-7BCategory=Open LLMs, Parameters=7B2026.07 | 18.1 | |
| Hephaestus-8B-BaseCategory=Open LLMs, Parameters=8B, Variant=Base2026.07 | 16 | |
| DeepSeek-V3Category=Open LLMs2026.07 | 14 | |
| DCPOCategory=RL Training2026.07 | 10.6 | |
| Qwen2.5-3B-InstructCategory=Open LLMs, Parameters=3B, Variant=Instruct2026.07 | 4.8 | |
| BAPOCategory=RL Training2026.07 | 4.8 |