List your company
ResearchFormerly WizWand

Operating System Interaction Agent Task on OS

53.6Success RateDeepSeek-R1
15.95225.72635.545.274Jul 8, 2026
Updated 3mo ago

Evaluation results

MethodLinks
DeepSeek-R1Category=Open LLMs2026.0753.6
DeepSeek-V3Category=Open LLMs2026.0753
Claude-Sonnet-4 ThinkingCategory=API LLMs, Variant=Thinking2026.0751
Claude-Sonnet-4Category=API LLMs2026.0745.3
o4-miniCategory=API LLMs2026.0741.8
Qwen2.5-32B-InstructCategory=Open LLMs, Parameters=32B, Variant=Instruct2026.0737
o3-miniCategory=API LLMs2026.0735.1
GPT-5Category=API LLMs2026.0734.5
EPPOCategory=RL Training2026.0732.8
AgentRLCategory=RL Training2026.0731.3
BAPOCategory=RL Training2026.0730
DCPOCategory=RL Training2026.0727.2
Qwen2.5-14B-InstructCategory=Open LLMs, Parameters=14B, Variant=Instruct2026.0726
Qwen2.5-3B-InstructCategory=Open LLMs, Parameters=3B, Variant=Instruct2026.0723.9
AgentLM-70BCategory=Open LLMs, Parameters=70B2026.0721.5
Hephaestus-8B-BaseCategory=Open LLMs, Parameters=8B, Variant=Base2026.0720.8
Hephaestus-8B-IFTCategory=Open LLMs, Parameters=8B, Variant=IFT2026.0720.8
AgentLM-13BCategory=Open LLMs, Parameters=13B2026.0718.1
AgentLM-7BCategory=Open LLMs, Parameters=7B2026.0717.4