List your company
ResearchFormerly WizWand

Interactive Decision Making on InterCode NL2Bash

79.6Success RateGRASP
65.87269.4367376.564Apr 20, 2026
Updated 6mo ago

Evaluation results

MethodLinks
GRASPModel=DeepSeek V3.22026.0479.611.2
ReAct + SkillsModel=DeepSeek V3.22026.0479.113.2
GRASPModel=Gemini 2.5 Pro2026.0478.312
ReAct + SkillsModel=Gemini 2.5 Pro2026.0477.714.1
GRASPModel=Claude-4-Sonnet2026.0477.613.2
GRASPModel=Kimi-K2.52026.0477.412.8
ReAct + SkillsModel=Kimi-K2.52026.0476.813.6
GRASPModel=o4 Mini2026.0476.713.8
ReAct + SkillsModel=Claude-4-Sonnet2026.0476.713.2
GRASPModel=GLM-52026.0476.713.4
ReflexionModel=DeepSeek V3.22026.0475.814
GRASPModel=GPT-4.12026.0475.812.5
ReAct + SkillsModel=GLM-52026.0475.614.6
ReAct + SkillsModel=GPT-4.12026.0475.113.3
ReflexionModel=Gemini 2.5 Pro2026.0474.314.6
ExpeLModel=DeepSeek V3.22026.0474.214.3
GRASPModel=Qwen3-235B2026.0474.213.4
ReflexionModel=Claude-4-Sonnet2026.0473.614
ReAct + SkillsModel=Qwen3-235B2026.0473.513.8
ExpeLModel=Gemini 2.5 Pro2026.0473.414.8
ExpeLModel=Claude-4-Sonnet2026.0473.413.6
ReflexionModel=Kimi-K2.52026.0473.414.2
ReActModel=Kimi-K2.52026.0473.215
ExpeLModel=Kimi-K2.52026.0473.214.6
ReActModel=DeepSeek V3.22026.0472.614.8
ReActModel=GPT-4.12026.0472.414.6
ReflexionModel=GPT-4.12026.0472.413.8
ReflexionModel=GLM-52026.0472.115.2
ReAct + SkillsModel=o4 Mini2026.0471.615.1
ReActModel=Qwen3-235B2026.0471.615.4
ReActModel=Gemini 2.5 Pro2026.0470.815.6
ExpeLModel=GPT-4.12026.0470.814.6
ReflexionModel=Qwen3-235B2026.0470.814.6
ReActModel=Claude-4-Sonnet2026.0470.814.8
ExpeLModel=Qwen3-235B2026.0470.614.2
ReflexionModel=o4 Mini2026.0470.215.8
ReActModel=GLM-52026.0468.416.5
ExpeLModel=o4 Mini2026.0467.916.2
ExpeLModel=GLM-52026.0467.916
ReActModel=o4 Mini2026.0466.417.2