List your company
ResearchFormerly WizWand

Bash Command Execution on InterCode Bash

72Execution Success RateGPT-5
25.907237.873649.8461.8064Apr 22, 2026May 1, 2026May 10, 2026May 19, 2026May 28, 2026Jun 6, 2026Jun 16, 2026
Updated 4mo ago

Evaluation results

MethodLinks
GPT-5Method=Few-Shot, Avg Latency (ms)=∼16,4902026.0472
Meta-Tool (3B)Method=Ours, Avg Latency (ms)=∼1,5762026.0454
QWEN3.5-397B-A17BBackbone LLM=QWEN3.5-397B-A17B2026.0651.34
OPD-Evolver-9BBackbone LLM=QWEN3.5-9B2026.0649.55
MemEvolveBackbone LLM=QWEN3.5-9B2026.0645.98
EvolveRBackbone LLM=QWEN3.5-9B2026.0644.64
STEP-3.5-FLASH (196B)Backbone LLM=STEP-3.5-FLASH (196B)2026.0644.2
MempBackbone LLM=QWEN3.5-9B2026.0644.2
Llama-3.2-3B-Ins.Method=Few-Shot, Avg Latency (ms)=∼1,6212026.0444
ReasoningBankBackbone LLM=QWEN3.5-9B2026.0643.75
CheatsheetBackbone LLM=QWEN3.5-9B2026.0641.96
No MemoryBackbone LLM=QWEN3.5-9B2026.0641.52
ExpeLBackbone LLM=QWEN3.5-9B2026.0640.18
AgentLM-7BMethod=Few-Shot, Avg Latency (ms)=∼8,8802026.0440
AWMBackbone LLM=QWEN3.5-9B2026.0639.73
OPD-Evolver-4BBackbone LLM=QWEN3-4B2026.0636.16
EvolveRBackbone LLM=QWEN3-4B2026.0633.93
MemEvolveBackbone LLM=QWEN3-4B2026.0633.93
CheatsheetBackbone LLM=QWEN3-4B2026.0633.04
MempBackbone LLM=QWEN3-4B2026.0631.7
ReasoningBankBackbone LLM=QWEN3-4B2026.0631.25
ExpeLBackbone LLM=QWEN3-4B2026.0630.8
No MemoryBackbone LLM=QWEN3-4B2026.0630.36
AWMBackbone LLM=QWEN3-4B2026.0627.68