List your company
ResearchFormerly WizWand

Agentic Task on AppWorld C (test)

48.4PR ScoreExpGraph
14.39223.22132.0540.879May 29, 2026
Updated 5mo ago

Evaluation results

MethodLinks
ExpGraphExecutor LLM=Gemini-3.1-Flash-Lite (Large LLM)2026.0548.414.8
ReasoningBankExecutor LLM=Gemini-3.1-Flash-Lite (Large LLM), Method Strategy Category=Retrieval-Centric Experience Learning Baselines2026.0541.225.5
S3Executor LLM=Gemini-3.1-Flash-Lite (Large LLM), Method Strategy Category=LLM-Centric Experience Learning Baselines2026.0540.822
Search-o1Executor LLM=Gemini-3.1-Flash-Lite (Large LLM), Method Strategy Category=LLM-Centric Experience Learning Baselines2026.0539.824.6
ExpGraphExecutor LLM=Qwen3-32B (Small LLM)2026.0539.313.6
IRCoTExecutor LLM=Qwen3-32B (Small LLM), Method Strategy Category=LLM-Centric Experience Learning Baselines2026.0536.117.8
S3Executor LLM=Qwen3-32B (Small LLM), Method Strategy Category=LLM-Centric Experience Learning Baselines2026.053515
IRCoTExecutor LLM=Gemini-3.1-Flash-Lite (Large LLM), Method Strategy Category=LLM-Centric Experience Learning Baselines2026.0534.925.4
ReActExecutor LLM=Gemini-3.1-Flash-Lite (Large LLM), Method Strategy Category=Prompt-based Agentic Baselines2026.0532.116.1
MemRLExecutor LLM=Gemini-3.1-Flash-Lite (Large LLM), Method Strategy Category=Retrieval-Centric Experience Learning Baselines2026.0531.716.7
AWMExecutor LLM=Gemini-3.1-Flash-Lite (Large LLM), Method Strategy Category=Retrieval-Centric Experience Learning Baselines2026.0531.525.8
ReflexionExecutor LLM=Gemini-3.1-Flash-Lite (Large LLM), Method Strategy Category=Prompt-based Agentic Baselines2026.0531.115.6
ExpeLExecutor LLM=Gemini-3.1-Flash-Lite (Large LLM), Method Strategy Category=Retrieval-Centric Experience Learning Baselines2026.0529.928.3
LightMemExecutor LLM=Gemini-3.1-Flash-Lite (Large LLM), Method Strategy Category=Retrieval-Centric Experience Learning Baselines2026.0529.734.2
MemRLExecutor LLM=Qwen3-32B (Small LLM), Method Strategy Category=Retrieval-Centric Experience Learning Baselines2026.0528.924.7
ExpeLExecutor LLM=Qwen3-32B (Small LLM), Method Strategy Category=Retrieval-Centric Experience Learning Baselines2026.0528.225.6
Search-o1Executor LLM=Qwen3-32B (Small LLM), Method Strategy Category=LLM-Centric Experience Learning Baselines2026.0526.514.2
LightMemExecutor LLM=Qwen3-32B (Small LLM), Method Strategy Category=Retrieval-Centric Experience Learning Baselines2026.0523.737.5
No-MemoryExecutor LLM=Gemini-3.1-Flash-Lite (Large LLM), Method Strategy Category=No-Memory2026.0523.236.6
Mem0Executor LLM=Gemini-3.1-Flash-Lite (Large LLM), Method Strategy Category=Retrieval-Centric Experience Learning Baselines2026.0522.934.8
AWMExecutor LLM=Qwen3-32B (Small LLM), Method Strategy Category=Retrieval-Centric Experience Learning Baselines2026.0522.833
ReasoningBankExecutor LLM=Qwen3-32B (Small LLM), Method Strategy Category=Retrieval-Centric Experience Learning Baselines2026.0522.224.3
ReflexionExecutor LLM=Qwen3-32B (Small LLM), Method Strategy Category=Prompt-based Agentic Baselines2026.0521.932.7
ReActExecutor LLM=Qwen3-32B (Small LLM), Method Strategy Category=Prompt-based Agentic Baselines2026.0521.831.9
No-MemoryExecutor LLM=Qwen3-32B (Small LLM), Method Strategy Category=No-Memory2026.0521.631.5
Mem0Executor LLM=Qwen3-32B (Small LLM), Method Strategy Category=Retrieval-Centric Experience Learning Baselines2026.0515.735.2