List your company
ResearchFormerly WizWand

Agentic Task on AppWorld N (test)

57.1PR ScoreExpGraph
19.34829.14938.9548.751May 29, 2026
Updated 5mo ago

Evaluation results

MethodLinks
ExpGraphExecutor LLM=Qwen3-32B (Small LLM)2026.0557.18.8
ExpGraphExecutor LLM=Gemini-3.1-Flash-Lite (Large LLM)2026.0557.112.8
S3Executor LLM=Qwen3-32B (Small LLM), Method Strategy Category=LLM-Centric Experience Learning Baselines2026.0555.49.7
S3Executor LLM=Gemini-3.1-Flash-Lite (Large LLM), Method Strategy Category=LLM-Centric Experience Learning Baselines2026.055318.5
IRCoTExecutor LLM=Qwen3-32B (Small LLM), Method Strategy Category=LLM-Centric Experience Learning Baselines2026.0552.811.4
Search-o1Executor LLM=Gemini-3.1-Flash-Lite (Large LLM), Method Strategy Category=LLM-Centric Experience Learning Baselines2026.0552.320.7
AWMExecutor LLM=Gemini-3.1-Flash-Lite (Large LLM), Method Strategy Category=Retrieval-Centric Experience Learning Baselines2026.0550.720.9
ReflexionExecutor LLM=Gemini-3.1-Flash-Lite (Large LLM), Method Strategy Category=Prompt-based Agentic Baselines2026.0548.221.9
IRCoTExecutor LLM=Gemini-3.1-Flash-Lite (Large LLM), Method Strategy Category=LLM-Centric Experience Learning Baselines2026.0546.918.7
Search-o1Executor LLM=Qwen3-32B (Small LLM), Method Strategy Category=LLM-Centric Experience Learning Baselines2026.0545.416.5
ReasoningBankExecutor LLM=Gemini-3.1-Flash-Lite (Large LLM), Method Strategy Category=Retrieval-Centric Experience Learning Baselines2026.0545.323.1
MemRLExecutor LLM=Qwen3-32B (Small LLM), Method Strategy Category=Retrieval-Centric Experience Learning Baselines2026.0545.218.6
ExpeLExecutor LLM=Gemini-3.1-Flash-Lite (Large LLM), Method Strategy Category=Retrieval-Centric Experience Learning Baselines2026.0543.621.1
MemRLExecutor LLM=Gemini-3.1-Flash-Lite (Large LLM), Method Strategy Category=Retrieval-Centric Experience Learning Baselines2026.0540.513.5
No-MemoryExecutor LLM=Gemini-3.1-Flash-Lite (Large LLM), Method Strategy Category=No-Memory2026.0534.233
LightMemExecutor LLM=Gemini-3.1-Flash-Lite (Large LLM), Method Strategy Category=Retrieval-Centric Experience Learning Baselines2026.0534.231.1
ReActExecutor LLM=Gemini-3.1-Flash-Lite (Large LLM), Method Strategy Category=Prompt-based Agentic Baselines2026.0532.724.8
Mem0Executor LLM=Gemini-3.1-Flash-Lite (Large LLM), Method Strategy Category=Retrieval-Centric Experience Learning Baselines2026.0532.731.5
ReflexionExecutor LLM=Qwen3-32B (Small LLM), Method Strategy Category=Prompt-based Agentic Baselines2026.0531.631.3
No-MemoryExecutor LLM=Qwen3-32B (Small LLM), Method Strategy Category=No-Memory2026.0530.130.8
ReActExecutor LLM=Qwen3-32B (Small LLM), Method Strategy Category=Prompt-based Agentic Baselines2026.053032.2
ReasoningBankExecutor LLM=Qwen3-32B (Small LLM), Method Strategy Category=Retrieval-Centric Experience Learning Baselines2026.0529.425.6
LightMemExecutor LLM=Qwen3-32B (Small LLM), Method Strategy Category=Retrieval-Centric Experience Learning Baselines2026.0528.534.2
AWMExecutor LLM=Qwen3-32B (Small LLM), Method Strategy Category=Retrieval-Centric Experience Learning Baselines2026.052828.8
ExpeLExecutor LLM=Qwen3-32B (Small LLM), Method Strategy Category=Retrieval-Centric Experience Learning Baselines2026.0526.735.1
Mem0Executor LLM=Qwen3-32B (Small LLM), Method Strategy Category=Retrieval-Centric Experience Learning Baselines2026.0520.833.4