List your company
ResearchFormerly WizWand

Task goal completion on AppWorld (test challenge)

32Goal Completion ScoreRLOO
-0.965927.5925416.15124.70946Mar 12, 2026Mar 27, 2026Apr 12, 2026Apr 27, 2026May 13, 2026May 28, 2026Jun 13, 2026
Updated 4mo ago

Evaluation results

MethodLinks
RLOOBase Model=Qwen-3-8B, Max Context Length=40962026.0332400.01
GSPOBase Model=Qwen-3-8B, Max Context Length=40962026.0332360
ADAPOBase Model=Qwen-3-8B, Max Context Length=40962026.0330330
DAPOBase Model=Qwen-3-8B, Max Context Length=40962026.0328290.06
GRPO + H bonusBase Model=Qwen-3-8B, Max Context Length=40962026.032628-0.01
REPO-RBase Model=Qwen-3-8B, Max Context Length=40962026.032527-0.01
GRPOBase Model=Qwen-3-8B, Max Context Length=40962026.032123-0.09
LOOPBase Model=Qwen-3-8B, Max Context Length=40962026.031922-0.08
ASSAYModel=GPT-4o-mini2026.060.854—4.3
ReActModel=GPT-4o-mini2026.060.811——
ASSAYModel=Sonnet 3.52026.060.753—5
ReActModel=Sonnet 3.52026.060.703——
ASSAYModel=DeepSeek-V32026.060.693—22.3
wt-tunedModel=Qwen2-14B2026.060.676——
ASSAYModel=GPT-3.52026.060.664—13.9
ASSAYModel=GPT-4-Turbo2026.060.64—13.6
ACEModel=DeepSeek-V32026.060.631—16.1
RLOOBase Model=Qwen-3-32B2026.030.610.71-0.18
ADAPOBase Model=Qwen-3-32B2026.030.580.620.52
CUGAModel=GPT-4-Turbo2026.060.576—7.2
ASSAYModel=Gemini 1.5 Pro2026.060.549—5.5
REPO-RBase Model=Qwen-3-32B2026.030.540.630.03
ReActModel=GPT-3.52026.060.525——
DAPOBase Model=Qwen-3-32B2026.030.520.551.52
GSPOBase Model=Qwen-3-32B2026.030.510.51-0.07
ReActModel=GPT-4-Turbo2026.060.504——
ACEModel=GPT-3.52026.060.499—2.6
ReActModel=Gemini 1.5 Pro2026.060.494——
GRPO + H bonusBase Model=Qwen-3-32B2026.030.490.510.14
ReActModel=DeepSeek-V32026.060.47——
GRPOBase Model=Qwen-3-32B2026.030.460.49-0.29
LOOPBase Model=Qwen-3-32B2026.030.450.47-0.31
ASSAYModel=GPT-4o2026.060.41—10.8
Gupta et al.Model=GPT-4o2026.060.389—8.7
ReActModel=GPT-4o2026.060.302——