Reasoning on SynthWorlds 2026 (test)
0.48ScoreCodeAct
Evaluation Results
| Method | Links | |
|---|---|---|
| CodeActBase Model=Llama-3.3 70B Instruct2026.05 | 0.48 | |
| RLMBase Model=Llama-3.3 70B Instruct2026.05 | 0.401 | |
| DOLORESBase Model=Llama-3.3 70B Instruct2026.05 | 0.359 | |
| DOLORESBase Model=Qwen3 32B Thinking2026.05 | 0.346 | |
| Deep ResearchBase Model=Llama-3.3 70B Instruct2026.05 | 0.308 | |
| DOLORESBase Model=Qwen3 8B Thinking2026.05 | 0.305 | |
| ReActBase Model=Llama-3.3 70B Instruct2026.05 | 0.3 | |
| CodeActBase Model=Qwen3 32B Thinking2026.05 | 0.281 | |
| Deep ResearchBase Model=Qwen3 32B Thinking2026.05 | 0.275 | |
| CodeActBase Model=Qwen3 8B Thinking2026.05 | 0.268 | |
| ReActBase Model=Qwen3 32B Thinking2026.05 | 0.228 | |
| Deep ResearchBase Model=Qwen3 8B Thinking2026.05 | 0.206 | |
| ReActBase Model=Qwen3 8B Thinking2026.05 | 0.176 | |
| RLMBase Model=Qwen3 32B Thinking2026.05 | 0.169 | |
| RLMBase Model=Qwen3 8B Thinking2026.05 | 0.058 |