Scientific Simulation on SciWorld
55.7Measure ScoreStraTA
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| StraTABackbone=Qwen2.5-7B-Instruct, Category=Training-Based2026.05 | 55.7 | 54.4 | 78.6 | 16.3 | 100 | 63.5 | |
| Claude-4-SonnetModel Type=Closed-Source Model, Category=Prompting-Based2026.05 | 45.4 | 54.7 | 60.3 | 45.6 | 90 | 57.4 | |
| Gemini-2.5-FlashModel Type=Closed-Source Model, Category=Prompting-Based2026.05 | 40.7 | 51.3 | 53.7 | 33 | 50 | 49.6 | |
| ScalingInterBackbone=Qwen2.5-7B-Instruct, Category=Training-Based2026.05 | 34 | 55.4 | 88.6 | 0 | 73.3 | 57 | |
| GPT-5.1Model Type=Closed-Source Model, Category=Prompting-Based2026.05 | 32.1 | 46.5 | 41.8 | 69.6 | 38.3 | 43 | |
| GRPOBackbone=Qwen2.5-7B-Instruct, Category=Training-Based2026.05 | 31.2 | 49.8 | 32.7 | 14.7 | 53.3 | 41.8 | |
| AgentGym-RLBackbone=Qwen2.5-7B-Instruct, Category=Training-Based2026.05 | 24.5 | 59 | 65.9 | 0 | 66.7 | 50.5 | |
| ReActBackbone=Qwen2.5-7B-Instruct, Category=Prompting-Based2026.05 | 12.2 | 14.4 | 33.6 | 9.6 | 28.9 | 20 | |
| VanillaBackbone=Qwen2.5-7B-Instruct, Category=Prompting-Based2026.05 | 2.1 | 12.3 | 21.2 | 9.6 | 25 | 13.9 | |
| PPOBackbone=Qwen2.5-7B-Instruct, Category=Training-Based2026.05 | 1.8 | 57.5 | 60.9 | 6.9 | 97.8 | 51.4 |