Interactive Decision Making on ScienceWorld Seen
88.8Success RateGRASP
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GRASPModel=Gemini 2.5 Pro2026.04 | 88.8 | 11.5 | |
| GRASPModel=DeepSeek V3.22026.04 | 84.9 | 11.9 | |
| BPOApproach=System-2, Evaluation Protocol=Fine-Tuned, Base Model=Llama-3.1-8B-Instruct2025.08 | 83.16 | — | |
| GRASPModel=Claude-4-Sonnet2026.04 | 82.4 | 13.8 | |
| GRASPModel=GPT-4.12026.04 | 81.6 | 13.7 | |
| STEP-HRLBackbone=Llama-3-8B, Evaluation Protocol=Fine-tuning (LoRA)2026.04 | 81.57 | — | |
| MPOApproach=System-2, Evaluation Protocol=Fine-Tuned, Base Model=Llama-3.1-8B-Instruct2025.08 | 80.91 | — | |
| STEP-HRLBackbone=Mistral-7B, Evaluation Protocol=Fine-tuning (LoRA)2026.04 | 80.28 | — | |
| GRASPModel=GLM-52026.04 | 79.8 | 14.8 | |
| ReAct + SkillsModel=DeepSeek V3.22026.04 | 79.6 | 15 | |
| GRASPModel=Kimi-K2.52026.04 | 79.5 | 15.2 | |
| ReAct + SkillsModel=GLM-52026.04 | 78.9 | 15 | |
| STEP-HRLBackbone=Gemma-7B, Evaluation Protocol=Fine-tuning (LoRA)2026.04 | 78.89 | — | |
| ReAct + SkillsModel=Gemini 2.5 Pro2026.04 | 78.8 | 13.8 | |
| ReAct + SkillsModel=Claude-4-Sonnet2026.04 | 78.8 | 14.2 | |
| GLIDERBackbone=Llama-3-8B, Evaluation Protocol=Fine-tuning (LoRA)2026.04 | 77.43 | — | |
| ReAct + SkillsModel=GPT-4.12026.04 | 76.7 | 14 | |
| ETOApproach=System-2, Evaluation Protocol=Fine-Tuned, Base Model=Llama-3.1-8B-Instruct2025.08 | 76.29 | — | |
| ReAct + SkillsModel=Kimi-K2.52026.04 | 75.5 | 15.6 | |
| ExpeLModel=GLM-52026.04 | 75.4 | 15.6 | |
| ExpeLModel=DeepSeek V3.22026.04 | 74.9 | 16 | |
| GRASPModel=o4 Mini2026.04 | 74.6 | 13.3 | |
| ExpeLModel=Claude-4-Sonnet2026.04 | 74.6 | 14.6 | |
| GRASPModel=Qwen3-235B2026.04 | 74.5 | 14.8 | |
| ReAct + SkillsModel=o4 Mini2026.04 | 74.1 | 13.3 | |
| ReAct + SkillsModel=Qwen3-235B2026.04 | 73 | 14.9 | |
| ExpeLModel=Gemini 2.5 Pro2026.04 | 72.8 | 15 | |
| ReflexionModel=DeepSeek V3.22026.04 | 72.4 | 16.3 | |
| ReflexionModel=GLM-52026.04 | 72.4 | 15.4 | |
| ExpeLModel=GPT-4.12026.04 | 72.1 | 14.4 | |
| ReflexionModel=Claude-4-Sonnet2026.04 | 71.2 | 15.3 | |
| ExpeLModel=Kimi-K2.52026.04 | 71.2 | 16 | |
| ReActModel=DeepSeek V3.22026.04 | 69.9 | 17.6 | |
| ExpeLModel=Qwen3-235B2026.04 | 69.4 | 15.3 | |
| ReflexionModel=Kimi-K2.52026.04 | 68.8 | 16 | |
| ReActModel=GLM-52026.04 | 68.2 | 16.6 | |
| SFTApproach=System-2, Evaluation Protocol=Fine-Tuned, Base Model=Llama-3.1-8B-Instruct2025.08 | 68.04 | — | |
| ExpeLModel=o4 Mini2026.04 | 68 | 13.7 | |
| ReflexionModel=o4 Mini2026.04 | 67.6 | 14.4 | |
| GLIDERBackbone=Mistral-7B, Evaluation Protocol=Fine-tuning (LoRA)2026.04 | 67.31 | — | |
| ReflexionModel=Gemini 2.5 Pro2026.04 | 66.8 | 16.9 | |
| ReflexionModel=GPT-4.12026.04 | 66.8 | 17 | |
| ReActModel=Claude-4-Sonnet2026.04 | 66.4 | 16.3 | |
| ReActModel=o4 Mini2026.04 | 64.9 | 15.1 | |
| GLIDERBackbone=Gemma-7B, Evaluation Protocol=Fine-tuning (LoRA)2026.04 | 63.67 | — | |
| ReflexionModel=Qwen3-235B2026.04 | 63.6 | 17.7 | |
| ReActModel=Kimi-K2.52026.04 | 63.6 | 17.4 | |
| WKMBackbone=Mistral-7B, Evaluation Protocol=Fine-tuning (LoRA)2026.04 | 62.12 | — | |
| WKMBackbone=Llama-3-8B, Evaluation Protocol=Fine-tuning (LoRA)2026.04 | 60.12 | — | |
| ReActModel=Gemini 2.5 Pro2026.04 | 58.2 | 18.4 | |
| ETOBackbone=Mistral-7B, Evaluation Protocol=Fine-tuning (LoRA)2026.04 | 58.17 | — | |
| ETOBackbone=Llama-3-8B, Evaluation Protocol=Fine-tuning (LoRA)2026.04 | 57.9 | — | |
| Qwen-3-8B-ThinkingApproach=System-2, Evaluation Protocol=Zero-shot, Model Category=Reasoning LLMs2025.08 | 57.45 | — | |
| ReActModel=GPT-4.12026.04 | 57.4 | 19.1 | |
| Deepseek-R1Approach=System-2, Evaluation Protocol=Zero-shot, Model Category=Reasoning LLMs2025.08 | 56.7 | — | |
| o3-miniApproach=System-2, Evaluation Protocol=Zero-shot, Model Category=Reasoning LLMs2025.08 | 56.28 | — | |
| ReActModel=Qwen3-235B2026.04 | 54.8 | 19.6 | |
| WKMBackbone=Gemma-7B, Evaluation Protocol=Fine-tuning (LoRA)2026.04 | 53.68 | — | |
| ETOBackbone=Gemma-7B, Evaluation Protocol=Fine-tuning (LoRA)2026.04 | 50.44 | — | |
| SwitchSageBackbone=Mistral-7B, Evaluation Protocol=Prompt-based2026.04 | 48.4 | — | |
| REACTBackbone=GPT-4, Evaluation Protocol=Prompt-based2026.04 | 44.29 | — | |
| Llama-3.1-8B-InstructApproach=System-1, Evaluation Protocol=Zero-shot, Model Category=Base Models2025.08 | 42.27 | — | |
| SwitchSageBackbone=Llama-3-8B, Evaluation Protocol=Prompt-based2026.04 | 42.22 | — | |
| SwitchSageBackbone=Gemma-7B, Evaluation Protocol=Prompt-based2026.04 | 33.43 | — | |
| Qwen-2.5-7B-InstructApproach=System-1, Evaluation Protocol=Zero-shot, Model Category=Base Models2025.08 | 28.35 | — | |
| ReflexionBackbone=Llama-3-8B, Evaluation Protocol=Prompt-based2026.04 | 27.23 | — | |
| ReActBackbone=Llama-3-8B, Evaluation Protocol=Prompt-based2026.04 | 24.76 | — | |
| ReflexionBackbone=Mistral-7B, Evaluation Protocol=Prompt-based2026.04 | 21.07 | — | |
| ReActBackbone=Mistral-7B, Evaluation Protocol=Prompt-based2026.04 | 20.72 | — | |
| REACTBackbone=GPT-3.5-Turbo, Evaluation Protocol=Prompt-based2026.04 | 8.57 | — | |
| ReflexionBackbone=Gemma-7B, Evaluation Protocol=Prompt-based2026.04 | 4.94 | — | |
| ReActBackbone=Gemma-7B, Evaluation Protocol=Prompt-based2026.04 | 3.58 | — |