Interactive Decision Making on ScienceWorld Unseen
85.15Success RateBPO
Evaluation Results
| Method | Links | |
|---|---|---|
| BPOApproach=System-2, Evaluation Protocol=Fine-Tuned, Base Model=Llama-3.1-8B-Instruct2025.08 | 85.15 | |
| STEP-HRLBackbone=Llama-3-8B, Evaluation Protocol=Fine-tuning (LoRA)2026.04 | 77.81 | |
| MPOApproach=System-2, Evaluation Protocol=Fine-Tuned, Base Model=Llama-3.1-8B-Instruct2025.08 | 77.46 | |
| ETOApproach=System-2, Evaluation Protocol=Fine-Tuned, Base Model=Llama-3.1-8B-Instruct2025.08 | 77.25 | |
| STEP-HRLBackbone=Mistral-7B, Evaluation Protocol=Fine-tuning (LoRA)2026.04 | 75.21 | |
| STEP-HRLBackbone=Gemma-7B, Evaluation Protocol=Fine-tuning (LoRA)2026.04 | 74.08 | |
| SFTApproach=System-2, Evaluation Protocol=Fine-Tuned, Base Model=Llama-3.1-8B-Instruct2025.08 | 72.04 | |
| GLIDERBackbone=Llama-3-8B, Evaluation Protocol=Fine-tuning (LoRA)2026.04 | 68.34 | |
| GLIDERBackbone=Mistral-7B, Evaluation Protocol=Fine-tuning (LoRA)2026.04 | 65.14 | |
| Deepseek-R1Approach=System-2, Evaluation Protocol=Zero-shot, Model Category=Reasoning LLMs2025.08 | 59.62 | |
| GLIDERBackbone=Gemma-7B, Evaluation Protocol=Fine-tuning (LoRA)2026.04 | 58.5 | |
| o3-miniApproach=System-2, Evaluation Protocol=Zero-shot, Model Category=Reasoning LLMs2025.08 | 55.45 | |
| WKMBackbone=Llama-3-8B, Evaluation Protocol=Fine-tuning (LoRA)2026.04 | 54.75 | |
| WKMBackbone=Mistral-7B, Evaluation Protocol=Fine-tuning (LoRA)2026.04 | 53.62 | |
| ETOBackbone=Llama-3-8B, Evaluation Protocol=Fine-tuning (LoRA)2026.04 | 52.33 | |
| ETOBackbone=Mistral-7B, Evaluation Protocol=Fine-tuning (LoRA)2026.04 | 51.85 | |
| Qwen-3-8B-ThinkingApproach=System-2, Evaluation Protocol=Zero-shot, Model Category=Reasoning LLMs2025.08 | 50.27 | |
| WKMBackbone=Gemma-7B, Evaluation Protocol=Fine-tuning (LoRA)2026.04 | 49.24 | |
| ETOBackbone=Gemma-7B, Evaluation Protocol=Fine-tuning (LoRA)2026.04 | 47.84 | |
| Llama-3.1-8B-InstructApproach=System-1, Evaluation Protocol=Zero-shot, Model Category=Base Models2025.08 | 46.92 | |
| SwitchSageBackbone=Mistral-7B, Evaluation Protocol=Prompt-based2026.04 | 45.25 | |
| SwitchSageBackbone=Llama-3-8B, Evaluation Protocol=Prompt-based2026.04 | 40.58 | |
| REACTBackbone=GPT-4, Evaluation Protocol=Prompt-based2026.04 | 38.05 | |
| Qwen-2.5-7B-InstructApproach=System-1, Evaluation Protocol=Zero-shot, Model Category=Base Models2025.08 | 33.18 | |
| SwitchSageBackbone=Gemma-7B, Evaluation Protocol=Prompt-based2026.04 | 30.9 | |
| ReflexionBackbone=Llama-3-8B, Evaluation Protocol=Prompt-based2026.04 | 25.41 | |
| ReActBackbone=Llama-3-8B, Evaluation Protocol=Prompt-based2026.04 | 22.66 | |
| ReflexionBackbone=Mistral-7B, Evaluation Protocol=Prompt-based2026.04 | 18.11 | |
| ReActBackbone=Mistral-7B, Evaluation Protocol=Prompt-based2026.04 | 17.65 | |
| REACTBackbone=GPT-3.5-Turbo, Evaluation Protocol=Prompt-based2026.04 | 5.97 | |
| ReflexionBackbone=Gemma-7B, Evaluation Protocol=Prompt-based2026.04 | 3.93 | |
| ReActBackbone=Gemma-7B, Evaluation Protocol=Prompt-based2026.04 | 3.51 |