Next-state prediction on SciWorld
98.64EM AccuracyLlama3.1-8B
Evaluation results
| Method | Links | |
|---|---|---|
| Llama3.1-8BEvaluation Protocol=SFT2025.12 | 98.64 | |
| Qwen2.5-7BEvaluation Protocol=SFT2025.12 | 98.6 | |
| Claude-sonnet-4.5Evaluation Protocol=Few-shot (3 shot)2025.12 | 73.08 | |
| Gemini-2.5-flashEvaluation Protocol=Few-shot (3 shot)2025.12 | 61.2 | |
| Claude-sonnet-4.5Evaluation Protocol=Zero-shot2025.12 | 56.83 | |
| GPT-4o-miniEvaluation Protocol=Few-shot (3 shot)2025.12 | 56.26 | |
| GPT-4.1Evaluation Protocol=Few-shot (3 shot)2025.12 | 51.56 | |
| GPT-4-turboEvaluation Protocol=Few-shot (3 shot)2025.12 | 50.08 | |
| GPT-5Evaluation Protocol=Few-shot (3 shot)2025.12 | 49.44 | |
| GPT-4oEvaluation Protocol=Few-shot (3 shot)2025.12 | 48.98 | |
| GPT-4oEvaluation Protocol=Zero-shot2025.12 | 45.78 | |
| Gemini-2.5-flashEvaluation Protocol=Zero-shot2025.12 | 44.81 | |
| GPT-4o-miniEvaluation Protocol=Zero-shot2025.12 | 40.68 | |
| GPT-4.1Evaluation Protocol=Zero-shot2025.12 | 35.65 | |
| GPT-4-turboEvaluation Protocol=Zero-shot2025.12 | 34.14 | |
| GPT-5Evaluation Protocol=Zero-shot2025.12 | 13.06 |