Multimodal Scientific Reasoning on SFE (test)
68.1AccuracySciOrch
Evaluation results
| Method | Links | |
|---|---|---|
| SciOrchOrchestrator=Qwen3-VL-8B2026.06 | 68.1 | |
| GPT-5.4Base Model=GPT-5.4, Evaluation Strategy=Vanilla2026.06 | 67.02 | |
| Gemini-3-ProBase Model=Gemini-3-Pro, Evaluation Strategy=Consistency2026.06 | 64.89 | |
| Gemini-3-ProBase Model=Gemini-3-Pro, Evaluation Strategy=Debate2026.06 | 64.89 | |
| GPT-5.4Base Model=GPT-5.4, Evaluation Strategy=Debate2026.06 | 62.77 | |
| Gemini-3-ProBase Model=Gemini-3-Pro, Evaluation Strategy=Vanilla2026.06 | 62.77 | |
| GPT-5.4Base Model=GPT-5.4, Evaluation Strategy=Consistency2026.06 | 60.64 | |
| P2LEvaluation Strategy=Routing-based2026.06 | 57.5 | |
| Claude-Sonnet-4.5Base Model=Claude-Sonnet-4.5, Evaluation Strategy=Consistency2026.06 | 57.45 | |
| Claude-Sonnet-4.5Base Model=Claude-Sonnet-4.5, Evaluation Strategy=Debate2026.06 | 52.13 | |
| Claude-Sonnet-4.5Base Model=Claude-Sonnet-4.5, Evaluation Strategy=Vanilla2026.06 | 48.94 | |
| Qwen3-VL-8BBase Model=Qwen3-VL-8B, Evaluation Strategy=Debate2026.06 | 44.68 | |
| Qwen3-VL-8BBase Model=Qwen3-VL-8B, Evaluation Strategy=Consistency2026.06 | 43.62 | |
| Qwen3-VL-8BBase Model=Qwen3-VL-8B, Evaluation Strategy=GRPO2026.06 | 41.3 | |
| Qwen3-VL-8BBase Model=Qwen3-VL-8B, Evaluation Strategy=SFT2026.06 | 39.4 | |
| Qwen3-VL-8BBase Model=Qwen3-VL-8B, Evaluation Strategy=Vanilla2026.06 | 31.9 |