Scientific Instruction Following on SciRIFF
53.6Accuracyo3
Evaluation results
| Method | Links | ||
|---|---|---|---|
| o3Reasoning Effort=High2025.08 | 53.6 | 1.8 | |
| Claude-Sonnet-4Reasoning Effort=Low2025.08 | 53.5 | — | |
| o4-miniReasoning Effort=High2025.08 | 52.2 | 1.6 | |
| o3Reasoning Effort=Low2025.08 | 51.8 | — | |
| o3-miniReasoning Effort=High2025.08 | 51.8 | 0.5 | |
| Gemini-2.5-ProReasoning Effort=Low2025.08 | 51.6 | — | |
| Gemini-2.5-ProReasoning Effort=High2025.08 | 51.4 | -0.2 | |
| o3-miniReasoning Effort=Low2025.08 | 51.3 | — | |
| Claude-Sonnet-4Reasoning Effort=High2025.08 | 50.9 | -2.6 | |
| o4-miniReasoning Effort=Low2025.08 | 50.6 | — | |
| GPT-5Reasoning Effort=High2025.08 | 50.1 | 3.3 | |
| GPT-5Reasoning Effort=Low2025.08 | 46.9 | — |