Scientific Reasoning on Principia OOD (test)
83.78Average Score @8CoT Self-Instruct
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| CoT Self-InstructCategory=ARB, Items=47, Training Data Source=CoT Self-Instruct2026.06 | 83.78 | 91.49 | 1.87 | |
| Combined (2× data)Category=ARB, Items=47, Training Data Source=Combined2026.06 | 83.24 | 93.62 | 1.33 | |
| Qwen3.5-4BCategory=ARB, Items=47, Training Data Source=Base2026.06 | 81.91 | 91.49 | — | |
| Agentic Self-InstructCategory=ARB, Items=47, Training Data Source=Agentic Self-Instruct2026.06 | 80.32 | 91.49 | -1.59 | |
| Agentic Self-InstructCategory=Physics, Items=110, Training Data Source=Agentic Self-Instruct2026.06 | 67.05 | 79.09 | 0.83 | |
| Combined (2× data)Category=Physics, Items=110, Training Data Source=Combined2026.06 | 66.25 | 72.73 | 0.03 | |
| Qwen3.5-4BCategory=Physics, Items=110, Training Data Source=Base2026.06 | 66.22 | 76.36 | — | |
| CoT Self-InstructCategory=Physics, Items=110, Training Data Source=CoT Self-Instruct2026.06 | 65.91 | 77.27 | -0.31 | |
| Agentic Self-InstructCategory=SuperGPQA, Items=1324, Training Data Source=Agentic Self-Instruct2026.06 | 56.82 | 64.95 | 0.82 | |
| Combined (2× data)Category=SuperGPQA, Items=1324, Training Data Source=Combined2026.06 | 56.5 | 65.11 | 0.5 | |
| CoT Self-InstructCategory=SuperGPQA, Items=1324, Training Data Source=CoT Self-Instruct2026.06 | 56.28 | 65.33 | 0.28 | |
| Qwen3.5-4BCategory=SuperGPQA, Items=1324, Training Data Source=Base2026.06 | 56 | 65.41 | — | |
| Agentic Self-InstructCategory=Overall, Items=2113, Training Data Source=Agentic Self-Instruct2026.06 | 51.47 | 59.91 | 1.04 | |
| Combined (2× data)Category=Overall, Items=2113, Training Data Source=Combined2026.06 | 51.17 | 59.91 | 0.74 | |
| CoT Self-InstructCategory=Overall, Items=2113, Training Data Source=CoT Self-Instruct2026.06 | 51.1 | 60.06 | 0.67 | |
| Qwen3.5-4BCategory=Overall, Items=2113, Training Data Source=Base2026.06 | 50.43 | 59.54 | — | |
| Agentic Self-InstructCategory=RealMath, Items=632, Training Data Source=Agentic Self-Instruct2026.06 | 35.43 | 43.67 | 1.75 | |
| CoT Self-InstructCategory=RealMath, Items=632, Training Data Source=CoT Self-Instruct2026.06 | 35.25 | 43.67 | 1.57 | |
| Combined (2× data)Category=RealMath, Items=632, Training Data Source=Combined2026.06 | 34.97 | 44.3 | 1.29 | |
| Qwen3.5-4BCategory=RealMath, Items=632, Training Data Source=Base2026.06 | 33.68 | 41.93 | — |