Multi-domain language model evaluation on ODA benchmark suite (test)
71.2General AccuracyODA-Mixture-500k
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| ODA-Mixture-500kBase Model=Qwen3-8B-Base, Size=506k2025.12 | 71.2 | 77.2 | 73 | 69.7 | 72.8 | 0.042 | |
| AM-Thinking (math)Base Model=Qwen3-8B-Base, Size=558k2025.12 | 65.9 | 79.7 | 59.5 | 63.2 | 67.1 | 0.028 | |
| Light-R1-SFTBase Model=Qwen3-8B-Base, Size=79k2025.12 | 64.9 | 71.8 | 59 | 63.6 | 64.8 | 0.168 | |
| AM-Thinking (code)Base Model=Qwen3-8B-Base, Size=324k2025.12 | 64.8 | 64.9 | 75.8 | 59.3 | 66.2 | 0.045 | |
| MiroMind-M1-SFTBase Model=Qwen3-8B-Base, Size=719k2025.12 | 64.5 | 77.2 | 63.6 | 65.8 | 67.8 | 0.023 | |
| ODA-Mixture-500kBase Model=Qwen2.5-7B-Base, Size=506k2025.12 | 63.4 | 72.8 | 66.7 | 59.6 | 65.6 | 0.039 | |
| LIMOBase Model=Qwen3-8B-Base, Size=8172025.12 | 61.7 | 46 | 52.7 | 54.1 | 53.6 | 0.49 | |
| ODA-Mixture-100kBase Model=Qwen3-8B-Base, Size=101k2025.12 | 61.1 | 77.3 | 73.2 | 64.7 | 69 | 0.177 | |
| LIMOBase Model=Qwen2.5-7B-Base, Size=8172025.12 | 60.7 | 44 | 57.9 | 53.8 | 54.1 | 9.92 | |
| SYNTHETIC-2-SFTBase Model=Qwen3-8B-Base, Size=105k2025.12 | 59.5 | 75.4 | 56.1 | 66.6 | 64.4 | 0.107 | |
| Qwen3-8B-BaseBase Model=Qwen3-8B-Base2025.12 | 58.7 | 51.2 | 52.4 | 50.6 | 53.2 | — | |
| AM-Thinking (math)Base Model=Qwen2.5-7B-Base, Size=558k2025.12 | 57.7 | 77.4 | 39.5 | 44.8 | 54.8 | 0.016 | |
| ODA-Mixture-100kBase Model=Qwen2.5-7B-Base, Size=101k2025.12 | 56.8 | 71.2 | 64.4 | 51.5 | 61 | 0.149 | |
| OmniThought-0528Base Model=Qwen3-8B-Base, Size=365k2025.12 | 55.8 | 78.3 | 68.1 | 66 | 67 | 0.043 | |
| Light-R1-SFTBase Model=Qwen2.5-7B-Base, Size=79k2025.12 | 55.5 | 64.4 | 38.8 | 51.9 | 52.7 | 0.084 | |
| MiroMind-M1-SFTBase Model=Qwen2.5-7B-Base, Size=719k2025.12 | 52 | 71 | 26.3 | 51.5 | 50.2 | 0.006 | |
| Qwen2.5-7B-BaseBase Model=Qwen2.5-7B-Base2025.12 | 51.4 | 39.8 | 50.1 | 42.7 | 46 | — | |
| SYNTHETIC-2-SFTBase Model=Qwen2.5-7B-Base, Size=105k2025.12 | 51.3 | 69.8 | 40.1 | 58.9 | 55 | 0.086 | |
| AM-Thinking (code)Base Model=Qwen2.5-7B-Base, Size=324k2025.12 | 49.9 | 52.3 | 68.7 | 44.4 | 53.8 | 0.024 | |
| OmniThought-0528Base Model=Qwen2.5-7B-Base, Size=365k2025.12 | 47.1 | 71.2 | 47.6 | 57.2 | 55.8 | 0.027 | |
| OpenThoughts3Base Model=Qwen2.5-7B-Base, Size=1.2M2025.12 | 45.5 | 71.8 | 67 | 54.3 | 59.6 | 0.011 |