Scientific Problem-Solving on SCIREAS Suite
55.8GPQAQwen3
Evaluation Results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3Base Model=Q3-Base, Training Methods=--, Trained by Us=No2025.08 | 55.8 | 23.32 | 73.36 | 36.99 | 28.51 | 54.05 | 81.51 | 37.99 | 47.23 | 30.98 | 46.97 | 19.05 | |
| Qwen3-thinkingBase Model=Q3-Base, Training Methods=--, Trained by Us=No2025.08 | 55.8 | 38.27 | 81.71 | 38.19 | 21.3 | 68.21 | 84.02 | 41.81 | 47.26 | 59.81 | 53.64 | 29.92 | |
| SCILIT01Base Model=Q3-Base, Training Methods=SFT, Trained by Us=Yes2025.08 | 50.89 | 30.11 | 76.57 | 35.07 | 43.78 | 61.27 | 80.6 | 39.46 | 44.01 | 52.28 | 51.41 | 24.84 | |
| OpenR1Base Model=Q2.5-Math, Training Methods=SFT, Trained by Us=No2025.08 | 44.42 | 31.9 | 60.86 | 27.14 | 53.03 | 61.85 | 43.64 | 28.45 | 29.17 | 50.3 | 43.08 | 26.43 | |
| Qwen-NemotronBase Model=Q2.5-Inst., Training Methods=SFT, Trained by Us=Yes2025.08 | 44.2 | 19.47 | 63.57 | 35.76 | 29.33 | 48.27 | 38.53 | 31.85 | 39.24 | 46.52 | 39.67 | 19.68 | |
| Qwen-BOTHBase Model=Q2.5-Inst., Training Methods=SFT, Trained by Us=Yes2025.08 | 40.63 | 20.33 | 65 | 33 | 34.55 | 47.11 | 72.36 | 32 | 41.81 | 40.03 | 42.68 | 21.11 | |
| SYNTHETIC-1-SFTBase Model=Q2.5-Inst., Training Methods=SFT, Trained by Us=No2025.08 | 38.84 | 22.39 | 56.21 | 28.61 | 40.75 | 51.59 | 46.41 | 19.13 | 28.57 | 43.96 | 37.64 | 19.44 | |
| Llama-NemotronBase Model=L3.1-Inst., Training Methods=SFT&RL, Trained by Us=No2025.08 | 37.95 | 29.39 | 65.64 | 27.78 | 37.62 | 57.66 | 68.67 | 30.69 | 34.01 | 45.92 | 43.53 | 23.75 | |
| General-ReasonerBase Model=Q2.5-Base, Training Methods=RL, Trained by Us=No2025.08 | 35.94 | 14.26 | 62.14 | 35.58 | 19.82 | 19.08 | 70.34 | 34.19 | 37.75 | 20.86 | 34.99 | 13.73 |