General-purpose Language Evaluation on SCAN
242.24Overall ScoreGPT-OSS-120B
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GPT-OSS-120BParameter Scale=>100B, Access Mode=Open-source2025.05 | 242.24 | 233.59 | 231.33 | 247.99 | 263.35 | 241.59 | 235.7 | |
| Qwen3-32BParameter Scale=<100B, Access Mode=Open-source2025.05 | 241.13 | 241.29 | 246.03 | 234.47 | 246.87 | 245.29 | 224.93 | |
| Deepseek-V3-250324Parameter Scale=>100B, Access Mode=Open-source2025.05 | 237.65 | 233.04 | 237.41 | 250.91 | 247.09 | 220.01 | 229.93 | |
| Deepseek-R1-250120Parameter Scale=>100B, Access Mode=Open-source2025.05 | 235.25 | 244.14 | 248.29 | 238.19 | 231.94 | 190.58 | 234.91 | |
| Gemma-3-27B-ITParameter Scale=<100B, Access Mode=Open-source2025.05 | 229.88 | 236.63 | 243.59 | 253.25 | 207.01 | 192.19 | 233.82 | |
| QwQ-32BParameter Scale=<100B, Access Mode=Open-source2025.05 | 229.38 | 232.26 | 250.41 | 237.71 | 224.73 | 187.67 | 228.24 | |
| claude-3-7-sonnet-20250219Parameter Scale=>100B, Access Mode=Closed-source2025.05 | 225.07 | 232.74 | 224.94 | 229.79 | 238.37 | 187.36 | 203.56 | |
| Qwen3-8BParameter Scale=<100B, Access Mode=Open-source2025.05 | 222.88 | 224.63 | 227.69 | 210.07 | 219.59 | 241.4 | 216.47 | |
| gpt-4o-2024-11-20Parameter Scale=>100B, Access Mode=Closed-source2025.05 | 214.8 | 225.9 | 234.42 | 219.31 | 202.03 | 171.82 | 210.5 | |
| GPT-OSS-20BParameter Scale=<100B, Access Mode=Open-source2025.05 | 213.72 | 196.23 | 176.09 | 230.79 | 251.59 | 225.26 | 212.92 | |
| Gemma-3-4B-ITParameter Scale=<100B, Access Mode=Open-source2025.05 | 203.43 | 224.3 | 230.57 | 219.62 | 167.27 | 137.9 | 207.15 | |
| Qwen2.5-72B-InstructParameter Scale=<100B, Access Mode=Open-source2025.05 | 190.42 | 197.99 | 179.91 | 194.18 | 190.53 | 171.61 | 194.16 | |
| qwen-max-2024-10-15Parameter Scale=>100B, Access Mode=Closed-source2025.05 | 182.59 | 187.44 | 174.14 | 192.56 | 183.33 | 164.28 | 178.68 | |
| doubao-1-5-pro-32k-250115Parameter Scale=>100B, Access Mode=Closed-source2025.05 | 182.23 | 175.36 | 173.19 | 192.4 | 187.05 | 192.35 | 181.46 | |
| Qwen2.5-32B-InstructParameter Scale=<100B, Access Mode=Open-source2025.05 | 176.1 | 185.19 | 170.58 | 174.2 | 174.39 | 162.31 | 173.53 | |
| Meta-Llama-3.1-70B-InstructParameter Scale=<100B, Access Mode=Open-source2025.05 | 170.51 | 183.13 | 178.49 | 167.73 | 165.25 | 133.37 | 170.07 | |
| Qwen2.5-7B-InstructParameter Scale=<100B, Access Mode=Open-source2025.05 | 166.67 | 179.1 | 164.64 | 162.77 | 161.99 | 146.67 | 163.15 | |
| Meta-Llama-3.1-8B-InstructParameter Scale=<100B, Access Mode=Open-source2025.05 | 159.01 | 176.65 | 159.26 | 152.3 | 154.93 | 119.67 | 159.35 | |
| hunyuan-standard-2025-02-10Parameter Scale=>100B, Access Mode=Closed-source2025.05 | 156.44 | 163.21 | 144.43 | 156.85 | 158.49 | 149.47 | 152.47 | |
| Mistral-7B-Instruct-v0.3Parameter Scale=<100B, Access Mode=Open-source2025.05 | 148.94 | 163.44 | 158.07 | 152.55 | 138.24 | 114.74 | 133.61 | |
| Phi-4-mini-instructParameter Scale=<100B, Access Mode=Open-source2025.05 | 143.99 | 150.48 | 144.18 | 142.36 | 143.21 | 123.93 | 148.08 |