Materials data quality evaluation on HEA property datasets
4.17Standard Deviation (Std.)gpt-5
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| gpt-5Size=-, external knowledge access=true2026.05 | 4.17 | — | — | — | — | 47.33 | 5.26 | 0.96 | |
| Qwen3Size=8B, Supervised Fine-Tuning (SFT)=true, Direct Preference Optimization (DPO)=true2026.05 | 4.56 | 5.67 | 7.69 | 5.67 | 0.9 | 32.67 | 5.71 | 0.94 | |
| gpt-5Size=-2026.05 | 4.7 | 11.27 | 15.42 | 11.27 | 0.6 | 42.67 | 5.91 | 0.92 | |
| HunyuanSize=7B, Supervised Fine-Tuning (SFT)=true, Direct Preference Optimization (DPO)=true2026.05 | 4.82 | 5.76 | 7.79 | 5.76 | 0.9 | 31 | 6.12 | 0.93 | |
| DeepSeek-R1Size=671B2026.05 | 4.92 | 14.68 | 19.47 | 14.68 | 0.36 | 28 | 6.16 | 0.85 | |
| DeepSeek-R1Size=671B, external knowledge access=true2026.05 | 5 | — | — | — | — | 29.67 | 6.27 | 0.93 | |
| Qwen3Size=4B, Supervised Fine-Tuning (SFT)=true, Direct Preference Optimization (DPO)=true2026.05 | 5.8 | 6.37 | 8.51 | 6.37 | 0.88 | 16.33 | 7.39 | 0.91 | |
| HunyuanSize=4B, Supervised Fine-Tuning (SFT)=true, Direct Preference Optimization (DPO)=true2026.05 | 6 | 6.57 | 8.66 | 6.57 | 0.87 | 15.67 | 7.62 | 0.91 | |
| Claude-sonnet-4.5Size=-2026.05 | 6.43 | 13.39 | 16.65 | 13.39 | 0.53 | 16 | 8.15 | 0.9 | |
| gpt-4oSize=-2026.05 | 6.69 | 14.25 | 17.86 | 14.25 | 0.46 | 13.33 | 8.51 | 0.89 | |
| HunyuanSize=1.8B, Supervised Fine-Tuning (SFT)=true, Direct Preference Optimization (DPO)=true2026.05 | 6.97 | 7.32 | 9.51 | 7.32 | 0.85 | 10.67 | 8.87 | 0.88 | |
| Qwen3Size=1.7B, Supervised Fine-Tuning (SFT)=true, Direct Preference Optimization (DPO)=true2026.05 | 7.63 | 7.83 | 9.98 | 7.83 | 0.83 | 6 | 9.66 | 0.87 | |
| Gemini-2.5-proSize=-2026.05 | 8.57 | 15.16 | 18.93 | 15.16 | 0.39 | 7.67 | 10.91 | 0.84 | |
| Hunyuan-7B-InstructSize=7B2026.05 | 17.16 | 18.77 | 23.2 | 18.77 | 0.09 | 2 | 21.8 | 0.59 | |
| Qwen3-8B-InstructSize=8B2026.05 | 17.65 | 18.26 | 22.58 | 18.26 | 0.13 | 1.33 | 22.39 | 0.59 |