Synthesis operation prediction on Material Synthesis Dataset (Split 2)
75Top-1 AccuracyGPT-5.1
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GPT-5.1n-shot=40-shot2026.02 | 75 | 1.03 | 1.77 | 2.33 | |
| MSP-LLMBackbone=Qwen-2.5-7B2026.02 | 8.77 | 18.47 | 22.57 | 32.18 | |
| MSP-LLMBackbone=LlaMA-3.1-8B2026.02 | 8.12 | 17.35 | 22.58 | 31.44 | |
| SPENDE2026.02 | 5.71 | — | — | — | |
| GPT-4o-minin-shot=40-shot2026.02 | 2.15 | 4.2 | 5.69 | 7.93 | |
| GPT-4o-minin-shot=20-shot2026.02 | 1.68 | 3.26 | 4.01 | 7.09 | |
| Retrieval based2026.02 | 1.12 | 3.54 | 5.5 | 11.19 | |
| GPT-5.1n-shot=20-shot2026.02 | 0.75 | 1.31 | 1.4 | 2.33 | |
| GPT-4on-shot=20-shot2026.02 | 0.47 | 1.03 | 1.77 | 3.82 | |
| Claude Haiku 4.5n-shot=20-shot2026.02 | 0.47 | 0.84 | 1.21 | 1.68 | |
| Claude Haiku 4.5n-shot=40-shot2026.02 | 0.47 | 1.49 | 2.52 | 3.26 | |
| GPT-4on-shot=40-shot2026.02 | 0.28 | 1.03 | 2.05 | 4.94 |