Synthesis operation prediction on Material Synthesis Dataset (Split 1)
7.86Top-1 AccuracyMSP-LLM
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| MSP-LLMBackbone=Qwen-2.5-7B2026.02 | 7.86 | 15.36 | 19.65 | 26.51 | |
| MSP-LLMBackbone=LlaMA-3.1-8B2026.02 | 6.4 | 13.71 | 19.2 | 26.51 | |
| SPENDE2026.02 | 3.66 | — | — | — | |
| GPT-4o-minin-shot=40-shot2026.02 | 1.83 | 2.74 | 3.56 | 5.94 | |
| GPT-4o-minin-shot=20-shot2026.02 | 1.73 | 3.11 | 4.2 | 5.85 | |
| Retrieval based2026.02 | 1.46 | 3.75 | 5.94 | 10.42 | |
| GPT-5.1n-shot=40-shot2026.02 | 0.91 | 1.74 | 1.74 | 2.74 | |
| Claude Haiku 4.5n-shot=40-shot2026.02 | 0.91 | 2.38 | 3.29 | 4.11 | |
| GPT-5.1n-shot=20-shot2026.02 | 0.73 | 1.55 | 1.65 | 2.93 | |
| GPT-4on-shot=40-shot2026.02 | 0.18 | 1.01 | 1.92 | 4.34 | |
| Claude Haiku 4.5n-shot=20-shot2026.02 | 0.18 | 0.73 | 1.56 | 2.56 | |
| GPT-4on-shot=20-shot2026.02 | 0.09 | 1.09 | 2.56 | 5.03 |