Material Synthesis Planning on MSP Dataset (Split 2)
6.72Top-1 AccuracyMSP-LLM
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| MSP-LLMbackbone=LLaMA-3.1-8B2026.02 | 6.72 | 12.12 | 16.6 | 23.23 | |
| MSP-LLMbackbone=Qwen-2.5-7B2026.02 | 6.62 | 13.34 | 16.79 | 22.48 | |
| GPT-5.1(PP) + GPT-4o-mini (SOP)pipeline=hybrid2026.02 | 1.4 | 2.71 | 3.64 | 5.41 | |
| GPT-5.1shot count=20-shot2026.02 | 0.65 | 0.75 | 0.75 | 0.75 | |
| Claude Haiku 4.5shot count=20-shot2026.02 | 0.56 | 0.65 | 0.65 | 0.75 | |
| GPT-4oshot count=20-shot2026.02 | 0.47 | 0.47 | 0.47 | 0.56 | |
| GPT-4oshot count=40-shot2026.02 | 0.37 | 0.84 | 0.93 | 1.12 | |
| GPT-5.1shot count=40-shot2026.02 | 0.37 | 0.37 | 0.47 | 0.47 | |
| Claude Haiku 4.5shot count=40-shot2026.02 | 0.37 | 0.84 | 0.93 | 1.12 | |
| GPT-4o-minishot count=40-shot2026.02 | 0.27 | 0.27 | 0.37 | 0.64 | |
| GPT-4o-minishot count=20-shot2026.02 | 0.19 | 0.28 | 0.28 | 0.47 |