Material Synthesis Planning on MSP Dataset (Split 1)
5.76Top-1 AccuracyMSP-LLM
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| MSP-LLMbackbone=Qwen-2.5-7B2026.02 | 5.76 | 10.79 | 13.62 | 18.56 | |
| MSP-LLMbackbone=LLaMA-3.1-8B2026.02 | 4.75 | 9.69 | 13.35 | 18.19 | |
| GPT-5.1(PP) + GPT-4o-mini (SOP)pipeline=hybrid2026.02 | 0.91 | 1.65 | 2.01 | 3.75 | |
| Claude Haiku 4.5shot count=20-shot2026.02 | 0.56 | 0.65 | 0.65 | 0.74 | |
| Claude Haiku 4.5shot count=40-shot2026.02 | 0.56 | 0.65 | 0.65 | 0.74 | |
| GPT-5.1shot count=40-shot2026.02 | 0.55 | 0.64 | 0.64 | 0.73 | |
| GPT-5.1shot count=20-shot2026.02 | 0.46 | 0.64 | 0.73 | 1.01 | |
| GPT-4oshot count=20-shot2026.02 | 0.27 | 0.55 | 0.64 | 0.73 | |
| GPT-4oshot count=40-shot2026.02 | 0.27 | 0.27 | 0.37 | 0.64 | |
| GPT-4o-minishot count=20-shot2026.02 | 0 | 0.09 | 0.09 | 0.19 | |
| GPT-4o-minishot count=40-shot2026.02 | 0 | 0 | 0.09 | 0.09 |