Text-based de novo Molecule Generation on ChEBI-20 (test)
78.32BLEUMolReasoner
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| MolReasonerSize=7B2025.08 | 78.32 | 7.46 | 26.0237 | 43.69 | 67.62 | 36.18 | 52.21 | 64.19 | 53.9 | 96.55 | |
| LLaMA3 + MSRSize=8B, Fine-tuning (ft)=true2025.08 | 33.82 | 10.73 | 31.2113 | 40.13 | 65.18 | 34.25 | 45.69 | 53.61 | 51.97 | 98.01 | |
| Mol-Instructions + LLaMA3Size=8B, Fine-tuning (ft)=true2025.08 | 33.43 | 7.4 | 37.7033 | 37.99 | 50.96 | 32.46 | 41.57 | 47.28 | 51.3 | 97.14 | |
| Mol-Instructions + LLaMA3Size=8B, Fine-tuning (ft)=false2025.08 | 33.23 | 7.38 | 38.1494 | 35.98 | 49.56 | 31.67 | 40.71 | 47.14 | 50.56 | 97.07 | |
| LLaMA3 + MSRSize=8B, n-shot=10-shot2025.08 | 31.34 | 0.19 | 43.7624 | 14.19 | 30.04 | 9.66 | 19.55 | 28.47 | 29.46 | 78.89 | |
| Mol-Instructions + LLaMA2Size=7B, Fine-tuning (ft)=false2025.08 | 30.49 | 4.37 | 39.4265 | 29.14 | 43.94 | 25.24 | 33.26 | 41.24 | 43.23 | 99.91 | |
| GPT-4oModel Type=Closed-Source2025.08 | 17.23 | 0.62 | 50.2363 | 9.28 | 20.64 | 8.44 | 12.82 | 17.4 | 36.78 | 32.24 | |
| LLaMA3.1-70B-InstructSize=70B2025.08 | 8.19 | 0.42 | 44.6099 | 8.31 | 23.44 | 7.75 | 13.67 | 19.45 | 35.22 | 46.65 | |
| GPT-4o-miniModel Type=Closed-Source2025.08 | 5.32 | 0.47 | 48.0824 | 8.58 | 20.57 | 8.75 | 9.28 | 12.35 | 38.6 | 19.32 | |
| Deepseek-R1Model Type=Closed-Source2025.08 | 1.45 | 0.63 | 55.3267 | 4.08 | 8.9 | 3.82 | 3.72 | 4.55 | 39.06 | 69.32 | |
| LLaMA3.1-8B-InstructSize=8B2025.08 | 1.33 | 0.26 | 40.8965 | 5.8 | 15.04 | 4.73 | 7.45 | 9.64 | 35.97 | 23.75 | |
| Qwen2.5-32B-InstructSize=32B2025.08 | 0.54 | 0.42 | 33.7951 | 12.04 | 25.66 | 11.63 | 12.57 | 15.86 | 35.42 | 17.22 | |
| Qwen2.5-7B-InstructSize=7B2025.08 | 0.01 | 0.31 | 35.2666 | 10.22 | 22.13 | 8.04 | 10.52 | 14.31 | 35.36 | 19.9 | |
| DeepSeek-R1-Qwen-7BSize=7B2025.08 | 0 | 0.22 | 49.8502 | 7.32 | 13.61 | 5.67 | 11.9 | 15.02 | 38.43 | 6.65 | |
| Qwen3-8BSize=8B2025.08 | 0 | 0.34 | 26.1532 | 35.78 | 46.05 | 30.34 | 35.04 | 37.28 | 52.99 | 1.19 | |
| Qwen2.5-72B-InstructSize=72B2025.08 | 0 | 0.44 | 17.8122 | 14.96 | 33.85 | 13.47 | 15.79 | 21.6 | 33.52 | 11.73 |