Text-guided molecule generation on ChEBI-20 (test)
100MACCS FTS SimilarityGround Truth
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Ground Truth2024.02 | 100 | 100 | 100 | 0 | 100 | 100 | 0 | 60.9 | 100 | |
| Ground Truth2022.04 | 100 | 100 | 100 | 0 | 100 | 100 | 0 | 60.9 | 100 | |
| LDMolFamily=DM2024.05 | 97.3 | 92.6 | 53 | 6.75 | 95 | 93.1 | 0.2 | — | 94.1 | |
| bioT5+Family=AR2024.05 | 90.7 | 87.2 | 52.2 | 12.776 | 83.5 | 77.9 | 0.35 | — | 100 | |
| MolReGPTbackbone=GPT-4-0314, shots=10-shot2023.06 | 90.3 | 85.7 | 28 | 17.14 | 80.5 | 73.9 | 0.41 | 59.3 | 89.9 | |
| bioT5Family=AR2024.05 | 88.6 | 86.7 | 41.3 | 15.097 | 80.1 | 73.4 | 0.43 | — | 100 | |
| PEIT-LLM-LLaMa3.1#Params=8B2024.12 | 87.5 | 83.6 | — | 18.03 | 77.6 | 66.1 | — | — | 97 | |
| TGM-DLMParameters=~180M, Correction Phase=Disabled2024.02 | 87.4 | 82.8 | 24.2 | 16.897 | 77.1 | 72.2 | 0.89 | 58.9 | 78.9 | |
| MolXPTNumber of Parameters=350M2023.05 | 85.9 | — | 21.5 | — | 75.7 | 66.7 | 0.45 | 57.8 | 98.3 | |
| TGM-DLMParameters=~180M, Correction Phase=Enabled2024.02 | 85.4 | 82.6 | 24.2 | 17.003 | 73.9 | 68.8 | 0.77 | 58.1 | 87.1 | |
| TGM-DLMFamily=DM2024.05 | 85.4 | 82.6 | 24.2 | 17.003 | 73.9 | 68.8 | 0.77 | — | 87.1 | |
| MolReGPTbackbone=GPT-3.5-turbo, shots=10-shot2023.06 | 84.7 | 79 | 13.9 | 24.91 | 70.8 | 62.4 | 0.57 | 57.1 | 88.7 | |
| MolReGPT#Params=N/A2024.12 | 84.7 | 79 | — | 24.91 | 70.8 | 62.4 | — | — | 88.7 | |
| MolT5Model Size=Large2022.04 | 83.4 | 85.4 | 31.1 | 16.071 | 74.6 | 68.4 | 1.2 | 55.4 | 90.5 | |
| MolT5-largeNumber of Parameters=800M2023.05 | 83.4 | — | 31.1 | — | 74.6 | 68.4 | 1.2 | 55.4 | 90.5 | |
| MolT5-largemodel_size=large2023.06 | 83.4 | 85.4 | 31.1 | 16.071 | 74.6 | 68.4 | 1.2 | 55.4 | 90.5 | |
| MolT5largeFamily=AR2024.05 | 83.4 | 85.4 | 31.1 | 16.071 | 74.6 | 68.4 | 1.2 | — | 90.5 | |
| PEIT-LLM-Qwen2.5#Params=7B2024.12 | 83.2 | 81 | — | 21.133 | 73.5 | 61.9 | — | — | 95 | |
| T5Model Size=Large2022.04 | 82.3 | 85.4 | 27.9 | 16.721 | 73.1 | 67 | 1.22 | 55.2 | 90.2 | |
| T5-largemodel_size=large2023.06 | 82.3 | 85.4 | 27.9 | 16.721 | 73.1 | 67 | 1.22 | 55.2 | 90.2 | |
| T5largeFamily=AR2024.05 | 82.3 | 85.4 | 27.9 | 16.721 | 73.1 | 67 | 1.22 | — | 90.2 | |
| MolReGPTbackbone=Llama-2-7B, shots=2-shot2023.06 | 80.8 | 69.3 | 2.2 | 36.77 | 71.7 | 60.9 | 4.9 | 14.9 | 76.1 | |
| MolXPTFamily=AR2024.05 | 75.7 | 85.9 | 45 | — | 66.7 | 21.5 | — | — | 98.3 | |
| GIT-MolFamily=AR2024.05 | 73.8 | 75.6 | 5.1 | 26.315 | 58.2 | 51.9 | — | — | 92.8 | |
| T5-BaseParameters=~220M, Pre-training=C4 dataset2024.02 | 73.1 | 76.2 | 6.9 | 24.95 | 60.5 | 54.5 | 2.48 | 49.9 | 66 | |
| T5Model Size=Base2022.04 | 73.1 | 76.2 | 6.9 | 24.95 | 60.5 | 54.5 | 2.48 | 49.9 | 66 | |
| T5-basemodel_size=base2023.06 | 73.1 | 76.2 | 6.9 | 24.95 | 60.5 | 54.5 | 2.48 | 49.9 | 66 | |
| T5baseFamily=AR2024.05 | 73.1 | 76.5 | 6.9 | 24.95 | 60.5 | 54.5 | 2.48 | — | 66 | |
| MolT5-BaseParameters=~220M, Pre-training=C4 and ZINC-152024.02 | 72.1 | 76.9 | 8.1 | 24.458 | 58.8 | 52.9 | 2.18 | 49.6 | 77.2 | |
| MolT5Model Size=Base2022.04 | 72.1 | 76.9 | 8.1 | 24.458 | 58.8 | 52.9 | 2.18 | 49.6 | 77.2 | |
| MolT5-mediumNumber of Parameters=250M2023.05 | 72.1 | — | 8.1 | — | 58.8 | 52.9 | 2.18 | 49.6 | 77.2 | |
| MolT5-basemodel_size=base2023.06 | 72.1 | 76.9 | 8.1 | 24.458 | 58.8 | 52.9 | 2.18 | 49.6 | 77.2 | |
| MolT5baseFamily=AR2024.05 | 72.1 | 76.9 | 8.1 | 24.458 | 58.8 | 52.9 | 2.18 | — | 77.2 | |
| GPT-3.5-turbomode=zero-shot2023.06 | 70.5 | 48.9 | 1.9 | 52.13 | 46.2 | 36.7 | 2.05 | 47.9 | 80.2 | |
| GPT-3.5-turbo#Params=N/A2024.12 | 70.5 | 48.9 | — | 52.13 | 46.2 | 36.7 | — | — | 80.2 | |
| T5Model Size=Small2022.04 | 70.4 | 74.1 | 6.4 | 27.703 | 57.8 | 52.5 | 2.89 | 47.9 | 60.8 | |
| MolT5Model Size=Small2022.04 | 70.3 | 75.5 | 7.9 | 25.988 | 56.8 | 51.7 | 2.49 | 48.2 | 72.1 | |
| MolT5-smallNumber of Parameters=77M2023.05 | 70.3 | — | 7.9 | — | 56.8 | 51.7 | 2.49 | 48.2 | 72.1 | |
| RNN2022.04 | 59.1 | 65.2 | 0.5 | 38.09 | 40 | 36.2 | 4.55 | 40.9 | 54.2 | |
| TransformerArchitecture=Vanilla Transformer, Layers=6 encoder/decoder2024.02 | 48 | 49.9 | 0 | 57.66 | 32 | 21.7 | 11.32 | 27.7 | 90.6 | |
| Transformer2022.04 | 48 | 49.9 | 0 | 57.66 | 32 | 21.7 | 11.32 | 27.7 | 90.6 | |
| Transformer2023.06 | 48 | 49.9 | 0 | 57.66 | 32 | 21.7 | 11.32 | 27.7 | 90.6 | |
| TransformerFamily=AR2024.05 | 48 | 49.9 | 0 | 57.66 | 32 | 21.7 | 11.32 | — | 90.6 | |
| Mol-Instructions#Params=8B2024.12 | 41.2 | 34.5 | — | 41.367 | 23.1 | 14.7 | — | — | 100 | |
| LLaMa3.1#Params=8B2024.12 | 41.1 | 27 | — | 43.183 | 24.8 | 13.8 | — | — | 36.8 | |
| Qwen2.5#Params=7B2024.12 | 40.3 | 21.7 | — | 50.55 | 27.6 | 11 | — | — | 24.5 | |
| LLaMa3#Params=7B2024.12 | 37.2 | 26.1 | — | 45.788 | 21.3 | 12.7 | — | — | 33 | |
| Llama-2-7Bmode=zero-shot2023.06 | 24.3 | 10.4 | 0 | 84.18 | 11.9 | 8.9 | 42.01 | 14.8 | 63.1 |