Molecule Captioning on ChEBI-20 (test)
0.68METEORBioT5+*
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| BioT5+*Model category=Semi-Generalist Models, Task objective=Cls. & Trans.2025.02 | 0.68 | — | 0.591 | — | — | 0.649 | — | |
| PEIT-GENData Size=0.48M2024.12 | 0.676 | 0.598 | 0.534 | 0.7 | 0.582 | 0.653 | — | |
| BioT5#Params.=252M2023.10 | 0.656 | 0.635 | 0.556 | 0.692 | 0.559 | 0.633 | 0.603 | |
| BioT5Data Size=33M2024.12 | 0.656 | 0.635 | 0.556 | 0.692 | 0.559 | 0.633 | — | |
| MolCA*Model category=Specialist Models2025.02 | 0.652 | — | 0.54 | — | — | 0.631 | — | |
| Text+Chem T5Model category=Specialist Models2025.02 | 0.648 | — | 0.542 | — | — | 0.622 | — | |
| Text+Chem-T5augmData Size=11.5M2024.12 | 0.648 | 0.625 | 0.529 | 0.682 | 0.543 | 0.622 | — | |
| MolCAData Size=2.3M, Backbone=Galactica2024.12 | 0.639 | 0.616 | 0.524 | 0.674 | 0.533 | 0.615 | — | |
| LAMOLT5Model Scale=Large2025.02 | 0.634 | 0.602 | 0.521 | 0.655 | 0.512 | 0.598 | 0.597 | |
| MolXPT#Params.=350M2023.10 | 0.626 | 0.594 | 0.505 | 0.66 | 0.511 | 0.597 | 0.594 | |
| MolXPTModel category=Specialist Models2025.02 | 0.626 | — | 0.505 | — | — | 0.597 | — | |
| MolXPTNumber of Parameters=350M2023.05 | 0.626 | 0.594 | 0.505 | 0.66 | 0.511 | 0.597 | 0.594 | |
| MolXPTData Size=30M2024.12 | 0.626 | 0.594 | 0.505 | 0.66 | 0.511 | 0.597 | — | |
| MolT5-large#Params.=783M2023.10 | 0.614 | 0.594 | 0.508 | 0.654 | 0.51 | 0.594 | 0.582 | |
| MolT5Model category=Specialist Models2025.02 | 0.614 | — | 0.508 | — | — | 0.594 | — | |
| MolT5Model Scale=Large2025.02 | 0.614 | 0.594 | 0.508 | 0.654 | 0.51 | 0.594 | 0.582 | |
| MolT5Model size=Large2022.04 | 0.614 | 0.594 | 0.508 | 0.654 | 0.51 | 0.594 | 0.582 | |
| MolT5-LargeNumber of Parameters=800M2023.05 | 0.614 | 0.594 | 0.508 | 0.654 | 0.51 | 0.594 | 0.582 | |
| MolT5-largeBackbone=MolT5-large2023.06 | 0.614 | 0.594 | 0.508 | 0.654 | 0.51 | 0.594 | 0.582 | |
| MolT5-largeData Size=100M2024.12 | 0.613 | 0.594 | 0.508 | 0.654 | 0.508 | 0.592 | — | |
| MolReGPT2025.02 | 0.61 | 0.607 | 0.525 | 0.634 | 0.476 | 0.562 | 0.585 | |
| GPT-4-0314 (10-shot MolReGPT)Prompting=10-shot, Framework=MolReGPT2023.06 | 0.61 | 0.607 | 0.525 | 0.634 | 0.476 | 0.562 | 0.585 | |
| Text+Chem T52025.02 | 0.604 | 0.58 | 0.49 | 0.647 | 0.498 | 0.586 | 0.567 | |
| Mol-LLMModel category=Generalist Models, Graph input configuration=Full model2025.02 | 0.599 | — | 0.493 | — | — | 0.439 | — | |
| LAMOLT5Model Scale=Base2025.02 | 0.596 | 0.574 | 0.485 | 0.634 | 0.478 | 0.564 | 0.599 | |
| Mol-LLMModel category=Generalist Models, Graph input configuration=w/o Graph2025.02 | 0.587 | — | 0.482 | — | — | 0.509 | — | |
| T5-large#Params.=783M2023.10 | 0.586 | 0.558 | 0.467 | 0.63 | 0.478 | 0.569 | 0.563 | |
| T5Model Scale=Large2025.02 | 0.586 | 0.558 | 0.467 | 0.63 | 0.478 | 0.569 | 0.563 | |
| T5Model size=Large2022.04 | 0.586 | 0.558 | 0.467 | 0.63 | 0.478 | 0.569 | 0.563 | |
| T5-largeBackbone=T5-large2023.06 | 0.586 | 0.558 | 0.467 | 0.63 | 0.478 | 0.569 | 0.563 | |
| GPT-3.5-turbo#Params.=>175B, mode=10-shot, context=MolReGPT2023.10 | 0.585 | 0.565 | 0.482 | 0.623 | 0.45 | 0.543 | 0.56 | |
| GPT-3.5-turbo (10-shot MolReGPT)Prompting=10-shot, Framework=MolReGPT2023.06 | 0.585 | 0.565 | 0.482 | 0.623 | 0.45 | 0.543 | 0.56 | |
| MolReGPT#Params=N/A2024.12 | 0.585 | 0.565 | 0.482 | 0.623 | 0.45 | 0.543 | — | |
| MolT5-base#Params.=248M2023.10 | 0.569 | 0.54 | 0.457 | 0.634 | 0.485 | 0.578 | 0.547 | |
| MolT5Model Scale=Base2025.02 | 0.569 | 0.54 | 0.457 | 0.634 | 0.485 | 0.578 | 0.547 | |
| MolT5Model size=Base2022.04 | 0.569 | 0.54 | 0.457 | 0.634 | 0.485 | 0.578 | 0.547 | |
| MolT5-baseNumber of Parameters=250M2023.05 | 0.569 | 0.54 | 0.457 | 0.634 | 0.485 | 0.578 | 0.547 | |
| MolT5-baseBackbone=MolT5-base2023.06 | 0.569 | 0.54 | 0.457 | 0.634 | 0.485 | 0.578 | 0.547 | |
| LAMOLT5Model Scale=Small2025.02 | 0.566 | 0.539 | 0.446 | 0.61 | 0.446 | 0.538 | 0.588 | |
| MolT5-small#Params.=77M2023.10 | 0.551 | 0.519 | 0.436 | 0.62 | 0.469 | 0.563 | 0.54 | |
| MolT5Model Scale=Small2025.02 | 0.551 | 0.519 | 0.436 | 0.62 | 0.469 | 0.563 | 0.54 | |
| MolT5Model size=Small2022.04 | 0.551 | 0.519 | 0.436 | 0.62 | 0.469 | 0.563 | 0.54 | |
| MolT5-smallNumber of Parameters=77M2023.05 | 0.551 | 0.519 | 0.436 | 0.62 | 0.469 | 0.563 | 0.54 | |
| T5-base#Params.=248M2023.10 | 0.539 | 0.511 | 0.423 | 0.607 | 0.451 | 0.55 | 0.523 | |
| T5Model Scale=Base2025.02 | 0.539 | 0.511 | 0.423 | 0.607 | 0.451 | 0.55 | 0.523 | |
| T5Model size=Base2022.04 | 0.539 | 0.511 | 0.423 | 0.607 | 0.451 | 0.55 | 0.523 | |
| T5-baseBackbone=T5-base2023.06 | 0.539 | 0.511 | 0.423 | 0.607 | 0.451 | 0.55 | 0.523 | |
| GIT-MolModel category=Specialist Models2025.02 | 0.533 | — | 0.263 | — | — | 0.56 | — | |
| GIT-MolPrompt Learning=true2023.08 | 0.533 | 0.352 | 0.263 | 0.575 | 0.485 | 0.56 | — | |
| GIT-MolData Size=4.8M2024.12 | 0.533 | 0.352 | 0.263 | 0.575 | 0.485 | 0.56 | — | |
| T5-small#Params.=77M2023.10 | 0.532 | 0.501 | 0.415 | 0.602 | 0.446 | 0.545 | 0.526 | |
| T5Model Scale=Small2025.02 | 0.532 | 0.501 | 0.415 | 0.602 | 0.446 | 0.545 | 0.526 | |
| T5Model size=Small2022.04 | 0.532 | 0.501 | 0.415 | 0.602 | 0.446 | 0.545 | 0.526 | |
| MolT5-base2023.08 | 0.529 | 0.316 | 0.247 | 0.572 | 0.48 | 0.545 | — | |
| GIT-Mol opPrompt Learning=false2023.08 | 0.525 | 0.312 | 0.237 | 0.556 | 0.468 | 0.535 | — | |
| InstructMolModel category=Specialist Models2025.02 | 0.509 | — | 0.371 | — | — | 0.502 | — | |
| InstructMol-GS#Params=7B2024.12 | 0.509 | 0.475 | 0.371 | 0.566 | 0.394 | 0.502 | — | |
| PEIT-LLM-LLaMa3.1#Params=8B2024.12 | 0.502 | 0.461 | 0.356 | 0.569 | 0.396 | 0.505 | — | |
| LLama-2-7B (2-shot MolReGPT)Prompting=2-shot, Framework=MolReGPT2023.06 | 0.495 | 0.489 | 0.409 | 0.535 | 0.374 | 0.472 | 0.466 | |
| MolT5-smallData Size=100M2024.12 | 0.492 | 0.513 | 0.398 | 0.567 | 0.412 | 0.501 | — | |
| GIT-Mol(Graph)Input Modality=Graph2023.08 | 0.491 | 0.29 | 0.21 | 0.54 | 0.445 | 0.512 | — | |
| MolT5-Large2026.04 | 0.48 | — | — | 0.539 | — | — | — | |
| MolReasonerSize=7B, Model Type=Open-Source Model2025.08 | 0.4767 | 0.4394 | 0.3233 | 0.5536 | 0.3674 | 0.4827 | — | |
| GIT-Mol(XTM)Pre-training Strategy=XTM only2023.08 | 0.471 | 0.264 | 0.187 | 0.521 | 0.421 | 0.494 | — | |
| Mol-LLM2026.04 | 0.471 | — | — | 0.57 | — | — | — | |
| LLaMA3 + MSRSize=8B, Model Type=Open-Source Model, fine-tuned=true2025.08 | 0.4701 | 0.2792 | 0.2151 | 0.414 | 0.2655 | 0.3428 | — | |
| PEIT-LLM-Qwen2.5#Params=7B2024.12 | 0.468 | 0.422 | 0.314 | 0.535 | 0.361 | 0.477 | — | |
| LlaSMol*Model category=Generalist Models2025.02 | 0.466 | — | 0.333 | — | — | 0.464 | — | |
| Mol-Instructions + LLaMA3Size=8B, Model Type=Open-Source Model, fine-tuned=true2025.08 | 0.4341 | 0.259 | 0.1995 | 0.3906 | 0.2472 | 0.2306 | — | |
| GIT-Mol(SMILES)Input Modality=SMILES2023.08 | 0.43 | 0.264 | 0.176 | 0.477 | 0.374 | 0.451 | — | |
| LlaSMol2026.04 | 0.426 | — | — | 0.494 | — | — | — | |
| ChemDFM#Params=13B2024.12 | 0.402 | 0.321 | 0.265 | 0.49 | 0.374 | 0.483 | — | |
| SciBERT2023.08 | 0.367 | 0.184 | 0.113 | 0.412 | 0.327 | 0.397 | — | |
| RNN#Params.=56M2023.10 | 0.363 | 0.251 | 0.176 | 0.45 | 0.278 | 0.394 | 0.426 | |
| RNN2025.02 | 0.363 | 0.251 | 0.176 | 0.45 | 0.278 | 0.394 | 0.426 | |
| RNN2022.04 | 0.363 | 0.251 | 0.176 | 0.45 | 0.278 | 0.394 | 0.426 | |
| 3D-MOLM*Model category=Generalist Models2025.02 | 0.326 | — | 0.171 | — | — | 0.287 | — | |
| BioT5+*Model category=Semi-Generalist Models, Task objective=Reg. & React.2025.02 | 0.323 | — | 0.216 | — | — | 0.364 | — | |
| GPT-4Model category=Generalist Models, Protocol=5-shot2025.02 | 0.32 | — | 0.158 | — | — | 0.303 | — | |
| BioMedGPT#Params=10B2024.12 | 0.308 | 0.234 | 0.141 | 0.386 | 0.206 | 0.332 | — | |
| ChemDFM2026.04 | 0.301 | — | — | 0.377 | — | — | — | |
| Mol-InstructionsModel category=Semi-Generalist Models2025.02 | 0.271 | — | 0.171 | — | — | 0.289 | — | |
| Mol-Instructions#Params=8B2024.12 | 0.254 | 0.217 | 0.143 | 0.337 | 0.196 | 0.291 | — | |
| MolDA2026.04 | 0.239 | — | — | 0.265 | — | — | — | |
| LLaMA3 + MSRSize=8B, Model Type=Open-Source Model, Few-shot=10-shot2025.08 | 0.2374 | 0.1843 | 0.1068 | 0.3142 | 0.1497 | 0.2525 | — | |
| 3D-MoLM2026.04 | 0.227 | — | — | 0.222 | — | — | — | |
| Deepseek-R1Model Type=Closed-Source Model2025.08 | 0.2189 | 0.1022 | 0.0354 | 0.2358 | 0.0666 | 0.1666 | — | |
| LLaMA3.1-8B-InstructSize=8B, Model Type=Open-Source Model2025.08 | 0.218 | 0.1687 | 0.0784 | 0.2838 | 0.1199 | 0.2272 | — | |
| Qwen2.5-7B-InstructSize=7B, Model Type=Open-Source Model2025.08 | 0.2125 | 0.0839 | 0.0287 | 0.2147 | 0.0633 | 0.153 | — | |
| Qwen2.5-32B-InstructSize=32B, Model Type=Open-Source Model2025.08 | 0.2032 | 0.095 | 0.0288 | 0.2278 | 0.0637 | 0.1614 | — | |
| Qwen2.5-72B-InstructSize=72B, Model Type=Open-Source Model2025.08 | 0.1933 | 0.1512 | 0.0638 | 0.2715 | 0.0936 | 0.2054 | — | |
| Mol-Instructions + LLaMA2Size=7B, Model Type=Open-Source Model, fine-tuned=false2025.08 | 0.19 | 0.1077 | 0.075 | 0.2789 | 0.1811 | 0.2569 | — | |
| Qwen2.5#Params=7B2024.12 | 0.188 | 0.049 | 0.007 | 0.177 | 0.029 | 0.112 | — | |
| LLaMA3.1-70B-InstructSize=70B, Model Type=Open-Source Model2025.08 | 0.1845 | 0.1485 | 0.0664 | 0.2753 | 0.1071 | 0.2211 | — | |
| Mol-Instructions + LLaMA3Size=8B, Model Type=Open-Source Model, fine-tuned=false2025.08 | 0.1801 | 0.1387 | 0.0951 | 0.2025 | 0.1163 | 0.1694 | — | |
| LLama-2-7BPrompting=zero-shot2023.06 | 0.175 | 0.094 | 0.039 | 0.169 | 0.054 | 0.142 | 0.153 | |
| Qwen3-8BSize=8B, Model Type=Open-Source Model2025.08 | 0.1729 | 0.0972 | 0.0288 | 0.2067 | 0.0502 | 0.1566 | — | |
| GPT-4oModel Type=Closed-Source Model2025.08 | 0.1656 | 0.1198 | 0.0433 | 0.2323 | 0.0735 | 0.1789 | — | |
| GPT-3.5-turbo#Params.=>175B, mode=zero-shot2023.10 | 0.161 | 0.103 | 0.05 | 0.261 | 0.088 | 0.204 | 0.352 | |
| GPT-3.5-turboPrompting=zero-shot2023.06 | 0.161 | 0.103 | 0.05 | 0.261 | 0.088 | 0.204 | 0.352 |