Molecule Captioning on ChEBI-20 MM (test)
0.73BLEU-2ChemAmp (Stage 1+2)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| ChemAmp (Stage 1+2)Setting=0-shot, Category=Agent with Tool Amplification2025.05 | 0.73 | 0.69 | 0.7 | 0.58 | 0.76 | |
| ChemAmp (Stage 1)Setting=0-shot, Category=Agent with Tool Amplification2025.05 | 0.64 | 0.56 | 0.45 | 0.29 | 0.55 | |
| Text+Chem T5Setting=0-shot, Category=Chemistry specialized models2025.05 | 0.63 | 0.54 | 0.68 | 0.54 | 0.62 | |
| ChemToolAgentSetting=0-shot, Category=Agent with Tool Orchestration, Tool count=292025.05 | 0.63 | 0.54 | 0.45 | 0.31 | 0.55 | |
| MolXPTSetting=0-shot, Category=Chemistry specialized models2025.05 | 0.59 | 0.5 | 0.66 | 0.51 | 0.6 | |
| InstructMolSetting=0-shot, Category=Chemistry specialized models2025.05 | 0.48 | 0.37 | 0.57 | 0.39 | 0.5 | |
| Deepseek-R1Setting=0-shot, Category=Generalist LLMs2025.05 | 0.4 | 0.25 | 0.1 | 0.02 | 0.21 | |
| ChemDFM-13bSetting=0-shot, Category=Chemistry specialized models2025.05 | 0.32 | 0.27 | 0.49 | 0.37 | 0.48 | |
| GPT-4oSetting=0-shot, Category=Generalist LLMs2025.05 | 0.26 | 0.17 | 0.1 | 0 | 0.3 | |
| Mol-InstructionSetting=0-shot, Category=Chemistry specialized models2025.05 | 0.25 | 0.17 | 0.33 | 0.29 | 0.27 | |
| ChemCrowSetting=0-shot, Category=Agent with Tool Orchestration, Tool count=182025.05 | 0.25 | 0.2 | 0.18 | 0.11 | 0.23 | |
| Llama3-70bSetting=0-shot, Category=Generalist LLMs2025.05 | 0.11 | 0.07 | 0.06 | 0 | 0.12 |