# Text-based de novo Molecule Generation on ChEBI-20 (test) Source: [Text-based de novo Molecule Generation on ChEBI-20 (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:37:04.231Z Evaluation of models generating novel molecules (de novo generation) from text prompts using the test split of the ChEBI-20 dataset. Metrics cover sequence similarity (BLEU, Levenshtein), chemical validity, and various fingerprint similarity scores (FTS, Frag). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Text-based de novo Molecule Generation | | Dataset (including any stated split) | ChEBI-20 (test) | | Metric | BLEU | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [BLEU](): Higher is better; unit: PERCENT. - [Exact Match Ratio](): Higher is better; unit: PERCENT. - [Levenshtein Distance](): Lower is better; unit: SCALAR. - [RDK FTS](): Higher is better; unit: PERCENT. - [MACCS FTS](): Higher is better; unit: PERCENT. - [MORGAN FTS](): Higher is better; unit: PERCENT. - [Frag Jaccard Similarity](): Higher is better; unit: PERCENT. - [Frag Recall](): Higher is better; unit: PERCENT. - [Fragment Match Score](): Higher is better; unit: PERCENT. - [Validity](): Higher is better; unit: PERCENT. ## Recorded results | Method | BLEU | Result date | Report paper | | --- | --- | --- | --- | | MolReasoner | 78.32 | Not recorded | 68fa7a691e209131cea830b9 | | LLaMA3 + MSR | 33.82 | Not recorded | 68fa7a691e209131cea830b9 | | Mol-Instructions + LLaMA3 | 33.43 | Not recorded | 68fa7a691e209131cea830b9 | | Mol-Instructions + LLaMA3 | 33.23 | Not recorded | 68fa7a691e209131cea830b9 | | LLaMA3 + MSR | 31.34 | Not recorded | 68fa7a691e209131cea830b9 | | Mol-Instructions + LLaMA2 | 30.49 | Not recorded | 68fa7a691e209131cea830b9 | | GPT-4o | 17.23 | Not recorded | 68fa7a691e209131cea830b9 | | LLaMA3.1-70B-Instruct | 8.19 | Not recorded | 68fa7a691e209131cea830b9 | | GPT-4o-mini | 5.32 | Not recorded | 68fa7a691e209131cea830b9 | | Deepseek-R1 | 1.45 | Not recorded | 68fa7a691e209131cea830b9 | | LLaMA3.1-8B-Instruct | 1.33 | Not recorded | 68fa7a691e209131cea830b9 | | Qwen2.5-32B-Instruct | 0.54 | Not recorded | 68fa7a691e209131cea830b9 | | Qwen2.5-7B-Instruct | 0.01 | Not recorded | 68fa7a691e209131cea830b9 | | DeepSeek-R1-Qwen-7B | 0 | Not recorded | 68fa7a691e209131cea830b9 | | Qwen3-8B | 0 | Not recorded | 68fa7a691e209131cea830b9 | | Qwen2.5-72B-Instruct | 0 | Not recorded | 68fa7a691e209131cea830b9 | ### MolReasoner BLEU: 78.32. Result ID: 699e1b3a5698665ec4d45185. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B"}. This record does not assert independent reproduction. ### LLaMA3 + MSR BLEU: 33.82. Result ID: 699e1b3a5698665ec4d45184. [Reporting paper]() Recorded evaluation configuration: {"Size":"8B","Fine-tuning (ft)":true}. This record does not assert independent reproduction. ### Mol-Instructions + LLaMA3 BLEU: 33.43. Result ID: 699e1b3a5698665ec4d45183. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"8B","Fine-tuning (ft)":true}. This record does not assert independent reproduction. ### Mol-Instructions + LLaMA3 BLEU: 33.23. Result ID: 699e1b3a5698665ec4d45182. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"8B","Fine-tuning (ft)":false}. This record does not assert independent reproduction. ### LLaMA3 + MSR BLEU: 31.34. Result ID: 699e1b3a5698665ec4d45180. [Reporting paper]() Recorded evaluation configuration: {"Size":"8B","n-shot":"10-shot"}. This record does not assert independent reproduction. ### Mol-Instructions + LLaMA2 BLEU: 30.49. Result ID: 699e1b3a5698665ec4d45181. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B","Fine-tuning (ft)":false}. This record does not assert independent reproduction. ### GPT-4o BLEU: 17.23. Result ID: 699e1b3a5698665ec4d45176. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Type":"Closed-Source"}. This record does not assert independent reproduction. ### LLaMA3.1-70B-Instruct BLEU: 8.19. Result ID: 699e1b3a5698665ec4d4517e. [Reporting paper]() Recorded evaluation configuration: {"Size":"70B"}. This record does not assert independent reproduction. ### GPT-4o-mini BLEU: 5.32. Result ID: 699e1b3a5698665ec4d45177. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Type":"Closed-Source"}. This record does not assert independent reproduction. ### Deepseek-R1 BLEU: 1.45. Result ID: 699e1b3a5698665ec4d45178. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Type":"Closed-Source"}. This record does not assert independent reproduction. ### LLaMA3.1-8B-Instruct BLEU: 1.33. Result ID: 699e1b3a5698665ec4d4517b. [Reporting paper]() Recorded evaluation configuration: {"Size":"8B"}. This record does not assert independent reproduction. ### Qwen2.5-32B-Instruct BLEU: 0.54. Result ID: 699e1b3a5698665ec4d4517d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"32B"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct BLEU: 0.01. Result ID: 699e1b3a5698665ec4d45179. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B"}. This record does not assert independent reproduction. ### DeepSeek-R1-Qwen-7B BLEU: 0. Result ID: 699e1b3a5698665ec4d4517a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B"}. This record does not assert independent reproduction. ### Qwen3-8B BLEU: 0. Result ID: 699e1b3a5698665ec4d4517c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"8B"}. This record does not assert independent reproduction. ### Qwen2.5-72B-Instruct BLEU: 0. Result ID: 699e1b3a5698665ec4d4517f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"72B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.