# Diagram-to-Code on DiagramGen (test) Source: [Diagram-to-Code on DiagramGen (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:57:21.665Z Evaluation of model performance on the test split of the DiagramGen dataset for converting diagrams into executable code, measured using metrics like Pass@1, codeBLEU, and ROUGE-L. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Diagram-to-Code | | Dataset (including any stated split) | DiagramGen (test) | | Metric | Pass@1 | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 15 of 15 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Pass@1](): Higher is better; unit: SCALAR. - [ROUGE-L](): Higher is better; unit: PERCENT. - [codeBLEU](): Higher is better; unit: PERCENT. - [Edit Distance](): Lower is better; unit: PERCENT. - [chrF](): Higher is better; unit: PERCENT. - [RUBY](): Higher is better; unit: PERCENT. ## Recorded results | Method | Pass@1 | Result date | Report paper | | --- | --- | --- | --- | | DiagramAgent | 68.89 | Not recorded | 68fa75bd1e209131cea69a25 | | GPT-4o | 64.07 | Not recorded | 68fa75bd1e209131cea69a25 | | DiagramAgent | 62.59 | Not recorded | 68fa75bd1e209131cea69a25 | | DiagramAgent | 53.33 | Not recorded | 68fa75bd1e209131cea69a25 | | DiagramAgent | 52.59 | Not recorded | 68fa75bd1e209131cea69a25 | | GLM-4-plus | 51.48 | Not recorded | 68fa75bd1e209131cea69a25 | | Deepseek-vl | 50.74 | Not recorded | 68fa75bd1e209131cea69a25 | | Qwen2-VL | 28.89 | Not recorded | 68fa75bd1e209131cea69a25 | | Llama-3.2-Vision | 27.78 | Not recorded | 68fa75bd1e209131cea69a25 | | Phi-3.5-vision | 24.07 | Not recorded | 68fa75bd1e209131cea69a25 | | Gemini-1.5-pro | 17.78 | Not recorded | 68fa75bd1e209131cea69a25 | | Llava-v1.6 | 8.89 | Not recorded | 68fa75bd1e209131cea69a25 | | Cogvlm2-llama3 | 3.7 | Not recorded | 68fa75bd1e209131cea69a25 | | Internlm-xcomposer2.5 | 3.33 | Not recorded | 68fa75bd1e209131cea69a25 | | Yi-VL | 2.22 | Not recorded | 68fa75bd1e209131cea69a25 | ### DiagramAgent Pass@1: 68.89. Result ID: 6993f8fa3a6c31484919f989. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B","configuration":"Full model"}. This record does not assert independent reproduction. ### GPT-4o Pass@1: 64.07. Result ID: 6993f8fa3a6c31484919f986. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"Not Specified"}. This record does not assert independent reproduction. ### DiagramAgent Pass@1: 62.59. Result ID: 6993f8fa3a6c31484919f98a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B","configuration":"w/o GPT-4o"}. This record does not assert independent reproduction. ### DiagramAgent Pass@1: 53.33. Result ID: 6993f8fa3a6c31484919f98b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B","configuration":"w/o Compiler"}. This record does not assert independent reproduction. ### DiagramAgent Pass@1: 52.59. Result ID: 6993f8fa3a6c31484919f98c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B","configuration":"w/o GPT-4o & Compiler"}. This record does not assert independent reproduction. ### GLM-4-plus Pass@1: 51.48. Result ID: 6993f8fa3a6c31484919f987. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"Not Specified"}. This record does not assert independent reproduction. ### Deepseek-vl Pass@1: 50.74. Result ID: 6993f8fa3a6c31484919f985. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"7B"}. This record does not assert independent reproduction. ### Qwen2-VL Pass@1: 28.89. Result ID: 6993f8fa3a6c31484919f97f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"8B"}. This record does not assert independent reproduction. ### Llama-3.2-Vision Pass@1: 27.78. Result ID: 6993f8fa3a6c31484919f981. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"11B"}. This record does not assert independent reproduction. ### Phi-3.5-vision Pass@1: 24.07. Result ID: 6993f8fa3a6c31484919f982. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"4B"}. This record does not assert independent reproduction. ### Gemini-1.5-pro Pass@1: 17.78. Result ID: 6993f8fa3a6c31484919f988. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"Not Specified"}. This record does not assert independent reproduction. ### Llava-v1.6 Pass@1: 8.89. Result ID: 6993f8fa3a6c31484919f983. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"34B"}. This record does not assert independent reproduction. ### Cogvlm2-llama3 Pass@1: 3.7. Result ID: 6993f8fa3a6c31484919f984. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"19B"}. This record does not assert independent reproduction. ### Internlm-xcomposer2.5 Pass@1: 3.33. Result ID: 6993f8fa3a6c31484919f980. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"7B"}. This record does not assert independent reproduction. ### Yi-VL Pass@1: 2.22. Result ID: 6993f8fa3a6c31484919f97e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"34B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.