# Diagram Editing on DiagramGenBenchmark 1.0 (test) Source: [Diagram Editing on DiagramGenBenchmark 1.0 (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:57:21.711Z Evaluation of model performance on the test split of DiagramGenBenchmark 1.0, assessing diagram editing capabilities using a combination of code generation metrics (Pass@1, CodeBLEU) and visual quality metrics (CLIP-FID, LPIPS, PSNR). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Diagram Editing | | Dataset (including any stated split) | DiagramGenBenchmark 1.0 (test) | | Metric | Pass@1 | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 17 of 17 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Pass@1](): Higher is better; unit: PERCENT. - [ROUGE-L](): Higher is better; unit: PERCENT. - [CodeBLEU](): Higher is better; unit: PERCENT. - [Edit Distance](): Lower is better; unit: SCALAR. - [chrF](): Higher is better; unit: PERCENT. - [RUBY](): Higher is better; unit: PERCENT. - [CLIP-FID](): Lower is better; unit: SCALAR. - [LPIPS](): Lower is better; unit: SCALAR. - [PSNR](): Higher is better; unit: SCALAR. - [MS-SSIM](): Higher is better; unit: PERCENT. ## Recorded results | Method | Pass@1 | Result date | Report paper | | --- | --- | --- | --- | | DiagramAgent Code Agent | 98 | Not recorded | 68fa75bd1e209131cea69a25 | | DeepSeek V2.5 | 95 | Not recorded | 68fa75bd1e209131cea69a25 | | GPT-4o | 92.42 | Not recorded | 68fa75bd1e209131cea69a25 | | GLM-4-plus | 92 | Not recorded | 68fa75bd1e209131cea69a25 | | DeepSeek-Coder-Instruct | 90.5 | Not recorded | 68fa75bd1e209131cea69a25 | | Yi-1.5-34B-chat | 90.5 | Not recorded | 68fa75bd1e209131cea69a25 | | Codegeex4-all | 90 | Not recorded | 68fa75bd1e209131cea69a25 | | WizardCoder | 87.5 | Not recorded | 68fa75bd1e209131cea69a25 | | Code-Llama | 87 | Not recorded | 68fa75bd1e209131cea69a25 | | Yi-Coder | 81.5 | Not recorded | 68fa75bd1e209131cea69a25 | | Qwen2-7B-Instruct | 81.5 | Not recorded | 68fa75bd1e209131cea69a25 | | Gemini | 72 | Not recorded | 68fa75bd1e209131cea69a25 | | Qwen2.5-Coder-7B | 71.5 | Not recorded | 68fa75bd1e209131cea69a25 | | Internlm2\_5-20b-chat | 57 | Not recorded | 68fa75bd1e209131cea69a25 | | Starcoder2 | 41 | Not recorded | 68fa75bd1e209131cea69a25 | | Baichuan2-13B-Chat | 39.5 | Not recorded | 68fa75bd1e209131cea69a25 | | Llama-3.1-8B-Instruct | 24 | Not recorded | 68fa75bd1e209131cea69a25 | ### DiagramAgent Code Agent Pass@1: 98. Result ID: 6993f8fa3a6c31484919f9ae. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B"}. This record does not assert independent reproduction. ### DeepSeek V2.5 Pass@1: 95. Result ID: 6993f8fa3a6c31484919f9ab. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"N/A"}. This record does not assert independent reproduction. ### GPT-4o Pass@1: 92.42. Result ID: 6993f8fa3a6c31484919f9aa. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"N/A"}. This record does not assert independent reproduction. ### GLM-4-plus Pass@1: 92. Result ID: 6993f8fa3a6c31484919f9ac. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"N/A"}. This record does not assert independent reproduction. ### DeepSeek-Coder-Instruct Pass@1: 90.5. Result ID: 6993f8fa3a6c31484919f99f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"33B"}. This record does not assert independent reproduction. ### Yi-1.5-34B-chat Pass@1: 90.5. Result ID: 6993f8fa3a6c31484919f9a8. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"34B"}. This record does not assert independent reproduction. ### Codegeex4-all Pass@1: 90. Result ID: 6993f8fa3a6c31484919f9a2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"9B"}. This record does not assert independent reproduction. ### WizardCoder Pass@1: 87.5. Result ID: 6993f8fa3a6c31484919f9a1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"15B"}. This record does not assert independent reproduction. ### Code-Llama Pass@1: 87. Result ID: 6993f8fa3a6c31484919f9a0. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"34B"}. This record does not assert independent reproduction. ### Yi-Coder Pass@1: 81.5. Result ID: 6993f8fa3a6c31484919f9a4. [Reporting paper]() Recorded evaluation configuration: {"Size":"9B"}. This record does not assert independent reproduction. ### Qwen2-7B-Instruct Pass@1: 81.5. Result ID: 6993f8fa3a6c31484919f9a9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B"}. This record does not assert independent reproduction. ### Gemini Pass@1: 72. Result ID: 6993f8fa3a6c31484919f9ad. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"N/A"}. This record does not assert independent reproduction. ### Qwen2.5-Coder-7B Pass@1: 71.5. Result ID: 6993f8fa3a6c31484919f99e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"7B"}. This record does not assert independent reproduction. ### Internlm2\_5-20b-chat Pass@1: 57. Result ID: 6993f8fa3a6c31484919f9a7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"20B"}. This record does not assert independent reproduction. ### Starcoder2 Pass@1: 41. Result ID: 6993f8fa3a6c31484919f9a3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"15B"}. This record does not assert independent reproduction. ### Baichuan2-13B-Chat Pass@1: 39.5. Result ID: 6993f8fa3a6c31484919f9a6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"13B"}. This record does not assert independent reproduction. ### Llama-3.1-8B-Instruct Pass@1: 24. Result ID: 6993f8fa3a6c31484919f9a5. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Size":"8B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.