BPMN Element Type Classification on BPMN Diagrams Dataset
95.3PrecisionBPMN Redrawer
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BPMN RedrawerPipeline=Only VLM, Protocol=Relaxed2025.11 | 95.3 | 90.6 | 92.1 | |
| Sketch2ProcessPipeline=Only VLM, Protocol=Relaxed2025.11 | 94.9 | 78.5 | 83.5 | |
| Sketch2ProcessPipeline=Only VLM, Protocol=Strict2025.11 | 93.7 | 71.2 | 78.1 | |
| BPMN RedrawerPipeline=Only VLM, Protocol=Strict2025.11 | 93.1 | 80.6 | 85.5 | |
| GPT-4.1-nanoPipeline=Only VLM, Protocol=Relaxed2025.11 | 90.8 | 80.3 | 82.9 | |
| GPT-4.1-nanoPipeline=VLM + Pix2Struct, Protocol=Relaxed2025.11 | 88 | 93.7 | 89.5 | |
| GPT-4o-miniPipeline=Only VLM, Protocol=Relaxed2025.11 | 87.1 | 97 | 90.8 | |
| GPT-4.1-nanoPipeline=Only VLM, Protocol=Strict2025.11 | 86.8 | 66.3 | 72 | |
| GPT-4.1-nanoPipeline=VLM + RapidOCR, Protocol=Relaxed2025.11 | 86.7 | 90.1 | 87.2 | |
| GPT-4o-miniPipeline=VLM + Pix2Struct, Protocol=Relaxed2025.11 | 86.3 | 97.9 | 90.9 | |
| GPT-4o-miniPipeline=VLM + RapidOCR, Protocol=Relaxed2025.11 | 86.3 | 97.9 | 90.9 | |
| GPT-4.1-nanoPipeline=VLM + Tesseract, Protocol=Relaxed2025.11 | 85.4 | 90.5 | 86.8 | |
| GPT-4o-miniPipeline=VLM + Tesseract, Protocol=Relaxed2025.11 | 85.4 | 98.5 | 90.7 | |
| GPT-4.1-miniPipeline=Only VLM, Protocol=Relaxed2025.11 | 84.2 | 87.4 | 85.3 | |
| GPT-4.1-miniPipeline=Only VLM, Protocol=Strict2025.11 | 83.9 | 87.8 | 85.5 | |
| GPT-4o-miniPipeline=Only VLM, Protocol=Strict2025.11 | 83.4 | 89.1 | 85 | |
| Mistral-small-3.1Pipeline=Only VLM, Protocol=Relaxed2025.11 | 83.1 | 85.7 | 83.7 | |
| GPT-4oPipeline=Only VLM, Protocol=Relaxed2025.11 | 82.3 | 85.7 | 83.5 | |
| GPT-4.1Pipeline=Only VLM, Protocol=Strict2025.11 | 82.2 | 85.5 | 83.5 | |
| Mistral-small-3.1Pipeline=Only VLM, Protocol=Strict2025.11 | 82 | 83.2 | 82.1 | |
| GPT-4.1Pipeline=Only VLM, Protocol=Relaxed2025.11 | 81.5 | 86.1 | 83.5 | |
| GPT-4oPipeline=Only VLM, Protocol=Strict2025.11 | 81.4 | 85.2 | 82.8 | |
| Gemma-12BPipeline=VLM + RapidOCR, Protocol=Relaxed2025.11 | 79 | 84.6 | 80.8 | |
| Gemma-12BPipeline=VLM + Tesseract, Protocol=Relaxed2025.11 | 79 | 84.6 | 80.8 | |
| GPT-4.1-miniPipeline=VLM + Tesseract, Protocol=Relaxed2025.11 | 78.9 | 86.6 | 81.9 | |
| GPT-4.1-miniPipeline=VLM + RapidOCR, Protocol=Relaxed2025.11 | 78.8 | 87.7 | 82.4 | |
| GPT-4oPipeline=VLM + Tesseract, Protocol=Relaxed2025.11 | 78.6 | 87.9 | 82.2 | |
| GPT-4o-miniPipeline=VLM + Pix2Struct, Protocol=Strict2025.11 | 78.5 | 92.8 | 83.6 | |
| GPT-4o-miniPipeline=VLM + RapidOCR, Protocol=Strict2025.11 | 78.5 | 92.8 | 83.6 | |
| GPT-4oPipeline=VLM + RapidOCR, Protocol=Relaxed2025.11 | 78.3 | 88.3 | 82.3 | |
| GPT-4.1-nanoPipeline=VLM + Pix2Struct, Protocol=Strict2025.11 | 77.7 | 82.9 | 77.7 | |
| GPT-4oPipeline=VLM + Pix2Struct, Protocol=Relaxed2025.11 | 77.5 | 87.6 | 81.4 | |
| GPT-4o-miniPipeline=VLM + Tesseract, Protocol=Strict2025.11 | 77.3 | 92.7 | 82.8 | |
| GPT-4.1-miniPipeline=VLM + Pix2Struct, Protocol=Relaxed2025.11 | 76.8 | 86.2 | 80.6 | |
| Mistral-small-3.1Pipeline=VLM + Tesseract, Protocol=Relaxed2025.11 | 76.8 | 87 | 80.4 | |
| GPT-4.1-nanoPipeline=VLM + RapidOCR, Protocol=Strict2025.11 | 76.5 | 81.3 | 77 | |
| Mistral-small-3.1Pipeline=VLM + RapidOCR, Protocol=Relaxed2025.11 | 76.5 | 88.3 | 80.7 | |
| GPT-4.1Pipeline=VLM + Pix2Struct, Protocol=Relaxed2025.11 | 76.1 | 85 | 79.7 | |
| Mistral-small-3.1Pipeline=VLM + Pix2Struct, Protocol=Relaxed2025.11 | 75.5 | 85.7 | 79.1 | |
| GPT-4.1Pipeline=VLM + RapidOCR, Protocol=Relaxed2025.11 | 75.1 | 84.6 | 78.8 | |
| GPT-4.1-nanoPipeline=VLM + Tesseract, Protocol=Strict2025.11 | 74.2 | 79.6 | 74.8 | |
| Pixtral-12BPipeline=Only VLM, Protocol=Relaxed2025.11 | 74.1 | 85.3 | 77.1 | |
| GPT-4.1Pipeline=VLM + Tesseract, Protocol=Relaxed2025.11 | 72.8 | 82.2 | 76.6 | |
| Gemma-12BPipeline=VLM + RapidOCR, Protocol=Strict2025.11 | 71.9 | 80.1 | 74.3 | |
| Gemma-12BPipeline=VLM + Tesseract, Protocol=Strict2025.11 | 71.9 | 80.1 | 74.3 | |
| Gemma-4BPipeline=VLM + RapidOCR, Protocol=Relaxed2025.11 | 71 | 100 | 82.1 | |
| Gemma-4BPipeline=VLM + Tesseract, Protocol=Relaxed2025.11 | 71 | 100 | 82.1 | |
| Aya-vision-8BPipeline=VLM + Tesseract, Protocol=Relaxed2025.11 | 70.7 | 99.3 | 81.6 | |
| Gemma-12BPipeline=Only VLM, Protocol=Relaxed2025.11 | 70.7 | 67.5 | 68.3 | |
| Aya-vision-8BPipeline=Only VLM, Protocol=Relaxed2025.11 | 70.4 | 99.3 | 81.4 | |
| Aya-vision-8BPipeline=VLM + RapidOCR, Protocol=Relaxed2025.11 | 70.2 | 98.9 | 81 | |
| GPT-4oPipeline=VLM + Tesseract, Protocol=Strict2025.11 | 70.2 | 87.9 | 76.9 | |
| Aya-vision-8BPipeline=VLM + Pix2Struct, Protocol=Relaxed2025.11 | 70.1 | 100 | 81.4 | |
| Mistral-small-3.1Pipeline=VLM + Pix2Struct, Protocol=Strict2025.11 | 70.1 | 82.1 | 74.2 | |
| Mistral-small-3.1Pipeline=VLM + Tesseract, Protocol=Strict2025.11 | 70 | 84.8 | 75.6 | |
| Gemma-4BPipeline=Only VLM, Protocol=Relaxed2025.11 | 69.8 | 96.4 | 80 | |
| Gemma-4BPipeline=VLM + Pix2Struct, Protocol=Relaxed2025.11 | 69.6 | 98 | 80.5 | |
| Pixtral-12BPipeline=VLM + Pix2Struct, Protocol=Relaxed2025.11 | 69.5 | 88.7 | 76.8 | |
| Mistral-small-3.1Pipeline=VLM + RapidOCR, Protocol=Strict2025.11 | 69.2 | 83.2 | 74.2 | |
| GPT-4oPipeline=VLM + Pix2Struct, Protocol=Strict2025.11 | 69.1 | 86.7 | 75.8 | |
| Gemma-12BPipeline=VLM + Pix2Struct, Protocol=Relaxed2025.11 | 68.6 | 69.3 | 68 | |
| GPT-4oPipeline=VLM + RapidOCR, Protocol=Strict2025.11 | 68.6 | 88.4 | 75.9 | |
| Gemma-12BPipeline=Only VLM, Protocol=Strict2025.11 | 68.3 | 58.5 | 61.7 | |
| Pixtral-12BPipeline=VLM + RapidOCR, Protocol=Relaxed2025.11 | 68.2 | 85.5 | 74.4 | |
| Pixtral-12BPipeline=Only VLM, Protocol=Strict2025.11 | 67.8 | 74.3 | 67.7 | |
| LLaMA-3.2-11BPipeline=VLM + RapidOCR, Protocol=Relaxed2025.11 | 67.5 | 96.5 | 78.1 | |
| GPT-4.1-miniPipeline=VLM + RapidOCR, Protocol=Strict2025.11 | 66.9 | 88.6 | 75.3 | |
| Qwen-7BPipeline=VLM + RapidOCR, Protocol=Relaxed2025.11 | 66.6 | 90.2 | 75.8 | |
| LLaMA-3.2-11BPipeline=Only VLM, Protocol=Relaxed2025.11 | 66.3 | 95.8 | 76.6 | |
| GPT-4.1-miniPipeline=VLM + Tesseract, Protocol=Strict2025.11 | 66.2 | 87.8 | 74.5 | |
| Qwen-7BPipeline=VLM + Tesseract, Protocol=Relaxed2025.11 | 66.2 | 88.2 | 74.8 | |
| GPT-4.1Pipeline=VLM + RapidOCR, Protocol=Strict2025.11 | 66.1 | 82.9 | 72.6 | |
| GPT-4.1Pipeline=VLM + Pix2Struct, Protocol=Strict2025.11 | 66 | 84.9 | 73.3 | |
| LLaMA-3.2-11BPipeline=VLM + Pix2Struct, Protocol=Relaxed2025.11 | 66 | 95.1 | 76.1 | |
| Gemma-4BPipeline=VLM + RapidOCR, Protocol=Strict2025.11 | 65.2 | 95.7 | 76.1 | |
| Gemma-4BPipeline=VLM + Tesseract, Protocol=Strict2025.11 | 65.2 | 95.7 | 76.1 | |
| LLaMA-3.2-11BPipeline=VLM + Tesseract, Protocol=Relaxed2025.11 | 65.2 | 95.1 | 75.5 | |
| GPT-4.1-miniPipeline=VLM + Pix2Struct, Protocol=Strict2025.11 | 64.8 | 87.3 | 73.3 | |
| Qwen-7BPipeline=VLM + Pix2Struct, Protocol=Relaxed2025.11 | 64.3 | 83.7 | 71.6 | |
| Pixtral-12BPipeline=VLM + Pix2Struct, Protocol=Strict2025.11 | 63.9 | 81.6 | 70.4 | |
| Gemma-4BPipeline=Only VLM, Protocol=Strict2025.11 | 63.4 | 92.2 | 73.8 | |
| GPT-4.1Pipeline=VLM + Tesseract, Protocol=Strict2025.11 | 63.2 | 81.6 | 70.1 | |
| Gemma-4BPipeline=VLM + Pix2Struct, Protocol=Strict2025.11 | 63 | 93.4 | 73.8 | |
| Qwen-7BPipeline=Only VLM, Protocol=Relaxed2025.11 | 62.9 | 81.9 | 70.5 | |
| Aya-vision-8BPipeline=VLM + RapidOCR, Protocol=Strict2025.11 | 62.2 | 92.5 | 72.5 | |
| Qwen-7BPipeline=Only VLM, Protocol=Strict2025.11 | 62.1 | 76.4 | 67.8 | |
| Aya-vision-8BPipeline=VLM + Tesseract, Protocol=Strict2025.11 | 61.7 | 94.1 | 73 | |
| Gemma-12BPipeline=VLM + Pix2Struct, Protocol=Strict2025.11 | 61.7 | 64.3 | 61.4 | |
| Qwen-7BPipeline=VLM + RapidOCR, Protocol=Strict2025.11 | 61.4 | 86.3 | 70.7 | |
| Qwen-7BPipeline=VLM + Tesseract, Protocol=Strict2025.11 | 61 | 85 | 70.1 | |
| Aya-vision-8BPipeline=Only VLM, Protocol=Strict2025.11 | 60.4 | 92.9 | 71.5 | |
| Pixtral-12BPipeline=VLM + Tesseract, Protocol=Relaxed2025.11 | 60.4 | 80.5 | 67.7 | |
| LLaMA-3.2-11BPipeline=VLM + Pix2Struct, Protocol=Strict2025.11 | 60.2 | 91.5 | 70 | |
| Aya-vision-8BPipeline=VLM + Pix2Struct, Protocol=Strict2025.11 | 60.1 | 93 | 71.4 | |
| Pixtral-12BPipeline=VLM + RapidOCR, Protocol=Strict2025.11 | 59.8 | 78.2 | 65.6 | |
| LLaMA-3.2-11BPipeline=VLM + RapidOCR, Protocol=Strict2025.11 | 59.7 | 91.3 | 69.7 | |
| LLaMA-3.2-11BPipeline=Only VLM, Protocol=Strict2025.11 | 58 | 91.2 | 68.5 | |
| LLaMA-3.2-11BPipeline=VLM + Tesseract, Protocol=Strict2025.11 | 57.3 | 90.1 | 67.4 | |
| Qwen-7BPipeline=VLM + Pix2Struct, Protocol=Strict2025.11 | 57.1 | 78.3 | 64.6 | |
| Pixtral-12BPipeline=VLM + Tesseract, Protocol=Strict2025.11 | 54.8 | 72.1 | 60.9 |