Procedural Graph Extraction on PAGED (test)
78.6Action F1text2flow
Evaluation Results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| text2flowModel backbone=Gemma3:27B2026.01 | 78.6 | 64.2 | 79.1 | 0.452 | 83.9 | 0.345 | 64.8 | 47.9 | 34.2 | 68.2 | |
| text2flowModel backbone=GPT-4o2026.01 | 78.2 | 68.8 | 85.9 | 0.485 | 83.1 | 0.356 | 69.2 | 48.4 | 35.7 | 71 | |
| text2flowModel backbone=Mistral3.1:24B2026.01 | 78.1 | 72.3 | 81.2 | 0.46 | 83.2 | 0.282 | 67.7 | 47.6 | 31.7 | 65.3 | |
| Few-ShotModel backbone=Mistral3.1:24B2026.01 | 77.8 | 71.1 | 77.6 | 0.461 | 76.5 | 0.192 | 61.3 | 42.3 | 28 | 61.1 | |
| Self-RefineModel backbone=Mistral3.1:24B2026.01 | 77.8 | 71.7 | 79 | 0.455 | 80.4 | 0.249 | 63.9 | 45 | 28.4 | 61.9 | |
| text2flowModel backbone=Qwen3:30B2026.01 | 77.6 | 64.4 | 84.3 | 0.411 | 79.2 | 0.238 | 58.1 | 42.9 | 27.6 | 68.4 | |
| Few-ShotModel backbone=Gemma3:27B2026.01 | 77.4 | 62.1 | 77.1 | 0.445 | 78.9 | 0.28 | 60.1 | 37.1 | 27.6 | 47.7 | |
| Self-RefineModel backbone=Gemma3:27B2026.01 | 77.4 | 61.2 | 73.4 | 0.434 | 78.4 | 0.239 | 64.2 | 47.5 | 28 | 63.8 | |
| Few-ShotModel backbone=GPT-4o2026.01 | 76.9 | 68.4 | 84.1 | 0.481 | 79.2 | 0.296 | 64.8 | 43.4 | 30.2 | 59.4 | |
| Self-RefineModel backbone=GPT-4o2026.01 | 76.9 | 69.1 | 83.8 | 0.483 | 79.6 | 0.299 | 67.3 | 47.6 | 30.6 | 67.3 | |
| Few-ShotModel backbone=Qwen3:30B2026.01 | 76.2 | 62.3 | 81.6 | 0.421 | 76.2 | 0.201 | 55.7 | 33.9 | 23.2 | 49.9 | |
| Self-RefineModel backbone=Qwen3:30B2026.01 | 76.2 | 65.3 | 84.6 | 0.421 | 79.2 | 0.201 | 58.7 | 33.9 | 25.2 | 49.9 | |
| Actor-CriticModel backbone=Mistral3.1:24B2026.01 | 75.4 | 66 | 70.9 | 0.335 | 81.5 | 0.256 | 63.3 | 45.5 | 29.1 | 55.6 | |
| Actor-CriticModel backbone=Qwen3:30B2026.01 | 75 | 51.8 | 71.4 | 0.388 | 73.4 | 0.237 | 58.4 | 34.4 | 24.6 | 55.5 | |
| text2flowModel backbone=Lamma3.1:8B2026.01 | 74.2 | 57.7 | 64.4 | 0.178 | 70.3 | 0.144 | 37 | 39.5 | 23.8 | 43.6 | |
| Actor-CriticModel backbone=Gemma3:27B2026.01 | 74.2 | 46.9 | 62.5 | 0.339 | 72.8 | 0.217 | 57.4 | 41.7 | 24.8 | 47.2 | |
| Self-RefineModel backbone=Lamma3.1:8B2026.01 | 70.8 | 56.8 | 56.2 | 0.172 | 70.2 | 0.152 | 34.7 | 36.8 | 20.4 | 39.9 | |
| Actor-CriticModel backbone=GPT-4o2026.01 | 70.6 | 61.7 | 76 | 0.441 | 72.6 | 0.225 | 65.1 | 42.5 | 23.7 | 58.1 | |
| Actor-CriticModel backbone=Lamma3.1:8B2026.01 | 67 | 50.4 | 55.6 | 0.165 | 67.3 | 0.15 | 32.7 | 32.1 | 18.2 | 37.1 | |
| Few-ShotModel backbone=Lamma3.1:8B2026.01 | 64.9 | 40.3 | 53.4 | 0.151 | 70.1 | 0.141 | 36.6 | 31.5 | 18.6 | 28.8 | |
| CISBackbone=GPT-32026.01 | 63.9 | 63.3 | — | — | 45.5 | — | — | 20.3 | 15.7 | — | |
| PETBackbone=Roberta-large2026.01 | 43 | 8.5 | 6.9 | — | 49.3 | — | — | 16.4 | 2.6 | — | |
| BPMN-Gen2026.01 | 38.7 | — | — | — | 46.3 | — | 19.8 | 9.1 | 2.2 | — | |
| MT-BPMN2026.01 | 30.8 | 2.8 | 21.3 | — | 48.5 | — | 27.9 | 5.6 | 4.7 | 1.7 | |
| BRP2026.01 | 27.6 | 2.7 | — | — | 46.9 | — | 33.7 | 7.4 | 6.1 | — |