Data Processing Workflow Generation on DataGovbench DAG-Level 1.0
34.77ATSClaude-4-sonnet
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Claude-4-sonnetAvg. Tokens=1,890.82, Generation Time (s)=2,007.23, Execution Time (s)=143.012025.12 | 34.77 | 54 | 76 | 54.92 | |
| o4-miniAvg. Tokens=2,075.26, Generation Time (s)=971.14, Execution Time (s)=91.312025.12 | 31.86 | 56 | 74 | 53.95 | |
| Grok-4Avg. Tokens=5,537.42, Generation Time (s)=4,706.45, Execution Time (s)=277.362025.12 | 31.38 | 50 | 66 | 49.13 | |
| o3Avg. Tokens=2,027.76, Generation Time (s)=1,410.07, Execution Time (s)=85.072025.12 | 31.22 | 46 | 64 | 47.07 | |
| o1Avg. Tokens=2,574.00, Generation Time (s)=3,270.06, Execution Time (s)=15.682025.12 | 27.79 | 52 | 80 | 53.26 | |
| Grok-3Avg. Tokens=854.72, Generation Time (s)=626.97, Execution Time (s)=194.632025.12 | 27.45 | 46 | 62 | 45.15 | |
| GPT-5Avg. Tokens=6,086.82, Generation Time (s)=7,121.52, Execution Time (s)=310.052025.12 | 27.18 | 46 | 86 | 53.06 | |
| Gemini-2.5-flashAvg. Tokens=7,383.40, Generation Time (s)=2,457.91, Execution Time (s)=295.212025.12 | 25.4 | 44 | 68 | 45.8 | |
| Kimi-K2-instructAvg. Tokens=1,107.94, Generation Time (s)=758.61, Execution Time (s)=80.782025.12 | 20.6 | 30 | 34 | 28.2 | |
| Claude-4-opusAvg. Tokens=1,759.84, Generation Time (s)=2,443.04, Execution Time (s)=74.242025.12 | 20.41 | 34 | 50 | 34.8 | |
| GPT-4oAvg. Tokens=754.82, Generation Time (s)=276.54, Execution Time (s)=52.942025.12 | 18.68 | 38 | 50 | 35.56 |