Data Unit Test Generation on EIDBench
715Passed CountprismaDV
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| prismaDVBackbone LLM=gpt-5, Exec.=44.5, Non-exec.=0.02026.04 | 715 | 112 | 368 | 305 | 70.5 | 86.6 | 77.4 | |
| prismaDVBackbone LLM=gemini-2.5-pro, Exec.=28.5, Non-exec.=0.02026.04 | 597 | 230 | 478 | 195 | 76.1 | 73 | 73.9 | |
| prismaDVBackbone LLM=gpt-5-mini, Exec.=42.9, Non-exec.=0.02026.04 | 560 | 267 | 484 | 189 | 76 | 68.9 | 71.7 | |
| prismaDVBackbone LLM=gemini-2.5-flash, Exec.=36.4, Non-exec.=0.02026.04 | 551 | 276 | 533 | 140 | 81.7 | 67 | 72.9 | |
| prismaDVBackbone LLM=gpt-4.1, Exec.=34.1, Non-exec.=0.02026.04 | 484 | 343 | 551 | 122 | 81.6 | 57.8 | 67 | |
| tensorflow-dv2026.04 | 352 | 475 | 456 | 217 | 65.1 | 42.7 | 50.3 | |
| few-shotBackbone LLM=gemini-2.5-pro, Exec.=9.6, Non-exec.=1.12026.04 | 230 | 597 | 566 | 107 | 69.9 | 26.8 | 43 | |
| few-shotBackbone LLM=gpt-5-mini, Exec.=18.8, Non-exec.=3.02026.04 | 215 | 612 | 570 | 103 | 67.1 | 25.2 | 51.3 | |
| zero-shotBackbone LLM=gemini-2.5-pro, Exec.=12.4, Non-exec.=2.82026.04 | 203 | 624 | 595 | 78 | 65 | 22.8 | 31 | |
| few-shotBackbone LLM=gpt-5, Exec.=25.8, Non-exec.=2.02026.04 | 191 | 636 | 614 | 59 | 75.8 | 21.4 | 47.2 | |
| swe-agentBackbone LLM=gpt-5, Exec.=24.1, Non-exec.=1.72026.04 | 179 | 648 | 622 | 51 | 78.6 | 20.3 | 47.2 | |
| zero-shotBackbone LLM=gpt-5-mini, Exec.=17.1, Non-exec.=4.72026.04 | 164 | 663 | 584 | 89 | 65.7 | 20 | 30.4 | |
| few-shotBackbone LLM=gpt-4.1, Exec.=23.2, Non-exec.=3.62026.04 | 152 | 675 | 613 | 60 | 66.8 | 18.5 | 40.4 | |
| deequExec.=67.0, Non-exec.=0.02026.04 | 134 | 693 | 560 | 113 | 65.8 | 14.4 | 24.2 | |
| few-shotBackbone LLM=gemini-2.5-flash, Exec.=33.8, Non-exec.=9.82026.04 | 131 | 696 | 615 | 58 | 65.8 | 15.2 | 35.5 | |
| zero-shotBackbone LLM=gpt-4.1, Exec.=26.1, Non-exec.=4.62026.04 | 100 | 727 | 594 | 79 | 51 | 12.7 | 18.7 | |
| zero-shotBackbone LLM=gpt-5, Exec.=34.1, Non-exec.=3.52026.04 | 75 | 752 | 661 | 12 | 80.3 | 8.7 | 31.9 | |
| zero-shotBackbone LLM=gemini-2.5-flash, Exec.=48.3, Non-exec.=20.32026.04 | 20 | 807 | 668 | 5 | 81.9 | 2.4 | 11 |