Patch generation on PatchBench 46 cases 1.0 (live)
79.6Semantic PassView Only
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| View OnlyActor=GPT-5-mini actor (strong)2026.07 | 79.6 | 92.8 | 0.7 | 0.3 | 718.1 | |
| PatchOpticActor=GPT-5-mini actor (strong)2026.07 | 77.9 | 92.7 | 0.2 | 0.1 | 718.6 | |
| PatchOpticActor=Local Mistral 7B actor (weak)2026.07 | 69 | 88.4 | 0 | 0 | 829.3 | |
| Verify OnlyActor=Local Mistral 7B actor (weak)2026.07 | 65.5 | 91.6 | 19.6 | 9 | 1,010.2 | |
| Schema OnlyActor=GPT-5-mini actor (strong)2026.07 | 62.1 | 91.6 | 25.9 | 11.9 | 805.5 | |
| Verify OnlyActor=GPT-5-mini actor (strong)2026.07 | 61.9 | 92.8 | 27.6 | 12.7 | 827.2 | |
| UnconstrainedActor=GPT-5-mini actor (strong)2026.07 | 60.9 | 92.5 | 31.5 | 14.5 | 817.5 | |
| FSM + ACLActor=GPT-5-mini actor (strong)2026.07 | 60.6 | 89.7 | 28.3 | 13 | 811.4 | |
| FSM + ACLActor=Local Mistral 7B actor (weak)2026.07 | 60.3 | 89.2 | 19.6 | 9 | 988.2 | |
| Schema OnlyActor=Local Mistral 7B actor (weak)2026.07 | 59 | 90.4 | 19.6 | 9 | 990.1 | |
| View OnlyActor=Local Mistral 7B actor (weak)2026.07 | 57.9 | 83.7 | 0 | 0 | 824.4 | |
| UnconstrainedActor=Local Mistral 7B actor (weak)2026.07 | 56 | 86.2 | 23.9 | 11 | 986.2 |