Bug Identification on Bounce Tail Student Programs 146K submissions, 108,583 unique
94AccuracyContrastive HoareLSTM + PTW
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Contrastive HoareLSTM + PTWsampling_agent=pre-trained play-to-win (PTW)2021.10 | 94 | 91 | 97.6 | 97.4 | 90.4 | |
| Code-as-text2021.10 | 68.4 | 85.9 | 44 | 62.4 | 92.8 | |
| Majority Class2021.10 | 50 | — | — | — | — |