Multi-step interactive programming on ColBench (test)
52.6Test Pass RateVANILLA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| VANILLAStep=1002025.12 | 52.6 | 39.36 | 7.8845 | |
| ABBELStep=1002025.12 | 46.55 | 32.86 | 3.8614 | |
| ABBEL-BGStep=1002025.12 | 45.77 | 32.62 | 3.4149 | |
| ABBEL-BGStep=502025.12 | 45.6 | 32.28 | 3.9693 | |
| VANILLAStep=502025.12 | 44.56 | 30.47 | 8.9805 | |
| ABBELStep=502025.12 | 38.44 | 26.51 | 3.4078 | |
| VANILLAStep=02025.12 | 28.27 | 17.48 | 4.5938 | |
| ABBELStep=02025.12 | 26.42 | 17.09 | 3.2953 |