Code Generation on MBPP extended (test)
57.6AccuracyLPW
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LPWBackbone=GPT-3.52024.11 | 57.6 | 4.8 | |
| MapCoderBackbone=GPT-3.52024.11 | 56.6 | 3.8 | |
| LPWBackbone=Llama-32024.11 | 56.4 | 21 | |
| SDBackbone=GPT-3.52024.11 | 56 | 3.2 | |
| LDBBackbone=GPT-3.52024.11 | 55.6 | 2.8 | |
| MapCoderBackbone=Llama-32024.11 | 55.6 | 20.2 | |
| BaselineBackbone=GPT-3.52024.11 | 52.8 | — | |
| SPBackbone=GPT-3.52024.11 | 52.4 | -0.4 | |
| SDBackbone=Llama-32024.11 | 50 | 14.6 | |
| LPWBackbone=Phi-32024.11 | 48.4 | 15.2 | |
| LDBBackbone=Llama-32024.11 | 44.8 | 9.4 | |
| LDBBackbone=Phi-32024.11 | 42.8 | 9.6 | |
| SPBackbone=Llama-32024.11 | 38.4 | 3 | |
| SPBackbone=Phi-32024.11 | 37.6 | 4.4 | |
| SDBackbone=Phi-32024.11 | 36.6 | 3.4 | |
| BaselineBackbone=Llama-32024.11 | 35.4 | — | |
| BaselineBackbone=Phi-32024.11 | 33.2 | — |