Code Generation on CodeXGLUE Python (test)
31.98CodeBLEUINFO-RAG
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| INFO-RAGBackbone=LLaMA-2-13B-chat, Zero-shot=true2024.02 | 31.98 | 46.55 | — | — | — | — | — | |
| INFO-RAGBackbone=LLaMA-2-13B, Zero-shot=true2024.02 | 29 | 41.04 | — | — | — | — | — | |
| INFO-RAGBackbone=LLaMA-2-7B-chat, Zero-shot=true2024.02 | 27.25 | 43.78 | — | — | — | — | — | |
| INFO-RAGBackbone=LLaMA-2-7B, Zero-shot=true2024.02 | 26.75 | 39.83 | — | — | — | — | — | |
| LLaMA-2-7B-chatZero-shot=true2024.02 | 22.46 | 40.83 | — | — | — | — | — | |
| LLaMA-2-13B-chatZero-shot=true2024.02 | 22.34 | 43.23 | — | — | — | — | — | |
| LLaMA-2-13BZero-shot=true2024.02 | 21.52 | 36.86 | — | — | — | — | — | |
| LLaMA-2-7BZero-shot=true2024.02 | 21.44 | 35.78 | — | — | — | — | — | |
| REDCODER2021.08 | 18.31 | — | 2.06 | 2.94 | 3.1 | 10.7 | 0.07 | |
| SCODE-R2021.08 | 16.66 | — | 2.09 | 3 | 3.16 | 11.56 | 0 |