Code Generation on CodeXGLUE Java (test)
38.12CodeBLEUINFO-RAG
Evaluation results
| Method | Links | ||
|---|---|---|---|
| INFO-RAGBackbone=LLaMA-2-13B-chat, Zero-shot=true2024.02 | 38.12 | 46.55 | |
| INFO-RAGBackbone=LLaMA-2-13B, Zero-shot=true2024.02 | 35.5 | 41.04 | |
| INFO-RAGBackbone=LLaMA-2-7B, Zero-shot=true2024.02 | 32.06 | 39.83 | |
| LLaMA-2-13B-chatZero-shot=true2024.02 | 30.96 | 43.23 | |
| LLaMA-2-13BZero-shot=true2024.02 | 29.16 | 36.86 | |
| INFO-RAGBackbone=LLaMA-2-7B-chat, Zero-shot=true2024.02 | 28.79 | 43.78 | |
| LLaMA-2-7B-chatZero-shot=true2024.02 | 23.9 | 40.83 | |
| LLaMA-2-7BZero-shot=true2024.02 | 22.99 | 35.78 |