Code Generation on CodeAlpaca
5.67Average Speed-upDReSD
Evaluation Results
| Method | Links | |
|---|---|---|
| DReSDBackbone=CodeLlama-13B, Retrieval Type=Static dense retrieval2025.02 | 5.67 | |
| DReSDBackbone=CodeLlama-7B, Retrieval Type=Static dense retrieval2025.02 | 5.47 | |
| DReSDBackbone=Average (All Models)2025.02 | 4.64 | |
| DReSDBackbone=Llama2-Chat-7B, Retrieval Type=Static dense retrieval2025.02 | 4.11 | |
| RESTBackbone=CodeLlama-7B, Retrieval Type=Static sparse retrieval2025.02 | 3.79 | |
| DARTModel=Qwen4B, Temperature=02026.01 | 3.45 | |
| DARTModel=Qwen14B, Temperature=02026.01 | 3.44 | |
| DARTModel=Qwen8B, Temperature=02026.01 | 3.4 | |
| DReSDBackbone=Llama2-Chat-13B, Retrieval Type=Static dense retrieval2025.02 | 3.33 | |
| DARTModel=Qwen14B, Temperature=12026.01 | 3.1 | |
| DARTModel=L2 7B, Temperature=02026.01 | 3.03 | |
| DARTModel=Qwen1.7B, Temperature=02026.01 | 2.9 | |
| RESTBackbone=Llama2-Chat-7B, Retrieval Type=Static sparse retrieval2025.02 | 2.89 | |
| HydraModel=L2 7B, Temperature=02026.01 | 2.89 | |
| DARTModel=Qwen32B, Temperature=02026.01 | 2.88 | |
| RESTBackbone=Average (All Models)2025.02 | 2.85 | |
| DARTModel=Qwen32B, Temperature=12026.01 | 2.78 | |
| RESTBackbone=CodeLlama-13B, Retrieval Type=Static sparse retrieval2025.02 | 2.6 | |
| EAGLE3Model=Qwen32B, Temperature=02026.01 | 2.38 | |
| EAGLE3Model=Qwen8B, Temperature=02026.01 | 2.34 | |
| EAGLE3Model=Qwen32B, Temperature=12026.01 | 2.3 | |
| MedusaModel=L2 7B, Temperature=02026.01 | 2.24 | |
| PLDBackbone=CodeLlama-7B, Retrieval Type=Dynamic sparse retrieval2025.02 | 2.21 | |
| EAGLE3Model=Qwen4B, Temperature=02026.01 | 2.21 | |
| RESTBackbone=Llama2-Chat-13B, Retrieval Type=Static sparse retrieval2025.02 | 2.13 | |
| EAGLE3Model=Qwen14B, Temperature=02026.01 | 2.08 | |
| EAGLE3Model=Qwen14B, Temperature=12026.01 | 2.01 | |
| PLDBackbone=CodeLlama-13B, Retrieval Type=Dynamic sparse retrieval2025.02 | 1.93 | |
| EAGLE3Model=Qwen1.7B, Temperature=02026.01 | 1.93 | |
| PLDModel=L2 7B, Temperature=02026.01 | 1.75 | |
| SDBackbone=CodeLlama-7B, Draft Model=Llama-Chat-68M2025.02 | 1.63 | |
| PLDBackbone=Average (All Models)2025.02 | 1.63 | |
| LookaheadModel=L2 7B, Temperature=02026.01 | 1.54 | |
| SDBackbone=Average (All Models)2025.02 | 1.42 | |
| SDBackbone=CodeLlama-13B, Draft Model=Llama-Chat-68M2025.02 | 1.4 | |
| SDBackbone=Llama2-Chat-13B, Draft Model=Llama-Chat-68M2025.02 | 1.33 | |
| SDBackbone=Llama2-Chat-7B, Draft Model=Llama-Chat-68M2025.02 | 1.32 | |
| PLDBackbone=Llama2-Chat-13B, Retrieval Type=Dynamic sparse retrieval2025.02 | 1.27 | |
| PLDBackbone=Llama2-Chat-7B, Retrieval Type=Dynamic sparse retrieval2025.02 | 1.11 | |
| SPSModel=Qwen32B, Temperature=02026.01 | 1.08 | |
| SPSModel=Qwen14B, Temperature=02026.01 | 1.07 |