Code Generation on LCA CodeGen (test)
19.3API PrecisionCodeComp
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CodeCompModel=DS-Coder, Cap.=0.62026.04 | 19.3 | 29.8 | 23.4 | 14.7 | 0.6 | |
| CodeCompModel=DS-Coder, Cap.=0.42026.04 | 17.6 | 23.9 | 20.2 | 12.4 | 0.628 | |
| No Comp.Model=Qwen-Coder, Cap.=1.02026.04 | 15.8 | 27.4 | 20.1 | 12.5 | 0.581 | |
| No Comp.Model=DS-Coder, Cap.=1.02026.04 | 15.1 | 24.2 | 18.6 | 11.8 | 0.601 | |
| ParallelCompModel=DS-Coder, Cap.=0.62026.04 | 13.2 | 15.4 | 14.2 | 8.6 | 0.705 | |
| ParallelCompModel=DS-Coder, Cap.=0.42026.04 | 10.9 | 16.2 | 13.1 | 8.1 | 0.712 | |
| CodeCompModel=Qwen-Coder, Cap.=0.42026.04 | 5.4 | 55.4 | 10 | 5.2 | 0.665 | |
| CodeCompModel=Qwen-Coder, Cap.=0.62026.04 | 5.4 | 56.7 | 9.9 | 5.2 | 0.67 | |
| SnapKVModel=Qwen-Coder, Cap.=0.42026.04 | 5.2 | 43.4 | 9.4 | 5 | 0.737 | |
| SnapKVModel=Qwen-Coder, Cap.=0.62026.04 | 5.2 | 50.2 | 9.5 | 5 | 0.733 | |
| SnapKVModel=DS-Coder, Cap.=0.62026.04 | 5 | 40.8 | 8.8 | 4.7 | 0.722 | |
| SnapKVModel=DS-Coder, Cap.=0.42026.04 | 4.8 | 36.5 | 8.5 | 4.4 | 0.716 | |
| ParallelCompModel=Qwen-Coder, Cap.=0.42026.04 | 4.1 | 38.6 | 7.4 | 3.8 | 0.742 | |
| ParallelCompModel=Qwen-Coder, Cap.=0.62026.04 | 3.4 | 31.5 | 6.8 | 3.2 | 0.748 |