Code Completion on HumanEval FIM
74Accuracy (HumanEval FIM)LLaDA 8B
Evaluation results
| Method | Links | |
|---|---|---|
| LLaDA 8BShots=2, E2E Latency (s)=21.8, Inference Speed (tokens/s)=162026.02 | 74 | |
| Evo 8BShots=2, E2E Latency (s)=8.6, Inference Speed (tokens/s)=522026.02 | 73.4 | |
| LLaMA3 8BShots=2, E2E Latency (s)=7.4, Inference Speed (tokens/s)=582026.02 | 73 | |
| BD3-LM 7BShots=0, E2E Latency (s)=14.2, Inference Speed (tokens/s)=282026.02 | 71.8 | |
| ARD 7BShots=2, E2E Latency (s)=32.5, Inference Speed (tokens/s)=122026.02 | 27.2 |