Code Generation on HumanEval (in-domain)
47.56Pass@1LoRA (r = 32)
Evaluation results
| Method | Links | |
|---|---|---|
| LoRA (r = 32)Backbone=Llama-3.1-8B, Rank=32, Trainable Parameters (M)=83.9, % Trainable Parameters=1.03%2025.06 | 47.56 | |
| MoRAMBackbone=Llama-3.1-8B, Trainable Parameters (M) (added/activated)=41.9/26.2, % Trainable Parameters=0.52%/0.32%2025.06 | 47.56 | |
| LoRA (r = 16)Backbone=Llama-3.1-8B, Rank=16, Trainable Parameters (M)=41.9, % Trainable Parameters=0.52%2025.06 | 45.73 | |
| LoRA (r = 8)Backbone=Llama-3.1-8B, Rank=8, Trainable Parameters (M)=21.0, % Trainable Parameters=0.26%2025.06 | 44.51 | |
| LoRA (r = 4)Backbone=Llama-3.1-8B, Rank=4, Trainable Parameters (M)=10.5, % Trainable Parameters=0.13%2025.06 | 41.46 | |
| Llama-3.1-8BBackbone=Llama-3.1-8B2025.06 | 38.4 |