Code Generation on HumanEval (Pass@1, Pass@5, Pass@10)
41.7Pass@1ID-LoRA
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| ID-LoRABase Model=LLaMA-3-8B, Trainable Parameters (%)=0.56%2026.02 | 41.7 | 54.5 | 58.5 | |
| LoRABase Model=LLaMA-3-8B, Trainable Parameters (%)=1.03%2026.02 | 39 | 48.8 | 51.9 | |
| DoRABase Model=LLaMA-3-8B, Trainable Parameters (%)=1.05%2026.02 | 38.3 | 50.2 | 55.7 | |
| ID-LoRABase Model=LLaMA-3-8B, # Params (%)=0.56%2026.02 | 37.5 | 49.7 | 55 | |
| LoRABase Model=LLaMA-3-8B, # Params (%)=1.03%2026.02 | 34.7 | 46.4 | 50.8 | |
| LoRABase Model=Mistral-7B, # Params (%)=1.15%2026.02 | 33.8 | 42.4 | 45.3 | |
| DoRABase Model=Mistral-7B, # Params (%)=1.16%2026.02 | 33.7 | 42.6 | 46.8 | |
| HydraLoRABase Model=Mistral-7B, # Params (%)=1.22%2026.02 | 33.5 | 41.4 | 44.2 | |
| ID-LoRABase Model=Mistral-7B, # Params (%)=0.62%2026.02 | 33.5 | 43.3 | 46.9 | |
| HydraLoRABase Model=LLaMA-3-8B, # Params (%)=1.10%2026.02 | 33.4 | 44.7 | 49.3 | |
| HydraLoRABase Model=LLaMA-3-8B, Trainable Parameters (%)=1.10%2026.02 | 33.2 | 43.9 | 47.8 | |
| DoRABase Model=LLaMA-3-8B, # Params (%)=1.05%2026.02 | 33.1 | 44 | 48.6 | |
| DoRABase Model=Mistral-7B, Trainable Parameters (%)=1.16%2026.02 | 32.5 | 40.1 | 43.3 | |
| HydraLoRABase Model=Mistral-7B, Trainable Parameters (%)=1.22%2026.02 | 32.4 | 41.7 | 45.6 | |
| ID-LoRABase Model=Mistral-7B, Trainable Parameters (%)=0.62%2026.02 | 32.4 | 41.8 | 45.7 | |
| LoRABase Model=Mistral-7B, Trainable Parameters (%)=1.15%2026.02 | 32.3 | 41 | 45.2 | |
| FFTBase Model=LLaMA-3-8B, Trainable Parameters (%)=100%2026.02 | 31.2 | 40.1 | 43.9 | |
| FFTBase Model=LLaMA-3-8B, # Params (%)=100%2026.02 | 30.5 | 39.3 | 41.7 | |
| MoELoRABase Model=Mistral-7B, Trainable Parameters (%)=1.21%2026.02 | 29.1 | 38.3 | 41.9 | |
| FFTBase Model=Mistral-7B, # Params (%)=100%2026.02 | 29.1 | 38.5 | 40.4 | |
| MoELoRABase Model=LLaMA-3-8B, Trainable Parameters (%)=1.09%2026.02 | 28.9 | 41.5 | 47.1 | |
| FFTBase Model=Mistral-7B, Trainable Parameters (%)=100%2026.02 | 23 | 30.2 | 33.9 |