Code Generation on MBPP base standard (test)
66.17Pass@1PiSSA
Evaluation results
| Method | Links | |
|---|---|---|
| PiSSAModel=Gemma-7B, Params=400M2026.02 | 66.17 | |
| LoRAModel=Gemma-7B, Params=400M2026.02 | 65.58 | |
| PiSSA + OursModel=Gemma-7B, Params=200M2026.02 | 62.97 | |
| PiSSAModel=Mistral-7B, Params=335M2026.02 | 62.55 | |
| LoRA + OursModel=Gemma-7B, Params=200M2026.02 | 62.07 | |
| LoRA + OursModel=Mistral-7B, Params=167M2026.02 | 61.8 | |
| PiSSA + OursModel=Mistral-7B, Params=167M2026.02 | 60.9 | |
| LoRAModel=Mistral-7B, Params=335M2026.02 | 58.46 | |
| Full FTModel=Gemma-7B, Params=8540M2026.02 | 55.67 | |
| Full FTModel=Mistral-7B, Params=7240M2026.02 | 51.46 | |
| PiSSA + OursModel=LLaMA 2-7B, Params=159M2026.02 | 40.5 | |
| LoRA + OursModel=LLaMA 2-7B, Params=159M2026.02 | 37.67 | |
| PiSSAModel=LLaMA 2-7B, Params=320M2026.02 | 37.24 | |
| Full FTModel=LLaMA 2-7B, Params=6738M2026.02 | 35.59 | |
| LoRAModel=LLaMA 2-7B, Params=320M2026.02 | 35.5 |