List your company
ResearchFormerly WizWand

Code Generation on MBPP (Score %)

66.17MBPP ScorePiSSA
12.433226.384140.33554.2859Apr 3, 2024Jul 22, 2024Nov 10, 2024Mar 1, 2025Jun 19, 2025Oct 8, 2025Jan 27, 2026
Updated 8mo ago

Evaluation results

MethodLinks
PiSSAModel=Gemma-7B2024.0466.17
Qwen2-7BParameters=7B2024.0765.9
LoRAModel=Gemma-7B, Initialization=Gaussian2024.0465.58
LoRAModel=Gemma-7B, Initialization=Kaiming2024.0465.25
PiSSAModel=Mistral-7B2024.0462.55
LoRAModel=Mistral-7B, Initialization=Gaussian2024.0458.46
LoRAModel=Mistral-7B, Initialization=Kaiming2024.0458.39
Full FTModel=Gemma-7B2024.0455.67
Llama-3-8BParameters=8B2024.0753.9
Qwen2-1.5Bnumber of parameters=1.5B2024.0751.9
Qwen1.5-7BParameters=7B2024.0751.6
Full FTModel=Mistral-7B2024.0451.46
Mistral-7BParameters=7B2024.0751.1
Gemma-7BParameters=7B2024.0750.6
PRINMIXalpha=1/162025.0650.2
PRINMIXalpha=1/642025.0650
Alignedalpha=12025.0648.9
LoRAalpha=1/162025.0647.7
PRINMIX-LoRAalpha=1/642025.0647.7
Backbonealpha=12025.0646
KEELTraining=SFT, Evaluation Setting=0-shot2026.0142.6
Qwen2-0.5Bnumber of parameters=0.5B2024.0737.8
Pre-LNTraining=SFT, Evaluation Setting=0-shot2026.0137.6
Qwen-1.5 7BRole=Teacher2024.0737.4
PiSSAModel=LLAMA 2-7B2024.0437.24
Full FTModel=LLAMA 2-7B2024.0435.59
LoRAModel=LLAMA 2-7B, Initialization=Gaussian2024.0435.5
LoRAModel=LLAMA 2-7B, Initialization=Kaiming2024.0435.47
Qwen1.5-1.8Bnumber of parameters=1.8B2024.0728.6
DDKBase Model=Qwen-1.5 1.8B, Teacher=Qwen-1.5 7B2024.0724.98
MiniLLMBase Model=Qwen-1.5 1.8B, Teacher=Qwen-1.5 7B2024.0723.31
KDBase Model=Qwen-1.5 1.8B, Teacher=Qwen-1.5 7B2024.0722.69
CPTBase Model=Qwen-1.5 1.8B, Teacher=Qwen-1.5 7B2024.0720.45
Qwen-1.5 1.8BRole=Student2024.0718
Qwen1.5-0.5Bnumber of parameters=0.5B2024.0714.5