List your company
ResearchFormerly WizWand

Python Programming on MBPP

95.5MBPP ScoreOkshot
-3.8221.96547.7573.535Jan 29, 2026Feb 22, 2026Mar 19, 2026Apr 12, 2026May 7, 2026May 31, 2026Jun 25, 2026
Updated 4mo ago

Evaluation results

MethodLinks
Okshotk-shot=102026.0695.5————0.28667.3
SOTA LLM2026.0686.2————0.001—
Qwen3-4B + FBS-Full (ours)Backbone=Qwen3-4B2026.0146.35320.736———
BaselineBase Model=Qwen1.5-MoE, Weight Bits=162026.0438.2———46.1——
GPTQBase Model=Qwen1.5-MoE, Weight Bits=32026.0430.18———37.48——
MoBiEBase Model=Qwen1.5-MoE, Weight Bits=1.352026.0429.27———37.54——
ARB-LLMBase Model=Qwen1.5-MoE, Weight Bits=1.112026.0421.88———30.6——
BaselineBase Model=OLMoE, Weight Bits=162026.0421.8———33.45——
BiLLMBase Model=Qwen1.5-MoE, Weight Bits=1.112026.0419.41———28.8——
NoWagBase Model=Qwen1.5-MoE, Weight Bits=2.042026.0416.76———26.94——
MoEQuantBase Model=Qwen1.5-MoE, Weight Bits=22026.0414.32———22.56——
MoBiEBase Model=OLMoE, Weight Bits=1.462026.0412.12———24.54——
GPTQBase Model=OLMoE, Weight Bits=32026.0412———24.85——
GPTQBase Model=Qwen1.5-MoE, Weight Bits=22026.049.24———16.96——
ARB-LLMBase Model=OLMoE, Weight Bits=1.112026.046.32———16.97——
AWQBase Model=Qwen1.5-MoE, Weight Bits=32026.046.31———15.1——
BiLLMBase Model=OLMoE, Weight Bits=1.112026.043.41———14.85——
NoWagBase Model=OLMoE, Weight Bits=2.042026.041.02———11.77——
AWQBase Model=OLMoE, Weight Bits=32026.040———3.88——
AWQBase Model=OLMoE, Weight Bits=22026.040———1.9——
GPTQBase Model=OLMoE, Weight Bits=22026.040———6.07——
MoEQuantBase Model=OLMoE, Weight Bits=22026.040———9.69——
AWQBase Model=Qwen1.5-MoE, Weight Bits=22026.040———12.1——