List your company
ResearchFormerly WizWand

Code Understanding on CodeMMLU

53.52Exact Match (EM)W/O (FP16)
19.293628.179337.06545.9507Nov 9, 2025
Updated 5mo ago

Evaluation results

MethodLinks
W/O (FP16)Backbone=Gemma-2-9B2025.1153.5217.21
W/O (FP16)Backbone=Qwen2-7B2025.1151.3214.19
GPTQBackbone=Gemma-2-9B2025.1151.275.75
TAQ-ISBackbone=Gemma-2-9B2025.1151.036.76
TAQ-KLBackbone=Gemma-2-9B2025.1150.836.76
AWQBackbone=Gemma-2-9B2025.1150.685.74
TAQ-OBackbone=Gemma-2-9B2025.1150.549.02
SliM-LLMBackbone=Gemma-2-9B2025.1150.448.31
W/O (FP16)Backbone=Qwen2.5-3B2025.1150.295.75
TAQ-OBackbone=Qwen2.5-7B2025.1149.519.05
SliM-LLMBackbone=Qwen2-7B2025.1149.325.83
TAQ-OBackbone=Qwen2-7B2025.1149.229.05
W/O (FP16)Backbone=Qwen2.5-7B2025.1148.7814.19
TAQ-ISBackbone=Qwen2-7B2025.1148.495.95
TAQ-OBackbone=Qwen2.5-3B2025.1147.413.2
TAQ-ISBackbone=Qwen2.5-7B2025.1147.225.95
SliM-LLMBackbone=Qwen2.5-7B2025.1147.175.83
TAQ-KLBackbone=Qwen2-7B2025.1147.125.95
SliM-LLMBackbone=Qwen2.5-3B2025.1147.042.77
TAQ-KLBackbone=Qwen2.5-7B2025.1147.025.95
TAQ-KLBackbone=Qwen2.5-3B2025.1146.832.25
TAQ-ISBackbone=Qwen2.5-3B2025.1146.292.25
AWQBackbone=Qwen2.5-7B2025.1140.285.19
AWQBackbone=Qwen2-7B2025.1140.095.19
GPTQBackbone=Qwen2-7B2025.1135.165.19
GPTQBackbone=Qwen2.5-7B2025.1134.915.19
GPTQBackbone=Qwen2.5-3B2025.1124.661.93
AWQBackbone=Qwen2.5-3B2025.1120.612.5