Code Understanding on CodeMMLU
53.52Exact Match (EM)W/O (FP16)
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| W/O (FP16)Backbone=Gemma-2-9B2025.11 | 53.52 | 17.21 | |
| W/O (FP16)Backbone=Qwen2-7B2025.11 | 51.32 | 14.19 | |
| GPTQBackbone=Gemma-2-9B2025.11 | 51.27 | 5.75 | |
| TAQ-ISBackbone=Gemma-2-9B2025.11 | 51.03 | 6.76 | |
| TAQ-KLBackbone=Gemma-2-9B2025.11 | 50.83 | 6.76 | |
| AWQBackbone=Gemma-2-9B2025.11 | 50.68 | 5.74 | |
| TAQ-OBackbone=Gemma-2-9B2025.11 | 50.54 | 9.02 | |
| SliM-LLMBackbone=Gemma-2-9B2025.11 | 50.44 | 8.31 | |
| W/O (FP16)Backbone=Qwen2.5-3B2025.11 | 50.29 | 5.75 | |
| TAQ-OBackbone=Qwen2.5-7B2025.11 | 49.51 | 9.05 | |
| SliM-LLMBackbone=Qwen2-7B2025.11 | 49.32 | 5.83 | |
| TAQ-OBackbone=Qwen2-7B2025.11 | 49.22 | 9.05 | |
| W/O (FP16)Backbone=Qwen2.5-7B2025.11 | 48.78 | 14.19 | |
| TAQ-ISBackbone=Qwen2-7B2025.11 | 48.49 | 5.95 | |
| TAQ-OBackbone=Qwen2.5-3B2025.11 | 47.41 | 3.2 | |
| TAQ-ISBackbone=Qwen2.5-7B2025.11 | 47.22 | 5.95 | |
| SliM-LLMBackbone=Qwen2.5-7B2025.11 | 47.17 | 5.83 | |
| TAQ-KLBackbone=Qwen2-7B2025.11 | 47.12 | 5.95 | |
| SliM-LLMBackbone=Qwen2.5-3B2025.11 | 47.04 | 2.77 | |
| TAQ-KLBackbone=Qwen2.5-7B2025.11 | 47.02 | 5.95 | |
| TAQ-KLBackbone=Qwen2.5-3B2025.11 | 46.83 | 2.25 | |
| TAQ-ISBackbone=Qwen2.5-3B2025.11 | 46.29 | 2.25 | |
| AWQBackbone=Qwen2.5-7B2025.11 | 40.28 | 5.19 | |
| AWQBackbone=Qwen2-7B2025.11 | 40.09 | 5.19 | |
| GPTQBackbone=Qwen2-7B2025.11 | 35.16 | 5.19 | |
| GPTQBackbone=Qwen2.5-7B2025.11 | 34.91 | 5.19 | |
| GPTQBackbone=Qwen2.5-3B2025.11 | 24.66 | 1.93 | |
| AWQBackbone=Qwen2.5-3B2025.11 | 20.61 | 2.5 |