Coding Tasks on MBPP+ instruct latest (test)
75.4AccuracyFP16
Evaluation Results
| Method | Links | |
|---|---|---|
| FP16W-bit=16, Prun. %=0%, R.size (×)=0×, Backbone=Llama-3.1-8B-Instruct2025.12 | 75.4 | |
| UniQLOne-pass=true, +FT=true, W-bit=4, Prun. %=0%, R.size (×)=4×, Backbone=Llama-3.1-8B-Instruct2025.12 | 64.8 | |
| UniQLOne-pass=true, +FT=true, W-bit=4, Prun. %=15%, R.size (×)=4.7×, Backbone=Llama-3.1-8B-Instruct2025.12 | 54.2 | |
| MoDeGPTOne-pass=false, +FT=false, W-bit=16, Prun. %=15%, R.size (×)=0.15×, Backbone=Llama-3.1-8B-Instruct2025.12 | 42.3 | |
| UniQLOne-pass=true, +FT=true, W-bit=4, Prun. %=25%, R.size (×)=5.3×, Backbone=Llama-3.1-8B-Instruct2025.12 | 33.8 | |
| SVD-LLMOne-pass=false, +FT=true, W-bit=4, Prun. %=15%, R.size (×)=4.7×, Backbone=Llama-3.1-8B-Instruct2025.12 | 24 |