Coding on HEval+
75AccuracyQSUN
Evaluation results
| Method | Links | |
|---|---|---|
| QSUNModel=Qwen3-8B-Base, # Bits=16/4, Decode Execution=Shared2026.03 | 75 | |
| Full-FTModel=Qwen3-8B-Base, # Bits=16/16, Decode Execution=Independent2026.03 | 74.3 | |
| AWQModel=Qwen3-8B-Base, # Bits=4/4, Decode Execution=Independent2026.03 | 72.6 | |
| Full-FTModel=LLaMA3.1-8B, # Bits=16/16, Decode Execution=Independent2026.03 | 45.7 | |
| ICaRus# Model=3, KV Sharing=O2026.02 | 43.9 | |
| QSUNModel=LLaMA3.1-8B, # Bits=16/4, Decode Execution=Shared2026.03 | 42.7 | |
| Coding Model# Model=1, KV Sharing=.2026.02 | 41.5 | |
| Multi Model# Model=3, KV Sharing=X2026.02 | 41.5 | |
| AWQModel=LLaMA3.1-8B, # Bits=4/4, Decode Execution=Independent2026.03 | 39.6 | |
| IF Model# Model=1, KV Sharing=.2026.02 | 39 | |
| Math Model# Model=1, KV Sharing=.2026.02 | 36.6 | |
| Base Model# Model=1, KV Sharing=.2026.02 | 29.9 |