Coding on HEval
84.8AccuracyQwen-2.5-7B
Evaluation Results
| Method | Links | |
|---|---|---|
| Qwen-2.5-7BType=AR, Tokens=18T, Few-shot setting=02026.04 | 84.8 | |
| Full-FTModel=Qwen3-8B-Base, # Bits=16/16, Decode Execution=Independent2026.03 | 83.5 | |
| QSUNModel=Qwen3-8B-Base, # Bits=16/4, Decode Execution=Shared2026.03 | 82.3 | |
| AWQModel=Qwen3-8B-Base, # Bits=4/4, Decode Execution=Independent2026.03 | 79.3 | |
| Gemma2 9BType=AR, Tokens=8T, Few-shot setting=02026.04 | 68.9 | |
| FLUID-7BType=Diff, Tokens=2.7B, Few-shot setting=02026.04 | 60.4 | |
| LLaMA-3-8BType=AR, Tokens=15T, Few-shot setting=02026.04 | 59.8 | |
| Dream-7BType=Diff, Tokens=0.6T, Few-shot setting=02026.04 | 55.5 | |
| LLaDA-1.5-8BType=Diff, Tokens=2.0T, Few-shot setting=02026.04 | 52.4 | |
| Full-FTModel=LLaMA3.1-8B, # Bits=16/16, Decode Execution=Independent2026.03 | 48.2 | |
| Coding Model# Model=1, KV Sharing=.2026.02 | 48.2 | |
| Multi Model# Model=3, KV Sharing=X2026.02 | 48.2 | |
| ICaRus# Model=3, KV Sharing=O2026.02 | 48.2 | |
| Deepseek 7BType=AR, Tokens=2T, Few-shot setting=-2026.04 | 48.2 | |
| QSUNModel=LLaMA3.1-8B, # Bits=16/4, Decode Execution=Shared2026.03 | 47.6 | |
| LLaDA-8BType=Diff, Tokens=2.0T, Few-shot setting=02026.04 | 47.6 | |
| AWQModel=LLaMA3.1-8B, # Bits=4/4, Decode Execution=Independent2026.03 | 45.7 | |
| IF Model# Model=1, KV Sharing=.2026.02 | 44.5 | |
| Math Model# Model=1, KV Sharing=.2026.02 | 42.7 | |
| Base Model# Model=1, KV Sharing=.2026.02 | 36.6 |