List your company
ResearchFormerly WizWand

Coding on HEval

84.8AccuracyQwen-2.5-7B
34.67247.68660.773.714Feb 27, 2026Mar 6, 2026Mar 13, 2026Mar 20, 2026Mar 27, 2026Apr 3, 2026Apr 11, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Qwen-2.5-7BType=AR, Tokens=18T, Few-shot setting=02026.0484.8
Full-FTModel=Qwen3-8B-Base, # Bits=16/16, Decode Execution=Independent2026.0383.5
QSUNModel=Qwen3-8B-Base, # Bits=16/4, Decode Execution=Shared2026.0382.3
AWQModel=Qwen3-8B-Base, # Bits=4/4, Decode Execution=Independent2026.0379.3
Gemma2 9BType=AR, Tokens=8T, Few-shot setting=02026.0468.9
FLUID-7BType=Diff, Tokens=2.7B, Few-shot setting=02026.0460.4
LLaMA-3-8BType=AR, Tokens=15T, Few-shot setting=02026.0459.8
Dream-7BType=Diff, Tokens=0.6T, Few-shot setting=02026.0455.5
LLaDA-1.5-8BType=Diff, Tokens=2.0T, Few-shot setting=02026.0452.4
Full-FTModel=LLaMA3.1-8B, # Bits=16/16, Decode Execution=Independent2026.0348.2
Coding Model# Model=1, KV Sharing=.2026.0248.2
Multi Model# Model=3, KV Sharing=X2026.0248.2
ICaRus# Model=3, KV Sharing=O2026.0248.2
Deepseek 7BType=AR, Tokens=2T, Few-shot setting=-2026.0448.2
QSUNModel=LLaMA3.1-8B, # Bits=16/4, Decode Execution=Shared2026.0347.6
LLaDA-8BType=Diff, Tokens=2.0T, Few-shot setting=02026.0447.6
AWQModel=LLaMA3.1-8B, # Bits=4/4, Decode Execution=Independent2026.0345.7
IF Model# Model=1, KV Sharing=.2026.0244.5
Math Model# Model=1, KV Sharing=.2026.0242.7
Base Model# Model=1, KV Sharing=.2026.0236.6