List your company

Zero-shot Language Understanding Suite (LMB, HellA, PIQA, ARC-E/C, WINO, Open, MMLU)

83.4ARC-E AccuracyBLT Distill (Teacher)
23.23638.855554.47570.0945Jan 27, 2026Jan 28, 2026Jan 29, 2026Jan 30, 2026Jan 31, 2026Feb 1, 2026

Evaluation results

MethodLinks
BLT Distill (Teacher)Model=Llama 2 7B, Role=Teacher2026.0283.4—80.780.755.2——66.3—————
Ours (Qwen 3 4B Teacher)Model=Qwen 3 4B, Role=Teacher (Baseline)2026.0278.969.173.677.751.370.440.673—————
Ours (Stage 2)Model=Qwen 3 4B, Role=Student, Bytes=95B2026.0277.166.269.775.452.666.441.268.52.19————
ALM (Teacher)Model=Llama 3.2 3B, Role=Teacher2026.0276.9————43.9—62.4—————
Ours (Stage 1)Model=Qwen 3 4B, Role=Student, Bytes=30B2026.0276.868.472.57750.26940.266.81.71————
Ours (Llama 3.2 3B Teacher)Model=Llama 3.2 3B, Role=Teacher (Baseline)2026.0274.670.173.776.84666.541.456—————
H-Net Distill (Teacher)Model=Llama 3.2 3B, Role=Teacher2026.0274.570.173.776.84666.541.456.1—————
ALM (Student)Model=Llama 3.2 3B, Role=Student (Byte)2026.0273.7————40.1—55.94.5————
Ours (Stage 1)Model=Llama 3.2 3B, Role=Student, Bytes=30B2026.0273.669.572.976.143.967.241.151.61.15————
H-Net Distill (Student)Model=Llama 3.2 3B, Role=Student (Byte), Bytes=189B2026.0272.163.470.276.143.366.541.451.99.85————
StableQATBits=4, Setting=20B Tokens & lr=1e-42026.0172.05—71.2877.5844.9767.6442——70.2191.567.152.67
ParetoQBits=4, Setting=20B Tokens & lr=1e-42026.0171.83—71.276.5845.4866.2542.4——70.1390.666.81—
BaselineBits=16, Setting=Baseline2026.0171.63—73.6177.4845.9969.8543——73.3992.768.46—
ParetoQBits=3, Setting=20B Tokens & lr=1e-42026.0171.47—7076.1545.1665.9342——70.289166.5—
StableQATBits=3, Setting=20B Tokens & lr=1e-42026.0171.04—70.8178.2944.9766.6944——69.0290.466.92.38
DSQBits=4, Setting=20B Tokens & lr=1e-42026.0170.19—64.5876.5141.7364.439.4——68.3990.9364.48—
Ours (Stage 2)Model=Llama 3.2 3B, Role=Student, Bytes=95B2026.0269.569.371.273.741.965.939.451.82.8————
DSQBits=2, Setting=30B Tokens & lr=1e-42026.0169.19—66.0374.9240.7863.6141.6——65.5488.663.78—
StableQATBits=2, Setting=30B Tokens & lr=1e-42026.0168.48—65.275.4140.8763.3841——63.068763.051.36
BLT Distill (Student)Model=Llama 2 7B, Role=Student (Byte), Bytes=220B2026.0266.6—76.177.445.8——63.77.34————
DSQBits=3, Setting=20B Tokens & lr=1e-42026.0166.58—66.5876.2841.6464.4840.6——70.2489.864.52—
ParetoQBits=2, Setting=30B Tokens & lr=1e-42026.0165.73—64.1374.6838.0861.4339.4——65.7384.361.69—
Baseline no QATBits=4, Setting=Baseline no QAT2026.0161.99—66.8774.4837.6361.9636.4——68.4489.462.15—
Baseline no QATBits=3, Setting=Baseline no QAT2026.0126.14—26.6152.525.4349.0128.4——44.012634.76—
Baseline no QATBits=2, Setting=Baseline no QAT2026.0125.55—26.0353.2624.9148.328.8——43.8821.434.02—