List your company
ResearchFormerly WizWand

Physical Reasoning on PIQA (Accuracy)

82.5AccuracyAAAC
49.781658.275866.7775.2642Jul 1, 2025Aug 23, 2025Oct 16, 2025Dec 9, 2025Jan 31, 2026Mar 26, 2026May 19, 2026
Updated 5mo ago

Evaluation results

MethodLinks
AAACModel=Qwen3.5-27B, Group Size (g)=162026.0582.5
RTNModel=Qwen3.5-27B, Group Size (g)=162026.0582.3
IF4Model=Qwen3.5-27B, Group Size (g)=162026.0582.2
BF16Model=Qwen3.5-27B2026.0582.1
AWQModel=Qwen3.5-27B, Group Size (g)=162026.0581.8
RSPruning ratio=25%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0781.34
Qwen2-57B-A14BPruning ratio=0%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0781.23
MoonlightPruning ratio=0%, Backbone=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0781.12
FLAPPruning ratio=25%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0781.12
MoNEPruning ratio=25%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0781.07
RSPruning ratio=25%, Backbone=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0781.01
NHA-Qwen3-30BA3BModel=NHA-Qwen3-30BA3B, Architecture Type=Native Hybrid Attention, Full-attention layers=32025.1080.96
MoNEPruning ratio=25%, Backbone=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0780.85
MC-SMoEPruning ratio=25%, Backbone=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0780.36
Qwen3-30B-A3BPruning ratio=0%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0780.3
BaselineModel Architecture=Qwen3-30B-A3B2026.0580.3
Deepseek-V2-LitePruning ratio=0%, Backbone=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0780.25
MC-SMoEPruning ratio=25%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0780.2
RSPruning ratio=25%, Backbone=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0780.14
IF4Model=Qwen3.5-9B, Group Size (g)=162026.0580
AWQModel=Qwen3.5-9B, Group Size (g)=162026.0580
AAACModel=Qwen3.5-9B, Group Size (g)=162026.0580
RSPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0779.92
BF16Model=Qwen3.5-9B2026.0579.9
OLMoEPruning ratio=0%, Backbone=OLMoE, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0779.76
MoNEPruning ratio=25%, Backbone=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0779.76
MC-SMoEPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0779.71
Qwen3-30BA3BModel=Qwen3-30BA3B, Architecture Type=Transformer2025.1079.49
MoNEPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0779.27
RTNModel=Qwen3.5-9B, Group Size (g)=162026.0579.2
FLAPPruning ratio=25%, Backbone=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0778.18
MoNEPruning ratio=25%, Backbone=OLMoE, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0778.07
FLAPPruning ratio=25%, Backbone=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0777.97
SSQR-2%Avg. Bitwidth=4.765, Backbone=Qwen3-8B2025.0777.97
TORQModel Architecture=Qwen3-30B-A3B2026.0577.93
BF16 BaselineAvg. Bitwidth=16.000, Backbone=Qwen3-8B2025.0777.8
GPTQAvg. Bitwidth=4.125, Backbone=Qwen3-8B2025.0777.58
HPTQAvg. Bitwidth=3.125, Backbone=Qwen3-8B2025.0777.53
SSQR-3%Avg. Bitwidth=5.085, Backbone=Qwen3-8B2025.0777.53
HPTQAvg. Bitwidth=4.125, Backbone=Qwen3-8B2025.0777.42
SSQR-5%Avg. Bitwidth=5.725, Backbone=Qwen3-8B2025.0777.42
FLAPPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0777.31
SSQR-3%Avg. Bitwidth=4.085, Backbone=Qwen3-8B2025.0777.31
SSQR-4%Avg. Bitwidth=5.405, Backbone=Qwen3-8B2025.0777.31
SSQR-1%Avg. Bitwidth=4.445, Backbone=Qwen3-8B2025.0777.04
HRTNAvg. Bitwidth=4.125, Backbone=Qwen3-8B2025.0776.99
SSQR-5%Avg. Bitwidth=4.725, Backbone=Qwen3-8B2025.0776.71
SSQR-4%Avg. Bitwidth=4.405, Backbone=Qwen3-8B2025.0776.61
SSQR-2%Avg. Bitwidth=3.765, Backbone=Qwen3-8B2025.0776.22
HRTNAvg. Bitwidth=3.125, Backbone=Qwen3-8B2025.0776.12
SOLARTraining Stage=SFT-1B2025.1275.9
MIDUS-HMLTraining Stage=SFT-1B2025.1275.9
RTNAvg. Bitwidth=4.125, Backbone=Qwen3-8B2025.0775.9
SSQR-1%Avg. Bitwidth=3.445, Backbone=Qwen3-8B2025.0775.84
MC-SMoEPruning ratio=25%, Backbone=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0775.57
OpT-DeUSTraining Stage=SFT-1B2025.1275.52
BaseTraining Stage=SFT-1B2025.1275.46
Llama ProTraining Stage=SFT-1B2025.1275.46
GPTQAvg. Bitwidth=3.125, Backbone=Qwen3-8B2025.0775.46
AngularPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0775.3
SOLARTraining Stage=CPT-1B2025.1275.24
LESATraining Stage=SFT-1B2025.1275.24
OSTQuantModel Architecture=Qwen3-30B-A3B2026.0575.09
BaseTraining Stage=CPT-1B2025.1275.08
MIDUS-HMLTraining Stage=CPT-1B2025.1275.08
Avg-DeUSTraining Stage=SFT-1B2025.1274.97
FLAPPruning ratio=25%, Backbone=OLMoE, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0774.81
LESATraining Stage=CPT-1B2025.1274.59
Llama ProTraining Stage=CPT-1B2025.1274.54
OpT-DeUSTraining Stage=CPT-1B2025.1274.54
Avg-DeUSTraining Stage=CPT-1B2025.1274.27
SSQR-5%Avg. Bitwidth=3.725, Backbone=Qwen3-8B2025.0773.23
SSQR-4%Avg. Bitwidth=3.405, Backbone=Qwen3-8B2025.0772.8
HPTQAvg. Bitwidth=2.125, Backbone=Qwen3-8B2025.0772.52
AngularPruning ratio=25%, Backbone=OLMoE, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0771.65
AngularPruning ratio=25%, Backbone=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0771.33
QuaRotModel Architecture=Qwen3-30B-A3B2026.0571.11
RTNAvg. Bitwidth=3.125, Backbone=Qwen3-8B2025.0770.89
AngularPruning ratio=25%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0770.02
SSQR-3%Avg. Bitwidth=3.085, Backbone=Qwen3-8B2025.0769.59
AngularPruning ratio=25%, Backbone=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0768.5
MC-SMoEPruning ratio=25%, Backbone=OLMoE, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0767.19
SSQR-2%Avg. Bitwidth=2.765, Backbone=Qwen3-8B2025.0766.76
RSPruning ratio=25%, Backbone=OLMoE, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0766.16
HRTNAvg. Bitwidth=2.125, Backbone=Qwen3-8B2025.0765.78
GPTQModel Architecture=Qwen3-30B-A3B2026.0562.81
SSQR-1%Avg. Bitwidth=2.445, Backbone=Qwen3-8B2025.0761.64
GPTQAvg. Bitwidth=2.125, Backbone=Qwen3-8B2025.0757.83
RTNAvg. Bitwidth=2.125, Backbone=Qwen3-8B2025.0751.63
RTNModel Architecture=Qwen3-30B-A3B2026.0551.04