Physical Reasoning on PIQA (Accuracy)
82.5AccuracyAAAC
Evaluation Results
| Method | Links | |
|---|---|---|
| AAACModel=Qwen3.5-27B, Group Size (g)=162026.05 | 82.5 | |
| RTNModel=Qwen3.5-27B, Group Size (g)=162026.05 | 82.3 | |
| IF4Model=Qwen3.5-27B, Group Size (g)=162026.05 | 82.2 | |
| BF16Model=Qwen3.5-27B2026.05 | 82.1 | |
| AWQModel=Qwen3.5-27B, Group Size (g)=162026.05 | 81.8 | |
| RSPruning ratio=25%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 81.34 | |
| Qwen2-57B-A14BPruning ratio=0%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 81.23 | |
| MoonlightPruning ratio=0%, Backbone=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 81.12 | |
| FLAPPruning ratio=25%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 81.12 | |
| MoNEPruning ratio=25%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 81.07 | |
| RSPruning ratio=25%, Backbone=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 81.01 | |
| NHA-Qwen3-30BA3BModel=NHA-Qwen3-30BA3B, Architecture Type=Native Hybrid Attention, Full-attention layers=32025.10 | 80.96 | |
| MoNEPruning ratio=25%, Backbone=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 80.85 | |
| MC-SMoEPruning ratio=25%, Backbone=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 80.36 | |
| Qwen3-30B-A3BPruning ratio=0%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 80.3 | |
| BaselineModel Architecture=Qwen3-30B-A3B2026.05 | 80.3 | |
| Deepseek-V2-LitePruning ratio=0%, Backbone=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 80.25 | |
| MC-SMoEPruning ratio=25%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 80.2 | |
| RSPruning ratio=25%, Backbone=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 80.14 | |
| IF4Model=Qwen3.5-9B, Group Size (g)=162026.05 | 80 | |
| AWQModel=Qwen3.5-9B, Group Size (g)=162026.05 | 80 | |
| AAACModel=Qwen3.5-9B, Group Size (g)=162026.05 | 80 | |
| RSPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 79.92 | |
| BF16Model=Qwen3.5-9B2026.05 | 79.9 | |
| OLMoEPruning ratio=0%, Backbone=OLMoE, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 79.76 | |
| MoNEPruning ratio=25%, Backbone=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 79.76 | |
| MC-SMoEPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 79.71 | |
| Qwen3-30BA3BModel=Qwen3-30BA3B, Architecture Type=Transformer2025.10 | 79.49 | |
| MoNEPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 79.27 | |
| RTNModel=Qwen3.5-9B, Group Size (g)=162026.05 | 79.2 | |
| FLAPPruning ratio=25%, Backbone=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 78.18 | |
| MoNEPruning ratio=25%, Backbone=OLMoE, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 78.07 | |
| FLAPPruning ratio=25%, Backbone=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 77.97 | |
| SSQR-2%Avg. Bitwidth=4.765, Backbone=Qwen3-8B2025.07 | 77.97 | |
| TORQModel Architecture=Qwen3-30B-A3B2026.05 | 77.93 | |
| BF16 BaselineAvg. Bitwidth=16.000, Backbone=Qwen3-8B2025.07 | 77.8 | |
| GPTQAvg. Bitwidth=4.125, Backbone=Qwen3-8B2025.07 | 77.58 | |
| HPTQAvg. Bitwidth=3.125, Backbone=Qwen3-8B2025.07 | 77.53 | |
| SSQR-3%Avg. Bitwidth=5.085, Backbone=Qwen3-8B2025.07 | 77.53 | |
| HPTQAvg. Bitwidth=4.125, Backbone=Qwen3-8B2025.07 | 77.42 | |
| SSQR-5%Avg. Bitwidth=5.725, Backbone=Qwen3-8B2025.07 | 77.42 | |
| FLAPPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 77.31 | |
| SSQR-3%Avg. Bitwidth=4.085, Backbone=Qwen3-8B2025.07 | 77.31 | |
| SSQR-4%Avg. Bitwidth=5.405, Backbone=Qwen3-8B2025.07 | 77.31 | |
| SSQR-1%Avg. Bitwidth=4.445, Backbone=Qwen3-8B2025.07 | 77.04 | |
| HRTNAvg. Bitwidth=4.125, Backbone=Qwen3-8B2025.07 | 76.99 | |
| SSQR-5%Avg. Bitwidth=4.725, Backbone=Qwen3-8B2025.07 | 76.71 | |
| SSQR-4%Avg. Bitwidth=4.405, Backbone=Qwen3-8B2025.07 | 76.61 | |
| SSQR-2%Avg. Bitwidth=3.765, Backbone=Qwen3-8B2025.07 | 76.22 | |
| HRTNAvg. Bitwidth=3.125, Backbone=Qwen3-8B2025.07 | 76.12 | |
| SOLARTraining Stage=SFT-1B2025.12 | 75.9 | |
| MIDUS-HMLTraining Stage=SFT-1B2025.12 | 75.9 | |
| RTNAvg. Bitwidth=4.125, Backbone=Qwen3-8B2025.07 | 75.9 | |
| SSQR-1%Avg. Bitwidth=3.445, Backbone=Qwen3-8B2025.07 | 75.84 | |
| MC-SMoEPruning ratio=25%, Backbone=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 75.57 | |
| OpT-DeUSTraining Stage=SFT-1B2025.12 | 75.52 | |
| BaseTraining Stage=SFT-1B2025.12 | 75.46 | |
| Llama ProTraining Stage=SFT-1B2025.12 | 75.46 | |
| GPTQAvg. Bitwidth=3.125, Backbone=Qwen3-8B2025.07 | 75.46 | |
| AngularPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 75.3 | |
| SOLARTraining Stage=CPT-1B2025.12 | 75.24 | |
| LESATraining Stage=SFT-1B2025.12 | 75.24 | |
| OSTQuantModel Architecture=Qwen3-30B-A3B2026.05 | 75.09 | |
| BaseTraining Stage=CPT-1B2025.12 | 75.08 | |
| MIDUS-HMLTraining Stage=CPT-1B2025.12 | 75.08 | |
| Avg-DeUSTraining Stage=SFT-1B2025.12 | 74.97 | |
| FLAPPruning ratio=25%, Backbone=OLMoE, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 74.81 | |
| LESATraining Stage=CPT-1B2025.12 | 74.59 | |
| Llama ProTraining Stage=CPT-1B2025.12 | 74.54 | |
| OpT-DeUSTraining Stage=CPT-1B2025.12 | 74.54 | |
| Avg-DeUSTraining Stage=CPT-1B2025.12 | 74.27 | |
| SSQR-5%Avg. Bitwidth=3.725, Backbone=Qwen3-8B2025.07 | 73.23 | |
| SSQR-4%Avg. Bitwidth=3.405, Backbone=Qwen3-8B2025.07 | 72.8 | |
| HPTQAvg. Bitwidth=2.125, Backbone=Qwen3-8B2025.07 | 72.52 | |
| AngularPruning ratio=25%, Backbone=OLMoE, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 71.65 | |
| AngularPruning ratio=25%, Backbone=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 71.33 | |
| QuaRotModel Architecture=Qwen3-30B-A3B2026.05 | 71.11 | |
| RTNAvg. Bitwidth=3.125, Backbone=Qwen3-8B2025.07 | 70.89 | |
| AngularPruning ratio=25%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 70.02 | |
| SSQR-3%Avg. Bitwidth=3.085, Backbone=Qwen3-8B2025.07 | 69.59 | |
| AngularPruning ratio=25%, Backbone=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 68.5 | |
| MC-SMoEPruning ratio=25%, Backbone=OLMoE, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 67.19 | |
| SSQR-2%Avg. Bitwidth=2.765, Backbone=Qwen3-8B2025.07 | 66.76 | |
| RSPruning ratio=25%, Backbone=OLMoE, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 66.16 | |
| HRTNAvg. Bitwidth=2.125, Backbone=Qwen3-8B2025.07 | 65.78 | |
| GPTQModel Architecture=Qwen3-30B-A3B2026.05 | 62.81 | |
| SSQR-1%Avg. Bitwidth=2.445, Backbone=Qwen3-8B2025.07 | 61.64 | |
| GPTQAvg. Bitwidth=2.125, Backbone=Qwen3-8B2025.07 | 57.83 | |
| RTNAvg. Bitwidth=2.125, Backbone=Qwen3-8B2025.07 | 51.63 | |
| RTNModel Architecture=Qwen3-30B-A3B2026.05 | 51.04 |