Physical Commonsense Reasoning on PIQA (Accuracy)
85.91AccuracyFanar-27B
Evaluation results
| Method | Links | |
|---|---|---|
| Fanar-27BSize=27B, Few-shot settings=0-shot2026.03 | 85.91 | |
| Llama 3.1-70BPrecision=BF162026.02 | 84.22 | |
| Llama-3.3-70b-InstructSize=70B, Few-shot settings=0-shot2026.03 | 84.11 | |
| Attn-QATModel=Llama 3.1-70B, Precision=Attn-QAT2026.02 | 83.51 | |
| AceGPT-v2-70B-ChatSize=70B, Few-shot settings=0-shot2026.03 | 83.24 | |
| Llama 3.1-70BPrecision=FP42026.02 | 83.08 | |
| AceGPT-v2-32B-ChatSize=32B, Few-shot settings=0-shot2026.03 | 82.75 | |
| Fanar-1-9b-instructSize=9B, Few-shot settings=0-shot2026.03 | 82.37 | |
| Qwen3-14BPrecision=BF162026.02 | 82.15 | |
| Mistral-7BRatio=0%, Evaluation Protocol=zero-shot2026.03 | 82.05 | |
| Qwen3-32BSize=32B, Few-shot settings=0-shot2026.03 | 81.94 | |
| Attn-QATModel=Qwen3-14B, Precision=Attn-QAT2026.02 | 81.88 | |
| Qwen2-57B-A14BPruning ratio=0%, Base Model=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 81.23 | |
| MoonlightPruning ratio=0%, Base Model=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 81.12 | |
| Allam-7B-Instruct-preview-v2Size=7B, Few-shot settings=0-shot2026.03 | 81.07 | |
| Qwen3-14BPrecision=FP42026.02 | 80.52 | |
| Qwen3-30B-A3BPruning ratio=0%, Base Model=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 80.3 | |
| Deepseek-V2-LitePruning ratio=0%, Base Model=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 80.25 | |
| Gemma-3-27B-itSize=27B, Few-shot settings=0-shot2026.03 | 80.14 | |
| OLMoEPruning ratio=0%, Base Model=OLMoE, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 79.76 | |
| RSPruning ratio=50%, Base Model=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 79.43 | |
| TWLABackbone=LLaMA2-70B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.06 | 79.4 | |
| Jais-2-70B-ChatSize=70B, Few-shot settings=0-shot2026.03 | 79.16 | |
| MoNEPruning ratio=50%, Base Model=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 79 | |
| TWLABackbone=Qwen3-32B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.06 | 78.99 | |
| MoNEPruning ratio=50%, Base Model=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 78.78 | |
| MoNEPruning ratio=50%, Base Model=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 78.41 | |
| RSPruning ratio=50%, Base Model=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 78.18 | |
| RSPruning ratio=50%, Base Model=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 78.02 | |
| MC-SMoEPruning ratio=50%, Base Model=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 77.53 | |
| TWLABackbone=LLaMA2-13B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.06 | 76.71 | |
| TWLABackbone=Qwen3-14B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.06 | 76.55 | |
| MoNEPruning ratio=50%, Base Model=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 75.3 | |
| AU-NetTokenizer=Byte, Parameter Scale=1.3B, Training Tokens=500B, Evaluation Protocol=Zero-shot2026.03 | 74.87 | |
| FLAPPruning ratio=50%, Base Model=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 74.76 | |
| TWLABackbone=LLaMA3-8B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.06 | 74.76 | |
| TWLABackbone=LLaMA2-7B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.06 | 74.65 | |
| GlobalGCBackbone=Llama-2 7B, Zero-Shot=true2025.02 | 74.59 | |
| FLAPPruning ratio=50%, Base Model=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 74.59 | |
| MC-SMoEPruning ratio=50%, Base Model=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 74.54 | |
| AdaGCBackbone=Llama-2 7B, Zero-Shot=true2025.02 | 74.32 | |
| ByteFlow NetTokenizer=Byte, Parameter Scale=1.3B, Training Tokens=500B, Evaluation Protocol=Zero-shot2026.03 | 74.25 | |
| TWLABackbone=Qwen3-8B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.06 | 73.78 | |
| SoLARatio=20%, Evaluation Protocol=zero-shot2026.03 | 73.67 | |
| KarnakSize=40B, Few-shot settings=0-shot2026.03 | 73.66 | |
| SVD-LLMRatio=20%, Evaluation Protocol=zero-shot2026.03 | 73.39 | |
| AdaGCBackbone=Mixtral 8x1B, Zero-Shot=true2025.02 | 73.12 | |
| AGCBackbone=Llama-2 7B, Zero-Shot=true2025.02 | 72.74 | |
| ClippyBackbone=Llama-2 7B, Zero-Shot=true2025.02 | 72.74 | |
| FLAPRatio=20%, Evaluation Protocol=zero-shot2026.03 | 72.31 | |
| ClipByValueBackbone=Llama-2 7B, Zero-Shot=true2025.02 | 71.65 | |
| GlobalGCBackbone=Mixtral 8x1B, Zero-Shot=true2025.02 | 71.6 | |
| FLAPPruning ratio=50%, Base Model=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 70.57 | |
| LLaMATokenizer=BPE, Parameter Scale=1.3B, Training Tokens=500B, Evaluation Protocol=Zero-shot2026.03 | 70.43 | |
| GlobalGCBackbone=Llama-2 1.3B, Zero-Shot=true2025.02 | 69.97 | |
| AdaGCBackbone=Llama-2 1.3B, Zero-Shot=true2025.02 | 69.7 | |
| MambaByteTokenizer=Byte, Parameter Scale=1.3B, Training Tokens=500B, Evaluation Protocol=Zero-shot2026.03 | 69.67 | |
| ClipByValueBackbone=Llama-2 1.3B, Zero-Shot=true2025.02 | 69.59 | |
| FLAPRatio=30%, Evaluation Protocol=zero-shot2026.03 | 69.59 | |
| FLAPPruning ratio=50%, Base Model=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 69.37 | |
| LlamaByteTokenizer=Byte, Parameter Scale=1.3B, Training Tokens=500B, Evaluation Protocol=Zero-shot2026.03 | 69.31 | |
| ClippyBackbone=Llama-2 1.3B, Zero-Shot=true2025.02 | 69.21 | |
| SpaceByteTokenizer=Byte, Parameter Scale=1.3B, Training Tokens=500B, Evaluation Protocol=Zero-shot2026.03 | 69.18 | |
| SoLARatio=30%, Evaluation Protocol=zero-shot2026.03 | 68.39 | |
| FLAPPruning ratio=50%, Base Model=OLMoE, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 67.57 | |
| MC-SMoEPruning ratio=50%, Base Model=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 67.3 | |
| SVD-LLMRatio=30%, Evaluation Protocol=zero-shot2026.03 | 64.91 | |
| AU-NetTokenizer=Byte, Parameter Scale=600M, Training Tokens=50B, Evaluation Protocol=Zero-shot2026.03 | 64.87 | |
| MoNEPruning ratio=50%, Base Model=OLMoE, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 63.33 | |
| ByteFlow NetTokenizer=Byte, Parameter Scale=600M, Training Tokens=50B, Evaluation Protocol=Zero-shot2026.03 | 62.25 | |
| MC-SMoEPruning ratio=50%, Base Model=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 61.37 | |
| SliceGPTRatio=20%, Evaluation Protocol=zero-shot2026.03 | 60.66 | |
| AngularPruning ratio=50%, Base Model=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 60.17 | |
| LLaMATokenizer=BPE, Parameter Scale=600M, Training Tokens=50B, Evaluation Protocol=Zero-shot2026.03 | 59.43 | |
| MambaByteTokenizer=Byte, Parameter Scale=600M, Training Tokens=50B, Evaluation Protocol=Zero-shot2026.03 | 58.67 | |
| LlamaByteTokenizer=Byte, Parameter Scale=600M, Training Tokens=50B, Evaluation Protocol=Zero-shot2026.03 | 58.31 | |
| AngularPruning ratio=50%, Base Model=OLMoE, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 58.27 | |
| SpaceByteTokenizer=Byte, Parameter Scale=600M, Training Tokens=50B, Evaluation Protocol=Zero-shot2026.03 | 58.18 | |
| AngularPruning ratio=50%, Base Model=Deepseek-V2-Lite, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 56.42 | |
| AngularPruning ratio=50%, Base Model=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 54.62 | |
| SliceGPTRatio=30%, Evaluation Protocol=zero-shot2026.03 | 54.41 | |
| MC-SMoEPruning ratio=50%, Base Model=OLMoE, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 53.92 | |
| RSPruning ratio=50%, Base Model=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 52.88 | |
| AngularPruning ratio=50%, Base Model=Moonlight, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 52.56 | |
| RSPruning ratio=50%, Base Model=OLMoE, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 52.34 |