Physical Common Sense Reasoning on PIQA 2020
80.96Accuracy (Normalized)Baseline
Evaluation Results
| Method | Links | |
|---|---|---|
| BaselineBackbone=Llama 3.1 8B Instruct, Compression Ratio=1.00x, Zero-shot=true2026.06 | 80.96 | |
| SigmaScaleBackbone=Llama 3.1 8B Instruct, Compression Ratio=0.90x, Zero-shot=true2026.06 | 79.54 | |
| ASVD+Backbone=Llama 3.1 8B Instruct, Compression Ratio=0.90x, Zero-shot=true2026.06 | 79.49 | |
| SigmaScale (KD)Backbone=Llama 3.1 8B Instruct, Compression Ratio=0.90x, Zero-shot=true2026.06 | 79.38 | |
| SigmaScaleBackbone=Qwen3-8B, Compression Ratio=0.90x, Zero-shot=true2026.06 | 77.75 | |
| BaselineBackbone=Qwen3-8B, Compression Ratio=1.00x, Zero-shot=true2026.06 | 77.69 | |
| SigmaScale (KD)Backbone=Qwen3-8B, Compression Ratio=0.90x, Zero-shot=true2026.06 | 77.64 | |
| ASVD+Backbone=Qwen3-8B, Compression Ratio=0.90x, Zero-shot=true2026.06 | 76.01 | |
| SVD-LLMBackbone=Qwen3-8B, Compression Ratio=0.90x, Zero-shot=true2026.06 | 75.57 | |
| SVD-LLMBackbone=Llama 3.1 8B Instruct, Compression Ratio=0.90x, Zero-shot=true2026.06 | 75.24 | |
| SigmaScaleBackbone=Qwen3-8B, Compression Ratio=0.75x, Zero-shot=true2026.06 | 73.94 | |
| SigmaScale (KD)Backbone=Qwen3-8B, Compression Ratio=0.75x, Zero-shot=true2026.06 | 73.88 | |
| SigmaScaleBackbone=Llama 3.1 8B Instruct, Compression Ratio=0.75x, Zero-shot=true2026.06 | 73.78 | |
| SigmaScale (KD)Backbone=Llama 3.1 8B Instruct, Compression Ratio=0.75x, Zero-shot=true2026.06 | 73.72 | |
| SVD-LLMBackbone=Qwen3-8B, Compression Ratio=0.75x, Zero-shot=true2026.06 | 72.69 | |
| ASVD+Backbone=Qwen3-8B, Compression Ratio=0.75x, Zero-shot=true2026.06 | 71.98 | |
| SVD-LLMBackbone=Llama 3.1 8B Instruct, Compression Ratio=0.75x, Zero-shot=true2026.06 | 70.67 | |
| ASVD+Backbone=Llama 3.1 8B Instruct, Compression Ratio=0.75x, Zero-shot=true2026.06 | 69.53 | |
| SVD-LLMBackbone=Qwen3-8B, Compression Ratio=0.50x, Zero-shot=true2026.06 | 66.76 | |
| SigmaScaleBackbone=Qwen3-8B, Compression Ratio=0.50x, Zero-shot=true2026.06 | 65.07 | |
| SigmaScale (KD)Backbone=Qwen3-8B, Compression Ratio=0.50x, Zero-shot=true2026.06 | 64.69 | |
| SVD-LLMBackbone=Llama 3.1 8B Instruct, Compression Ratio=0.50x, Zero-shot=true2026.06 | 62.13 | |
| SigmaScale (KD)Backbone=Llama 3.1 8B Instruct, Compression Ratio=0.50x, Zero-shot=true2026.06 | 61.81 | |
| SigmaScaleBackbone=Llama 3.1 8B Instruct, Compression Ratio=0.50x, Zero-shot=true2026.06 | 61.59 | |
| ASVD+Backbone=Qwen3-8B, Compression Ratio=0.50x, Zero-shot=true2026.06 | 59.3 | |
| ASVD+Backbone=Llama 3.1 8B Instruct, Compression Ratio=0.50x, Zero-shot=true2026.06 | 57.89 |