Zero-shot reasoning on ZeroShot 7
71.57AccuracyFP
Evaluation Results
| Method | Links | |
|---|---|---|
| FP#Bits=16.0, Model=Qwen3-30B-A3B, Evaluation protocol=0-shot2026.05 | 71.57 | |
| FP#Bits=16.0, Model=Mixtral-8×7B, Evaluation protocol=0-shot2026.05 | 70.97 | |
| FPModel=LLaMA3.1-70B, Compression Ratio=1.02026.02 | 67 | |
| SAES-SVDModel=LLaMA3.1-70B, Compression Ratio=0.22026.02 | 66 | |
| GEMQ#Bits=2.5, Model=Qwen3-30B-A3B, Evaluation protocol=0-shot2026.05 | 65.69 | |
| Uniform#Bits=2.5, Model=Qwen3-30B-A3B, Evaluation protocol=0-shot2026.05 | 65.58 | |
| Uniform#Bits=2.5, Model=Mixtral-8×7B, Evaluation protocol=0-shot2026.05 | 65.49 | |
| GEMQ#Bits=2.5, Model=Mixtral-8×7B, Evaluation protocol=0-shot2026.05 | 65.13 | |
| FPModel=Qwen2.5-32B, Compression Ratio=1.02026.02 | 65 | |
| PMQ#Bits=2.5, Model=Mixtral-8×7B, Evaluation protocol=0-shot2026.05 | 64.34 | |
| FP#Bits=16.0, Model=DeepSeekV2-Lite, Evaluation protocol=0-shot2026.05 | 64.24 | |
| SAES-SVDModel=Qwen2.5-32B, Compression Ratio=0.22026.02 | 64 | |
| SVD-LLMModel=LLaMA3.1-70B, Compression Ratio=0.22026.02 | 63 | |
| FP#Bits=16.0, Model=Qwen1.5-MoE-A2.7B, Evaluation protocol=0-shot2026.05 | 62.32 | |
| PMQ#Bits=2.5, Model=Qwen3-30B-A3B, Evaluation protocol=0-shot2026.05 | 61.96 | |
| Uniform#Bits=2.0, Model=Qwen3-30B-A3B, Evaluation protocol=0-shot2026.05 | 61.28 | |
| FPModel=LLaMA3.1-8B, Compression Ratio=1.02026.02 | 61 | |
| SVD-LLMModel=Qwen2.5-32B, Compression Ratio=0.22026.02 | 61 | |
| GEMQ#Bits=2.5, Model=DeepSeekV2-Lite, Evaluation protocol=0-shot2026.05 | 60.73 | |
| GEMQ#Bits=2.0, Model=Qwen3-30B-A3B, Evaluation protocol=0-shot2026.05 | 60.53 | |
| PMQ#Bits=2.0, Model=Mixtral-8×7B, Evaluation protocol=0-shot2026.05 | 60.38 | |
| FPModel=Qwen2.5-7B, Compression Ratio=1.02026.02 | 60 | |
| GEMQ#Bits=2.0, Model=Mixtral-8×7B, Evaluation protocol=0-shot2026.05 | 59.98 | |
| Uniform#Bits=2.5, Model=DeepSeekV2-Lite, Evaluation protocol=0-shot2026.05 | 59.81 | |
| PMQ#Bits=2.5, Model=DeepSeekV2-Lite, Evaluation protocol=0-shot2026.05 | 59.6 | |
| SAES-SVDModel=LLaMA3.1-8B, Compression Ratio=0.22026.02 | 59 | |
| GEMQ#Bits=2.5, Model=Qwen1.5-MoE-A2.7B, Evaluation protocol=0-shot2026.05 | 58.38 | |
| SAES-SVDModel=Qwen2.5-7B, Compression Ratio=0.22026.02 | 58 | |
| SVD-LLMModel=LLaMA3.1-8B, Compression Ratio=0.22026.02 | 57 | |
| SVD-LLMModel=Qwen2.5-7B, Compression Ratio=0.22026.02 | 56 | |
| Uniform#Bits=2.0, Model=Mixtral-8×7B, Evaluation protocol=0-shot2026.05 | 55.72 | |
| GEMQ#Bits=2.0, Model=Qwen1.5-MoE-A2.7B, Evaluation protocol=0-shot2026.05 | 54.87 | |
| PMQ#Bits=2.5, Model=Qwen1.5-MoE-A2.7B, Evaluation protocol=0-shot2026.05 | 54.57 | |
| ASVDModel=LLaMA3.1-70B, Compression Ratio=0.22026.02 | 54 | |
| GEMQ#Bits=2.0, Model=DeepSeekV2-Lite, Evaluation protocol=0-shot2026.05 | 53.8 | |
| PMQ#Bits=2.0, Model=DeepSeekV2-Lite, Evaluation protocol=0-shot2026.05 | 52.52 | |
| GEMQ#Bits=1.5, Model=Qwen3-30B-A3B, Evaluation protocol=0-shot2026.05 | 52.45 | |
| PMQ#Bits=2.0, Model=Qwen1.5-MoE-A2.7B, Evaluation protocol=0-shot2026.05 | 52.41 | |
| ASVDModel=Qwen2.5-32B, Compression Ratio=0.22026.02 | 52 | |
| GEMQ#Bits=1.5, Model=Mixtral-8×7B, Evaluation protocol=0-shot2026.05 | 52 | |
| PMQ#Bits=1.5, Model=Mixtral-8×7B, Evaluation protocol=0-shot2026.05 | 51.78 | |
| PMQ#Bits=2.0, Model=Qwen3-30B-A3B, Evaluation protocol=0-shot2026.05 | 51.39 | |
| GEMQ#Bits=1.5, Model=Qwen1.5-MoE-A2.7B, Evaluation protocol=0-shot2026.05 | 50.55 | |
| Uniform#Bits=2.0, Model=DeepSeekV2-Lite, Evaluation protocol=0-shot2026.05 | 50.06 | |
| ASVDModel=Qwen2.5-7B, Compression Ratio=0.22026.02 | 50 | |
| Uniform#Bits=1.5, Model=Qwen3-30B-A3B, Evaluation protocol=0-shot2026.05 | 49.42 | |
| ASVDModel=LLaMA3.1-8B, Compression Ratio=0.22026.02 | 49 | |
| PMQ#Bits=1.5, Model=Qwen1.5-MoE-A2.7B, Evaluation protocol=0-shot2026.05 | 47.58 | |
| Uniform#Bits=1.5, Model=Mixtral-8×7B, Evaluation protocol=0-shot2026.05 | 47.45 | |
| GEMQ#Bits=1.5, Model=DeepSeekV2-Lite, Evaluation protocol=0-shot2026.05 | 47.27 | |
| PMQ#Bits=1.5, Model=DeepSeekV2-Lite, Evaluation protocol=0-shot2026.05 | 46.91 | |
| Uniform#Bits=1.5, Model=DeepSeekV2-Lite, Evaluation protocol=0-shot2026.05 | 45.07 | |
| Uniform#Bits=2.5, Model=Qwen1.5-MoE-A2.7B, Evaluation protocol=0-shot2026.05 | 44.12 | |
| PMQ#Bits=1.5, Model=Qwen3-30B-A3B, Evaluation protocol=0-shot2026.05 | 42.08 | |
| Uniform#Bits=2.0, Model=Qwen1.5-MoE-A2.7B, Evaluation protocol=0-shot2026.05 | 41.06 | |
| Uniform#Bits=1.5, Model=Qwen1.5-MoE-A2.7B, Evaluation protocol=0-shot2026.05 | 35.96 |