Symbolic and Logical Reasoning on BBH
85.01AccuracyDense
Evaluation results
| Method | Links | |
|---|---|---|
| DenseSparsity=0, Model Scale=72B, Model Family=Qwen 2.52025.10 | 85.01 | |
| DenseSparsity=0, Model Scale=32B, Model Family=Qwen 2.52025.10 | 81.72 | |
| ARMORSparsity=2:4+2.4%, Model Scale=72B, Model Family=Qwen 2.52025.10 | 79.42 | |
| ARMORSparsity=2:4+3.44%, Model Scale=32B, Model Family=Qwen 2.52025.10 | 76.56 | |
| NoWag-PSparsity=2:4, Model Scale=72B, Model Family=Qwen 2.52025.10 | 76.04 | |
| WandaSparsity=2:4, Model Scale=72B, Model Family=Qwen 2.52025.10 | 75.96 | |
| SparseGPTSparsity=2:4, Model Scale=72B, Model Family=Qwen 2.52025.10 | 75.31 | |
| DenseSparsity=0, Model Scale=14B, Model Family=Qwen 2.52025.10 | 75.18 | |
| SparseGPTSparsity=2:4, Model Scale=32B, Model Family=Qwen 2.52025.10 | 71.31 | |
| WandaSparsity=2:4, Model Scale=32B, Model Family=Qwen 2.52025.10 | 70.17 | |
| NoWag-PSparsity=2:4, Model Scale=32B, Model Family=Qwen 2.52025.10 | 69.53 | |
| DenseSparsity=0, Model Scale=7B, Model Family=Qwen 2.52025.10 | 69.16 | |
| ARMORSparsity=2:4+4.17%, Model Scale=14B, Model Family=Qwen 2.52025.10 | 67.24 | |
| SparseGPTSparsity=2:4, Model Scale=14B, Model Family=Qwen 2.52025.10 | 56.44 | |
| ARMORSparsity=2:4+4.95%, Model Scale=7B, Model Family=Qwen 2.52025.10 | 55.11 | |
| WandaSparsity=2:4, Model Scale=14B, Model Family=Qwen 2.52025.10 | 54.51 | |
| NoWag-PSparsity=2:4, Model Scale=14B, Model Family=Qwen 2.52025.10 | 52.1 | |
| SparseGPTSparsity=2:4, Model Scale=7B, Model Family=Qwen 2.52025.10 | 46.31 | |
| WandaSparsity=2:4, Model Scale=7B, Model Family=Qwen 2.52025.10 | 41.39 | |
| NoWag-PSparsity=2:4, Model Scale=7B, Model Family=Qwen 2.52025.10 | 39.98 |