Commonsense Reasoning on StrategyQA (accuracy)
69.5Accuracy (StrategyQA)Dense
Evaluation Results
| Method | Links | |
|---|---|---|
| DenseBackbone Model=Llama-3-8B-Instruct, Sparsity=0%2026.04 | 69.5 | |
| CAPBackbone Model=Llama-3-8B-Instruct, Sparsity=10%2026.04 | 65.5 | |
| CAPBackbone Model=Mistral-7B-Instruct, Sparsity=10%2026.04 | 65.3 | |
| WandaBackbone Model=Mistral-7B-Instruct, Sparsity=10%2026.04 | 65.1 | |
| WandaBackbone Model=Llama-3-8B-Instruct, Sparsity=10%2026.04 | 65 | |
| CAPBackbone Model=Mistral-7B-Instruct, Sparsity=20%2026.04 | 64.8 | |
| DenseBackbone Model=Mistral-7B-Instruct, Sparsity=0%2026.04 | 64.3 | |
| WandaBackbone Model=Mistral-7B-Instruct, Sparsity=20%2026.04 | 63.8 | |
| CAPBackbone Model=Llama-3-8B-Instruct, Sparsity=20%2026.04 | 63.1 | |
| WandaBackbone Model=Llama-3-8B-Instruct, Sparsity=20%2026.04 | 60.5 | |
| CAPBackbone Model=Mistral-7B-Instruct, Sparsity=50%2026.04 | 53 | |
| WandaBackbone Model=Llama-3-8B-Instruct, Sparsity=50%2026.04 | 51.7 | |
| WandaBackbone Model=Mistral-7B-Instruct, Sparsity=50%2026.04 | 49.5 | |
| CAPBackbone Model=Llama-3-8B-Instruct, Sparsity=50%2026.04 | 6.9 |