Language Generation on QASPER
15.35AccuracyE2E Caprese
Evaluation Results
| Method | Links | |
|---|---|---|
| E2E CapreseBase Model=Llama 3.1 8B Instruct, Framework=GRIFFIN, Prompting=0-shot, FF Sparsity=50%2025.05 | 15.35 | |
| Llama 3.1 8B InstructPrompting=0-shot, FF Sparsity=50%2025.05 | 15.16 | |
| Llama 3.2 1B InstructPrompting=0-shot, FF Sparsity=50%2025.05 | 14.43 | |
| E2E CapreseBase Model=Llama 3.1 8B Instruct, Framework=CATS, Prompting=0-shot, FF Sparsity=50%2025.05 | 14.42 | |
| Layer-wise CapreseBase Model=Llama 3.2 1B Instruct, Framework=CATS, Prompting=0-shot, FF Sparsity=50%2025.05 | 14.35 | |
| Layer-wise CapreseBase Model=Llama 3.1 8B Instruct, Framework=CATS, Prompting=0-shot, FF Sparsity=50%2025.05 | 14.34 | |
| Layer-wise CapreseBase Model=Llama 3.1 8B Instruct, Framework=GRIFFIN, Prompting=0-shot, FF Sparsity=50%2025.05 | 14.21 | |
| GRIFFINBase Model=Llama 3.2 1B Instruct, Prompting=0-shot, FF Sparsity=50%2025.05 | 14.11 | |
| E2E CapreseBase Model=Llama 3.2 1B Instruct, Framework=GRIFFIN, Prompting=0-shot, FF Sparsity=50%2025.05 | 13.88 | |
| CATSBase Model=Llama 3.2 1B Instruct, Prompting=0-shot, FF Sparsity=50%2025.05 | 13.88 | |
| E2E CapreseBase Model=Llama 3.2 1B Instruct, Framework=CATS, Prompting=0-shot, FF Sparsity=50%2025.05 | 13.75 | |
| Layer-wise CapreseBase Model=Llama 3.2 1B Instruct, Framework=GRIFFIN, Prompting=0-shot, FF Sparsity=50%2025.05 | 13.4 | |
| GRIFFINBase Model=Llama 3.1 8B Instruct, Prompting=0-shot, FF Sparsity=50%2025.05 | 12.63 | |
| GRIFFINBase Model=Llama 3.2 3B Instruct, Prompting=0-shot, FF Sparsity=50%2025.05 | 12.52 | |
| Llama 3.2 3B InstructPrompting=0-shot, FF Sparsity=50%2025.05 | 12.45 | |
| E2E CapreseBase Model=Llama 3.2 3B Instruct, Framework=CATS, Prompting=0-shot, FF Sparsity=50%2025.05 | 12.36 | |
| E2E CapreseBase Model=Llama 3.2 3B Instruct, Framework=GRIFFIN, Prompting=0-shot, FF Sparsity=50%2025.05 | 12.35 | |
| Layer-wise CapreseBase Model=Llama 3.2 3B Instruct, Framework=CATS, Prompting=0-shot, FF Sparsity=50%2025.05 | 12.26 | |
| CATSBase Model=Llama 3.1 8B Instruct, Prompting=0-shot, FF Sparsity=50%2025.05 | 12.26 | |
| Layer-wise CapreseBase Model=Llama 3.2 3B Instruct, Framework=GRIFFIN, Prompting=0-shot, FF Sparsity=50%2025.05 | 11.6 | |
| CATSBase Model=Llama 3.2 3B Instruct, Prompting=0-shot, FF Sparsity=50%2025.05 | 11.36 | |
| GRIFFINBase Model=Gemma 2 2B Instruct, Prompting=0-shot, FF Sparsity=50%2025.05 | 11.07 | |
| E2E CapreseBase Model=Gemma 2 2B Instruct, Framework=GRIFFIN, Prompting=0-shot, FF Sparsity=50%2025.05 | 11.05 | |
| E2E CapreseBase Model=Gemma 2 2B Instruct, Framework=CATS, Prompting=0-shot, FF Sparsity=50%2025.05 | 11.03 | |
| Gemma 2 2B InstructPrompting=0-shot, FF Sparsity=50%2025.05 | 10.96 | |
| Layer-wise CapreseBase Model=Gemma 2 2B Instruct, Framework=CATS, Prompting=0-shot, FF Sparsity=50%2025.05 | 10.82 | |
| Layer-wise CapreseBase Model=Gemma 2 2B Instruct, Framework=GRIFFIN, Prompting=0-shot, FF Sparsity=50%2025.05 | 10.75 | |
| Layer-wise CapreseBase Model=Gemma 2 9B Instruct, Framework=CATS, Prompting=0-shot, FF Sparsity=50%2025.05 | 10.46 | |
| GRIFFINBase Model=Gemma 2 9B Instruct, Prompting=0-shot, FF Sparsity=50%2025.05 | 10.14 | |
| CATSBase Model=Gemma 2 2B Instruct, Prompting=0-shot, FF Sparsity=50%2025.05 | 10.11 | |
| E2E CapreseBase Model=Gemma 2 9B Instruct, Framework=CATS, Prompting=0-shot, FF Sparsity=50%2025.05 | 10.02 | |
| E2E CapreseBase Model=Gemma 2 9B Instruct, Framework=GRIFFIN, Prompting=0-shot, FF Sparsity=50%2025.05 | 9.92 | |
| Gemma 2 9B InstructPrompting=0-shot, FF Sparsity=50%2025.05 | 9.91 | |
| Layer-wise CapreseBase Model=Gemma 2 9B Instruct, Framework=GRIFFIN, Prompting=0-shot, FF Sparsity=50%2025.05 | 9.82 | |
| CATSBase Model=Gemma 2 9B Instruct, Prompting=0-shot, FF Sparsity=50%2025.05 | 9.78 |