List your company
ResearchFormerly WizWand

Language Generation on QASPER

15.35AccuracyE2E Caprese
9.557211.061112.56514.0689May 8, 2025
Updated 8mo ago

Evaluation results

MethodLinks
E2E CapreseBase Model=Llama 3.1 8B Instruct, Framework=GRIFFIN, Prompting=0-shot, FF Sparsity=50%2025.0515.35
Llama 3.1 8B InstructPrompting=0-shot, FF Sparsity=50%2025.0515.16
Llama 3.2 1B InstructPrompting=0-shot, FF Sparsity=50%2025.0514.43
E2E CapreseBase Model=Llama 3.1 8B Instruct, Framework=CATS, Prompting=0-shot, FF Sparsity=50%2025.0514.42
Layer-wise CapreseBase Model=Llama 3.2 1B Instruct, Framework=CATS, Prompting=0-shot, FF Sparsity=50%2025.0514.35
Layer-wise CapreseBase Model=Llama 3.1 8B Instruct, Framework=CATS, Prompting=0-shot, FF Sparsity=50%2025.0514.34
Layer-wise CapreseBase Model=Llama 3.1 8B Instruct, Framework=GRIFFIN, Prompting=0-shot, FF Sparsity=50%2025.0514.21
GRIFFINBase Model=Llama 3.2 1B Instruct, Prompting=0-shot, FF Sparsity=50%2025.0514.11
E2E CapreseBase Model=Llama 3.2 1B Instruct, Framework=GRIFFIN, Prompting=0-shot, FF Sparsity=50%2025.0513.88
CATSBase Model=Llama 3.2 1B Instruct, Prompting=0-shot, FF Sparsity=50%2025.0513.88
E2E CapreseBase Model=Llama 3.2 1B Instruct, Framework=CATS, Prompting=0-shot, FF Sparsity=50%2025.0513.75
Layer-wise CapreseBase Model=Llama 3.2 1B Instruct, Framework=GRIFFIN, Prompting=0-shot, FF Sparsity=50%2025.0513.4
GRIFFINBase Model=Llama 3.1 8B Instruct, Prompting=0-shot, FF Sparsity=50%2025.0512.63
GRIFFINBase Model=Llama 3.2 3B Instruct, Prompting=0-shot, FF Sparsity=50%2025.0512.52
Llama 3.2 3B InstructPrompting=0-shot, FF Sparsity=50%2025.0512.45
E2E CapreseBase Model=Llama 3.2 3B Instruct, Framework=CATS, Prompting=0-shot, FF Sparsity=50%2025.0512.36
E2E CapreseBase Model=Llama 3.2 3B Instruct, Framework=GRIFFIN, Prompting=0-shot, FF Sparsity=50%2025.0512.35
Layer-wise CapreseBase Model=Llama 3.2 3B Instruct, Framework=CATS, Prompting=0-shot, FF Sparsity=50%2025.0512.26
CATSBase Model=Llama 3.1 8B Instruct, Prompting=0-shot, FF Sparsity=50%2025.0512.26
Layer-wise CapreseBase Model=Llama 3.2 3B Instruct, Framework=GRIFFIN, Prompting=0-shot, FF Sparsity=50%2025.0511.6
CATSBase Model=Llama 3.2 3B Instruct, Prompting=0-shot, FF Sparsity=50%2025.0511.36
GRIFFINBase Model=Gemma 2 2B Instruct, Prompting=0-shot, FF Sparsity=50%2025.0511.07
E2E CapreseBase Model=Gemma 2 2B Instruct, Framework=GRIFFIN, Prompting=0-shot, FF Sparsity=50%2025.0511.05
E2E CapreseBase Model=Gemma 2 2B Instruct, Framework=CATS, Prompting=0-shot, FF Sparsity=50%2025.0511.03
Gemma 2 2B InstructPrompting=0-shot, FF Sparsity=50%2025.0510.96
Layer-wise CapreseBase Model=Gemma 2 2B Instruct, Framework=CATS, Prompting=0-shot, FF Sparsity=50%2025.0510.82
Layer-wise CapreseBase Model=Gemma 2 2B Instruct, Framework=GRIFFIN, Prompting=0-shot, FF Sparsity=50%2025.0510.75
Layer-wise CapreseBase Model=Gemma 2 9B Instruct, Framework=CATS, Prompting=0-shot, FF Sparsity=50%2025.0510.46
GRIFFINBase Model=Gemma 2 9B Instruct, Prompting=0-shot, FF Sparsity=50%2025.0510.14
CATSBase Model=Gemma 2 2B Instruct, Prompting=0-shot, FF Sparsity=50%2025.0510.11
E2E CapreseBase Model=Gemma 2 9B Instruct, Framework=CATS, Prompting=0-shot, FF Sparsity=50%2025.0510.02
E2E CapreseBase Model=Gemma 2 9B Instruct, Framework=GRIFFIN, Prompting=0-shot, FF Sparsity=50%2025.059.92
Gemma 2 9B InstructPrompting=0-shot, FF Sparsity=50%2025.059.91
Layer-wise CapreseBase Model=Gemma 2 9B Instruct, Framework=GRIFFIN, Prompting=0-shot, FF Sparsity=50%2025.059.82
CATSBase Model=Gemma 2 9B Instruct, Prompting=0-shot, FF Sparsity=50%2025.059.78