List your company

Zero-shot Accuracy on Downstream Suite (BoolQ, HellaSwag, PIQA, RACE, WinoGrande)

82.4BoolQ AccuracyBaseline
61.49666.92372.3577.777Oct 7, 2024

Evaluation results

MethodLinks
BaselineBackbone=Llama2-13B, Compression Ratio=0%2024.1082.460.279.546.871.968.2
BaselineBackbone=Llama2-7B, Compression Ratio=0%2024.1079.257.178.14469.565.6
ESPACEBackbone=Nemotron4-15B, Compression Ratio=25%2024.1078.959.98046.472.867.6
ESPACEBackbone=Llama2-13B, Compression Ratio=20%2024.1078.360.579.54372.866.8
BaselineBackbone=Nemotron4-15B, Compression Ratio=0%2024.1078.362.181.14775.268.7
RetrainedBackbone=Llama2-7B, Compression Ratio=0%2024.1078.257.97843.770.665.7
ESPACEBackbone=Nemotron4-15B, Compression Ratio=50%2024.1077.95777.842.469.965
ESPACEBackbone=Llama2-7B, Compression Ratio=21%2024.1077.157.178.742.769.265
ESPACEBackbone=GPT3-22B, Compression Ratio=40%2024.1076.657.379.540.270.264.8
BaselineBackbone=GPT3-22B, Compression Ratio=0%2024.1076.457.279.340.770.564.8
ESPACEBackbone=Llama2-13B, Compression Ratio=50%2024.1075.756.27841.569.164.1
ESPACEBackbone=GPT3-22B, Compression Ratio=55%2024.1072.255.879.340.169.763.4
ESPACEBackbone=Llama2-7B, Compression Ratio=50%2024.1072.25276.53863.560.4
ESPACEBackbone=GPT3-8B, Compression Ratio=21%2024.1070.355.378.940.769.362.9
BaselineBackbone=GPT3-8B, Compression Ratio=0%2024.106954.278.141.467.862.1
ESPACEBackbone=GPT3-1.3B, Compression Ratio=20%2024.1066.645.17336.463.955.6
ESPACEBackbone=GPT3-8B, Compression Ratio=50%2024.1066.552.377.638.966.960.4
BaselineBackbone=GPT3-1.3B, Compression Ratio=0%2024.1064.343.574.237.658.155.5
ESPACEBackbone=GPT3-1.3B, Compression Ratio=47%2024.1062.339.971.634.558.753.4