List your company
ResearchFormerly WizWand

Inference Efficiency on Model Profiling

15.9Total GEMM Latency (ms)ESPACE
-0.024107.463214.95322.437Oct 7, 2024
Updated 8mo ago

Evaluation results

MethodLinks
ESPACEBackbone=GPT3-1.3B, Compression Ratio=47%, # of Weights=6.42 x 10^82024.1015.931.7
ESPACEBackbone=GPT3-1.3B, Compression Ratio=20%, # of Weights=9.71 x 10^82024.1020.636.1
BaselineBackbone=GPT3-1.3B, Compression Ratio=0%, # of Weights=1.21 x 10^92024.1024.239.8
ESPACEBackbone=GPT3-8B, Compression Ratio=50%, # of Weights=4.08 x 10^92024.1076.8122
ESPACEBackbone=GPT3-8B, Compression Ratio=21%, # of Weights=6.33 x 10^92024.10110155
ESPACEBackbone=Llama2-7B, Compression Ratio=50%, # of Weights=3.24 x 10^92024.10113266
BaselineBackbone=GPT3-8B, Compression Ratio=0%, # of Weights=8.05 x 10^92024.10136186
ESPACEBackbone=Llama2-7B, Compression Ratio=21%, # of Weights=5.11 x 10^92024.10169322
ESPACEBackbone=GPT3-22B, Compression Ratio=55%, # of Weights=9.74 x 10^92024.10181261
BaselineBackbone=Llama2-7B, Compression Ratio=0%, # of Weights=6.48 x 10^92024.10210368
RetrainedBackbone=Llama2-7B, Compression Ratio=0%, # of Weights=6.48 x 10^92024.10210368
ESPACEBackbone=Nemotron4-15B, Compression Ratio=50%, # of Weights=6.25 x 10^92024.10223545
ESPACEBackbone=GPT3-22B, Compression Ratio=40%, # of Weights=1.30 x 10^102024.10229313
ESPACEBackbone=Llama2-13B, Compression Ratio=50%, # of Weights=6.34 x 10^92024.10259447
ESPACEBackbone=Nemotron4-15B, Compression Ratio=25%, # of Weights=9.54 x 10^92024.10324655
ESPACEBackbone=Llama2-13B, Compression Ratio=20%, # of Weights=1.01 x 10^102024.10336562
BaselineBackbone=GPT3-22B, Compression Ratio=0%, # of Weights=2.17 x 10^102024.10354457
BaselineBackbone=Llama2-13B, Compression Ratio=0%, # of Weights=1.27 x 10^102024.10406643
BaselineBackbone=Nemotron4-15B, Compression Ratio=0%, # of Weights=1.25 x 10^102024.10414741