ResearchBenchmarksLatency Measurement on LLaMA-7B linear layers (4096 × 11008)Follow0.0514Latency (ms)MBOK-0.0720390.7609031.5938452.426787May 28, 2025Evaluation ResultsMethodMethodLinksLatency (ms)Speed-upMBOKWEIGHT SIZE=4096 × 110...WEIGHT SIZE=4096 × 11008, Hardware=A100 GPU2025.050.05145.44FP16WEIGHT SIZE=4096 × 110...WEIGHT SIZE=4096 × 11008, Hardware=A100 GPU2025.050.2794—QUIP#WEIGHT SIZE=4096 × 110...WEIGHT SIZE=4096 × 11008, Hardware=A100 GPU2025.050.55530.5QTIPWEIGHT SIZE=4096 × 110...WEIGHT SIZE=4096 × 11008, Hardware=A100 GPU2025.053.13630.09