ResearchBenchmarksInference Latency on VLM prefill 1024 tokensFollow92.7Latency (ms)W4A892.04896.449100.85105.251Dec 27, 2024Evaluation ResultsMethodMethodLinksLatency (ms)W4A8Model=LLaVA-onevision-...Model=LLaVA-onevision-7B, Hardware=RTX 4090, Quantization=W4A82024.1292.7FP16Model=LLaVA-onevision-...Model=LLaVA-onevision-7B, Hardware=RTX 4090, Quantization=FP162024.12109