ResearchBenchmarksInference Latency on VLM prefill 512 tokensFollow59.4Prefill Latency (ms)W4A859.02461.56264.166.638Dec 27, 2024Evaluation ResultsMethodMethodLinksPrefill Latency (ms)W4A8Model=LLaVA-onevision-...Model=LLaVA-onevision-7B, Hardware=RTX 4090, Quantization=W4A82024.1259.4FP16Model=LLaVA-onevision-...Model=LLaVA-onevision-7B, Hardware=RTX 4090, Quantization=FP162024.1268.8