ResearchBenchmarksInference Latency on ViT encoder prefill 729 tokensInference Latency on ViT encoder prefill 729 tokensFollow9.7Latency (ms)W4A89.6410.04510.4510.855Dec 27, 2024Evaluation resultsMethodMethodLinksLatency (ms)W4A8Model=LLaVA-onevision-...Model=LLaVA-onevision-7B, Hardware=RTX 4090, Quantization=W4A82024.129.7FP16Model=LLaVA-onevision-...Model=LLaVA-onevision-7B, Hardware=RTX 4090, Quantization=FP162024.1211.2