List your company

VLM Inference Latency on Qualcomm Snapdragon 8 Gen 3 SoC llama.cpp quantization

6.82VE Latency (ms/patch)MobileVLM-336
6.75567.19037.6258.0597Dec 28, 2023

Evaluation results

MethodLinks
MobileVLM-336Language Model=MobileLLaMA 1.4B, Size (GB)=1.40, Quantization=4-bit, Input Resolution=336x3362023.126.8234,89234.9321.5418.51
LLaVA-v1.5-336Language Model=TinyLLaMA 1B, Size (GB)=1.18, Quantization=4-bit, Input Resolution=336x3362023.127.7731,37041.718.420.7
LLaVA-v1.5-336Language Model=OpenLLaMA 3B, Size (GB)=2.88, Quantization=4-bit, Input Resolution=336x3362023.127.9827,5308.957.2284.43
LLaVA-v1.5-336Language Model=Vicuna 7B, Size (GB)=4.70, Quantization=4-bit, Input Resolution=336x3362023.128.2317,3475.360.25329.89
MobileVLM-336Language Model=MobileLLaMA 2.7B, Size (GB)=2.14, Quantization=4-bit, Input Resolution=336x3362023.128.4327,66018.3612.2133.1