Visual-Language Model Inference on VILA Jetson Orin
35.6Throughput (tokens/sec)VILA-7B-AWQ
Evaluation results
| Method | Links | |
|---|---|---|
| VILA-7B-AWQPrecision=W4A162023.06 | 35.6 | |
| VILA-13B-AWQPrecision=W4A162023.06 | 17.5 | |
| VILA-7BPrecision=FP162023.06 | 11.5 | |
| VILA-13BPrecision=FP162023.06 | 6.1 |