Visual-Language Model Inference on VILA A100
155.3Throughput (tokens/sec)VILA-7B-AWQ
Evaluation results
| Method | Links | |
|---|---|---|
| VILA-7B-AWQPrecision=W4A162023.06 | 155.3 | |
| VILA-13B-AWQPrecision=W4A162023.06 | 102.1 | |
| VILA-7BPrecision=FP162023.06 | 81.6 | |
| VILA-13BPrecision=FP162023.06 | 48.5 |