Large Vision-Language Model evaluation on LLaVA Evaluation Suite 1.5
68.5MMBenchLoRA
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| LoRAModel=LLaVA-1.5-13B, Training Bits=16, Inference Bits=162024.07 | 68.5 | 77.1 | 38.3 | 71.2 | 63.8 | |
| QLoRAModel=LLaVA-1.5-13B, Training Bits=4+16, Inference Bits=162024.07 | 67.6 | 76.9 | 36 | 69.9 | 62.7 | |
| EfficientQATModel=LLaVA-1.5-13B, Training Bits=4, Inference Bits=42024.07 | 67.5 | 74.8 | 35.6 | 70.2 | 62 | |
| QLoRA + Block-APModel=LLaVA-1.5-13B, Training Bits=4+16, Inference Bits=42024.07 | 67.4 | 76.6 | 35.6 | 69.3 | 62.4 | |
| EfficientQATModel=LLaVA-1.5-13B, Training Bits=3, Inference Bits=32024.07 | 67.4 | 74.8 | 35.3 | 69.3 | 61.7 | |
| QLoRA + Block-APModel=LLaVA-1.5-13B, Training Bits=4+16, Inference Bits=32024.07 | 66.8 | 75.5 | 34.5 | 68.4 | 61.3 | |
| LoRAModel=LLaVA-1.5-7B, Training Bits=16, Inference Bits=162024.07 | 66.1 | 73.8 | 30.2 | 68.4 | 59.6 | |
| EfficientQATModel=LLaVA-1.5-7B, Training Bits=4, Inference Bits=42024.07 | 64.4 | 73.2 | 30.3 | 68.1 | 58.8 | |
| QLoRAModel=LLaVA-1.5-7B, Training Bits=4+16, Inference Bits=162024.07 | 64.1 | 72.8 | 30.3 | 68 | 58.8 | |
| EfficientQATModel=LLaVA-1.5-13B, Training Bits=2, Inference Bits=22024.07 | 63.9 | 73.1 | 33.9 | 68.6 | 59.9 | |
| QLoRA + Block-APModel=LLaVA-1.5-7B, Training Bits=4+16, Inference Bits=42024.07 | 63.6 | 72 | 29.8 | 67.7 | 58.3 | |
| EfficientQATModel=LLaVA-1.5-7B, Training Bits=3, Inference Bits=32024.07 | 63.2 | 71.4 | 30.9 | 67.3 | 58.2 | |
| QLoRA + Block-APModel=LLaVA-1.5-7B, Training Bits=4+16, Inference Bits=32024.07 | 62.9 | 71.8 | 29.7 | 66.4 | 57.7 | |
| QLoRA + Block-APModel=LLaVA-1.5-13B, Training Bits=4+16, Inference Bits=22024.07 | 62.5 | 72.1 | 32.5 | 65 | 58 | |
| EfficientQATModel=LLaVA-1.5-7B, Training Bits=2, Inference Bits=22024.07 | 62.3 | 68 | 27.8 | 63.4 | 55.4 | |
| QLoRA + Block-APModel=LLaVA-1.5-7B, Training Bits=4+16, Inference Bits=22024.07 | 53.7 | 64.3 | 28.9 | 60.7 | 51.9 |