Compositional Reasoning on GQA
62Accuracy (GQA Compositional Reasoning)FP16 (Baseline)
Evaluation Results
| Method | Links | |
|---|---|---|
| FP16 (Baseline)Backbone=LLaVA-1.5 7B, Avg. Bit width=162025.09 | 62 | |
| AWQBackbone=LLaVA-1.5 7B, Avg. Bit width=42025.09 | 61.5 | |
| GPTQBackbone=LLaVA-1.5 7B, Avg. Bit width=42025.09 | 61.4 | |
| FP16 (Baseline)Backbone=Qwen 2.5 VL 7B, Avg. Bit width=162025.09 | 60.5 | |
| AWQBackbone=Qwen 2.5 VL 7B, Avg. Bit width=42025.09 | 60.5 | |
| GPTQBackbone=Qwen 2.5 VL 7B, Avg. Bit width=42025.09 | 60.1 | |
| GPTQBackbone=LLaVA-1.5 7B, Avg. Bit width=32025.09 | 58.8 | |
| LUQBackbone=LLaVA-1.5 7B, Avg. Bit width=2.54, Quantized Layers=16 layer2025.09 | 58.2 | |
| OmniQuantBackbone=Qwen 2.5 VL 7B, Avg. Bit width=42025.09 | 57.5 | |
| GPTQBackbone=Qwen 2.5 VL 7B, Avg. Bit width=32025.09 | 53.4 | |
| LUQBackbone=Qwen 2.5 VL 7B, Avg. Bit width=2.75, Quantized Layers=12 layer2025.09 | 52.9 | |
| OmniQuantBackbone=LLaVA-1.5 7B, Avg. Bit width=42025.09 | 39.2 | |
| SlimLLMBackbone=LLaVA-1.5 7B, Avg. Bit width=42025.09 | 37.7 | |
| SlimLLMBackbone=LLaVA-1.5 7B, Avg. Bit width=32025.09 | 34.4 | |
| BiLLMBackbone=LLaVA-1.5 7B, Avg. Bit width=1.082025.09 | 22.7 | |
| OmniQuantBackbone=LLaVA-1.5 7B, Avg. Bit width=32025.09 | 15.1 | |
| OmniQuantBackbone=Qwen 2.5 VL 7B, Avg. Bit width=32025.09 | 12.8 | |
| BiLLMBackbone=Qwen 2.5 VL 7B, Avg. Bit width=1.082025.09 | 4.3 |