Visual Question Answering on VizWiz (val test)
39.76AccuracyFloating Point
Evaluation Results
| Method | Links | |
|---|---|---|
| Floating PointBackbone=OpenFlamingo-3B, Shots=32, Bitwidth=FP2024.10 | 39.76 | |
| Q-VLMBackbone=OpenFlamingo-3B, Shots=32, Bitwidth=8-bit2024.10 | 37.6 | |
| Q-LoRABackbone=OpenFlamingo-3B, Shots=32, Bitwidth=8-bit2024.10 | 36.38 | |
| Q-VLMBackbone=OpenFlamingo-3B, Shots=32, Bitwidth=4-bit2024.10 | 35.52 | |
| Q-LoRABackbone=OpenFlamingo-3B, Shots=32, Bitwidth=4-bit2024.10 | 31.64 | |
| Floating PointBackbone=OpenFlamingo-3B, Shots=4, Bitwidth=FP2024.10 | 27.05 | |
| Q-VLMBackbone=OpenFlamingo-3B, Shots=4, Bitwidth=8-bit2024.10 | 26.59 | |
| Q-LoRABackbone=OpenFlamingo-3B, Shots=4, Bitwidth=8-bit2024.10 | 25.83 | |
| Q-VLMBackbone=OpenFlamingo-3B, Shots=4, Bitwidth=4-bit2024.10 | 24.55 | |
| Q-LoRABackbone=OpenFlamingo-3B, Shots=4, Bitwidth=4-bit2024.10 | 24.17 | |
| Floating PointBackbone=OpenFlamingo-3B, Shots=0, Bitwidth=FP2024.10 | 23.79 | |
| Q-VLMBackbone=OpenFlamingo-3B, Shots=0, Bitwidth=8-bit2024.10 | 21.47 | |
| Q-LoRABackbone=OpenFlamingo-3B, Shots=0, Bitwidth=8-bit2024.10 | 21.24 | |
| Q-VLMBackbone=OpenFlamingo-3B, Shots=0, Bitwidth=4-bit2024.10 | 18.69 | |
| Q-LoRABackbone=OpenFlamingo-3B, Shots=0, Bitwidth=4-bit2024.10 | 17.62 |