List your company
ResearchFormerly WizWand

Visual Question Answering on VizWiz (val test)

39.76AccuracyFloating Point
16.734422.712228.6934.6678Oct 10, 2024
Updated 8mo ago

Evaluation results

MethodLinks
Floating PointBackbone=OpenFlamingo-3B, Shots=32, Bitwidth=FP2024.1039.76
Q-VLMBackbone=OpenFlamingo-3B, Shots=32, Bitwidth=8-bit2024.1037.6
Q-LoRABackbone=OpenFlamingo-3B, Shots=32, Bitwidth=8-bit2024.1036.38
Q-VLMBackbone=OpenFlamingo-3B, Shots=32, Bitwidth=4-bit2024.1035.52
Q-LoRABackbone=OpenFlamingo-3B, Shots=32, Bitwidth=4-bit2024.1031.64
Floating PointBackbone=OpenFlamingo-3B, Shots=4, Bitwidth=FP2024.1027.05
Q-VLMBackbone=OpenFlamingo-3B, Shots=4, Bitwidth=8-bit2024.1026.59
Q-LoRABackbone=OpenFlamingo-3B, Shots=4, Bitwidth=8-bit2024.1025.83
Q-VLMBackbone=OpenFlamingo-3B, Shots=4, Bitwidth=4-bit2024.1024.55
Q-LoRABackbone=OpenFlamingo-3B, Shots=4, Bitwidth=4-bit2024.1024.17
Floating PointBackbone=OpenFlamingo-3B, Shots=0, Bitwidth=FP2024.1023.79
Q-VLMBackbone=OpenFlamingo-3B, Shots=0, Bitwidth=8-bit2024.1021.47
Q-LoRABackbone=OpenFlamingo-3B, Shots=0, Bitwidth=8-bit2024.1021.24
Q-VLMBackbone=OpenFlamingo-3B, Shots=0, Bitwidth=4-bit2024.1018.69
Q-LoRABackbone=OpenFlamingo-3B, Shots=0, Bitwidth=4-bit2024.1017.62