List your company
ResearchFormerly WizWand

Multimodal Classification on SNLI-VE (test)

81.64AccuracyALBEF+VAWI+SBS
31.397644.441357.48570.5287Dec 15, 2022Jan 3, 2023Jan 23, 2023Feb 12, 2023Mar 4, 2023Mar 24, 2023Apr 13, 2023
Updated 8mo ago

Evaluation results

MethodLinks
ALBEF+VAWI+SBStraining_data_percentage=100%2022.1281.64
ALBEFtraining_data_percentage=100%2022.1280.91
ALBEF+VAWI+SBStraining_data_percentage=50%2022.1276.14
ALBEFtraining_data_percentage=50%2022.1275.47
MMBT*Updated Param. (Million)=196.5, Train Memory Usage (GB)=37.87, Inference Memory Usage (GB)=3.48, Backbone=bert-base / vit-base2023.0474.69
MBT*Updated Param. (Million)=196.0, Train Memory Usage (GB)=38, Inference Memory Usage (GB)=4.06, Backbone=bert-base / vit-base2023.0474.02
PMF-large (M=4, Lf=22)Updated Param. (Million)=4.5, Train Memory Usage (GB)=18.44, Inference Memory Usage (GB)=6.42, Backbone=bert-large / vit-large, M (prompt length)=4, Lf (starting fusion layer)=222023.0472.1
PMF (M=4, Lf=10)Updated Param. (Million)=2.5, Train Memory Usage (GB)=12.84, Inference Memory Usage (GB)=4.08, Backbone=bert-base / vit-base, M (prompt length)=4, Lf (starting fusion layer)=102023.0471.92
LateConcatUpdated Param. (Million)=196.0, Train Memory Usage (GB)=38.54, Inference Memory Usage (GB)=3.36, Backbone=bert-base / vit-base2023.0470.01
BERTUpdated Param. (Million)=109.0, Train Memory Usage (GB)=30.82, Inference Memory Usage (GB)=2.79, Backbone=bert-base2023.0469.82
ALBEF+VAWI+SBStraining_data_percentage=20%2022.1268.23
P-MMBTUpdated Param. (Million)=0.9, Train Memory Usage (GB)=30.9, Inference Memory Usage (GB)=3.48, Backbone=bert-base / vit-base, Prompt Length=102023.0467.58
ALBEFtraining_data_percentage=20%2022.1267.52
ALBEF+VAWI+SBStraining_data_percentage=10%2022.1265.94
BlindPromptUpdated Param. (Million)=< 0.1, Train Memory Usage (GB)=29.57, Inference Memory Usage (GB)=3.65, Backbone=bert-base / vit-base, Prompt Length=202023.0465.54
ALBEFtraining_data_percentage=10%2022.1265.46
PromptFuseUpdated Param. (Million)=< 0.1, Train Memory Usage (GB)=29.57, Inference Memory Usage (GB)=3.55, Backbone=bert-base / vit-base, Prompt Length=202023.0464.53
P-BERTUpdated Param. (Million)=< 0.1, Train Memory Usage (GB)=28.13, Inference Memory Usage (GB)=2.99, Backbone=bert-base, Prompt Length=102023.0463.28
P-LateConcatUpdated Param. (Million)=0.3, Train Memory Usage (GB)=30.82, Inference Memory Usage (GB)=3.43, Backbone=bert-base / vit-base, Prompt Length=102023.0463.05
LinearUpdated Param. (Million)=< 0.1, Train Memory Usage (GB)=3.76, Inference Memory Usage (GB)=3.23, Backbone=bert-base / vit-base2023.0450.05
ViTUpdated Param. (Million)=86.5, Train Memory Usage (GB)=9.36, Inference Memory Usage (GB)=1.99, Backbone=vit-base2023.0433.33
VPTUpdated Param. (Million)=< 0.1, Train Memory Usage (GB)=6.12, Inference Memory Usage (GB)=2.01, Backbone=vit-base, Prompt Length=102023.0433.33