Multimodal Sequencing on RecipeQA Golden (test)
92.12AccuracyHuman Performance
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Human PerformanceModality=Multimodal2021.10 | 92.12 | 83.13 | 4.82 | 4.53 | 0.95 | 0.45 | |
| Human PerformanceModality=Text-Only2021.10 | 88.92 | 78.56 | 4.76 | 4.41 | 0.93 | 0.7 | |
| CLIP-ViLModality=Multimodal, Encoders=CLIP-ViL, Sequence-aware Pretraining=Y2021.10 | 82.6 | 68 | 4.61 | 4.1 | 0.88 | 1.1 | |
| Human PerformanceModality=Image-Only2021.10 | 80.4 | 64.5 | 4.54 | 4.02 | 0.86 | 1.29 | |
| CLIP-ViLModality=Multimodal, Encoders=CLIP-ViL, Sequence-aware Pretraining=N2021.10 | 79.2 | 60 | 4.57 | 3.93 | 0.85 | 1.29 | |
| VisualBERTModality=Multimodal, Encoders=VisualBERT, Sequence-aware Pretraining=Y2021.10 | 78.2 | 60 | 4.56 | 3.91 | 0.85 | 1.44 | |
| RoBERTaModality=Text-Only, Encoders=RoBERTa, Sequence-aware Pretraining=Y2021.10 | 77 | 57 | 4.49 | 3.81 | 0.84 | 1.48 | |
| VisualBERTModality=Multimodal, Encoders=VisualBERT, Sequence-aware Pretraining=N2021.10 | 76.2 | 58 | 4.49 | 3.85 | 0.83 | 1.58 | |
| RoBERTaModality=Text-Only, Encoders=RoBERTa, Sequence-aware Pretraining=N2021.10 | 74 | 52 | 4.45 | 3.68 | 0.83 | 1.64 | |
| CLIPModality=Image-Only, Encoders=CLIP, Sequence-aware Pretraining=Y2021.10 | 47.2 | 16 | 3.68 | 2.4 | 0.52 | 4.12 | |
| CLIPModality=Image-Only, Encoders=CLIP, Sequence-aware Pretraining=N2021.10 | 38.4 | 8 | 3.39 | 2.02 | 0.35 | 5.44 | |
| ResNetModality=Image-Only, Encoders=ResNet, Sequence-aware Pretraining=N2021.10 | 31.2 | 5 | 3.27 | 2.07 | 0.27 | 6.1 |