Multimodal Abstractive Summarization on How2 (test)
67.7ROUGE-1SOV-MAS (BART)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SOV-MAS (BART)Input Modality=T+V, Backbone=BART2022.12 | 67.7 | 50.9 | 62.8 | |
| VG-BARTInput Modality=T+V, Backbone=BART2022.12 | 66.3 | 49.4 | 61.4 | |
| SOV-MAS (T5)Input Modality=T+V, Backbone=T52022.12 | 64.8 | 46.7 | 59.5 | |
| BARTInput Modality=T, Backbone=BART2022.12 | 64 | 46.4 | 58.9 | |
| VG-T5Input Modality=T+V, Backbone=T52022.12 | 63.3 | 45.3 | 58 | |
| T5Input Modality=T, Backbone=T52022.12 | 62.8 | 45 | 57.5 | |
| MFFG (RNN)Input Modality=T+V, Backbone=RNN2022.12 | 62.3 | 46.1 | 58.2 | |
| MFFG (Trans.)Input Modality=T+V, Backbone=Trans.2022.12 | 61.6 | 45.1 | 57.4 | |
| HA (RNN)Input Modality=T+V, Backbone=RNN2022.12 | 60.3 | 42.5 | 55.7 | |
| HA (Trans.)Input Modality=T+V, Backbone=Trans.2022.12 | 60.2 | 43.1 | 55.9 | |
| Transf.Input Modality=T2022.12 | 59 | 41 | 54.3 | |
| S2SInput Modality=T2022.12 | 58.6 | 40.6 | 53.8 | |
| PGInput Modality=T2022.12 | 57.2 | 39.5 | 52.8 | |
| READBackbone=VG-T5, #params (M)=1.91 (0.57%)2023.12 | 40.12 | 18.71 | 34.42 | |
| AdapterBackbone=VG-T5, #params (M)=1.95 (0.59%)2023.12 | 37.83 | 17.45 | 33.87 | |
| LoRABackbone=VG-T5, #params (M)=1.93 (0.58%)2023.12 | 36.63 | 16.72 | 32.77 | |
| FullBackbone=VG-T5, #params (M)=333.16 (100%)2023.12 | 32.37 | 8.07 | 26.53 | |
| ProjBackbone=VG-T5, #params (M)=24.67 (7.40%)2023.12 | 29.16 | 5.5 | 26.76 | |
| PromptBackbone=VG-T5, #params (M)=0.18 (0.05%)2023.12 | 28.33 | 5.12 | 11.75 | |
| PartialBackbone=VG-T5, #params (M)=16.52 (4.96%)2023.12 | 27.83 | 4.92 | 10.49 | |
| BiasBackbone=VG-T5, #params (M)=0.07 (0.02%)2023.12 | 27.03 | 4.53 | 19.51 |