Multimodal Abstractive Summarization on MM-Sum mid-high-resource
41.51ROUGE-1SOV-MAS
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SOV-MASTraining Paradigm=Multilingual Training, Language=Chinese, Visual Features=ViT2022.12 | 41.51 | 26.34 | 34.41 | |
| SOV-MASTraining Paradigm=Monolingual Training, Language=Chinese, Visual Features=ViT2022.12 | 40.61 | 25.37 | 33.39 | |
| VG-mT5Training Paradigm=Monolingual Training, Language=Chinese, Visual Features=ViT2022.12 | 40.31 | 25.45 | 33.51 | |
| mT5Training Paradigm=Multilingual Training, Language=Chinese, Visual Features=ViT2022.12 | 40.3 | 24.97 | 33.04 | |
| mT5Training Paradigm=Monolingual Training, Language=Chinese, Visual Features=ViT2022.12 | 40.2 | 25.39 | 33.49 | |
| VG-mT5Training Paradigm=Multilingual Training, Language=Chinese, Visual Features=ViT2022.12 | 40.19 | 25.31 | 33.35 | |
| SOV-MASTraining Paradigm=Multilingual Training, Language=Urdu, Visual Features=ViT2022.12 | 39.22 | 18.31 | 32.62 | |
| mT5Training Paradigm=Monolingual Training, Language=Urdu, Visual Features=ViT2022.12 | 38.22 | 17.25 | 31.37 | |
| SOV-MASTraining Paradigm=Monolingual Training, Language=Urdu, Visual Features=ViT2022.12 | 38.19 | 17.12 | 31.38 | |
| mT5Training Paradigm=Multilingual Training, Language=Urdu, Visual Features=ViT2022.12 | 38.15 | 17.12 | 31.36 | |
| VG-mT5Training Paradigm=Monolingual Training, Language=Urdu, Visual Features=ViT2022.12 | 38.11 | 17.27 | 31.51 | |
| VG-mT5Training Paradigm=Multilingual Training, Language=Urdu, Visual Features=ViT2022.12 | 38.01 | 17.21 | 31.55 | |
| SOV-MASTraining Paradigm=Multilingual Training, Language=English, Visual Features=ViT2022.12 | 37.77 | 15.14 | 29.81 | |
| SOV-MASTraining Paradigm=Multilingual Training, Language=Persian, Visual Features=ViT2022.12 | 37.65 | 16.92 | 30.58 | |
| SOV-MASTraining Paradigm=Multilingual Training, Language=Indonesian, Visual Features=ViT2022.12 | 37.46 | 17.13 | 31.18 | |
| SOV-MASTraining Paradigm=Monolingual Training, Language=English, Visual Features=ViT2022.12 | 37.29 | 15.18 | 29.82 | |
| VG-mT5Training Paradigm=Monolingual Training, Language=English, Visual Features=ViT2022.12 | 37.25 | 14.97 | 29.54 | |
| mT5Training Paradigm=Monolingual Training, Language=English, Visual Features=ViT2022.12 | 36.99 | 15.18 | 29.64 | |
| VG-mT5Training Paradigm=Multilingual Training, Language=Persian, Visual Features=ViT2022.12 | 36.88 | 16.34 | 29.93 | |
| VG-mT5Training Paradigm=Multilingual Training, Language=English, Visual Features=ViT2022.12 | 36.69 | 14.16 | 28.79 | |
| mT5Training Paradigm=Multilingual Training, Language=English, Visual Features=ViT2022.12 | 36.65 | 13.91 | 28.53 | |
| VG-mT5Training Paradigm=Multilingual Training, Language=Indonesian, Visual Features=ViT2022.12 | 36.55 | 16.38 | 30.19 | |
| mT5Training Paradigm=Multilingual Training, Language=Persian, Visual Features=ViT2022.12 | 36.39 | 15.84 | 29.45 | |
| SOV-MASTraining Paradigm=Monolingual Training, Language=Persian, Visual Features=ViT2022.12 | 36.37 | 15.75 | 29.35 | |
| SOV-MASTraining Paradigm=Multilingual Training, Language=Hindi, Visual Features=ViT2022.12 | 36.33 | 14.91 | 29.68 | |
| mT5Training Paradigm=Monolingual Training, Language=Persian, Visual Features=ViT2022.12 | 36.14 | 15.55 | 29.25 | |
| VG-mT5Training Paradigm=Monolingual Training, Language=Persian, Visual Features=ViT2022.12 | 36.01 | 15.45 | 29.08 | |
| mT5Training Paradigm=Multilingual Training, Language=Indonesian, Visual Features=ViT2022.12 | 35.84 | 15.66 | 29.4 | |
| VG-mT5Training Paradigm=Multilingual Training, Language=Hindi, Visual Features=ViT2022.12 | 35.66 | 14.26 | 28.97 | |
| mT5Training Paradigm=Multilingual Training, Language=Hindi, Visual Features=ViT2022.12 | 35.5 | 13.91 | 28.52 | |
| VG-mT5Training Paradigm=Monolingual Training, Language=Indonesian, Visual Features=ViT2022.12 | 35.16 | 15.49 | 29.09 | |
| SOV-MASTraining Paradigm=Monolingual Training, Language=Indonesian, Visual Features=ViT2022.12 | 35.14 | 15.31 | 28.81 | |
| mT5Training Paradigm=Monolingual Training, Language=Indonesian, Visual Features=ViT2022.12 | 35.1 | 15.44 | 28.91 | |
| SOV-MASTraining Paradigm=Monolingual Training, Language=Hindi, Visual Features=ViT2022.12 | 34.78 | 13.55 | 28.11 | |
| SOV-MASTraining Paradigm=Multilingual Training, Language=Vietnamese, Visual Features=ViT2022.12 | 34.75 | 17.82 | 27.09 | |
| SOV-MASTraining Paradigm=Multilingual Training, Language=Arabic, Visual Features=ViT2022.12 | 34.69 | 14.39 | 28.54 | |
| VG-mT5Training Paradigm=Monolingual Training, Language=Hindi, Visual Features=ViT2022.12 | 34.55 | 13.47 | 28.26 | |
| mT5Training Paradigm=Multilingual Training, Language=Arabic, Visual Features=ViT2022.12 | 34.34 | 14.3 | 28.43 | |
| SOV-MASTraining Paradigm=Multilingual Training, Language=Avg., Visual Features=ViT2022.12 | 33.87 | 15.27 | 27.69 | |
| SOV-MASTraining Paradigm=Monolingual Training, Language=Arabic, Visual Features=ViT2022.12 | 33.86 | 14.53 | 28.06 | |
| VG-mT5Training Paradigm=Monolingual Training, Language=Arabic, Visual Features=ViT2022.12 | 33.79 | 14.11 | 27.95 | |
| VG-mT5Training Paradigm=Multilingual Training, Language=Vietnamese, Visual Features=ViT2022.12 | 33.79 | 17.08 | 26.34 | |
| mT5Training Paradigm=Multilingual Training, Language=Vietnamese, Visual Features=ViT2022.12 | 33.71 | 16.72 | 25.97 | |
| mT5Training Paradigm=Monolingual Training, Language=Arabic, Visual Features=ViT2022.12 | 33.67 | 14.06 | 27.83 | |
| mT5Training Paradigm=Monolingual Training, Language=Hindi, Visual Features=ViT2022.12 | 33.66 | 13.14 | 27.71 | |
| VG-mT5Training Paradigm=Multilingual Training, Language=Arabic, Visual Features=ViT2022.12 | 33.4 | 13.49 | 27.51 | |
| VG-mT5Training Paradigm=Multilingual Training, Language=Avg., Visual Features=ViT2022.12 | 32.89 | 14.53 | 26.84 | |
| mT5Training Paradigm=Multilingual Training, Language=Avg., Visual Features=ViT2022.12 | 32.88 | 14.3 | 26.61 | |
| SOV-MASTraining Paradigm=Monolingual Training, Language=Vietnamese, Visual Features=ViT2022.12 | 32.87 | 16.59 | 25.24 | |
| SOV-MASTraining Paradigm=Multilingual Training, Language=Turkish, Visual Features=ViT2022.12 | 32.67 | 15.7 | 28.77 | |
| SOV-MASTraining Paradigm=Monolingual Training, Language=Avg., Visual Features=ViT2022.12 | 32.49 | 14.26 | 26.4 | |
| VG-mT5Training Paradigm=Monolingual Training, Language=Avg., Visual Features=ViT2022.12 | 32.25 | 14.16 | 26.32 | |
| VG-mT5Training Paradigm=Monolingual Training, Language=Vietnamese, Visual Features=ViT2022.12 | 32.19 | 15.99 | 24.87 | |
| mT5Training Paradigm=Monolingual Training, Language=Vietnamese, Visual Features=ViT2022.12 | 32.18 | 15.84 | 24.83 | |
| mT5Training Paradigm=Monolingual Training, Language=Avg., Visual Features=ViT2022.12 | 32.14 | 14.16 | 26.26 | |
| mT5Training Paradigm=Multilingual Training, Language=Turkish, Visual Features=ViT2022.12 | 31.93 | 14.69 | 27.76 | |
| SOV-MASTraining Paradigm=Multilingual Training, Language=Portuguese, Visual Features=ViT2022.12 | 31.77 | 11.76 | 23.79 | |
| SOV-MASTraining Paradigm=Monolingual Training, Language=Russian, Visual Features=ViT2022.12 | 31.58 | 12.77 | 24.96 | |
| SOV-MASTraining Paradigm=Multilingual Training, Language=Russian, Visual Features=ViT2022.12 | 31.57 | 13.12 | 25.21 | |
| VG-mT5Training Paradigm=Multilingual Training, Language=Turkish, Visual Features=ViT2022.12 | 31.41 | 14.71 | 27.7 | |
| mT5Training Paradigm=Multilingual Training, Language=Russian, Visual Features=ViT2022.12 | 31.12 | 12.33 | 24.67 | |
| VG-mT5Training Paradigm=Multilingual Training, Language=Portuguese, Visual Features=ViT2022.12 | 31.01 | 11.22 | 23.11 | |
| VG-mT5Training Paradigm=Monolingual Training, Language=Russian, Visual Features=ViT2022.12 | 31.01 | 12.43 | 24.52 | |
| SOV-MASTraining Paradigm=Multilingual Training, Language=Spanish, Visual Features=ViT2022.12 | 31 | 11.56 | 23.58 | |
| SOV-MASTraining Paradigm=Monolingual Training, Language=Turkish, Visual Features=ViT2022.12 | 30.91 | 14.6 | 27.16 | |
| mT5Training Paradigm=Multilingual Training, Language=Portuguese, Visual Features=ViT2022.12 | 30.84 | 10.92 | 22.64 | |
| VG-mT5Training Paradigm=Multilingual Training, Language=Russian, Visual Features=ViT2022.12 | 30.55 | 12.65 | 24.58 | |
| VG-mT5Training Paradigm=Monolingual Training, Language=Turkish, Visual Features=ViT2022.12 | 30.44 | 14.4 | 26.77 | |
| VG-mT5Training Paradigm=Multilingual Training, Language=Spanish, Visual Features=ViT2022.12 | 30.37 | 10.94 | 23.02 | |
| mT5Training Paradigm=Monolingual Training, Language=Turkish, Visual Features=ViT2022.12 | 30.37 | 14.39 | 26.79 | |
| mT5Training Paradigm=Monolingual Training, Language=Portuguese, Visual Features=ViT2022.12 | 30.13 | 10.32 | 22.06 | |
| mT5Training Paradigm=Monolingual Training, Language=Russian, Visual Features=ViT2022.12 | 30.01 | 12.47 | 24.28 | |
| mT5Training Paradigm=Multilingual Training, Language=Spanish, Visual Features=ViT2022.12 | 29.91 | 10.7 | 22.66 | |
| SOV-MASTraining Paradigm=Monolingual Training, Language=Portuguese, Visual Features=ViT2022.12 | 29.77 | 10.01 | 21.55 | |
| mT5Training Paradigm=Monolingual Training, Language=Spanish, Visual Features=ViT2022.12 | 29.51 | 10.48 | 22.51 | |
| VG-mT5Training Paradigm=Monolingual Training, Language=Portuguese, Visual Features=ViT2022.12 | 29.46 | 9.72 | 21.91 | |
| VG-mT5Training Paradigm=Monolingual Training, Language=Spanish, Visual Features=ViT2022.12 | 29.37 | 10.59 | 22.52 | |
| SOV-MASTraining Paradigm=Monolingual Training, Language=Spanish, Visual Features=ViT2022.12 | 29.19 | 10.32 | 22.37 | |
| SOV-MASTraining Paradigm=Multilingual Training, Language=Tamil, Visual Features=ViT2022.12 | 24.01 | 10.82 | 21.89 | |
| SOV-MASTraining Paradigm=Multilingual Training, Language=Ukrainian, Visual Features=ViT2022.12 | 23.84 | 9.94 | 20.49 | |
| VG-mT5Training Paradigm=Multilingual Training, Language=Tamil, Visual Features=ViT2022.12 | 23.14 | 10.29 | 20.98 | |
| mT5Training Paradigm=Multilingual Training, Language=Tamil, Visual Features=ViT2022.12 | 22.96 | 10.05 | 20.75 | |
| SOV-MASTraining Paradigm=Monolingual Training, Language=Tamil, Visual Features=ViT2022.12 | 22.8 | 10.51 | 20.62 | |
| mT5Training Paradigm=Multilingual Training, Language=Ukrainian, Visual Features=ViT2022.12 | 22.79 | 9.13 | 19.46 | |
| VG-mT5Training Paradigm=Multilingual Training, Language=Ukrainian, Visual Features=ViT2022.12 | 22.79 | 9.39 | 19.75 | |
| mT5Training Paradigm=Monolingual Training, Language=Tamil, Visual Features=ViT2022.12 | 22.31 | 10.08 | 20.36 | |
| VG-mT5Training Paradigm=Monolingual Training, Language=Tamil, Visual Features=ViT2022.12 | 22.29 | 10.14 | 20.38 | |
| SOV-MASTraining Paradigm=Monolingual Training, Language=Ukrainian, Visual Features=ViT2022.12 | 21.77 | 8.61 | 18.77 | |
| VG-mT5Training Paradigm=Monolingual Training, Language=Ukrainian, Visual Features=ViT2022.12 | 21.69 | 8.78 | 18.65 | |
| mT5Training Paradigm=Monolingual Training, Language=Ukrainian, Visual Features=ViT2022.12 | 21.57 | 8.66 | 18.64 |