Grounded Situation Recognition on SWiG (test)
75.95Value AccuracyCoFormer
Evaluation results
| Method | Links | |||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CoFormerEvaluation Setting=Ground-Truth Verb2022.03 | 75.95 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 41.87 | 60.11 | 22.12 | |
| SituFormerVerb Evaluation Setting=Ground-Truth-Verb2021.12 | 75.85 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 42.13 | 61.89 | 24.89 | |
| GSRTRVerb prediction scenario=ground-truth verb2021.11 | 74.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 39 | 57.45 | 19.67 | |
| GSRTREvaluation Setting=Ground-Truth Verb2022.03 | 74.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 39 | 57.45 | 19.67 | |
| JSLVerb prediction scenario=ground-truth verb2021.11 | 73.21 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 37.82 | 56.57 | 18.45 | |
| JSLVerb Evaluation Setting=Ground-Truth-Verb2021.12 | 73.21 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 37.82 | 56.57 | 18.45 | |
| JSLEvaluation Setting=Ground-Truth Verb2022.03 | 73.21 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 37.82 | 56.57 | 18.45 | |
| ISLVerb prediction scenario=ground-truth verb2021.11 | 72.42 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 37.1 | 52.19 | 14.58 | |
| ISLVerb Evaluation Setting=Ground-Truth-Verb2021.12 | 72.42 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 37.1 | 52.19 | 14.58 | |
| ISLEvaluation Setting=Ground-Truth Verb2022.03 | 72.42 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 37.1 | 52.19 | 14.58 | |
| GSRTR w/o VEVerb prediction scenario=ground-truth verb2021.11 | 72.32 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 36.75 | 56.03 | 18.02 | |
| CoFormerEvaluation Setting=Top-5 Predicted Verbs2022.03 | 57.76 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 73.31 | 33.98 | 46.25 | 18.37 | |
| SituFormerVerb Evaluation Setting=Top-5-Verb2021.12 | 55.75 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 71.21 | 33.27 | 46 | 20.1 | |
| GSRTRVerb prediction scenario=top-5 predicted verbs2021.11 | 54.13 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 69.81 | 31.01 | 42.5 | 15.88 | |
| GSRTREvaluation Setting=Top-5 Predicted Verbs2022.03 | 54.13 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 69.81 | 31.01 | 42.5 | 15.88 | |
| GSRTR w/o VEVerb prediction scenario=top-5 predicted verbs2021.11 | 53.25 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 69.75 | 29.67 | 41.65 | 14.93 | |
| JSLVerb prediction scenario=top-5 predicted verbs2021.11 | 51.88 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 67.6 | 29.39 | 40.6 | 14.72 | |
| JSLVerb Evaluation Setting=Top-5-Verb2021.12 | 51.88 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 67.6 | 29.39 | 40.6 | 14.72 | |
| JSLEvaluation Setting=Top-5 Predicted Verbs2022.03 | 51.88 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 67.6 | 29.39 | 40.6 | 14.72 | |
| ISLVerb prediction scenario=top-5 predicted verbs2021.11 | 50.16 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 65.51 | 28.47 | 36.6 | 11.56 | |
| ISLVerb Evaluation Setting=Top-5-Verb2021.12 | 50.16 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 65.51 | 28.47 | 36.6 | 11.56 | |
| ISLEvaluation Setting=Top-5 Predicted Verbs2022.03 | 50.16 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 65.51 | 28.47 | 36.6 | 11.56 | |
| CoFormerEvaluation Setting=Top-1 Predicted Verb2022.03 | 35.98 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 44.66 | 22.22 | 29.05 | 12.21 | |
| SituFormerVerb Evaluation Setting=Top-1-Verb2021.12 | 35.24 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 44.2 | 21.86 | 29.22 | 13.41 | |
| GSRTRVerb prediction scenario=top-1 predicted verb2021.11 | 32.15 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 40.63 | 19.28 | 25.49 | 10.1 | |
| GSRTREvaluation Setting=Top-1 Predicted Verb2022.03 | 32.15 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 40.63 | 19.28 | 25.49 | 10.1 | |
| GSRTR w/o VEVerb prediction scenario=top-1 predicted verb2021.11 | 31.87 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 40.61 | 19.01 | 25.21 | 9.69 | |
| JSLVerb prediction scenario=top-1 predicted verb2021.11 | 31.44 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 39.94 | 18.87 | 24.86 | 9.66 | |
| JSLVerb Evaluation Setting=Top-1-Verb2021.12 | 31.44 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 39.94 | 18.87 | 24.86 | 9.66 | |
| JSLEvaluation Setting=Top-1 Predicted Verb2022.03 | 31.44 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 39.94 | 18.87 | 24.86 | 9.66 | |
| ISLVerb prediction scenario=top-1 predicted verb2021.11 | 30.09 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 39.36 | 18.62 | 22.73 | 7.72 | |
| ISLVerb Evaluation Setting=Top-1-Verb2021.12 | 30.09 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 39.36 | 18.62 | 22.73 | 7.72 | |
| ISLEvaluation Setting=Top-1 Predicted Verb2022.03 | 30.09 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 39.36 | 18.62 | 22.73 | 7.72 | |
| ClipSitu XTF2025.01 | — | — | 40.01 | 15.03 | — | — | — | 49.78 | 25.22 | — | — | 53.36 | 33.2 | — | — | — | — | — | — | |
| CoFormer2025.01 | — | — | 29.05 | 12.21 | — | — | — | 46.25 | 18.37 | — | — | 60.11 | 22.12 | — | — | — | — | — | — | |
| ISLModel Architecture Category=RNN based models, Implementation=Authors' implementation2020.03 | — | 39.36 | 30.09 | 18.62 | 22.73 | 7.72 | 65.51 | 50.16 | 28.47 | 36.6 | 11.56 | 72.42 | 37.1 | 52.19 | 14.58 | — | — | — | — | |
| ISLour implementation=true2020.03 | — | 39.36 | 30.09 | 18.62 | 22.73 | 7.72 | 65.51 | 50.16 | 28.47 | 36.6 | 11.56 | 72.42 | 37.1 | 52.19 | 14.58 | — | — | — | — | |
| JSLModel Architecture Category=RNN based models, Implementation=Authors' implementation2020.03 | — | 39.94 | 31.44 | 18.87 | 24.86 | 9.66 | 67.6 | 51.88 | 29.39 | 40.6 | 14.72 | 73.21 | 37.82 | 56.57 | 18.45 | — | — | — | — | |
| JSLour implementation=true2020.03 | — | 39.94 | 31.44 | 18.87 | 24.86 | 9.66 | 67.6 | 51.88 | 29.39 | 40.6 | 14.72 | 73.21 | 37.82 | 56.57 | 18.45 | — | — | — | — | |
| Ours (CoFormer+)Augmentation=Attention Feature Augmentation, Base Model=CoFormer2025.01 | — | — | 41.32 | 18.16 | — | — | — | 57.02 | 24.72 | — | — | 65.11 | 27.28 | — | — | — | — | — | — | |
| SituFormer2025.01 | — | — | 29.22 | 13.41 | — | — | — | 46 | 20.1 | — | — | 61.89 | 24.89 | — | — | — | — | — | — | |
| Updated RNNModel Architecture Category=RNN based models, Implementation=Authors' implementation2020.03 | — | 39.36 | 30.09 | 18.62 | — | — | 65.51 | 50.16 | 28.47 | — | — | 72.42 | 37.1 | — | — | — | — | — | — | |
| Updated RNNour implementation=true2020.03 | — | 39.36 | 30.09 | 18.62 | — | — | 65.51 | 50.16 | 28.47 | — | — | 72.42 | 37.1 | — | — | — | — | — | — |