Audio-Visual Question Answering on MUSIC-AVQA (test)
80.7Acc (Avg)VAST
Evaluation results
| Method | Links | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| VAST0-shot=false2023.12 | 80.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Crab2025.03 | 78.94 | — | — | — | — | — | 76.58 | 90.73 | 74.13 | — | — | — | — | — | — | — | |
| VALOR0-shot=false2023.12 | 78.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TSPM2025.03 | 76.79 | — | — | — | — | — | 76.91 | 83.61 | 73.51 | — | — | — | — | — | — | — | |
| CAE-AVVisual Encoder=Swin-V2-L, Audio Encoder=HTS-AT2026.02 | 76.7 | — | — | — | — | — | 78.2 | 83.9 | 72.7 | — | — | — | — | — | — | — | |
| MoLTVisual Encoder=Swin-V2-L, Audio Encoder=HTS-AT, Trainable Params(%)=25.9, Total Params(M)=441.4, Memory (GB)=22025.11 | 76.1 | — | — | — | — | — | 77.4 | 83.1 | 72.3 | — | — | — | — | — | — | — | |
| QSTar2026.01 | 75.98 | 83.81 | 72.72 | 79.76 | 70.03 | 72.38 | — | — | — | 85.64 | 72.05 | 80.63 | 83.46 | 84.9 | 84.17 | 78.98 | |
| CoPL2026.02 | 75.9 | — | — | — | — | — | 76.8 | 77.1 | 75.2 | — | — | — | — | — | — | — | |
| AVMoEVisual Encoder=Swin-V2-L, Audio Encoder=HTS-AT, Trainable Params(%)=42.7, Total Params(M)=456.6, Memory (GB)=11.42025.11 | 75.7 | — | — | — | — | — | 77.6 | 82.7 | 71.9 | — | — | — | — | — | — | — | |
| AVMoEVisual Encoder=Swin-V2-L, Audio Encoder=HTS-AT2026.02 | 75.7 | — | — | — | — | — | 77.6 | 82.7 | 71.9 | — | — | — | — | — | — | — | |
| MoLTVisual Encoder=Swin-V2-L (shared), Audio Encoder=Swin-V2-L (shared), Trainable Params(%)=19.5, Total Params(M)=366.2, Memory (GB)=1.72025.11 | 74.9 | — | — | — | — | — | 75.8 | 81.3 | 71.5 | — | — | — | — | — | — | — | |
| DG-SCTVisual Encoder=Swin-V2-L, Audio Encoder=HTS-AT, Trainable Params(%)=50, Total Params(M)=520.2, Memory (GB)=11.32025.11 | 74.8 | — | — | — | — | — | 77.4 | 81.9 | 70.7 | — | — | — | — | — | — | — | |
| DG-SCTVisual Encoder=Swin-V2-L, Audio Encoder=HTS-AT2026.02 | 74.8 | — | — | — | — | — | 77.4 | 81.9 | 70.7 | — | — | — | — | — | — | — | |
| LAVISH2025.03 | 74.46 | — | — | — | — | — | 75.97 | 80.22 | 71.26 | — | — | — | — | — | — | — | |
| AV-Master2025.10 | 74.22 | 83.6 | 72.39 | 79.13 | 64.21 | 70.8 | — | — | — | 87.02 | 67.85 | 79.95 | 86.55 | 86.61 | 86.58 | 78.51 | |
| LAVisHVisual Encoder=Swin-V2-L (shared), Audio Encoder=Swin-V2-L (shared), Trainable Params(%)=8.4, Total Params(M)=249.8, Memory (GB)=10.82025.11 | 74 | — | — | — | — | — | 75.7 | 80.4 | 70.4 | — | — | — | — | — | — | — | |
| LAVisHBackbone=Swin-V2-L (shared)2026.02 | 74 | — | — | — | — | — | 75.7 | 80.4 | 70.4 | — | — | — | — | — | — | — | |
| CoQoAudio Encoder=Same as AV-Master2025.10 | 73.92 | — | — | — | — | — | — | — | — | — | — | 78.9 | — | — | 83.7 | 77.4 | |
| Q-TriM2026.07 | 73.78 | 82.29 | 71.85 | 80.87 | 62.58 | 69.83 | — | — | — | 85.05 | 65.49 | 77.84 | 84.46 | 87.02 | 85.76 | 77.68 | |
| QA-TIGER2026.01 | 73.74 | 83.1 | 72.5 | 78.58 | 63.94 | 69.59 | — | — | — | 84.86 | 67.85 | 78.58 | 83.96 | 86.29 | 85.14 | 77.62 | |
| QA-TIGER2025.10 | 73.74 | 83.1 | 72.5 | 78.58 | 63.94 | 69.59 | — | — | — | 84.86 | 67.85 | 78.58 | 83.96 | 86.29 | 85.14 | 77.62 | |
| QA-Tiger2026.07 | 73.74 | 83.1 | 72.5 | 78.58 | 63.94 | 69.59 | — | — | — | 84.86 | 67.85 | 78.58 | 83.96 | 86.29 | 85.14 | 77.62 | |
| LAVisHVisual Encoder=Swin-V2-L, Audio Encoder=HTS-AT, Trainable Params(%)=28.7, Total Params(M)=367.4, Note=results reported in [6]2025.11 | 73.6 | — | — | — | — | — | 75.4 | 79.6 | 70.1 | — | — | — | — | — | — | — | |
| LAVisHBackbone=Swin-V2-B (shared)2026.02 | 73.6 | — | — | — | — | — | 75.4 | 79.6 | 70.1 | — | — | — | — | — | — | — | |
| PSTP-Net2025.03 | 73.52 | — | — | — | — | — | 70.91 | 77.26 | 72.57 | — | — | — | — | — | — | — | |
| TSPM2026.01 | 73.51 | 82.19 | 71.85 | 76.21 | 65.76 | 71.17 | — | — | — | 84.07 | 64.65 | 76.91 | 82.29 | 84.9 | 83.61 | 76.79 | |
| TSPM2025.10 | 73.51 | 82.19 | 71.85 | 76.21 | 65.76 | 71.17 | — | — | — | 84.07 | 64.65 | 76.91 | 82.29 | 84.9 | 83.61 | 76.79 | |
| TSPM2026.07 | 73.51 | 82.19 | 71.85 | 76.21 | 65.76 | 71.17 | — | — | — | 84.07 | 64.65 | 76.91 | 82.29 | 84.9 | 83.61 | 76.79 | |
| MettleVisual Encoder=Swin-V2-L, Audio Encoder=HTS-AT, Trainable Params(%)=29.4, Total Params(M)=452.2, Memory (GB)=3.1, Note=reproduced in this paper environment2025.11 | 72.9 | — | — | — | — | — | 74.9 | 79 | 69.4 | — | — | — | — | — | — | — | |
| PSOT2026.01 | 72.61 | — | — | — | — | — | — | — | — | — | — | 78.22 | — | — | 80.07 | 75.29 | |
| PSOTAudio Encoder=Same as AV-Master2025.10 | 72.61 | — | — | — | — | — | — | — | — | — | — | 78.22 | — | — | 80.07 | 75.29 | |
| PSTP2026.01 | 72.57 | 76.18 | 71.8 | 72.23 | 71.79 | 69 | — | — | — | 73.97 | 65.59 | 70.91 | 77.15 | 77.36 | 77.26 | 73.52 | |
| PSTP-Net2026.07 | 72.57 | 76.18 | 71.8 | 72.23 | 71.79 | 69 | — | — | — | 73.97 | 65.59 | 70.91 | 77.15 | 77.36 | 77.26 | 73.52 | |
| COCA2025.03 | 72.33 | — | — | — | — | — | 75.42 | 75.23 | 69.96 | — | — | — | — | — | — | — | |
| AVAF-Net2025.10 | 72.12 | 84.51 | 68.37 | 75.05 | 61.94 | 70.07 | — | — | — | 83.09 | 69.7 | 78.15 | 80.2 | 84.49 | 82.37 | 75.9 | |
| ST-AVQA2025.03 | 71.59 | — | — | — | — | — | 73.87 | 74.4 | 69.53 | — | — | — | — | — | — | — | |
| MAVQA0-shot=false2023.12 | 71.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ST-AVQAVisual Encoder=ResNet-18, Audio Encoder=VGG-like, Trainable Params(%)=11.2, Total Params(M)=94.42025.11 | 71.5 | — | — | — | — | — | 74.1 | 74 | 69.5 | — | — | — | — | — | — | — | |
| ST-AVQA2026.02 | 71.5 | — | — | — | — | — | 74.1 | 74 | 69.5 | — | — | — | — | — | — | — | |
| LAVISH2026.01 | 71.26 | 81.71 | 66.13 | 75.51 | 63.77 | 67.96 | — | — | — | 82.09 | 65.56 | 75.97 | 78.98 | 81.43 | 80.22 | 74.46 | |
| LAVISH2025.10 | 71.26 | 81.71 | 66.13 | 75.51 | 63.77 | 67.96 | — | — | — | 82.09 | 65.56 | 75.97 | 78.98 | 81.43 | 80.22 | 74.46 | |
| LAVISH2026.07 | 71.26 | 81.71 | 66.13 | 75.51 | 63.77 | 67.96 | — | — | — | 82.09 | 65.56 | 75.97 | 78.98 | 81.43 | 80.22 | 74.46 | |
| APL2026.01 | 70.96 | 82.99 | 66.68 | 73.29 | 64.76 | 65.95 | — | — | — | 82.4 | 70.71 | 78.09 | 76.52 | 82.74 | 79.69 | 74.53 | |
| APL2025.10 | 70.96 | 82.99 | 66.68 | 73.29 | 64.76 | 65.95 | — | — | — | 82.4 | 70.71 | 78.09 | 76.52 | 82.74 | 79.69 | 74.53 | |
| APL2026.07 | 70.96 | 82.99 | 66.68 | 73.29 | 64.76 | 65.95 | — | — | — | 82.4 | 70.71 | 78.09 | 76.52 | 82.74 | 79.69 | 74.53 | |
| SHMamba2025.10 | 70.64 | 82.89 | 67.93 | 72.65 | 61.31 | 68.37 | — | — | — | 82.3 | 63.64 | 75.42 | 78.53 | 81.31 | 79.93 | 74.12 | |
| COCA2026.01 | 69.96 | 83.5 | 69.72 | 66.63 | 64.12 | 65.57 | — | — | — | 79.35 | 67.68 | 75.42 | 75.1 | 75.43 | 75.23 | 72.33 | |
| COCA2026.07 | 69.96 | 83.5 | 69.72 | 66.63 | 64.12 | 65.57 | — | — | — | 79.35 | 67.68 | 75.42 | 75.1 | 75.43 | 75.23 | 72.33 | |
| Spatio-Temporal Grounded Audio-Visual NetworkModel Domain=AVQA2022.03 | 69.54 | 81.81 | 64.51 | 70.8 | 66.01 | 63.23 | — | — | — | — | — | — | — | — | — | — | |
| ST-AVQA2025.10 | 69.54 | 81.81 | 64.51 | 70.8 | 66.01 | 63.23 | — | — | — | 78.18 | 67.05 | 74.06 | 71.56 | 76.38 | 74 | 71.52 | |
| ST-AVQA2026.07 | 69.54 | 81.81 | 64.51 | 70.8 | 66.01 | 63.23 | — | — | — | 78.18 | 67.05 | 74.06 | 71.56 | 76.38 | 74 | 71.52 | |
| AVST2026.01 | 69.53 | 82.49 | 64.24 | 69.88 | 64.67 | 65.82 | — | — | — | 77.78 | 67.17 | 73.87 | 73.52 | 75.27 | 74.4 | 71.59 | |
| QAGL2025.10 | 69.43 | 82.29 | 62.83 | 72.73 | 63.4 | 64.36 | — | — | — | 82.99 | 71.04 | 78.58 | 80.12 | 77.88 | 78.89 | 73.58 | |
| Pano-AVQAVisual Encoder=Faster RCNN, Audio Encoder=VGG-like2025.11 | 68.9 | — | — | — | — | — | 70.7 | 72.6 | 66.6 | — | — | — | — | — | — | — | |
| Pano-AVQA2026.02 | 68.9 | — | — | — | — | — | 70.7 | 72.6 | 66.6 | — | — | — | — | — | — | — | |
| AVSDVisual Encoder=VGG-19, Audio Encoder=VGG-like2025.11 | 67.4 | — | — | — | — | — | 68.5 | 70.8 | 65.5 | — | — | — | — | — | — | — | |
| AVSD2026.02 | 67.4 | — | — | — | — | — | 68.5 | 70.8 | 65.5 | — | — | — | — | — | — | — | |
| MCCD2026.01 | 67.13 | 80.87 | 51.63 | 71.46 | 64.67 | 64.6 | — | — | — | 83.87 | 71.04 | 79.14 | 79.78 | 76.73 | 78.24 | 72.2 | |
| MCCD2026.07 | 67.13 | 80.87 | 71.46 | 51.63 | 64.67 | 64.6 | — | — | — | 83.87 | 71.04 | 79.14 | 79.78 | 76.73 | 78.24 | 72.2 | |
| PanaAVQA2026.01 | 66.97 | 82.09 | 61.3 | 65.38 | 63.67 | 62.04 | — | — | — | 75.71 | 65.99 | 72.13 | 70.51 | 75.76 | 73.16 | 69.53 | |
| Pano-AVQAModel Domain=AVQA2022.03 | 66.64 | 81.21 | 59.33 | 64.91 | 64.22 | 63.23 | — | — | — | — | — | — | — | — | — | — | |
| Pano-AVQA2025.10 | 66.64 | 81.21 | 59.33 | 64.91 | 64.22 | 63.23 | — | — | — | 74.36 | 64.56 | 70.73 | 69.39 | 75.65 | 72.56 | 68.93 | |
| Pano-AVQA2026.07 | 66.64 | 81.21 | 59.33 | 64.91 | 64.22 | 63.23 | — | — | — | 74.36 | 64.56 | 70.73 | 69.39 | 75.65 | 72.56 | 68.93 | |
| AVSDModel Domain=AVQA2022.03 | 65.49 | 81.61 | 58.79 | 63.89 | 61.52 | 61.41 | — | — | — | — | — | — | — | — | — | — | |
| AVSD2025.10 | 65.49 | 81.61 | 58.79 | 63.89 | 61.52 | 61.41 | — | — | — | 72.41 | 61.9 | 68.52 | 67.39 | 74.19 | 70.83 | 67.44 | |
| AVSD2026.07 | 65.49 | 81.61 | 58.79 | 63.89 | 61.52 | 61.41 | — | — | — | 72.41 | 61.9 | 68.52 | 67.39 | 74.19 | 70.83 | 67.44 | |
| AVSD2026.01 | 65.44 | 80.77 | 57.93 | 64.03 | 62.85 | 61.07 | — | — | — | 72.47 | 62.46 | 68.78 | 66 | 74.53 | 70.31 | 67.32 | |
| GRUModel Domain=AudioQA2022.03 | 65.18 | 81.71 | 59.44 | 62.64 | 61.88 | 60.07 | — | — | — | — | — | — | — | — | — | — | |
| HME2026.01 | 64.78 | 80.87 | 54.89 | 63.64 | 63.03 | 60.58 | — | — | — | 73.65 | 63.74 | 69.89 | 67.42 | 70.2 | 68.83 | 66.75 | |
| HMEModel Domain=VideoQA2022.03 | 64.05 | 80.3 | 53.18 | 63.19 | 62.69 | 59.83 | — | — | — | — | — | — | — | — | — | — | |
| HME2025.10 | 64.05 | 80.3 | 53.18 | 63.19 | 62.69 | 59.83 | — | — | — | 74.76 | 63.56 | 70.61 | 67.97 | 69.46 | 68.76 | 66.45 | |
| HME2026.07 | 64.05 | 80.3 | 53.18 | 63.19 | 62.69 | 59.83 | — | — | — | 74.76 | 63.56 | 70.61 | 67.97 | 69.46 | 68.76 | 66.45 | |
| PSAC2026.01 | 63.6 | 79.76 | 55.22 | 61.66 | 61.13 | 59.85 | — | — | — | 75.02 | 66.84 | 72 | 68 | 70.78 | 69.41 | 66.62 | |
| PSACModel Domain=VisualQA2022.03 | 63.52 | 77.59 | 55.02 | 63.42 | 61.17 | 59.47 | — | — | — | — | — | — | — | — | — | — | |
| PSAC2025.10 | 63.52 | 77.59 | 55.02 | 63.42 | 61.17 | 59.47 | — | — | — | 75.64 | 66.06 | 72.09 | 68.64 | 69.79 | 69.22 | 66.54 | |
| PSAC2026.07 | 63.52 | 77.59 | 55.02 | 63.42 | 61.17 | 59.47 | — | — | — | 75.64 | 66.06 | 72.09 | 68.64 | 69.79 | 69.22 | 66.54 | |
| CONVLSTMModel Domain=AudioQA2022.03 | 62.24 | 82.91 | 50.81 | 63.03 | 60.27 | 51.58 | — | — | — | — | — | — | — | — | — | — | |
| MCANModel Domain=VisualQA2022.03 | 61.58 | 80.4 | 54.48 | 64.91 | 57.22 | 47.57 | — | — | — | — | — | — | — | — | — | — | |
| MCAN2025.10 | 61.58 | 80.4 | 54.48 | 64.91 | 57.22 | 47.57 | — | — | — | 77.5 | 55.24 | 69.25 | 71.56 | 70.93 | 71.24 | 65.49 | |
| MCAN2026.07 | 61.58 | 80.4 | 54.48 | 64.91 | 57.22 | 47.57 | — | — | — | 77.5 | 55.24 | 69.25 | 71.56 | 70.93 | 71.24 | 65.49 | |
| MCAN2026.01 | 61.52 | 80.77 | 54.57 | 65.22 | 56.77 | 46.84 | — | — | — | 78.07 | 57.74 | 70.58 | 71.76 | 71.76 | 71.76 | 65.83 | |
| FCNLSTM2026.01 | 60.42 | 82.29 | 46.2 | 59.92 | 62.94 | 47.45 | — | — | — | 70.8 | 65.66 | 68.9 | 64.58 | 48.08 | 56.23 | 60.81 | |
| HCAttnModel Domain=VisualQA2022.03 | 60.19 | 79.1 | 49.51 | 59.97 | 55.25 | 56.43 | — | — | — | — | — | — | — | — | — | — | |
| HCAttn2026.07 | 60.19 | 79.1 | 49.51 | 59.97 | 55.25 | 56.43 | — | — | — | 70.25 | 54.91 | 64.57 | 64.05 | 66.37 | 65.22 | 62.3 | |
| FCNLSTMModel Domain=AudioQA2022.03 | 60.06 | 82.01 | 46.28 | 59.34 | 62.15 | 47.33 | — | — | — | — | — | — | — | — | — | — | |
| FCNLSTM2026.07 | 60.06 | 82.01 | 46.28 | 59.34 | 62.15 | 47.33 | — | — | — | 70.45 | 66.22 | 68.88 | 63.89 | 46.74 | 55.21 | 60.34 | |
| BiLSTM AttnModel Domain=VisualQA2022.03 | 57.1 | 78.39 | 45.85 | 56.91 | 53.09 | 49.76 | — | — | — | — | — | — | — | — | — | — | |
| BiLSTM2026.07 | 57.1 | 78.39 | 45.85 | 56.91 | 53.09 | 49.76 | — | — | — | 70.35 | 47.92 | 62.05 | 64.64 | 64.33 | 64.48 | 59.92 | |
| HCRNModel Domain=VideoQA2022.03 | 50.26 | 54.47 | 41.53 | 53.38 | 52.11 | 47.69 | — | — | — | — | — | — | — | — | — | — | |
| HCRN2025.10 | 50.26 | 54.47 | 41.53 | 53.38 | 52.11 | 47.69 | — | — | — | 68.59 | 50.92 | 62.05 | 64.39 | 61.81 | 63.08 | 55.73 | |
| HCRN2026.07 | 50.26 | 54.47 | 41.53 | 53.38 | 52.11 | 47.69 | — | — | — | 68.59 | 50.92 | 62.05 | 64.39 | 61.81 | 63.08 | 55.73 | |
| HCRN2026.01 | 49.82 | 54.15 | 41.74 | 53.28 | 51.04 | 46.72 | — | — | — | 71.29 | 50.67 | 63.69 | 65.33 | 64.98 | 65.15 | 56.34 | |
| OneLLM-7B0-shot=true2023.12 | 47.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ChatBridge-13B0-shot=true2023.12 | 43 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AudioZero-shot=true, Method Category=Individual Modalities, Modality=Audio2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 27.93 | |
| AV-MasterV-Enc.=CLIPL, A-Enc.=VGGish, Z-S=✕, PT=N/A, Params=34M2025.10 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 78.5 | |
| AV-MasterParams=34.07M, FLOPs=6.20G, Latency=89.24ms2025.10 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 78.51 | |
| AV-MasterSize=0.5B2026.05 | — | — | — | — | — | — | 79.9 | 86.5 | 74.2 | — | — | — | — | — | — | 78.5 | |
| AVAF-NetSize=0.5B2026.05 | — | — | — | — | — | — | 78.1 | 82.3 | 72.1 | — | — | — | — | — | — | 75.9 | |
| AVicunaV-Enc.=CLIPL, A-Enc.=CLAP, Z-S=✓, PT=1.1M, Params=7B2025.10 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 49.6 |