Response Type Prediction on Short-Clips (test)
62.21Affirmation ScoreMM-When2Speak
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| MM-When2SpeakModality=Video + Audio + Text, Metric=Precision (P), Evaluation Protocol=fine-tuned2025.05 | 62.21 | 64.35 | 63.15 | 63.29 | 46.26 | 50.52 | 37.78 | 68.15 | 68.78 | |
| MM-When2SpeakModality=Video + Audio + Text, Metric=Recall (R), Evaluation Protocol=fine-tuned2025.05 | 59.86 | 61.99 | 60.79 | 60.44 | 43.91 | 46.25 | 35.45 | 65.79 | 66.42 | |
| VITA-1.5Modality=Video + Audio + Text, Metric=Precision (P), Evaluation Protocol=zero-shot2025.05 | 38.03 | 39.68 | 39.16 | 38.81 | 28.46 | 31.23 | 23.32 | 31.36 | 35.32 | |
| VITA-1.5Modality=Video + Audio + Text, Metric=Recall (R), Evaluation Protocol=zero-shot2025.05 | 35.14 | 36.79 | 36.27 | 35.92 | 25.61 | 28.37 | 20.5 | 28.5 | 32.44 | |
| Gemini-3.1Modality=Video + Text, Metric=Precision (P), Evaluation Protocol=zero-shot2025.05 | 33.77 | 35.4 | 31.66 | 28.47 | 23.52 | 25.37 | 27.37 | 29.5 | 34.27 | |
| ChatGPT-5.2Modality=Video + Text, Metric=Precision (P), Evaluation Protocol=zero-shot2025.05 | 33.16 | 29.38 | 33.59 | 35.61 | 28.66 | 29.31 | 26.88 | 33.81 | 39.42 | |
| Gemini-3.1Modality=Video + Text, Metric=Recall (R), Evaluation Protocol=zero-shot2025.05 | 31.57 | 33.19 | 29.46 | 26.28 | 21.34 | 23.19 | 25.18 | 27.3 | 32.96 | |
| ChatGPT-5.2Modality=Video + Text, Metric=Recall (R), Evaluation Protocol=zero-shot2025.05 | 30.79 | 27.02 | 31.22 | 33.23 | 26.3 | 26.95 | 24.53 | 31.44 | 37.04 | |
| Gemini-3.1Modality=Audio + Text, Metric=Precision (P), Evaluation Protocol=zero-shot2025.05 | 30.49 | 30.38 | 26.24 | 24.25 | 18.06 | 19.21 | 19.3 | 28.08 | 28.84 | |
| Qwen2-AudioModality=Audio + Text, Metric=Precision (P), Evaluation Protocol=zero-shot2025.05 | 29.49 | 24.83 | 21.09 | 19.02 | 18.76 | 19 | 15.43 | 25.13 | 30.22 | |
| Gemini-3.1Modality=Audio + Text, Metric=Recall (R), Evaluation Protocol=zero-shot2025.05 | 27.81 | 27.7 | 23.58 | 21.6 | 15.46 | 16.6 | 16.69 | 25.41 | 26.17 | |
| ChatGPT-5.2Modality=Audio + Text, Metric=Precision (P), Evaluation Protocol=zero-shot2025.05 | 26.85 | 20.76 | 27.76 | 23.68 | 21.07 | 21.57 | 20.7 | 27.05 | 31.43 | |
| Qwen2-AudioModality=Audio + Text, Metric=Recall (R), Evaluation Protocol=zero-shot2025.05 | 26.82 | 22.18 | 18.46 | 16.41 | 16.15 | 16.39 | 12.86 | 22.48 | 27.54 | |
| VITA-1.5Modality=Video + Text, Metric=Precision (P), Evaluation Protocol=zero-shot2025.05 | 25.04 | 24.23 | 20.88 | 26.81 | 20.69 | 25.09 | 18.56 | 33.17 | 34.96 | |
| VITA-1.5Modality=Audio + Text, Metric=Precision (P), Evaluation Protocol=zero-shot2025.05 | 24.64 | 20.34 | 26.03 | 25.59 | 16.86 | 24.24 | 16.97 | 30.7 | 32.69 | |
| ChatGPT-5.2Modality=Audio + Text, Metric=Recall (R), Evaluation Protocol=zero-shot2025.05 | 24.19 | 18.14 | 25.09 | 21.04 | 18.44 | 18.94 | 18.08 | 24.39 | 28.75 | |
| VITA-1.5Modality=Video + Text, Metric=Recall (R), Evaluation Protocol=zero-shot2025.05 | 23.12 | 22.31 | 18.97 | 24.88 | 18.78 | 23.17 | 16.66 | 31.23 | 33.02 | |
| ChatGPT-5.2Modality=Text, Metric=Precision (P), Evaluation Protocol=zero-shot2025.05 | 22.51 | 19.63 | 21.54 | 20.5 | 18.04 | 17.29 | 17.12 | 22.98 | 28.55 | |
| VITA-1.5Modality=Audio + Text, Metric=Recall (R), Evaluation Protocol=zero-shot2025.05 | 21.99 | 17.72 | 23.37 | 22.94 | 14.27 | 21.59 | 14.38 | 28.02 | 30 | |
| Gemini-3.1Modality=Text, Metric=Precision (P), Evaluation Protocol=zero-shot2025.05 | 21.91 | 21.63 | 25.59 | 17.49 | 13.3 | 14.08 | 16.99 | 22.21 | 26.15 | |
| Qwen2.5-VLModality=Video + Text, Metric=Precision (P), Evaluation Protocol=zero-shot2025.05 | 21.89 | 23.87 | 20.32 | 19.69 | 19.3 | 20.88 | 21.19 | 23.78 | 24.52 | |
| Qwen-2.5Modality=Text, Metric=Precision (P), Evaluation Protocol=zero-shot2025.05 | 19.74 | 16.76 | 14.37 | 13.05 | 13.03 | 10.76 | 14.39 | 20.64 | 21.88 | |
| ChatGPT-5.2Modality=Text, Metric=Recall (R), Evaluation Protocol=zero-shot2025.05 | 18.47 | 15.65 | 17.53 | 16.51 | 14.1 | 13.37 | 13.2 | 18.94 | 24.44 | |
| Gemini-3.1Modality=Text, Metric=Recall (R), Evaluation Protocol=zero-shot2025.05 | 18.16 | 17.88 | 21.8 | 13.82 | 9.75 | 10.5 | 13.33 | 18.45 | 22.34 | |
| Qwen2.5-VLModality=Video + Text, Metric=Recall (R), Evaluation Protocol=zero-shot2025.05 | 17.65 | 19.83 | 17.25 | 17.14 | 16.01 | 15.42 | 16.98 | 20.09 | 19.49 | |
| Qwen-2.5Modality=Text, Metric=Recall (R), Evaluation Protocol=zero-shot2025.05 | 16.44 | 13.51 | 11.17 | 9.89 | 9.87 | 7.68 | 11.19 | 17.33 | 18.55 | |
| VITA-1.5Modality=Text, Metric=Precision (P), Evaluation Protocol=zero-shot2025.05 | 15.32 | 17.08 | 15.19 | 14.04 | 14.78 | 12.68 | 13.34 | 20.46 | 21.49 | |
| DeepSeek-V3Modality=Text, Metric=Precision (P), Evaluation Protocol=zero-shot2025.05 | 13.74 | 19.9 | 20.63 | 22.39 | 12.85 | 12.26 | 12.32 | 20.25 | 21.91 | |
| VITA-1.5Modality=Text, Metric=Recall (R), Evaluation Protocol=zero-shot2025.05 | 12.1 | 13.82 | 11.97 | 10.85 | 11.57 | 9.53 | 10.17 | 17.15 | 18.17 | |
| DeepSeek-V3Modality=Text, Metric=Recall (R), Evaluation Protocol=zero-shot2025.05 | 10.56 | 16.6 | 17.32 | 19.06 | 9.69 | 9.12 | 9.18 | 16.94 | 18.58 |