Multi-party Multi-modal Social Interaction (MMSI) on Ego4D
65.1STI AccuracyOnline-MMSI-VLM (LLaMA)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Online-MMSI-VLM (LLaMA)Backbone=LLaMA2025.03 | 65.1 | 59.8 | 43.4 | 56.1 | |
| Online-MMSI-VLM (Qwen)Backbone=Qwen2025.03 | 61.7 | 60.7 | 39.5 | 53.9 | |
| Vanilla LLaMABackbone=LLaMA2025.03 | 60.2 | 54.5 | 38.2 | 51 | |
| Lee et al. (2024a) (Online)Temporal Context=Online2025.03 | 59.6 | 55.4 | 41 | 52 | |
| Vanilla QwenBackbone=Qwen2025.03 | 57.7 | 48.2 | 32.9 | 46.3 | |
| Lee et al. (2024a) (Offline)Temporal Context=Offline2025.03 | 52.8 | 41.1 | 32.1 | 42 | |
| Vanilla InternVLBackbone=InternVL2025.03 | 52.8 | 55.2 | 36.8 | 48.2 | |
| GPT-4o (Zero-shot)Evaluation Protocol=Zero-shot2025.03 | 44 | 50.9 | 31.6 | 42.2 | |
| Gemini 2.5 Pro (Zero-shot)Evaluation Protocol=Zero-shot2025.03 | 30.5 | 29.4 | 30.2 | 30 |