Theory of Mind reasoning on MMToM-QA Video-only
69.1Belief Inference 1.1Human
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| HumanModality=Video-only2024.01 | 69.1 | 64.3 | 86.4 | 73.3 | 58.5 | 60 | 76.7 | 63.3 | 64.6 | 68.9 | |
| HumanModality=video only2025.06 | 69.1 | 64.3 | 86.4 | 73.3 | 58.5 | 60 | 76.7 | 63.3 | 64.6 | 68.9 | |
| BIP-ALMModality=Video-only, Backbone=LLAMA 22024.01 | 69 | 63 | 60 | 64 | 62.7 | 54.7 | 53.3 | 62.7 | 58.3 | 61.2 | |
| BIPALM w/ Llama2-7BModality=video only2025.06 | 69 | 63 | 60 | 64 | 62.7 | 54.7 | 53.3 | 62.7 | 58.3 | 61.2 | |
| GPT-4VModality=Video-only2024.01 | 64 | 34 | 39 | 45.7 | 54.7 | 26.7 | 48 | 56 | 46.3 | 46 | |
| GPT-4VModality=video only2025.06 | 64 | 34 | 39 | 45.7 | 54.7 | 26.7 | 48 | 56 | 46.3 | 46 | |
| BIP-ALMModality=Video-only, Backbone=GPT-J2024.01 | 63 | 57 | 72 | 64 | 45.3 | 62.7 | 50.7 | 62.7 | 55.3 | 59.7 | |
| BIPALM w/ GPT-J-6BModality=video only2025.06 | 63 | 57 | 72 | 64 | 45.3 | 62.7 | 50.7 | 62.7 | 55.3 | 59.7 | |
| InstructBLIPModality=Video-only2024.01 | 56 | 50 | 42 | 49.3 | 56 | 45.3 | 54.7 | 53.3 | 52.3 | 50.8 | |
| LLaVAModality=Video-only2024.01 | 33 | 15 | 69 | 39 | 44 | 24 | 56 | 57.3 | 45.3 | 42.2 | |
| LLaVA-7BModality=video only2025.06 | 33 | 15 | 69 | 39 | 44 | 24 | 56 | 57.3 | 45.3 | 42.2 | |
| Video-LLaMA 2Modality=Video-only2024.01 | 24 | 32 | 67 | 41 | 50.7 | 45.3 | 56 | 52 | 51 | 46 | |
| Video-Llama2-13BModality=video only2025.06 | 24 | 32 | 67 | 41 | 50.7 | 45.3 | 56 | 52 | 51 | 46 |