Social Reasoning on GRASP-Bench (test)
42.9T1 AccuracyGemini 3.1 Pro
Evaluation results
| Method | Links | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Gemini 3.1 ProMethod Category=Proprietary Models2026.05 | 42.9 | 50.5 | 41.8 | 54.4 | 49.1 | 40.3 | 28.4 | 29.8 | 67.6 | 63.1 | 84.7 | 61.2 | 56.1 | 48.4 | 68.2 | 44.3 | 42.6 | 50 | 44.7 | 44.6 | |
| GPT-5.4Method Category=Proprietary Models2026.05 | 38.6 | 43.9 | 34.9 | 50.9 | 26.4 | 23.9 | 27.3 | 38.1 | 60.4 | 63.1 | 69.4 | 55.1 | 51.2 | 38.7 | 59.1 | 39 | 39.1 | 31.8 | 40.8 | 39.2 | |
| Qwen3-VL-8B + SGRMethod Category=Socially-Grounded RL (Ours), Backbone=Qwen3-VL-8B, Training Policy=SGR2026.05 | 37.1 | 50.4 | 46.3 | 65.8 | 47.3 | 41.8 | 43.2 | 33.3 | 58 | 50.8 | 90.1 | 38.8 | 39 | 25.8 | 59.1 | 48.1 | 53.9 | 22.7 | 51.3 | 43.2 | |
| Qwen3.5-9B + SGRMethod Category=Socially-Grounded RL (Ours), Backbone=Qwen3.5-9B, Training Policy=SGR2026.05 | 37.1 | 52.6 | 48 | 70.2 | 50 | 46.3 | 46.6 | 27.4 | 64.4 | 54.9 | 90.1 | 59.2 | 46.3 | 41.9 | 63.6 | 45.6 | 48.7 | 40.9 | 48.7 | 39.2 | |
| VideoChat-R1.5-7BMethod Category=RL Post-Training Models2026.05 | 32.9 | 33.7 | 32.6 | 39.5 | 25.5 | 35.8 | 30.7 | 32.1 | 35.4 | 44.3 | 29.7 | 36.7 | 43.9 | 12.9 | 27.3 | 33.4 | 33.9 | 27.3 | 31.6 | 36.5 | |
| Qwen3.5-9B (Thinking Mode)Method Category=RL Post-Training Models, Evaluation Protocol=Thinking Mode2026.05 | 32.9 | 31.7 | 23.6 | 21.9 | 14.5 | 19.4 | 22.7 | 34.5 | 45.7 | 45.9 | 66.7 | 32.7 | 29.3 | 25.8 | 27.3 | 28.2 | 24.3 | 31.8 | 30.3 | 31.1 | |
| Qwen3-VL-8B-ThinkingMethod Category=RL Post-Training Models, Evaluation Protocol=Thinking Mode2026.05 | 31.4 | 36.9 | 28.1 | 36 | 23.6 | 16.4 | 30.7 | 27.4 | 48.4 | 41 | 74.8 | 40.8 | 34.1 | 19.4 | 40.9 | 38 | 40.9 | 36.4 | 39.5 | 32.4 | |
| Qwen2.5-VL-7B-InstructMethod Category=Supervised Fine-Tuned Models, Evaluation Protocol=Instruct Mode2026.05 | 30 | 36.2 | 34.3 | 55.3 | 27.3 | 28.4 | 27.3 | 31 | 40.7 | 41 | 48.6 | 36.7 | 34.1 | 22.6 | 45.5 | 33.8 | 33.9 | 31.8 | 31.6 | 36.5 | |
| InternVL3.5-8B-InstructMethod Category=Supervised Fine-Tuned Models, Evaluation Protocol=Instruct Mode2026.05 | 30 | 36.6 | 31.9 | 31.6 | 30 | 28.4 | 36.4 | 34.5 | 42 | 46.7 | 53.2 | 26.5 | 34.1 | 19.4 | 40.9 | 38.3 | 43.5 | 45.5 | 31.6 | 35.1 | |
| Video-R1-7BMethod Category=RL Post-Training Models2026.05 | 30 | 36.3 | 37.1 | 51.8 | 38.2 | 28.4 | 35.2 | 31 | 34.8 | 42.6 | 29.7 | 36.7 | 26.8 | 25.8 | 40.9 | 36.6 | 36.5 | 22.7 | 30.3 | 47.3 | |
| LongVILA-R1-7BMethod Category=RL Post-Training Models2026.05 | 28.6 | 30.1 | 29.1 | 29.8 | 32.7 | 20.9 | 30.7 | 28.6 | 28.5 | 29.5 | 26.1 | 28.6 | 24.4 | 32.3 | 36.4 | 34.1 | 37.4 | 36.4 | 38.2 | 24.3 | |
| LLaVA-OV-1.5-8B-InstructMethod Category=Supervised Fine-Tuned Models, Evaluation Protocol=Instruct Mode2026.05 | 24.3 | 37.3 | 30 | 36.8 | 33.6 | 17.9 | 30.7 | 29.8 | 47.1 | 35.2 | 82.9 | 32.7 | 22 | 35.5 | 27.3 | 38 | 42.6 | 31.8 | 35.5 | 35.1 | |
| Qwen3-VL-8B-InstructMethod Category=Supervised Fine-Tuned Models, Evaluation Protocol=Instruct Mode2026.05 | 24.3 | 38.3 | 28.3 | 35.1 | 25.5 | 14.9 | 31.8 | 33.3 | 52.7 | 45.1 | 86.5 | 28.6 | 41.5 | 22.6 | 40.9 | 38 | 33.9 | 31.8 | 35.5 | 48.6 | |
| VL-Rethinker-7BMethod Category=RL Post-Training Models2026.05 | 24.3 | 34.2 | 29.6 | 36 | 22.7 | 25.4 | 39.8 | 27.4 | 38 | 36.1 | 49.5 | 38.8 | 29.3 | 25.8 | 22.7 | 37.6 | 32.2 | 27.3 | 36.8 | 50 | |
| Qwen3.5-9B (Instruct Mode)Method Category=Supervised Fine-Tuned Models, Evaluation Protocol=Instruct Mode2026.05 | 22.9 | 43 | 32.8 | 52.6 | 30 | 22.4 | 34.1 | 25 | 59 | 51.6 | 87.4 | 40.8 | 46.3 | 25.8 | 68.2 | 40.8 | 40 | 31.8 | 39.5 | 45.9 | |
| LLaVA-OV-1.5-8B-RLMethod Category=RL Post-Training Models2026.05 | 22.9 | 34.2 | 28.3 | 30.7 | 30 | 26.9 | 27.3 | 29.8 | 41.5 | 31.1 | 76.6 | 32.7 | 19.5 | 12.9 | 22.7 | 35.5 | 42.6 | 40.9 | 30.3 | 28.4 | |
| InternVL3.5-8BMethod Category=RL Post-Training Models2026.05 | 20 | 33.9 | 25.9 | 27.2 | 31.8 | 19.4 | 28.4 | 23.8 | 42.8 | 46.7 | 55.9 | 32.7 | 31.7 | 16.1 | 36.4 | 36.9 | 36.5 | 31.8 | 38.2 | 37.8 | |
| Claude Sonnet 4.6Method Category=Proprietary Models2026.05 | 17.1 | 37 | 30.2 | 56.1 | 18.2 | 11.9 | 30.7 | 35.7 | 46.5 | 42.6 | 64 | 30.6 | 39 | 29 | 54.5 | 37.3 | 38.3 | 18.2 | 42.1 | 36.5 |