Human Behavior Simulation on SOUL (Social Understanding and Learning) (test)
95FanToMDITTO
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DITTOModel Size=8B2026.05 | 95 | 78 | 93 | 51.2 | 80 | 47 | 56 | 71.3 | 93 | 61 | 72.6 | |
| GRPOModel Size=8B2026.05 | 94 | 77 | 82 | 54.1 | 69 | 42.3 | 58 | 68.3 | 86.3 | 47 | 67.8 | |
| GPT-5.42026.05 | 90 | 70 | 88 | 65.9 | 87 | 30 | 57 | 53.6 | 57.5 | 64 | 66.3 | |
| HumanLMModel Size=8B2026.05 | 78 | 51 | 72 | 22.7 | 66 | 27.5 | 52 | 48.1 | 31.8 | 38 | 48.7 | |
| Qwen3-VL-8B-InstructModel Size=8B, Configuration=Base2026.05 | 78 | 58 | 68 | 43.5 | 67 | 27.7 | 46 | 54.7 | 46.9 | 43 | 53.3 | |
| GPT-5 nano2026.05 | 72 | 37 | 85 | 35.2 | 60 | 31 | 58 | 35.8 | 32.4 | 23 | 46.9 | |
| HERModel Size=32B2026.05 | 55 | 56 | 63 | 41.9 | 75 | 29 | 35 | 36.1 | 53.7 | 39 | 48.4 | |
| Sotopia-RLModel Size=7B2026.05 | 15 | 32 | 61 | 29.6 | 67 | 31.2 | 18 | 36.7 | 40.4 | 34 | 36.5 | |
| CoserModel Size=8B2026.05 | 3 | 2 | 43 | 30.5 | 52 | 24.1 | 0 | 31.4 | 42 | 7 | 23.5 |