Multimodal Role-playing on MMRole (out-of-domain)
1.164IAClaude-Opus-4
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Claude-Opus-4Backbone=Closed large MLLMs2026.05 | 1.164 | 1.12 | 1.195 | 1.201 | 1.185 | 1.239 | 1.189 | 1.246 | 1.192 | |
| SFTBackbone=Qwen3-VL-8B2026.05 | 1.136 | 1.146 | 1.141 | 1.123 | 1.125 | 1.078 | 1.109 | 1.05 | 1.114 | |
| SFTBackbone=Gemma-3n-E4B2026.05 | 1.12 | 1.121 | 1.118 | 1.11 | 1.111 | 1.088 | 1.104 | 1.071 | 1.105 | |
| CAVIBackbone=Qwen3-VL-8B2026.05 | 1.111 | 1.047 | 1.143 | 1.178 | 1.126 | 1.227 | 1.177 | 1.221 | 1.154 | |
| VCDBackbone=Qwen3-VL-8B2026.05 | 1.095 | 1.027 | 1.12 | 1.135 | 1.113 | 1.206 | 1.152 | 1.206 | 1.132 | |
| VCDBackbone=Gemma-3n-E4B2026.05 | 1.095 | 1.034 | 1.126 | 1.129 | 1.11 | 1.179 | 1.127 | 1.197 | 1.125 | |
| GPT-5Backbone=Closed large MLLMs2026.05 | 1.092 | 1.093 | 1.136 | 1.146 | 1.127 | 1.205 | 1.139 | 1.201 | 1.142 | |
| CAVIBackbone=Gemma-3n-E4B2026.05 | 1.086 | 1.047 | 1.143 | 1.157 | 1.132 | 1.2 | 1.152 | 1.191 | 1.139 | |
| AIRBackbone=Qwen3-VL-8B2026.05 | 1.085 | 1.004 | 1.1 | 1.14 | 1.114 | 1.204 | 1.146 | 1.195 | 1.124 | |
| Gemini-Pro-3Backbone=Closed large MLLMs2026.05 | 1.076 | 1.103 | 1.116 | 1.089 | 1.082 | 1.087 | 1.084 | 1.093 | 1.091 | |
| ClearSightBackbone=Gemma-3n-E4B2026.05 | 1.075 | 1.018 | 1.108 | 1.117 | 1.104 | 1.177 | 1.12 | 1.167 | 1.111 | |
| MemVRBackbone=Gemma-3n-E4B2026.05 | 1.073 | 1.05 | 1.116 | 1.124 | 1.107 | 1.17 | 1.125 | 1.174 | 1.117 | |
| AIRBackbone=Gemma-3n-E4B2026.05 | 1.073 | 1.05 | 1.116 | 1.124 | 1.107 | 1.17 | 1.125 | 1.174 | 1.117 | |
| MemVRBackbone=Qwen3-VL-8B2026.05 | 1.066 | 0.992 | 1.071 | 1.118 | 1.094 | 1.213 | 1.138 | 1.197 | 1.111 | |
| ClearSightBackbone=Qwen3-VL-8B2026.05 | 1.056 | 0.998 | 1.073 | 1.113 | 1.078 | 1.2 | 1.134 | 1.204 | 1.107 | |
| BaseBackbone=Qwen3-VL-8B2026.05 | 1.039 | 0.96 | 1.042 | 1.078 | 1.048 | 1.155 | 1.097 | 1.144 | 1.07 | |
| BaseBackbone=Gemma-3n-E4B2026.05 | 1.02 | 0.996 | 1.063 | 1.07 | 1.05 | 1.118 | 1.063 | 1.109 | 1.061 |