Multi-modal Role-playing on MMRole in-domain
1.199IAClaude-Opus-4
Evaluation Results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Claude-Opus-4Backbone=Closed large MLLMs2026.05 | 1.199 | 1.142 | 1.216 | 1.238 | 1.223 | 1.296 | 1.249 | 1.288 | 1.231 | |
| SFTBackbone=Qwen3-VL-8B2026.05 | 1.143 | 1.149 | 1.145 | 1.119 | 1.139 | 1.109 | 1.117 | 1.098 | 1.127 | |
| CAVI (Ours)Backbone=Gemma-3n-E4B2026.05 | 1.135 | 1.066 | 1.156 | 1.195 | 1.151 | 1.217 | 1.179 | 1.209 | 1.164 | |
| CAVI (Ours)Backbone=Qwen3-VL-8B2026.05 | 1.122 | 1.062 | 1.152 | 1.208 | 1.149 | 1.248 | 1.204 | 1.241 | 1.173 | |
| GPT-5Backbone=Closed large MLLMs2026.05 | 1.12 | 1.094 | 1.142 | 1.157 | 1.163 | 1.232 | 1.172 | 1.227 | 1.163 | |
| SFTBackbone=Gemma-3n-E4B2026.05 | 1.112 | 1.116 | 1.12 | 1.104 | 1.109 | 1.123 | 1.108 | 1.104 | 1.112 | |
| VCDBackbone=Qwen3-VL-8B2026.05 | 1.109 | 1.031 | 1.12 | 1.154 | 1.135 | 1.229 | 1.171 | 1.233 | 1.148 | |
| VCDBackbone=Gemma-3n-E4B2026.05 | 1.104 | 1.048 | 1.129 | 1.137 | 1.122 | 1.204 | 1.137 | 1.191 | 1.134 | |
| ClearSightBackbone=Gemma-3n-E4B2026.05 | 1.098 | 1.02 | 1.11 | 1.115 | 1.108 | 1.176 | 1.12 | 1.173 | 1.115 | |
| ClearSightBackbone=Qwen3-VL-8B2026.05 | 1.097 | 1.011 | 1.102 | 1.132 | 1.099 | 1.219 | 1.146 | 1.22 | 1.128 | |
| Gemini-Pro-3Backbone=Closed large MLLMs2026.05 | 1.097 | 1.103 | 1.12 | 1.105 | 1.119 | 1.163 | 1.116 | 1.162 | 1.123 | |
| MemVRBackbone=Qwen3-VL-8B2026.05 | 1.09 | 1.001 | 1.099 | 1.133 | 1.107 | 1.218 | 1.145 | 1.213 | 1.126 | |
| AIRBackbone=Qwen3-VL-8B2026.05 | 1.087 | 1.016 | 1.102 | 1.132 | 1.096 | 1.204 | 1.132 | 1.199 | 1.121 | |
| MemVRBackbone=Gemma-3n-E4B2026.05 | 1.073 | 1.018 | 1.104 | 1.121 | 1.09 | 1.169 | 1.111 | 1.169 | 1.107 | |
| AIRBackbone=Gemma-3n-E4B2026.05 | 1.073 | 1.018 | 1.104 | 1.121 | 1.09 | 1.169 | 1.111 | 1.169 | 1.107 | |
| BaseBackbone=Qwen3-VL-8B2026.05 | 1.062 | 0.987 | 1.072 | 1.097 | 1.078 | 1.18 | 1.122 | 1.176 | 1.097 | |
| BaseBackbone=Gemma-3n-E4B2026.05 | 1.043 | 0.99 | 1.074 | 1.086 | 1.059 | 1.137 | 1.079 | 1.138 | 1.076 |