# Multi-modal Role-playing on MMRole in-domain

Source: [Multi-modal Role-playing on MMRole in-domain](<https://www.sota2.com/research/sota/multi-modal-role-playing-on-mmrole-in-domain>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-12T12:11:49.970Z

Evaluation of multi-modal role-playing capabilities using the in-domain split of the MMRole dataset, assessed using specific component metrics (IA, Flu, Coh, etc.) and an Overall score.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Multi-modal Role-playing |
| Dataset (including any stated split) | MMRole in-domain |
| Metric | IA |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 17 of 17 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [IA](<https://www.sota2.com/research/sota/multi-modal-role-playing-on-mmrole-in-domain.md?metric=IA>): Higher is better; unit: SCALAR.

- [Fluency](<https://www.sota2.com/research/sota/multi-modal-role-playing-on-mmrole-in-domain.md?metric=Fluency>): Higher is better; unit: SCALAR.

- [Coherence](<https://www.sota2.com/research/sota/multi-modal-role-playing-on-mmrole-in-domain.md?metric=Coherence>): Higher is better; unit: SCALAR.

- [ITR](<https://www.sota2.com/research/sota/multi-modal-role-playing-on-mmrole-in-domain.md?metric=ITR>): Higher is better; unit: SCALAR.

- [RA](<https://www.sota2.com/research/sota/multi-modal-role-playing-on-mmrole-in-domain.md?metric=RA>): Higher is better; unit: SCALAR.

- [PC](<https://www.sota2.com/research/sota/multi-modal-role-playing-on-mmrole-in-domain.md?metric=PC>): Higher is better; unit: SCALAR.

- [KC](<https://www.sota2.com/research/sota/multi-modal-role-playing-on-mmrole-in-domain.md?metric=KC>): Higher is better; unit: SCALAR.

- [TC](<https://www.sota2.com/research/sota/multi-modal-role-playing-on-mmrole-in-domain.md?metric=TC>): Higher is better; unit: SCALAR.

- [Overall Score](<https://www.sota2.com/research/sota/multi-modal-role-playing-on-mmrole-in-domain.md?metric=Overall%20Score>): Higher is better; unit: SCALAR.

## Recorded results

| Method | IA | Result date | Report paper |
| --- | --- | --- | --- |
| Claude-Opus-4 | 1.199 | Not recorded | 6a02b469dbc836b87d824b21 |
| SFT | 1.143 | Not recorded | 6a02b469dbc836b87d824b21 |
| CAVI (Ours) | 1.135 | Not recorded | 6a02b469dbc836b87d824b21 |
| CAVI (Ours) | 1.122 | Not recorded | 6a02b469dbc836b87d824b21 |
| GPT-5 | 1.12 | Not recorded | 6a02b469dbc836b87d824b21 |
| SFT | 1.112 | Not recorded | 6a02b469dbc836b87d824b21 |
| VCD | 1.109 | Not recorded | 6a02b469dbc836b87d824b21 |
| VCD | 1.104 | Not recorded | 6a02b469dbc836b87d824b21 |
| ClearSight | 1.098 | Not recorded | 6a02b469dbc836b87d824b21 |
| ClearSight | 1.097 | Not recorded | 6a02b469dbc836b87d824b21 |
| Gemini-Pro-3 | 1.097 | Not recorded | 6a02b469dbc836b87d824b21 |
| MemVR | 1.09 | Not recorded | 6a02b469dbc836b87d824b21 |
| AIR | 1.087 | Not recorded | 6a02b469dbc836b87d824b21 |
| MemVR | 1.073 | Not recorded | 6a02b469dbc836b87d824b21 |
| AIR | 1.073 | Not recorded | 6a02b469dbc836b87d824b21 |
| Base | 1.062 | Not recorded | 6a02b469dbc836b87d824b21 |
| Base | 1.043 | Not recorded | 6a02b469dbc836b87d824b21 |

### Claude-Opus-4

IA: 1.199. Result ID: 6a03185247f34ce924e7264a.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

Recorded evaluation configuration: {"Backbone":"Closed large MLLMs"}. This record does not assert independent reproduction.

### SFT

IA: 1.143. Result ID: 6a03185247f34ce924e7263b.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B"}. This record does not assert independent reproduction.

### CAVI (Ours)

IA: 1.135. Result ID: 6a03185247f34ce924e72647.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

Recorded evaluation configuration: {"Backbone":"Gemma-3n-E4B"}. This record does not assert independent reproduction.

### CAVI (Ours)

IA: 1.122. Result ID: 6a03185247f34ce924e72640.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B"}. This record does not assert independent reproduction.

### GPT-5

IA: 1.12. Result ID: 6a03185247f34ce924e72648.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

[Method origin paper](<https://www.sota2.com/research/paper/695f39f053fdd5c361743ecb.md>)

Recorded evaluation configuration: {"Backbone":"Closed large MLLMs"}. This record does not assert independent reproduction.

### SFT

IA: 1.112. Result ID: 6a03185247f34ce924e72642.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

Recorded evaluation configuration: {"Backbone":"Gemma-3n-E4B"}. This record does not assert independent reproduction.

### VCD

IA: 1.109. Result ID: 6a03185247f34ce924e7263e.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71461e209131cea5053d.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B"}. This record does not assert independent reproduction.

### VCD

IA: 1.104. Result ID: 6a03185247f34ce924e72645.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71461e209131cea5053d.md>)

Recorded evaluation configuration: {"Backbone":"Gemma-3n-E4B"}. This record does not assert independent reproduction.

### ClearSight

IA: 1.098. Result ID: 6a03185247f34ce924e72644.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78dd1e209131cea7adf7.md>)

[Implementation](<https://github.com/ustc-hyin/ClearSight>)

Recorded evaluation configuration: {"Backbone":"Gemma-3n-E4B"}. This record does not assert independent reproduction.

### ClearSight

IA: 1.097. Result ID: 6a03185247f34ce924e7263d.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78dd1e209131cea7adf7.md>)

[Implementation](<https://github.com/ustc-hyin/ClearSight>)

Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B"}. This record does not assert independent reproduction.

### Gemini-Pro-3

IA: 1.097. Result ID: 6a03185247f34ce924e72649.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

Recorded evaluation configuration: {"Backbone":"Closed large MLLMs"}. This record does not assert independent reproduction.

### MemVR

IA: 1.09. Result ID: 6a03185247f34ce924e7263c.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78821e209131cea78bff.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B"}. This record does not assert independent reproduction.

### AIR

IA: 1.087. Result ID: 6a03185247f34ce924e7263f.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a519692d603e6de3e3c851.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B"}. This record does not assert independent reproduction.

### MemVR

IA: 1.073. Result ID: 6a03185247f34ce924e72643.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78821e209131cea78bff.md>)

Recorded evaluation configuration: {"Backbone":"Gemma-3n-E4B"}. This record does not assert independent reproduction.

### AIR

IA: 1.073. Result ID: 6a03185247f34ce924e72646.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a519692d603e6de3e3c851.md>)

Recorded evaluation configuration: {"Backbone":"Gemma-3n-E4B"}. This record does not assert independent reproduction.

### Base

IA: 1.062. Result ID: 6a03185247f34ce924e7263a.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-VL-8B"}. This record does not assert independent reproduction.

### Base

IA: 1.043. Result ID: 6a03185247f34ce924e72641.

[Reporting paper](<https://www.sota2.com/research/paper/6a02b469dbc836b87d824b21.md>)

Recorded evaluation configuration: {"Backbone":"Gemma-3n-E4B"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.