User Simulation on UserLLM
93Primary Metric ScoreDITTO
Evaluation results
| Method | Links | |
|---|---|---|
| DITTOBackbone=Qwen3-VL-8B-Instruct2026.05 | 93 | |
| OSIM 8BModel=OSIM, Parameter Count=8B, Training Stage=Final2026.06 | 90.1 | |
| GRPOBackbone=Qwen3-VL-8B-Instruct2026.05 | 86.3 | |
| Qwen 3.6 Plus2026.06 | 72.1 | |
| Gemini 3.1 Pro2026.06 | 67.7 | |
| GPT 5.52026.06 | 65.3 | |
| Claude Opus 4.72026.06 | 57.6 | |
| GPT-5.42026.05 | 57.5 | |
| HER-32B2026.05 | 53.7 | |
| OSIM 8B-MidModel=OSIM, Parameter Count=8B, Training Stage=Mid-trained2026.06 | 49.5 | |
| Qwen3-VL-8B-InstructRole=Base2026.05 | 46.9 | |
| Qwen3 8B InstModel=Qwen3-8B-Instruct, Parameter Count=8B2026.06 | 46 | |
| Others *2026.06 | 44.6 | |
| GPT-5-nano2026.05 | 32.4 | |
| Base 8BModel=Qwen3-8B-Base, Parameter Count=8B2026.06 | 31 |