User Profile Summarization on HPR-Bench sum
75.28AccExUserGPT
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| UserGPTTraining Protocol=RL post-training2026.05 | 75.28 | 97.47 | 19.31 | 6.91 | 82.15 | 6.3559 | 9.8996 | 6.5882 | 6.0502 | |
| Qwen3-235B-A22B-Thinking-2507Model Type=Thinking2026.05 | 70.14 | 96.38 | 18.8 | 5.92 | 79.17 | 6.2549 | 9.8097 | 6.1296 | 7.177 | |
| UserGPTSFTTraining Protocol=SFT2026.05 | 68.99 | 95.87 | 16.31 | 3.78 | 78.54 | 6.0447 | 9.8406 | 6.1253 | 6.4236 | |
| GLM-5Model Type=Thinking2026.05 | 67.75 | 94.22 | 18.04 | 5.14 | 81.13 | 6.0432 | 9.8734 | 6.19 | 6.5677 | |
| Qwen3-235B-A22B-Instruct-2507Model Type=Non-Thinking2026.05 | 64.23 | 94.71 | 16.16 | 4.27 | 76.7 | 5.9192 | 9.821 | 6.086 | 6.9236 | |
| Kimi-K2.5Model Type=Thinking2026.05 | 63.56 | 93.32 | 14.91 | 3.18 | 78.67 | 5.6517 | 9.8624 | 5.7825 | 6.4214 | |
| DeepSeek-V3.2Model Type=Thinking2026.05 | 63.04 | 93.41 | 16.65 | 4.48 | 80.62 | 5.6963 | 9.8493 | 6.0052 | 5.9607 | |
| DeepSeek-R1-0528Model Type=Thinking2026.05 | 62.55 | 92.21 | 11.85 | 2.32 | 76.66 | 5.6062 | 9.8428 | 5.4998 | 5.3581 | |
| Qwen3.6-PlusModel Type=Thinking2026.05 | 57.13 | 90.46 | 13.71 | 2.62 | 77.28 | 5.4473 | 9.8839 | 5.5883 | 7.8214 | |
| Qwen3-8B ThinkingModel Type=Thinking2026.05 | 51.92 | 88.01 | 8.17 | 1.53 | 77.32 | 4.7062 | 9.8515 | 4.8489 | 4.9192 | |
| Qwen3-8B Non-ThinkingModel Type=Non-Thinking2026.05 | 51.6 | 88.21 | 8.28 | 1.75 | 78.11 | 4.7568 | 9.7795 | 4.9616 | 4.4236 |