Smart Home Agent Interaction on SmartHome-Bench
96.22Quality Task 1 (QT1)HomeFlow-RL
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| HomeFlow-RLModel Size=4B2026.05 | 96.22 | 72.22 | 88.17 | 92.83 | 76.84 | 84.6 | 2.06 | 89.07 | |
| HomeFlow-SFTModel Size=4B2026.05 | 94.99 | 64.7 | 89.5 | 86.75 | 76.8 | 81.5 | 2.06 | 90.48 | |
| HomeFlow-SFTModel Size=8B2026.05 | 94.99 | 73.89 | 90.68 | 85.71 | 81.05 | 82.45 | 2.01 | 113.88 | |
| HomeFlow-RLModel Size=8B2026.05 | 94.72 | 79.63 | 90.68 | 88.31 | 82.11 | 87.03 | 2.07 | 89.11 | |
| DeepSeek-V4-Flash2026.05 | 93.14 | 78.15 | 84.59 | 87.79 | 84.21 | 85.16 | 2.5 | 26.98 | |
| Role-playModel Size=8B2026.05 | 91.56 | 60.19 | 84.95 | 85.45 | 70.53 | 77.78 | 1.97 | 119.65 | |
| GLM-52026.05 | 91.03 | 74.07 | 90.68 | 87.27 | 87.37 | 84.4 | 2.32 | 86.06 | |
| Role-playModel Size=4B2026.05 | 89.71 | 55.56 | 82.08 | 85.45 | 74.74 | 75.62 | 1.96 | 119.01 | |
| Qwen-3.5 Plus2026.05 | 89.45 | 73.89 | 88.53 | 89.87 | 85.26 | 84.15 | 2.17 | 84.03 | |
| Claude-4.6-Sonnet2026.05 | 88.65 | 79.26 | 88.53 | 83.9 | 81.05 | 84.09 | 2.16 | 98.09 | |
| MiniMax M2.72026.05 | 73.88 | 55.56 | 61.65 | 81.3 | 81.05 | 68.06 | 2.1 | 77.9 | |
| GPT-5.52026.05 | 72.82 | 90.19 | 94.27 | 88.05 | 78.95 | 85.8 | 2.13 | 61.16 | |
| Qwen3Model Size=4B2026.05 | 38 | 9.44 | 13.62 | 65.45 | 53.68 | 30.85 | 1.51 | 68.43 | |
| Qwen3Model Size=8B2026.05 | 27.7 | 10.19 | 13.26 | 68.83 | 60 | 31.87 | 1.43 | 64.32 |