Device control on MemHome (test)
61.82No-Memory F1GPT-5
Evaluation Results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-52026.04 | 61.82 | 60.99 | 73.58 | 69.42 | 68.37 | 70 | 78.85 | 75.37 | 79.31 | 71.2 | 69.45 | 73.26 | |
| Deepseek-V3.22026.04 | 58.38 | 57.51 | 71.7 | 63.24 | 61.69 | 61.36 | 80.21 | 76.34 | 75 | 67.64 | 65.49 | 66.84 | |
| Gemini-2.5 Pro2026.04 | 58.11 | 57.25 | 73.58 | 78.35 | 76.91 | 77.73 | 85.02 | 81.16 | 82.76 | 77.58 | 75.5 | 78.66 | |
| Qwen3-235B2026.04 | 50.5 | 49.67 | 66.04 | 70.57 | 68.81 | 65.45 | 80.11 | 76.59 | 74.14 | 70.68 | 68.52 | 68.12 | |
| Qwen2.5-72B2026.04 | 46.61 | 45.62 | 56.6 | 64.12 | 62.17 | 60.45 | 84.86 | 81.22 | 76.72 | 67.92 | 65.6 | 64.78 |