Device Control on MemHomeLife
72.36F1 ScoreGemini-2.5 Pro
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Gemini-2.5 Pro2026.04 | 72.36 | 68.34 | 68.33 | |
| GPT-52026.04 | 72.04 | 66.9 | 64.17 | |
| Deepseek-V3.22026.04 | 67.46 | 61.97 | 55.83 | |
| Qwen3-235B2026.04 | 62.7 | 56.84 | 53.33 |
Loading the SOTA2 catalog…