Question Answering on Household (full)
80.2AccuracyAutoToM
Evaluation results
| Method | Links | ||
|---|---|---|---|
| AutoToMBackbone=Gemini-3-Flash2026.05 | 80.2 | — | |
| MindZeroBackbone=Llama-3.2-3B2026.05 | 77.8 | 4.4 | |
| AutoToMBackbone=GPT-5.22026.05 | 76.5 | — | |
| MindZeroBackbone=Llama-3.1-8B2026.05 | 76.2 | 12.9 | |
| GPT-5.2mode=Think2026.05 | 73.5 | — | |
| MindZeroBackbone=Qwen3-4B2026.05 | 72.7 | 13.1 | |
| ThoughtTracingBackbone=Gemini-3-Flash2026.05 | 72.3 | — | |
| ThoughtTracingBackbone=GPT-5.22026.05 | 68 | — | |
| AutoToMBackbone=Qwen3-235B-A22B2026.05 | 67.5 | 389.9 | |
| Gemini-3-Flash2026.05 | 67.2 | — | |
| GPT-5.22026.05 | 65 | — | |
| Gemini-3-Pro2026.05 | 60.8 | — | |
| ThoughtTracingBackbone=Qwen3-235B-A22B2026.05 | 59.8 | 2,097.9 | |
| AutoToMBackbone=Qwen3-4B2026.05 | 54.7 | 177.5 | |
| Qwen3-235B-A22B2026.05 | 54.5 | 80.4 | |
| ThoughtTracingBackbone=Qwen3-4B2026.05 | 54.5 | 291.2 | |
| Qwen3-235B-A22Bmode=Think2026.05 | 54 | 2,663 | |
| AutoToMBackbone=Llama-3.1-8B2026.05 | 54 | 136.3 | |
| AutoToMBackbone=Llama-3.2-3B2026.05 | 51 | 23.4 | |
| Qwen3-4Bmode=Think2026.05 | 45 | 41.3 | |
| ThoughtTracingBackbone=Llama-3.1-8B2026.05 | 44.3 | 571.7 | |
| ThoughtTracingBackbone=Llama-3.2-3B2026.05 | 43.5 | 232.9 | |
| Qwen3-4B2026.05 | 42.8 | 10.9 | |
| Llama-3.1-8B2026.05 | 41.3 | 12.9 | |
| Llama-3.2-3B2026.05 | 34.8 | 4 |