Multi-turn information acquisition on Home
89.78Success Rate (SR)CUP
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CUPModel=Llama-3.1-8B2026.04 | 89.78 | 3.75 | |
| CUPModel=Qwen3-4B2026.04 | 84.14 | 3.67 | |
| CUPModel=Mistral-7B-v0.32026.04 | 83.87 | 3.62 | |
| MISQ-HFModel=Llama-3.1-8B2026.04 | 75 | 4.12 | |
| MISQ-HFModel=Mistral-7B-v0.32026.04 | 71.77 | 4.06 | |
| MISQ-HFModel=Qwen3-4B2026.04 | 69.89 | 4.1 | |
| BED-LLMModel=Llama-3.1-8B2026.04 | 69.35 | 4.35 | |
| BED-LLMModel=Qwen3-4B2026.04 | 65.05 | 4.31 | |
| BED-LLMModel=Mistral-7B-v0.32026.04 | 63.44 | 4.34 | |
| UoTModel=Llama-3.1-8B2026.04 | 60.48 | 4.61 | |
| UoTModel=Qwen3-4B2026.04 | 59.68 | 4.69 | |
| UoTModel=Mistral-7B-v0.32026.04 | 59.13 | 4.71 | |
| ATDModel=Llama-3.1-8B2026.04 | 56.99 | 4.56 | |
| ATDModel=Mistral-7B-v0.32026.04 | 54.3 | 4.5 | |
| ATDModel=Qwen3-4B2026.04 | 53.76 | 4.58 | |
| LLM-planningModel=Mistral-7B-v0.32026.04 | 52.69 | 4.62 | |
| LLM-planningModel=Qwen3-4B2026.04 | 51.34 | 4.65 | |
| LLM-planningModel=Llama-3.1-8B2026.04 | 49.19 | 4.7 | |
| CoTModel=Qwen3-4B2026.04 | 48.39 | 3.88 | |
| CoTModel=Llama-3.1-8B2026.04 | 47.31 | 3.94 | |
| CoTModel=Mistral-7B-v0.32026.04 | 45.98 | 3.93 | |
| Direct PromptingModel=Qwen3-4B2026.04 | 41.92 | 4.05 | |
| Direct PromptingModel=Mistral-7B-v0.32026.04 | 39.52 | 4.11 | |
| Direct PromptingModel=Llama-3.1-8B2026.04 | 38.17 | 4.15 | |
| SBERT2026.04 | 18.82 | 4.73 |