Multi-turn information acquisition on Inspired
92.86Success Rate (SR)CUP
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CUPModel=Llama-3.1-8B2026.04 | 92.86 | 3.86 | |
| CUPModel=Mistral-7B-v0.32026.04 | 91.84 | 4.01 | |
| CUPModel=Qwen3-4B2026.04 | 89.8 | 3.93 | |
| MISQ-HFModel=Llama-3.1-8B2026.04 | 77.55 | 4.32 | |
| MISQ-HFModel=Mistral-7B-v0.32026.04 | 74.49 | 4.28 | |
| BED-LLMModel=Llama-3.1-8B2026.04 | 71.43 | 4.39 | |
| MISQ-HFModel=Qwen3-4B2026.04 | 67.35 | 4.3 | |
| BED-LLMModel=Mistral-7B-v0.32026.04 | 66.33 | 4.58 | |
| BED-LLMModel=Qwen3-4B2026.04 | 65.31 | 4.41 | |
| UoTModel=Llama-3.1-8B2026.04 | 62.23 | 4.71 | |
| UoTModel=Mistral-7B-v0.32026.04 | 61.22 | 4.66 | |
| UoTModel=Qwen3-4B2026.04 | 59.18 | 4.7 | |
| ATDModel=Qwen3-4B2026.04 | 58.16 | 4.68 | |
| ATDModel=Llama-3.1-8B2026.04 | 58.16 | 4.64 | |
| ATDModel=Mistral-7B-v0.32026.04 | 57.14 | 4.63 | |
| LLM-planningModel=Llama-3.1-8B2026.04 | 50 | 4.77 | |
| LLM-planningModel=Qwen3-4B2026.04 | 48.98 | 4.73 | |
| LLM-planningModel=Mistral-7B-v0.32026.04 | 48.98 | 4.7 | |
| CoTModel=Qwen3-4B2026.04 | 45.92 | 4.01 | |
| CoTModel=Llama-3.1-8B2026.04 | 40.82 | 3.99 | |
| CoTModel=Mistral-7B-v0.32026.04 | 36.73 | 4.09 | |
| Direct PromptingModel=Qwen3-4B2026.04 | 32.65 | 4.22 | |
| Direct PromptingModel=Llama-3.1-8B2026.04 | 28.57 | 4.16 | |
| Direct PromptingModel=Mistral-7B-v0.32026.04 | 27.55 | 4.23 | |
| SBERT2026.04 | 11.22 | 4.85 |