Automatic Diagnosis on Dxy
0.1Number of TurnsGemini-Pro
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Gemini-ProLearning Setting=Zero-shot Setting2024.07 | 0.1 | — | — | — | 57.7 | 56.7 | |
| GPT-3.5Learning Setting=Zero-shot Setting2024.07 | 0.4 | — | — | — | 45.6 | 46.2 | |
| Yi-34B-ChatLearning Setting=Zero-shot Setting2024.07 | 0.5 | — | — | — | 52.9 | 50.5 | |
| GPT-4Learning Setting=Zero-shot Setting2024.07 | 0.6 | — | — | — | 62.5 | 65.4 | |
| DiagnosisGPTLearning Setting=Zero-shot Setting, Confidence threshold (τ)=0.42024.07 | 0.6 | — | — | — | 60.5 | 68.2 | |
| ERNIE BotLearning Setting=Zero-shot Setting2024.07 | 0.8 | — | — | — | 51.9 | 51.9 | |
| DiagnosisGPTLearning Setting=Zero-shot Setting, Confidence threshold (τ)=0.52024.07 | 1.5 | — | — | — | 60.5 | 72.6 | |
| Mixtral-8x7B-Instruct-v0.1Learning Setting=Zero-shot Setting2024.07 | 1.7 | — | — | — | 47.1 | 55.8 | |
| Claude-3-OpusLearning Setting=Zero-shot Setting2024.07 | 1.8 | — | — | — | 62.5 | 73.1 | |
| H-DQN+Max Turns=52023.07 | 1.9 | 70 | 15 | 0.25 | — | — | |
| HRLLearning Setting=Supervised Learning2024.07 | 1.9 | — | — | — | — | 70.2 | |
| H-DQN+Max Turns=102023.07 | 2.3 | 72 | 16 | 0.26 | — | — | |
| H-DQN+Max Turns=152023.07 | 2.3 | 72 | 16 | 0.26 | — | — | |
| DQN+limited turns=202023.07 | 2.4 | 72 | 0.32 | 0.44 | — | — | |
| H-DQN+limited turns=202023.07 | 2.4 | 70 | 0.16 | 0.26 | — | — | |
| DQN+Max Turns=52023.07 | 2.5 | 65 | 31 | 0.42 | — | — | |
| Basic DQNLearning Setting=Supervised Learning2024.07 | 2.5 | — | — | — | — | 64.7 | |
| PPOMax Turns=52023.07 | 2.6 | 73 | 24 | 0.36 | — | — | |
| DQN+Max Turns=102023.07 | 2.7 | 72 | 32 | 0.44 | — | — | |
| DQN+Max Turns=152023.07 | 2.7 | 71 | 32 | 0.44 | — | — | |
| DiagnosisGPTLearning Setting=Zero-shot Setting, Confidence threshold (τ)=0.62024.07 | 2.8 | — | — | — | 60.5 | 75.4 | |
| PPOMax Turns=102023.07 | 3.6 | 75 | 30 | 0.43 | — | — | |
| PPOMax Turns=152023.07 | 4.1 | 77 | 30 | 0.43 | — | — | |
| CoAD_w/o sMax Turns=52023.07 | 4.4 | 75 | 56 | 0.64 | — | — | |
| PPOlimited turns=202023.07 | 4.7 | 78 | 0.31 | 0.45 | — | — | |
| DiaformerMax Turns=52023.07 | 4.8 | 77 | 55 | 0.64 | — | — | |
| GPT-2disMax Turns=52023.07 | 4.8 | 72 | 58 | 0.64 | — | — | |
| DiaformerLearning Setting=Supervised Learning2024.07 | 4.8 | — | — | — | — | 76.6 | |
| CoADMax Turns=52023.07 | 4.9 | 79 | 56 | 0.65 | — | — | |
| CoAD_w/o dMax Turns=52023.07 | 5 | 75 | 59 | 0.65 | — | — | |
| MTDiagLearning Setting=Supervised Learning2024.07 | 5 | — | — | — | — | 76.1 | |
| CoADMax Turns=102023.07 | 9.3 | 85 | 80 | 0.83 | — | — | |
| DiaformerMax Turns=102023.07 | 9.6 | 81 | 78 | 0.79 | — | — | |
| CoAD_w/o sMax Turns=102023.07 | 9.6 | 78 | 79 | 0.78 | — | — | |
| GPT-2disMax Turns=102023.07 | 9.7 | 80 | 79 | 0.79 | — | — | |
| CoAD_w/o dMax Turns=102023.07 | 9.7 | 82 | 83 | 0.82 | — | — | |
| CoAD*limited turns=202023.07 | 10.5 | 85 | 0.93 | 0.89 | — | — | |
| DiaformerMax Turns=152023.07 | 12.4 | 83 | 83 | 0.83 | — | — | |
| Diaformerlimited turns=202023.07 | 13.1 | 83 | 0.83 | 0.83 | — | — | |
| GPT-2disMax Turns=152023.07 | 13.5 | 83 | 85 | 0.84 | — | — | |
| CoADMax Turns=152023.07 | 13.5 | 85 | 90 | 0.88 | — | — | |
| CoAD_w/o sMax Turns=152023.07 | 14.3 | 82 | 89 | 0.85 | — | — | |
| CoAD_w/o dMax Turns=152023.07 | 14.5 | 84 | 91 | 0.88 | — | — | |
| CoAD w/o slimited turns=202023.07 | 14.9 | 84 | 0.84 | 0.84 | — | — | |
| CoAD w/o dlimited turns=202023.07 | 15.1 | 83 | 0.93 | 0.88 | — | — | |
| GPT-2 dislimited turns=202023.07 | 15.8 | 83 | 0.9 | 0.87 | — | — | |
| BERT fulllimited turns=202023.07 | — | 83 | — | — | — | — |