Medical Question Answering on GenMedGPT-5k 1.0 (test)
100Diversity & Integrity Win RateMindMap
Evaluation results
| Method | Links | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MindMapBaseline Model (Comparator)=GPT-3.5, Evaluation Rater=GPT-42023.08 | 100 | 0 | 0 | 80.11 | 0 | 19.89 | 84.73 | 0.14 | 15.13 | 88 | 5 | 7 | 88.21 | 1.285 | 10.505 | |
| MindMapBaseline Model (Comparator)=BM25 Retriever, Evaluation Rater=GPT-42023.08 | 100 | 0 | 0 | 66.67 | 0 | 33.33 | 75.91 | 1.26 | 22.83 | 87 | 8 | 5 | 82.395 | 2.315 | 15.29 | |
| MindMapBaseline Model (Comparator)=Embedding Retriever, Evaluation Rater=GPT-42023.08 | 100 | 0 | 0 | 76.05 | 0 | 23.95 | 77.03 | 1.96 | 21.01 | 72 | 13 | 15 | 81.27 | 3.74 | 14.99 | |
| MindMapBaseline Model (Comparator)=KG Retriever, Evaluation Rater=GPT-42023.08 | 100 | 0 | 0 | 73.53 | 0 | 26.47 | 66.67 | 2.94 | 30.39 | 74 | 19 | 7 | 78.55 | 5.485 | 15.965 | |
| MindMapBaseline Model (Comparator)=GPT-4, Evaluation Rater=GPT-42023.08 | 100 | 0 | 0 | 75.77 | 0 | 24.23 | 73.11 | 1.4 | 25.49 | 83 | 8 | 9 | 82.97 | 2.35 | 14.68 | |
| MindMapBaseline Model (Comparator)=TOT, Evaluation Rater=GPT-42023.08 | 100 | 0 | 0 | 78.5 | 0 | 21.5 | 75 | 0.3 | 24.6 | 87 | 5 | 8 | 80.17 | 14.8 | 9.93 |