Section Identification on MedSecId Corpus
96.86AccuracyGPT-4
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GPT-4Prompting Strategy=One Shot Prompt2024.04 | 96.86 | 100 | 92.93 | 96.24 | 96.11 | |
| GPT-4Prompting Strategy=Chain-of-Thought (COT)2024.04 | 94.9 | 100 | 88.62 | 93.97 | 92.47 | |
| GPT-4Prompting Strategy=Zero Shot Prompt2024.04 | 94.41 | 100 | 87.61 | 93.4 | 92.05 | |
| BILSTM-CRF2024.04 | 82.2 | 95 | 95 | 95 | — | |
| GPT-4Prompting Strategy=Close Ended Prompt2024.04 | 73.23 | 100 | 73.23 | 84.55 | 73.2 | |
| MedSpacy2024.04 | 56.63 | 100 | 38.29 | 55.38 | 62.63 | |
| LLaMa-2Prompting Strategy=Zero Shot Prompt2024.04 | 52.29 | 94.61 | 32.92 | 48.82 | 62.25 | |
| Regex2024.04 | 49.24 | 100 | 30.07 | 46.24 | 50.8 | |
| LLaMa-2Prompting Strategy=Close Ended Prompt2024.04 | 39.96 | 100 | 39.96 | 57.1 | 39.94 | |
| LLaMa-2Prompting Strategy=Chain-of-Thought (CoT)2024.04 | 38.21 | 93.95 | 21.11 | 34.48 | 46.95 | |
| Keyword Based2024.04 | 36.07 | 100 | 36.07 | 53.01 | 36.05 | |
| LLaMa-2Prompting Strategy=One Shot Prompt2024.04 | 13.95 | 94.57 | 6.86 | 12.8 | 16.86 | |
| MistralPrompting Strategy=Chain-of-Thought (CoT)2024.04 | 11.99 | 98.61 | 5.64 | 10.67 | 15.53 | |
| MistralPrompting Strategy=Zero Shot Prompt2024.04 | 11.51 | 97.43 | 5.23 | 9.93 | 14.45 | |
| MistralPrompting Strategy=One Shot Prompt2024.04 | 8.41 | 98.61 | 4.07 | 7.82 | 10.48 | |
| MistralPrompting Strategy=Close Ended Prompt2024.04 | 5.24 | 100 | 5.24 | 9.96 | 5.24 |