Rare disease mention extraction on MIMIC3-RD Entity n=117 docs (test)
59.2Micro-F1Mistral 24B
Evaluation results
| Method | Links | |
|---|---|---|
| Mistral 24BApproach=RAG2025.07 | 59.2 | |
| Mistral 24BApproach=RDMA2025.07 | 59.2 | |
| Qwen3 32BApproach=RDMA2025.07 | 52.8 | |
| GPT-5Approach=RDMA2025.07 | 51.3 | |
| GPT-5Approach=RAG2025.07 | 50.5 | |
| GPT-5Approach=Zeroshot2025.07 | 49.7 | |
| Llama-3 70BApproach=RDMA2025.07 | 44.2 | |
| DictionaryApproach=Baselines2025.07 | 43.1 | |
| Qwen3 32BApproach=Zeroshot2025.07 | 32.6 | |
| Mistral 24BApproach=Zeroshot2025.07 | 22.8 | |
| Llama-3 70BApproach=Zeroshot2025.07 | 22.3 | |
| Llama-3 8BApproach=Zeroshot2025.07 | 13.4 | |
| Qwen3 32BApproach=RAG2025.07 | 13.3 | |
| Llama-3 70BApproach=RAG2025.07 | 12.5 | |
| Llama-3 8BApproach=RDMA2025.07 | 7.2 | |
| BioClinicalBERTApproach=Baselines2025.07 | 5 | |
| Llama-3 8BApproach=RAG2025.07 | 4.7 | |
| BioBERT-MRCApproach=Baselines2025.07 | 4.6 | |
| i2b2Approach=Baselines2025.07 | 2 |