Named Entity Recognition on MIMIC-III (test)
85.6Strict F1Centralized fine-tuning
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Centralized fine-tuningBackbone=LLaMA3-8B2026.01 | 85.6 | 95.4 | |
| Centralized fine-tuningBackbone=DeepSeek-R1-Distill-8B2026.01 | 85.2 | 95.2 | |
| Fed-MedLoRA+Backbone=LLaMA3-8B2026.01 | 85 | 94.9 | |
| Fed-MedLoRABackbone=LLaMA3-8B2026.01 | 84.7 | 94.2 | |
| Fed-MedLoRA+Backbone=DeepSeek-R1-Distill-8B2026.01 | 84.5 | 94.5 | |
| Fed-MedLoRABackbone=DeepSeek-R1-Distill-8B2026.01 | 84.1 | 93.2 | |
| LLaMA3-8B CentralizedBackbone=LLaMA3-8B, Strategy=Centralized (Upper bound)2026.01 | 84 | 92.5 | |
| DeepSeek-R1-Distill-8B CentralizedBackbone=DeepSeek-R1-Distill-8B, Strategy=Centralized (Upper bound)2026.01 | 83 | 92.5 | |
| Bio_ClinicalBERT CentralizedBackbone=Bio_ClinicalBERT, Strategy=Centralized (Upper bound)2026.01 | 81 | 91.9 | |
| Fed-MedLoRABackbone=LLaMA3-8B2026.01 | 80.9 | 89.3 | |
| Centralized fine-tuningBackbone=Bio_ClinicalBERT2026.01 | 80.7 | 90.7 | |
| Single site fine-tuningBackbone=LLaMA3-8B2026.01 | 80.3 | 91.8 | |
| Fed-MedLoRA+Backbone=LLaMA3-8B2026.01 | 80.3 | 90 | |
| Fed-MedLoRA+Backbone=DeepSeek-R1-Distill-8B2026.01 | 80.2 | 88.8 | |
| Fed-MedLoRABackbone=DeepSeek-R1-Distill-8B2026.01 | 79.9 | 86.5 | |
| Single site fine-tuningBackbone=DeepSeek-R1-Distill-8B2026.01 | 79.7 | 90.2 | |
| Bio_ClinicalBERT Single siteBackbone=Bio_ClinicalBERT, Strategy=Single site (Avg, baseline)2026.01 | 78.2 | 84.8 | |
| DeepSeek-R1-Distill-8B Single siteBackbone=DeepSeek-R1-Distill-8B, Strategy=Single site (Avg, baseline)2026.01 | 77.8 | 89.5 | |
| LLaMA3-8B Single siteBackbone=LLaMA3-8B, Strategy=Single site (Avg, baseline)2026.01 | 77.5 | 87.6 | |
| Single site fine-tuningBackbone=Bio_ClinicalBERT2026.01 | 75.3 | 86.7 | |
| Zero-shotBackbone=GPT-4o2026.01 | 55.6 | 81.5 | |
| GPT-4o Zero-shotBackbone=GPT-4o, Strategy=Zero-shot (Baseline)2026.01 | 55.6 | 81.5 | |
| Zero-shotBackbone=LLaMA3-8B2026.01 | 34.5 | 50.3 | |
| LLaMA3-8B Zero-shotBackbone=LLaMA3-8B, Strategy=Zero-shot (Baseline)2026.01 | 34.5 | 50.3 | |
| Zero-shotBackbone=DeepSeek-R1-Distill-8B2026.01 | 29.1 | 43.8 | |
| DeepSeek-R1-Distill-8B Zero-shotBackbone=DeepSeek-R1-Distill-8B, Strategy=Zero-shot (Baseline)2026.01 | 29.1 | 43.8 |