Named Entity Recognition on I2B2 Two-site experiment (test)
86.4Strict Micro F1Centralized (Upper bound)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Centralized (Upper bound)Backbone=DeepSeek-R1-Distill-8B2026.01 | 86.4 | 92.6 | |
| Centralized (Upper bound)Backbone=LLaMA3-8B2026.01 | 86.3 | 94.5 | |
| Fed-MedLoRA+Backbone=LLaMA3-8B2026.01 | 86 | 93.9 | |
| Fed-MedLoRA+Backbone=DeepSeek-R1-Distill-8B2026.01 | 85.5 | 91.7 | |
| Fed-MedLoRABackbone=LLaMA3-8B2026.01 | 85 | 93 | |
| Fed-MedLoRABackbone=DeepSeek-R1-Distill-8B2026.01 | 84.5 | 91.5 | |
| Single site (Avg, baseline)Backbone=LLaMA3-8B2026.01 | 80.1 | 92.4 | |
| Single site (Avg, baseline)Backbone=DeepSeek-R1-Distill-8B2026.01 | 76.9 | 90.6 | |
| Centralized (Upper bound)Backbone=Bio_ClinicalBERT2026.01 | 76.8 | 83.8 | |
| Single site (Avg, baseline)Backbone=Bio_ClinicalBERT2026.01 | 74.7 | 80.8 | |
| Zero-shotBackbone=GPT-4o2026.01 | 60.2 | 76.6 | |
| Zero-shot (Baseline)Backbone=LLaMA3-8B2026.01 | 33 | 49.6 | |
| Zero-shot (Baseline)Backbone=DeepSeek-R1-Distill-8B2026.01 | 32.1 | 55.2 |