Named Entity Recognition on I2B2 Three-site experiment (test)
85.2Strict Micro F1Centralized (Upper bound)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Centralized (Upper bound)Backbone=LLaMA3-8B2026.01 | 85.2 | 91.6 | |
| Fed-MedLoRA+Backbone=LLaMA3-8B2026.01 | 84.4 | 88.9 | |
| Fed-MedLoRABackbone=LLaMA3-8B2026.01 | 83.3 | 89.6 | |
| Centralized (Upper bound)Backbone=DeepSeek-R1-Distill-8B2026.01 | 82.2 | 90.7 | |
| Fed-MedLoRA+Backbone=DeepSeek-R1-Distill-8B2026.01 | 79.9 | 89.6 | |
| Fed-MedLoRABackbone=DeepSeek-R1-Distill-8B2026.01 | 79.1 | 86.8 | |
| Single site (Avg, baseline)Backbone=DeepSeek-R1-Distill-8B2026.01 | 78.9 | 86.2 | |
| Centralized (Upper bound)Backbone=Bio_ClinicalBERT2026.01 | 78.7 | 86.3 | |
| Single site (Avg, baseline)Backbone=LLaMA3-8B2026.01 | 75.6 | 87.3 | |
| Single site (Avg, baseline)Backbone=Bio_ClinicalBERT2026.01 | 75.2 | 81.6 | |
| Zero-shotBackbone=GPT-4o2026.01 | 60.2 | 76.6 | |
| Zero-shot (Baseline)Backbone=LLaMA3-8B2026.01 | 33 | 49.6 | |
| Zero-shot (Baseline)Backbone=DeepSeek-R1-Distill-8B2026.01 | 32.1 | 55.2 |