Relation Extraction on I2B2 Two-site experiment (test)
74.6Strict Micro F1Centralized (Upper bound)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Centralized (Upper bound)Backbone=LLaMA3-8B2026.01 | 74.6 | 75.5 | |
| Fed-MedLoRA+Backbone=LLaMA3-8B2026.01 | 74 | 73.6 | |
| Fed-MedLoRA+Backbone=DeepSeek-R1-Distill-8B2026.01 | 72.4 | 80.1 | |
| Centralized (Upper bound)Backbone=DeepSeek-R1-Distill-8B2026.01 | 72 | 81.6 | |
| Fed-MedLoRABackbone=DeepSeek-R1-Distill-8B2026.01 | 71.1 | 78.2 | |
| Fed-MedLoRABackbone=LLaMA3-8B2026.01 | 69.3 | 72.6 | |
| Single site (Avg, baseline)Backbone=LLaMA3-8B2026.01 | 66.7 | 71.5 | |
| Single site (Avg, baseline)Backbone=DeepSeek-R1-Distill-8B2026.01 | 66.5 | 68.8 | |
| Centralized (Upper bound)Backbone=Bio_ClinicalBERT2026.01 | 35.4 | 53.6 | |
| Single site (Avg, baseline)Backbone=Bio_ClinicalBERT2026.01 | 23.2 | 41.9 | |
| Zero-shot (Baseline)Backbone=DeepSeek-R1-Distill-8B2026.01 | 15.5 | 18.9 | |
| Zero-shotBackbone=GPT-4o2026.01 | 10.6 | 19.2 | |
| Zero-shot (Baseline)Backbone=LLaMA3-8B2026.01 | 9.6 | 11.4 |