Relation Extraction on I2B2 Three-site experiment (test)
0.745Strict Micro F1Centralized (Upper bound)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Centralized (Upper bound)Backbone=LLaMA3-8B2026.01 | 0.745 | 0.798 | |
| Centralized (Upper bound)Backbone=DeepSeek-R1-Distill-8B2026.01 | 0.73 | 0.877 | |
| Fed-MedLoRA+Backbone=LLaMA3-8B2026.01 | 0.713 | 0.761 | |
| Fed-MedLoRABackbone=LLaMA3-8B2026.01 | 0.711 | 0.748 | |
| Fed-MedLoRABackbone=DeepSeek-R1-Distill-8B2026.01 | 0.692 | 0.764 | |
| Fed-MedLoRA+Backbone=DeepSeek-R1-Distill-8B2026.01 | 0.685 | 0.761 | |
| Single site (Avg, baseline)Backbone=LLaMA3-8B2026.01 | 0.659 | 0.708 | |
| Single site (Avg, baseline)Backbone=DeepSeek-R1-Distill-8B2026.01 | 0.649 | 0.729 | |
| Centralized (Upper bound)Backbone=Bio_ClinicalBERT2026.01 | 0.352 | 0.579 | |
| Single site (Avg, baseline)Backbone=Bio_ClinicalBERT2026.01 | 0.219 | 0.495 | |
| Zero-shot (Baseline)Backbone=DeepSeek-R1-Distill-8B2026.01 | 0.155 | 0.189 | |
| Zero-shotBackbone=GPT-4o2026.01 | 0.106 | 0.192 | |
| Zero-shot (Baseline)Backbone=LLaMA3-8B2026.01 | 0.096 | 0.114 |