Relation Extraction on MIMIC-III (test)
86.7Strict F1Centralized fine-tuning
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Centralized fine-tuningBackbone=LLaMA3-8B2026.01 | 86.7 | 88.9 | |
| Centralized fine-tuningBackbone=DeepSeek-R1-Distill-8B2026.01 | 86.3 | 84.2 | |
| Fed-MedLoRA+Backbone=LLaMA3-8B2026.01 | 86 | 86.8 | |
| LLaMA3-8B CentralizedBackbone=LLaMA3-8B, Strategy=Centralized (Upper bound)2026.01 | 85.1 | 88.5 | |
| Fed-MedLoRABackbone=LLaMA3-8B2026.01 | 85 | 86 | |
| DeepSeek-R1-Distill-8B CentralizedBackbone=DeepSeek-R1-Distill-8B, Strategy=Centralized (Upper bound)2026.01 | 84.3 | 88.5 | |
| Fed-MedLoRA+Backbone=DeepSeek-R1-Distill-8B2026.01 | 83.1 | 83.1 | |
| Fed-MedLoRABackbone=DeepSeek-R1-Distill-8B2026.01 | 79.9 | 82.6 | |
| Fed-MedLoRA+Backbone=LLaMA3-8B2026.01 | 77.1 | 81.4 | |
| Fed-MedLoRA+Backbone=DeepSeek-R1-Distill-8B2026.01 | 75.2 | 89 | |
| Fed-MedLoRABackbone=LLaMA3-8B2026.01 | 74.4 | 80.1 | |
| Fed-MedLoRABackbone=DeepSeek-R1-Distill-8B2026.01 | 73.7 | 81.1 | |
| LLaMA3-8B Single siteBackbone=LLaMA3-8B, Strategy=Single site (Avg, baseline)2026.01 | 65.6 | 69.9 | |
| Single site fine-tuningBackbone=LLaMA3-8B2026.01 | 64.8 | 69.8 | |
| DeepSeek-R1-Distill-8B Single siteBackbone=DeepSeek-R1-Distill-8B, Strategy=Single site (Avg, baseline)2026.01 | 60.5 | 73.2 | |
| Bio_ClinicalBERT CentralizedBackbone=Bio_ClinicalBERT, Strategy=Centralized (Upper bound)2026.01 | 59.8 | 74.2 | |
| Centralized fine-tuningBackbone=Bio_ClinicalBERT2026.01 | 59 | 73.5 | |
| Single site fine-tuningBackbone=DeepSeek-R1-Distill-8B2026.01 | 58.9 | 64.4 | |
| Single site fine-tuningBackbone=Bio_ClinicalBERT2026.01 | 43.4 | 59.5 | |
| Bio_ClinicalBERT Single siteBackbone=Bio_ClinicalBERT, Strategy=Single site (Avg, baseline)2026.01 | 34.3 | 47 | |
| Zero-shotBackbone=LLaMA3-8B2026.01 | 20.3 | 23.9 | |
| LLaMA3-8B Zero-shotBackbone=LLaMA3-8B, Strategy=Zero-shot (Baseline)2026.01 | 20.3 | 23.9 | |
| Zero-shotBackbone=DeepSeek-R1-Distill-8B2026.01 | 16.3 | 19 | |
| DeepSeek-R1-Distill-8B Zero-shotBackbone=DeepSeek-R1-Distill-8B, Strategy=Zero-shot (Baseline)2026.01 | 16.3 | 19 | |
| Zero-shotBackbone=GPT-4o2026.01 | 12.4 | 28.9 | |
| GPT-4o Zero-shotBackbone=GPT-4o, Strategy=Zero-shot (Baseline)2026.01 | 12.4 | 28.9 |