Relation Extraction on MTSamples (test)
83.1Strict F1 ScoreCentralized fine-tuning
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Centralized fine-tuningBackbone=LLaMA3-8B2026.01 | 83.1 | 91.3 | |
| Centralized fine-tuningBackbone=DeepSeek-R1-Distill-8B2026.01 | 82.9 | 91.1 | |
| Fed-MedLoRA+Backbone=LLaMA3-8B2026.01 | 82.6 | 85.6 | |
| Fed-MedLoRABackbone=LLaMA3-8B2026.01 | 81.7 | 84.9 | |
| LLaMA3-8B CentralizedBackbone=LLaMA3-8B, Strategy=Centralized (Upper bound)2026.01 | 78.8 | 85.5 | |
| DeepSeek-R1-Distill-8B CentralizedBackbone=DeepSeek-R1-Distill-8B, Strategy=Centralized (Upper bound)2026.01 | 78.5 | 85.5 | |
| Fed-MedLoRA+Backbone=DeepSeek-R1-Distill-8B2026.01 | 77.3 | 90.7 | |
| Fed-MedLoRABackbone=DeepSeek-R1-Distill-8B2026.01 | 76.7 | 89.2 | |
| Fed-MedLoRA+Backbone=LLaMA3-8B2026.01 | 76.2 | 81.6 | |
| Fed-MedLoRABackbone=LLaMA3-8B2026.01 | 75.9 | 79.7 | |
| Fed-MedLoRABackbone=DeepSeek-R1-Distill-8B2026.01 | 73.9 | 82.1 | |
| Fed-MedLoRA+Backbone=DeepSeek-R1-Distill-8B2026.01 | 73.6 | 89.6 | |
| DeepSeek-R1-Distill-8B Single siteBackbone=DeepSeek-R1-Distill-8B, Strategy=Single site (Avg, baseline)2026.01 | 72.5 | 79.9 | |
| Single site fine-tuningBackbone=LLaMA3-8B2026.01 | 72.1 | 79.8 | |
| Single site fine-tuningBackbone=DeepSeek-R1-Distill-8B2026.01 | 71.9 | 79.3 | |
| LLaMA3-8B Single siteBackbone=LLaMA3-8B, Strategy=Single site (Avg, baseline)2026.01 | 70.3 | 78 | |
| Centralized fine-tuningBackbone=Bio_ClinicalBERT2026.01 | 58.4 | 65.7 | |
| Bio_ClinicalBERT CentralizedBackbone=Bio_ClinicalBERT, Strategy=Centralized (Upper bound)2026.01 | 51.5 | 62.1 | |
| Single site fine-tuningBackbone=Bio_ClinicalBERT2026.01 | 47.4 | 54.9 | |
| Bio_ClinicalBERT Single siteBackbone=Bio_ClinicalBERT, Strategy=Single site (Avg, baseline)2026.01 | 43.3 | 50 | |
| Zero-shotBackbone=DeepSeek-R1-Distill-8B2026.01 | 10.6 | 13.9 | |
| DeepSeek-R1-Distill-8B Zero-shotBackbone=DeepSeek-R1-Distill-8B, Strategy=Zero-shot (Baseline)2026.01 | 10.6 | 13.9 | |
| Zero-shotBackbone=GPT-4o2026.01 | 9.6 | 17.8 | |
| GPT-4o Zero-shotBackbone=GPT-4o, Strategy=Zero-shot (Baseline)2026.01 | 9.6 | 17.8 | |
| Zero-shotBackbone=LLaMA3-8B2026.01 | 5.6 | 7.4 | |
| LLaMA3-8B Zero-shotBackbone=LLaMA3-8B, Strategy=Zero-shot (Baseline)2026.01 | 5.6 | 7.4 |