Relation Extraction on UTP Two-site experiment (test)
0.924Strict Micro F1LLaMA3-8B Centralized
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LLaMA3-8B CentralizedBackbone=LLaMA3-8B, Strategy=Centralized (Upper bound)2026.01 | 0.924 | 0.932 | |
| Fed-MedLoRABackbone=LLaMA3-8B2026.01 | 0.916 | 0.901 | |
| Fed-MedLoRA+Backbone=LLaMA3-8B2026.01 | 0.913 | 0.91 | |
| DeepSeek-R1-Distill-8B CentralizedBackbone=DeepSeek-R1-Distill-8B, Strategy=Centralized (Upper bound)2026.01 | 0.913 | 0.932 | |
| Fed-MedLoRA+Backbone=DeepSeek-R1-Distill-8B2026.01 | 0.894 | 0.918 | |
| Fed-MedLoRABackbone=DeepSeek-R1-Distill-8B2026.01 | 0.89 | 0.914 | |
| Centralized (Upper bound)Backbone=LLaMA3-8B2026.01 | 0.803 | 0.887 | |
| Centralized (Upper bound)Backbone=DeepSeek-R1-Distill-8B2026.01 | 0.797 | 0.93 | |
| Fed-MedLoRA+Backbone=LLaMA3-8B2026.01 | 0.793 | 0.854 | |
| Fed-MedLoRABackbone=DeepSeek-R1-Distill-8B2026.01 | 0.789 | 0.901 | |
| Fed-MedLoRABackbone=LLaMA3-8B2026.01 | 0.778 | 0.815 | |
| LLaMA3-8B Single siteBackbone=LLaMA3-8B, Strategy=Single site (Avg, baseline)2026.01 | 0.778 | 0.812 | |
| DeepSeek-R1-Distill-8B Single siteBackbone=DeepSeek-R1-Distill-8B, Strategy=Single site (Avg, baseline)2026.01 | 0.756 | 0.807 | |
| Fed-MedLoRA+Backbone=DeepSeek-R1-Distill-8B2026.01 | 0.751 | 0.928 | |
| Single site (Avg, baseline)Backbone=LLaMA3-8B2026.01 | 0.73 | 0.822 | |
| Single site (Avg, baseline)Backbone=DeepSeek-R1-Distill-8B2026.01 | 0.687 | 0.719 | |
| Bio_ClinicalBERT CentralizedBackbone=Bio_ClinicalBERT, Strategy=Centralized (Upper bound)2026.01 | 0.413 | 0.571 | |
| Centralized (Upper bound)Backbone=Bio_ClinicalBERT2026.01 | 0.288 | 0.529 | |
| Single site (Avg, baseline)Backbone=Bio_ClinicalBERT2026.01 | 0.27 | 0.441 | |
| Bio_ClinicalBERT Single siteBackbone=Bio_ClinicalBERT, Strategy=Single site (Avg, baseline)2026.01 | 0.249 | 0.276 | |
| Zero-shot (Baseline)Backbone=DeepSeek-R1-Distill-8B2026.01 | 0.11 | 0.128 | |
| DeepSeek-R1-Distill-8B Zero-shotBackbone=DeepSeek-R1-Distill-8B, Strategy=Zero-shot (Baseline)2026.01 | 0.11 | 0.128 | |
| Zero-shot (Baseline)Backbone=LLaMA3-8B2026.01 | 0.091 | 0.116 | |
| Zero-shotBackbone=GPT-4o2026.01 | 0.091 | 0.213 | |
| LLaMA3-8B Zero-shotBackbone=LLaMA3-8B, Strategy=Zero-shot (Baseline)2026.01 | 0.091 | 0.112 | |
| GPT-4o Zero-shotBackbone=GPT-4o, Strategy=Zero-shot (Baseline)2026.01 | 0.091 | 0.213 |