Named Entity Recognition on UTP Two-site experiment (test)
90.9Strict Micro F1LLaMA3-8B Centralized
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LLaMA3-8B CentralizedBackbone=LLaMA3-8B, Strategy=Centralized (Upper bound)2026.01 | 90.9 | 91.3 | |
| DeepSeek-R1-Distill-8B CentralizedBackbone=DeepSeek-R1-Distill-8B, Strategy=Centralized (Upper bound)2026.01 | 90.7 | 90.6 | |
| Fed-MedLoRA+Backbone=LLaMA3-8B2026.01 | 89.8 | 93.6 | |
| Fed-MedLoRABackbone=LLaMA3-8B2026.01 | 89.7 | 90.6 | |
| Fed-MedLoRA+Backbone=DeepSeek-R1-Distill-8B2026.01 | 87.9 | 94 | |
| Centralized (Upper bound)Backbone=LLaMA3-8B2026.01 | 85.2 | 93.2 | |
| Fed-MedLoRABackbone=DeepSeek-R1-Distill-8B2026.01 | 84.8 | 92 | |
| Centralized (Upper bound)Backbone=DeepSeek-R1-Distill-8B2026.01 | 84.4 | 91.3 | |
| Fed-MedLoRA+Backbone=LLaMA3-8B2026.01 | 84 | 92.7 | |
| Fed-MedLoRABackbone=LLaMA3-8B2026.01 | 83.8 | 92.1 | |
| Fed-MedLoRA+Backbone=DeepSeek-R1-Distill-8B2026.01 | 82.5 | 91.2 | |
| Bio_ClinicalBERT CentralizedBackbone=Bio_ClinicalBERT, Strategy=Centralized (Upper bound)2026.01 | 82.3 | 88.4 | |
| Fed-MedLoRABackbone=DeepSeek-R1-Distill-8B2026.01 | 81.9 | 90.1 | |
| Single site (Avg, baseline)Backbone=LLaMA3-8B2026.01 | 80.6 | 88.4 | |
| DeepSeek-R1-Distill-8B Single siteBackbone=DeepSeek-R1-Distill-8B, Strategy=Single site (Avg, baseline)2026.01 | 80.6 | 86.1 | |
| LLaMA3-8B Single siteBackbone=LLaMA3-8B, Strategy=Single site (Avg, baseline)2026.01 | 79.8 | 87.8 | |
| Single site (Avg, baseline)Backbone=DeepSeek-R1-Distill-8B2026.01 | 78.8 | 89.2 | |
| Bio_ClinicalBERT Single siteBackbone=Bio_ClinicalBERT, Strategy=Single site (Avg, baseline)2026.01 | 72 | 83.7 | |
| Centralized (Upper bound)Backbone=Bio_ClinicalBERT2026.01 | 69.2 | 85.1 | |
| Single site (Avg, baseline)Backbone=Bio_ClinicalBERT2026.01 | 67.8 | 84.6 | |
| Zero-shotBackbone=GPT-4o2026.01 | 47.7 | 67.6 | |
| GPT-4o Zero-shotBackbone=GPT-4o, Strategy=Zero-shot (Baseline)2026.01 | 47.7 | 67.6 | |
| Zero-shot (Baseline)Backbone=LLaMA3-8B2026.01 | 25.6 | 51.3 | |
| LLaMA3-8B Zero-shotBackbone=LLaMA3-8B, Strategy=Zero-shot (Baseline)2026.01 | 25.6 | 51.3 | |
| Zero-shot (Baseline)Backbone=DeepSeek-R1-Distill-8B2026.01 | 25.1 | 43.3 | |
| DeepSeek-R1-Distill-8B Zero-shotBackbone=DeepSeek-R1-Distill-8B, Strategy=Zero-shot (Baseline)2026.01 | 25.1 | 43.3 |