Named Entity Recognition on MTSamples (test)
87Strict F1Centralized fine-tuning
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Centralized fine-tuningBackbone=LLaMA3-8B2026.01 | 87 | 95.8 | |
| LLaMA3-8B CentralizedBackbone=LLaMA3-8B, Strategy=Centralized (Upper bound)2026.01 | 86.7 | 92.6 | |
| Fed-MedLoRA+Backbone=LLaMA3-8B2026.01 | 86.6 | 93.7 | |
| Centralized fine-tuningBackbone=DeepSeek-R1-Distill-8B2026.01 | 86.2 | 96.3 | |
| Fed-MedLoRA+Backbone=LLaMA3-8B2026.01 | 86.2 | 91.4 | |
| Fed-MedLoRABackbone=LLaMA3-8B2026.01 | 86.1 | 90.1 | |
| Fed-MedLoRA+Backbone=DeepSeek-R1-Distill-8B2026.01 | 86 | 95.1 | |
| Fed-MedLoRABackbone=LLaMA3-8B2026.01 | 85.8 | 92.9 | |
| Fed-MedLoRABackbone=DeepSeek-R1-Distill-8B2026.01 | 85.8 | 93.6 | |
| DeepSeek-R1-Distill-8B CentralizedBackbone=DeepSeek-R1-Distill-8B, Strategy=Centralized (Upper bound)2026.01 | 84.6 | 92.6 | |
| Single site fine-tuningBackbone=LLaMA3-8B2026.01 | 84.3 | 92.6 | |
| LLaMA3-8B Single siteBackbone=LLaMA3-8B, Strategy=Single site (Avg, baseline)2026.01 | 84.1 | 86.7 | |
| Bio_ClinicalBERT CentralizedBackbone=Bio_ClinicalBERT, Strategy=Centralized (Upper bound)2026.01 | 83.8 | 91.2 | |
| Single site fine-tuningBackbone=DeepSeek-R1-Distill-8B2026.01 | 83.6 | 92.1 | |
| Centralized fine-tuningBackbone=Bio_ClinicalBERT2026.01 | 82.7 | 90.5 | |
| DeepSeek-R1-Distill-8B Single siteBackbone=DeepSeek-R1-Distill-8B, Strategy=Single site (Avg, baseline)2026.01 | 82.5 | 85.5 | |
| Fed-MedLoRA+Backbone=DeepSeek-R1-Distill-8B2026.01 | 81.8 | 92.5 | |
| Fed-MedLoRABackbone=DeepSeek-R1-Distill-8B2026.01 | 80.3 | 91 | |
| Single site fine-tuningBackbone=Bio_ClinicalBERT2026.01 | 79.2 | 88.8 | |
| Bio_ClinicalBERT Single siteBackbone=Bio_ClinicalBERT, Strategy=Single site (Avg, baseline)2026.01 | 79 | 86.9 | |
| Zero-shotBackbone=GPT-4o2026.01 | 60.2 | 83.4 | |
| GPT-4o Zero-shotBackbone=GPT-4o, Strategy=Zero-shot (Baseline)2026.01 | 60.2 | 83.4 | |
| Zero-shotBackbone=LLaMA3-8B2026.01 | 43.7 | 64.4 | |
| LLaMA3-8B Zero-shotBackbone=LLaMA3-8B, Strategy=Zero-shot (Baseline)2026.01 | 43.7 | 64.4 | |
| Zero-shotBackbone=DeepSeek-R1-Distill-8B2026.01 | 30.3 | 49.9 | |
| DeepSeek-R1-Distill-8B Zero-shotBackbone=DeepSeek-R1-Distill-8B, Strategy=Zero-shot (Baseline)2026.01 | 30.3 | 49.9 |