Question Answering on PubMedQA (Accuracy)
83.67AccuracyTL++ secure
Evaluation results
| Method | Links | |
|---|---|---|
| TL++ secureAdaptation Method=Full fine-tuning2026.06 | 83.67 | |
| TL++ secureAdaptation Method=LoRA2026.06 | 82.6 | |
| CentralizedAdaptation Method=Full fine-tuning2026.06 | 81.33 | |
| TL++ baseAdaptation Method=LoRA2026.06 | 81.2 | |
| CentralizedAdaptation Method=LoRA2026.06 | 81.07 | |
| TL++ baseAdaptation Method=Full fine-tuning2026.06 | 80.87 | |
| Pre-trainedBackbone=Llama-3.1-8B2026.06 | 76.6 | |
| Pre-trainedBackbone=Llama-3.2-3B2026.06 | 75 | |
| Pre-trainedBackbone=Qwen-3-4B2026.06 | 74.6 | |
| Pre-trainedBackbone=Gemma-2-2B2026.06 | 73.2 | |
| METISBackbone=Llama-3.1-8B2026.06 | 72.8 | |
| TIESBackbone=Qwen-3-4B2026.06 | 72.8 | |
| METISBackbone=Qwen-3-4B2026.06 | 72.8 | |
| TIESBackbone=Gemma-2-2B2026.06 | 72 | |
| METISBackbone=Gemma-2-2B2026.06 | 71.8 | |
| DAREBackbone=Qwen-3-4B2026.06 | 70.8 | |
| Task ArithmeticBackbone=Qwen-3-4B2026.06 | 68.8 | |
| ConsensusTABackbone=Qwen-3-4B2026.06 | 68.8 | |
| SplitFedAdaptation Method=Full fine-tuning2026.06 | 67.44 | |
| Standard SLAdaptation Method=LoRA2026.06 | 67.11 | |
| FedAvgAdaptation Method=Full fine-tuning2026.06 | 65.67 | |
| DAREBackbone=Llama-3.1-8B2026.06 | 65.4 | |
| FedProxAdaptation Method=Full fine-tuning2026.06 | 65.33 | |
| ConsensusTABackbone=Gemma-2-2B2026.06 | 64 | |
| DAREBackbone=Gemma-2-2B2026.06 | 62.2 | |
| SplitFedAdaptation Method=LoRA2026.06 | 61.44 | |
| METISBackbone=Llama-3.2-3B2026.06 | 59.6 | |
| FedProxAdaptation Method=LoRA2026.06 | 59.44 | |
| TIESBackbone=Llama-3.1-8B2026.06 | 57.4 | |
| FedAvgAdaptation Method=LoRA2026.06 | 55.89 | |
| Task ArithmeticBackbone=Gemma-2-2B2026.06 | 55.8 | |
| DAREBackbone=Llama-3.2-3B2026.06 | 55.6 | |
| ConsensusTABackbone=Llama-3.2-3B2026.06 | 55.6 | |
| Standard SLAdaptation Method=Full fine-tuning2026.06 | 55.44 | |
| Task ArithmeticBackbone=Llama-3.1-8B2026.06 | 55.4 | |
| ConsensusTABackbone=Llama-3.1-8B2026.06 | 55.4 | |
| Task ArithmeticBackbone=Llama-3.2-3B2026.06 | 55.2 | |
| TIESBackbone=Llama-3.2-3B2026.06 | 55.2 | |
| SCAFFOLDAdaptation Method=LoRA2026.06 | 53.67 | |
| SCAFFOLDAdaptation Method=Full fine-tuning2026.06 | 51.67 |