Domain Adaptation on PubMedQA
8.3PPL Delta (%)LoRA
Evaluation Results
| Method | Links | |
|---|---|---|
| LoRAModel=Llama-2-70B, Params=1.3M, Learning Rate=10−32026.04 | 8.3 | |
| LoRAModel=Mistral-7B, Params=262K, Learning Rate=10−32026.04 | 5.4 | |
| LoRAModel=Llama-2-13B, Params=410K, Learning Rate=10−32026.04 | 1.4 | |
| GAINModel=Llama-2-70B, Params=655K2026.04 | -1.3 | |
| GAINModel=Llama-2-13B, Params=205K2026.04 | -2.4 | |
| GAINModel=Mistral-7B, Params=131K2026.04 | -3.1 | |
| LoRAModel=Mistral-7B, Params=262K, Learning Rate=3×10−42026.04 | -3.7 | |
| LoRAModel=Llama-2-13B, Params=410K, Learning Rate=3×10−42026.04 | -4 | |
| LoRAModel=Llama-2-70B, Params=1.3M, Learning Rate=3×10−42026.04 | -4.1 |