Language Modeling on PubMedQA
8.3PPL Change (%)GAIN-FFN
Evaluation results
| Method | Links | |
|---|---|---|
| GAIN-FFNLR=3×10−3, Params=230K2026.04 | 8.3 | |
| LoRALR=10−2, Params=92K2026.04 | 7.7 | |
| LoRALR=10−3, Params=92K2026.04 | 7.4 | |
| GAIN-FFNLR=10−3, Params=230K2026.04 | 5.1 | |
| GAINLR=3×10−3, Params=46K2026.04 | 4.2 | |
| LoRALR=3×10−4, Params=92K2026.04 | 3.2 | |
| GAINLR=10−3, Params=46K2026.04 | 2 | |
| GAINLR=3×10−4, Params=46K2026.04 | 0.7 | |
| LoRALR=10−4, Params=92K2026.04 | 0.5 |