Language Modeling on Specialized Corpora (MIMIC-III, FinGPT, Asylex)
8.25Perplexity (MIMIC-III)Nirvana
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| NirvanaFine-tuning=3 epochs, Model Scale=1.3B2025.10 | 8.25 | 7.88 | 7.22 | 7.78 | |
| Gated DeltaNetFine-tuning=3 epochs, Model Scale=1.3B2025.10 | 9.02 | 9.72 | 8.89 | 9.21 | |
| Mamba2Fine-tuning=3 epochs, Model Scale=1.3B2025.10 | 9.13 | 9.97 | 9.07 | 9.39 | |
| Nirvana-noTriggerFine-tuning=3 epochs, Model Scale=1.3B2025.10 | 9.19 | 9.87 | 8.84 | 9.3 | |
| SambaFine-tuning=3 epochs, Model Scale=1.3B2025.10 | 9.27 | 9.5 | 8.74 | 9.17 | |
| Transformer++Fine-tuning=3 epochs, Model Scale=1.3B2025.10 | 9.28 | 10.7 | 8.82 | 9.6 |