Language Modeling on Hospitality (HS)
0.2PPL Change vs BaselineGAIN
Evaluation results
| Method | Links | |
|---|---|---|
| GAINBackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+Med2026.04 | 0.2 | |
| GAINBackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+PG2026.04 | 0.2 | |
| GAINBackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+WT2026.04 | 0.3 | |
| GAINBackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+MdQ2026.04 | 0.3 | |
| GAINBackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+Fin2026.04 | 0.4 | |
| GAINBackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+LB2026.04 | 0.5 | |
| GAINBackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+SST2026.04 | 0.5 | |
| LoRABackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+Med2026.04 | 0.6 | |
| LoRABackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+Fin2026.04 | 3 | |
| LoRABackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+PG2026.04 | 3.4 | |
| LoRABackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+MdQ2026.04 | 6.9 | |
| GAINBackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+HS2026.04 | 8.5 | |
| LoRABackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+WT2026.04 | 8.7 | |
| GAINBackbone=Mistral-7B, Learning Rate=10⁻³, Tokens per domain=200K, Training Stage=+HS2026.04 | 9.3 | |
| LoRABackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+LB2026.04 | 10 | |
| LoRABackbone=Mistral-7B, Learning Rate=10⁻³, Tokens per domain=200K, Training Stage=+HS2026.04 | 19 | |
| LoRABackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+HS2026.04 | 20.8 | |
| LoRABackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+SST2026.04 | 24.5 |