List your company
ResearchFormerly WizWand

Language Modeling on Medical (Med)

0.1PPL Change (%) vs BaselineLoRA
-1.1967.55216.325.048Apr 6, 2026
Updated 7mo ago

Evaluation results

MethodLinks
LoRABackbone=Mistral-7B, Learning Rate=10⁻³, Tokens per domain=200K, Training Stage=+Med2026.040.1
GAINBackbone=Mistral-7B, Learning Rate=10⁻³, Tokens per domain=200K, Training Stage=+HS2026.040.8
GAINBackbone=Mistral-7B, Learning Rate=10⁻³, Tokens per domain=200K, Training Stage=+SST2026.040.9
GAINBackbone=Mistral-7B, Learning Rate=10⁻³, Tokens per domain=200K, Training Stage=+LB2026.041
GAINBackbone=Mistral-7B, Learning Rate=10⁻³, Tokens per domain=200K, Training Stage=+Fin2026.041.1
GAINBackbone=Mistral-7B, Learning Rate=10⁻³, Tokens per domain=200K, Training Stage=+PG2026.041.1
GAINBackbone=Mistral-7B, Learning Rate=10⁻³, Tokens per domain=200K, Training Stage=+MdQ2026.041.1
GAINBackbone=Mistral-7B, Learning Rate=10⁻³, Tokens per domain=200K, Training Stage=+Med2026.041.2
GAINBackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+SST2026.042.1
GAINBackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+LB2026.042.2
GAINBackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+HS2026.042.2
GAINBackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+MdQ2026.042.4
GAINBackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+PG2026.042.8
GAINBackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+Med2026.042.9
GAINBackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+Fin2026.042.9
GAINBackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+WT2026.043
LoRABackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+PG2026.044.7
LoRABackbone=Mistral-7B, Learning Rate=10⁻³, Tokens per domain=200K, Training Stage=+Fin2026.046.3
LoRABackbone=Mistral-7B, Learning Rate=10⁻³, Tokens per domain=200K, Training Stage=+PG2026.046.7
LoRABackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+Fin2026.047.2
LoRABackbone=Mistral-7B, Learning Rate=10⁻³, Tokens per domain=200K, Training Stage=+MdQ2026.049.7
LoRABackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+Med2026.049.8
LoRABackbone=Mistral-7B, Learning Rate=10⁻³, Tokens per domain=200K, Training Stage=+LB2026.0411
LoRABackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+LB2026.0414.5
LoRABackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+MdQ2026.0415
LoRABackbone=Mistral-7B, Learning Rate=10⁻³, Tokens per domain=200K, Training Stage=+SST2026.0420
LoRABackbone=Mistral-7B, Learning Rate=10⁻³, Tokens per domain=200K, Training Stage=+HS2026.0421.8
LoRABackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+HS2026.0422.6
LoRABackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+SST2026.0430.7
LoRABackbone=GPT-2 Large (774M), Learning rate=10^-3, Training sequence stage=+WT2026.0432.5