Biomedical domain adaptation on Open Medical-LLM leaderboard macro-average
72.3Macro Average ScoreOriginal tokenization
Evaluation results
| Method | Links | |
|---|---|---|
| Original tokenizationBackbone=Llama3.1-8B-i2025.05 | 72.3 | |
| Token DistillationBackbone=Llama3.1-8B-i2025.05 | 71 | |
| Token Distillation + αNTPBackbone=Llama3.1-8B-i2025.05 | 70.9 | |
| Original tokenizationBackbone=Llama3-8B-i2025.05 | 70.6 | |
| Token Distillation + NTPBackbone=Llama3.1-8B-i2025.05 | 70.6 | |
| Original tokenizationBackbone=Llama3-8B2025.05 | 69.8 | |
| Original tokenizationBackbone=Llama3.1-8B2025.05 | 69.2 | |
| NTP (Lampinen et al., 2018)Backbone=Llama3.1-8B-i2025.05 | 68.9 | |
| Token Distillation + αNTPBackbone=Llama3-8B-i2025.05 | 67.8 | |
| Token DistillationBackbone=Llama3-8B-i2025.05 | 67.6 | |
| Token Distillation + αNTPBackbone=Llama3-8B2025.05 | 67.6 | |
| Token Distillation + αNTPBackbone=Llama3.1-8B2025.05 | 67.4 | |
| Token DistillationBackbone=Llama3-8B2025.05 | 67.3 | |
| Token DistillationBackbone=Llama3.1-8B2025.05 | 67.3 | |
| Token Distillation + NTPBackbone=Llama3-8B2025.05 | 67.2 | |
| Token Distillation + NTPBackbone=Llama3.1-8B2025.05 | 67.2 | |
| Mean (Gee et al., 2022)Backbone=Llama3.1-8B-i2025.05 | 66.9 | |
| NTP (tune all embeddings)Backbone=Llama3.1-8B-i2025.05 | 66.8 | |
| Token Distillation + NTPBackbone=Llama3-8B-i2025.05 | 66.7 | |
| Original tokenizationBackbone=Avg2025.05 | 66.5 | |
| ZeTT (Minixhofer et al., 2024)Backbone=Llama3-8B-i2025.05 | 66.3 | |
| ZeTT (Minixhofer et al., 2024)Backbone=Llama3-8B2025.05 | 66.1 | |
| NTP (Lampinen et al., 2018)Backbone=Llama3.1-8B2025.05 | 66 | |
| NTP (Lampinen et al., 2018)Backbone=Llama3-8B2025.05 | 65.6 | |
| NTP (Lampinen et al., 2018)Backbone=Llama3-8B-i2025.05 | 65.3 | |
| Token Distillation + αNTPBackbone=Avg2025.05 | 64.7 | |
| Token DistillationBackbone=Avg2025.05 | 64.6 | |
| Original tokenizationBackbone=Mistral-7B2025.05 | 64.5 | |
| Original tokenizationBackbone=Llama3.2-3B-i2025.05 | 64.4 | |
| Token Distillation + NTPBackbone=Avg2025.05 | 64.3 | |
| NTP (tune all embeddings)Backbone=Llama3-8B2025.05 | 63.8 | |
| NTP (tune all embeddings)Backbone=Llama3-8B-i2025.05 | 63.8 | |
| Mean (Gee et al., 2022)Backbone=Llama3-8B2025.05 | 63.8 | |
| Mean (Gee et al., 2022)Backbone=Llama3.1-8B2025.05 | 63.7 | |
| Mean (Gee et al., 2022)Backbone=Llama3-8B-i2025.05 | 63.4 | |
| Token DistillationBackbone=Llama3.2-3B-i2025.05 | 63.1 | |
| NTP (Lampinen et al., 2018)Backbone=Avg2025.05 | 63 | |
| Token Distillation + NTPBackbone=Mistral-7B2025.05 | 63 | |
| NTP (tune all embeddings)Backbone=Llama3.1-8B2025.05 | 62.9 | |
| Token DistillationBackbone=Mistral-7B2025.05 | 62.8 | |
| Token Distillation + αNTPBackbone=Mistral-7B2025.05 | 62.8 | |
| ZeTT (Minixhofer et al., 2024)Backbone=Mistral-7B2025.05 | 62.7 | |
| Random (Hewitt, 2021)Backbone=Llama3.1-8B-i2025.05 | 62.6 | |
| Token Distillation + αNTPBackbone=Llama3.2-3B-i2025.05 | 62.5 | |
| Token Distillation + NTPBackbone=Llama3.2-3B-i2025.05 | 62.2 | |
| Original tokenizationBackbone=OLMo2-7B-i2025.05 | 61.3 | |
| NTP (Lampinen et al., 2018)Backbone=Llama3.2-3B-i2025.05 | 61.3 | |
| NTP (Lampinen et al., 2018)Backbone=Mistral-7B2025.05 | 61.2 | |
| Token DistillationBackbone=OLMo2-7B-i2025.05 | 61.2 | |
| Token Distillation + αNTPBackbone=OLMo2-7B-i2025.05 | 61.2 | |
| Mean (Gee et al., 2022)Backbone=Avg2025.05 | 60.8 | |
| Random (Hewitt, 2021)Backbone=Llama3-8B-i2025.05 | 60.6 | |
| Original tokenizationBackbone=Llama3.2-3B2025.05 | 60.1 | |
| NTP (tune all embeddings)Backbone=Avg2025.05 | 59.8 | |
| Token Distillation + NTPBackbone=OLMo2-7B-i2025.05 | 59.8 | |
| Random (Hewitt, 2021)Backbone=Llama3.1-8B2025.05 | 59.3 | |
| Random (Hewitt, 2021)Backbone=Llama3-8B2025.05 | 58.8 | |
| NTP (Lampinen et al., 2018)Backbone=OLMo2-7B-i2025.05 | 58.7 | |
| Mean (Gee et al., 2022)Backbone=Llama3.2-3B-i2025.05 | 58.4 | |
| Mean (Gee et al., 2022)Backbone=Mistral-7B2025.05 | 58.3 | |
| NTP (tune all embeddings)Backbone=OLMo2-7B-i2025.05 | 58 | |
| Mean (Gee et al., 2022)Backbone=OLMo2-7B-i2025.05 | 58 | |
| Token Distillation + αNTPBackbone=Llama3.2-3B2025.05 | 57.9 | |
| Token Distillation + NTPBackbone=Llama3.2-3B2025.05 | 57.6 | |
| Random (Hewitt, 2021)Backbone=Avg2025.05 | 57.5 | |
| Random (Hewitt, 2021)Backbone=Mistral-7B2025.05 | 57 | |
| NTP (Lampinen et al., 2018)Backbone=Llama3.2-3B2025.05 | 56.6 | |
| Token DistillationBackbone=Llama3.2-3B2025.05 | 56.2 | |
| Random (Hewitt, 2021)Backbone=Llama3.2-3B-i2025.05 | 55.8 | |
| NTP (tune all embeddings)Backbone=Mistral-7B2025.05 | 55.7 | |
| NTP (tune all embeddings)Backbone=Llama3.2-3B-i2025.05 | 55.5 | |
| Mean (Gee et al., 2022)Backbone=Llama3.2-3B2025.05 | 54.2 | |
| Random (Hewitt, 2021)Backbone=OLMo2-7B-i2025.05 | 53.9 | |
| Random (Hewitt, 2021)Backbone=Llama3.2-3B2025.05 | 51.6 | |
| NTP (tune all embeddings)Backbone=Llama3.2-3B2025.05 | 51.5 |