Dialogue Generation on Vicuna
15.05Rouge-LXPERT-OLMoE
Evaluation Results
| Method | Links | |
|---|---|---|
| XPERT-OLMoE#Params=391M, #Size=dim=2048 8 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 15.05 | |
| XPERT-OLMoE#Params=480M, #Size=dim=2048 12 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 14.83 | |
| XPERT-OLMoE#Params=570M, #Size=dim=2048 16 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 14.57 | |
| XPERT-OLMoE#Params=270M, #Size=dim=1024 16 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 14.46 | |
| Distillation#Params=480M, #Size=dim=2048 12 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 14.39 | |
| XPERT-DeepSeek#Params=570M, #Size=dim=2048 16 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 14.37 | |
| Scratch#Params=391M, #Size=dim=2048 8 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 14.25 | |
| Scratch#Params=480M, #Size=dim=2048 12 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 14.2 | |
| Distillation#Params=570M, #Size=dim=2048 16 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 14.09 | |
| XPERT-DeepSeek#Params=391M, #Size=dim=2048 8 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 13.99 | |
| XPERT-DeepSeek#Params=480M, #Size=dim=2048 12 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 13.92 | |
| Scratch#Params=270M, #Size=dim=1024 16 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 13.77 | |
| Distillation#Params=270M, #Size=dim=1024 16 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 13.52 | |
| Scratch#Params=570M, #Size=dim=2048 16 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 13.42 | |
| XPERT-DeepSeek#Params=270M, #Size=dim=1024 16 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 13.25 | |
| Distillation#Params=391M, #Size=dim=2048 8 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 13.01 |