Dialogue Generation on S-NI
19.82Rouge-LXPERT-OLMoE
Evaluation Results
| Method | Links | |
|---|---|---|
| XPERT-OLMoE#Params=570M, #Size=dim=2048 16 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 19.82 | |
| XPERT-OLMoE#Params=480M, #Size=dim=2048 12 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 19.43 | |
| XPERT-DeepSeek#Params=480M, #Size=dim=2048 12 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 18.01 | |
| XPERT-DeepSeek#Params=570M, #Size=dim=2048 16 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 17.99 | |
| XPERT-OLMoE#Params=391M, #Size=dim=2048 8 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 17.39 | |
| XPERT-DeepSeek#Params=391M, #Size=dim=2048 8 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 17.34 | |
| Distillation#Params=391M, #Size=dim=2048 8 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 16.63 | |
| XPERT-OLMoE#Params=270M, #Size=dim=1024 16 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 16.58 | |
| Scratch#Params=480M, #Size=dim=2048 12 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 16.57 | |
| Distillation#Params=570M, #Size=dim=2048 16 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 16.06 | |
| Distillation#Params=480M, #Size=dim=2048 12 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 15.7 | |
| Scratch#Params=391M, #Size=dim=2048 8 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 15.69 | |
| Scratch#Params=570M, #Size=dim=2048 16 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 15.61 | |
| XPERT-DeepSeek#Params=270M, #Size=dim=1024 16 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 15.2 | |
| Distillation#Params=270M, #Size=dim=1024 16 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 15 | |
| Scratch#Params=270M, #Size=dim=1024 16 Layers, Pre-training Tokens=5B, Fine-tuning Dataset=Dolly2026.05 | 14.67 |