Language Modeling on Qwen3 (val)
2.493Validation LossDana-MK4
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Dana-MK4Size=2.47B, kappa=0.852026.02 | 2.493 | — | |
| ADanaSize=2.47B, kappa=0.852026.02 | 2.496 | — | |
| Dana-Star-MK4Size=2.47B, kappa=0.852026.02 | 2.496 | — | |
| Dana-MK4Size=2.47B, kappa=0.752026.02 | 2.501 | — | |
| Dana-Star-MK4Size=2.47B, kappa=0.752026.02 | 2.512 | — | |
| Dana-MK4Size=2.09B, kappa=0.852026.02 | 2.527 | — | |
| Dana-Star-MK4Size=2.09B, kappa=0.852026.02 | 2.53 | — | |
| ADanaSize=2.09B, kappa=0.852026.02 | 2.531 | — | |
| AdamWSize=2.47B2026.02 | 2.536 | — | |
| Dana-MK4Size=2.09B, kappa=0.752026.02 | 2.542 | — | |
| Dana-Star-MK4Size=2.09B, kappa=0.752026.02 | 2.543 | — | |
| AdemamixSize=2.09B2026.02 | 2.566 | — | |
| AdamWSize=2.09B2026.02 | 2.572 | — | |
| Dana-MK4Size=1.44B, kappa=0.852026.02 | 2.601 | — | |
| ADanaSize=1.44B, kappa=0.852026.02 | 2.608 | — | |
| Dana-Star-MK4Size=1.44B, kappa=0.852026.02 | 2.609 | — | |
| Dana-MK4Size=1.44B, kappa=0.752026.02 | 2.618 | — | |
| Dana-Star-MK4Size=1.44B, kappa=0.752026.02 | 2.62 | — | |
| AdemamixSize=1.44B2026.02 | 2.641 | — | |
| AdamWSize=1.44B2026.02 | 2.644 | — | |
| Dana-MK4Size=947M, kappa=0.852026.02 | 2.692 | — | |
| ADanaSize=947M, kappa=0.852026.02 | 2.696 | — | |
| Dana-Star-MK4Size=947M, kappa=0.852026.02 | 2.699 | — | |
| Dana-MK4Size=947M, kappa=0.752026.02 | 2.707 | — | |
| Dana-Star-MK4Size=947M, kappa=0.752026.02 | 2.709 | — | |
| AdemamixSize=947M2026.02 | 2.73 | — | |
| AdamWSize=947M2026.02 | 2.74 | — | |
| Dana-MK4Size=588M, kappa=0.852026.02 | 2.815 | — | |
| ADanaSize=588M, kappa=0.852026.02 | 2.816 | — | |
| Dana-Star-MK4Size=588M, kappa=0.852026.02 | 2.82 | — | |
| Dana-MK4Size=588M, kappa=0.752026.02 | 2.834 | — | |
| Dana-Star-MK4Size=588M, kappa=0.752026.02 | 2.839 | — | |
| AdemamixSize=588M2026.02 | 2.868 | — | |
| AdamWSize=588M2026.02 | 2.871 | — | |
| ADanaSize=338M, kappa=0.852026.02 | 2.965 | — | |
| Dana-MK4Size=338M, kappa=0.852026.02 | 2.965 | — | |
| Dana-MK4Size=338M, kappa=0.752026.02 | 2.971 | — | |
| Dana-Star-MK4Size=338M, kappa=0.852026.02 | 2.977 | — | |
| Dana-Star-MK4Size=338M, kappa=0.752026.02 | 2.98 | — | |
| AdemamixSize=338M2026.02 | 2.999 | — | |
| AdamWSize=338M2026.02 | 3.017 | — | |
| ADanaSize=176M, kappa=0.852026.02 | 3.182 | — | |
| Dana-MK4Size=176M, kappa=0.852026.02 | 3.183 | — | |
| Dana-MK4Size=176M, kappa=0.752026.02 | 3.185 | — | |
| Dana-Star-MK4Size=176M, kappa=0.752026.02 | 3.199 | — | |
| Dana-Star-MK4Size=176M, kappa=0.852026.02 | 3.202 | — | |
| AdemamixSize=176M2026.02 | 3.214 | — | |
| AdemamixSize=2.47B2026.02 | 3.226 | — | |
| AdamWSize=176M2026.02 | 3.228 | — | |
| CR-NetTraining steps=10K, Training tokens (B)=1.32025.09 | — | 25.99 | |
| CR-NetTraining steps=20K, Training tokens (B)=2.62025.09 | — | 21.73 | |
| CR-NetTraining steps=30K, Training tokens (B)=3.92025.09 | — | 19.32 | |
| CR-NetTraining steps=40K, Training tokens (B)=5.22025.09 | — | 18.12 | |
| Full-rankTraining steps=10K, Training tokens (B)=1.32025.09 | — | 24.31 | |
| Full-rankTraining steps=20K, Training tokens (B)=2.62025.09 | — | 20.53 | |
| Full-rankTraining steps=30K, Training tokens (B)=3.92025.09 | — | 18.84 | |
| Full-rankTraining steps=40K, Training tokens (B)=5.22025.09 | — | 17.85 |