Language Modeling on FineWeb-edu 12 text passages (held-out)
1.683Average LossFull Qwen 3B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Full Qwen 3BNumber of layers=362026.04 | 1.683 | — | — | |
| 17-layer baselineNumber of layers=172026.04 | 5.69 | — | 0 | |
| OUROBOROS V22026.04 | 5.961 | 3 | 0 |