Language Modeling on GPT-2
0.44Delta PPLRandom
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| RandomModel=Large (36L), k=12026.04 | 0.44 | — | |
| SpectralModel=Large (36L), k=12026.04 | 0.5 | 1.5 | |
| WorstModel=Large (36L), k=12026.04 | 0.62 | — | |
| WorstModel=Medium (24L), k=12026.04 | 0.73 | — | |
| Last-NModel=Large (36L), k=12026.04 | 0.75 | — | |
| SpectralModel=Medium (24L), k=12026.04 | 0.84 | 2.4 | |
| SpectralModel=Large (36L), k=22026.04 | 1.04 | 1.9 | |
| RandomModel=Medium (24L), k=12026.04 | 1.06 | — | |
| RandomModel=Large (36L), k=22026.04 | 1.06 | — | |
| RandomModel=Large (36L), k=32026.04 | 1.51 | — | |
| SpectralModel=Large (36L), k=32026.04 | 1.68 | 2 | |
| Last-NModel=Large (36L), k=22026.04 | 1.92 | — | |
| Last-NModel=Medium (24L), k=12026.04 | 1.98 | — | |
| RandomModel=Large (36L), k=42026.04 | 2.18 | — | |
| RandomModel=Medium (24L), k=22026.04 | 2.36 | — | |
| WorstModel=Medium (24L), k=22026.04 | 2.66 | — | |
| SpectralModel=Medium (24L), k=22026.04 | 2.67 | 2.3 | |
| SpectralModel=Large (36L), k=42026.04 | 2.96 | 1.9 | |
| Last-NModel=Large (36L), k=32026.04 | 3.41 | — | |
| RandomModel=Medium (24L), k=32026.04 | 3.79 | — | |
| SpectralModel=Medium (24L), k=32026.04 | 4.89 | 2.8 | |
| Last-NModel=Large (36L), k=42026.04 | 5.55 | — | |
| RandomModel=Medium (24L), k=42026.04 | 5.62 | — | |
| Last-NModel=Medium (24L), k=22026.04 | 6.04 | — | |
| SpectralModel=Medium (24L), k=42026.04 | 9.65 | 2.6 | |
| Last-NModel=Medium (24L), k=32026.04 | 13.49 | — | |
| WorstModel=Medium (24L), k=32026.04 | 19.49 | — | |
| WorstModel=Large (36L), k=22026.04 | 21.2 | — | |
| Last-NModel=Medium (24L), k=42026.04 | 25.06 | — | |
| WorstModel=Large (36L), k=32026.04 | 26.8 | — | |
| WorstModel=Large (36L), k=42026.04 | 33.36 | — | |
| WorstModel=Medium (24L), k=42026.04 | 229.15 | — |