Text Reconstruction on OpenWebText (val)
2.15Perplexity (PPL)LaDiff
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LaDiffCompression=1×, Regularization strategy=MildAug, N_cont=200, N_disc=10242026.05 | 2.15 | 74.5 | |
| LaDiffCompression=2×, Regularization strategy=MildAug, N_cont=200, N_disc=10242026.05 | 2.25 | 75.2 | |
| LaDiffCompression=4×, Regularization strategy=MildAug, N_cont=200, N_disc=10242026.05 | 2.58 | 68.3 | |
| LaDiffCompression=1×, Regularization strategy=Standard, N_cont=200, N_disc=10242026.05 | 2.87 | 69.7 | |
| LaDiffCompression=2×, Regularization strategy=Standard, N_cont=200, N_disc=10242026.05 | 3.25 | 69.4 | |
| LaDiffCompression=8×, Regularization strategy=MildAug, N_cont=200, N_disc=10242026.05 | 3.28 | 61 | |
| LaDiffCompression=32×, Regularization strategy=MildAug, N_cont=200, N_disc=10242026.05 | 6.83 | 53.3 | |
| LaDiffCompression=16×, Regularization strategy=MildAug, N_cont=200, N_disc=10242026.05 | 6.83 | 53.3 | |
| MDLMN_cont=200, N_disc=10242026.05 | 30 | 45.2 |