Language Modeling on Paloma-C4 (val)
58.48PerplexityPartially Reverse BDLM Mamba-H
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Partially Reverse BDLM Mamba-HLR=8×10−32026.07 | 58.48 | 1.28 | |
| BDLM attentionLR=4×10−32026.07 | 70.81 | 1.34 | |
| Full-sequence DiffuMamba-HLR=4×10−32026.07 | 80.42 | 1.38 | |
| Full-sequence attentionLR=8×10−32026.07 | 83.25 | 1.39 |