Language Modeling on WikiText-2 context length 8192 (test)
6.5PerplexityLlama3.1-8B-Instruct Baseline
Evaluation results
| Method | Links | |
|---|---|---|
| Llama3.1-8B-Instruct BaselineMP=false, bits=16, Backbone=Llama3.1-8B-Instruct2026.02 | 6.5 | |
| ScaleBITS + RTNMP=true, bits=3.1, Backbone=Llama3.1-8B-Instruct2026.02 | 7.32 | |
| RTN-g128MP=false, bits=3.1, Backbone=Llama3.1-8B-Instruct2026.02 | 11.13 | |
| ScaleBITS + RTNMP=true, bits=2.1, Backbone=Llama3.1-8B-Instruct2026.02 | 12.75 | |
| RTN-g128MP=false, bits=2.1, Backbone=Llama3.1-8B-Instruct2026.02 | 1,000,000 |