Next Token Prediction on Amber 1.2T tokens
4.28BPDLLAMA2-7B
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| LLAMA2-7Bquantization=2-bit post-training QuIP2024.07 | 4.28 | 0.475 | 0.1256 | |
| LLAMA2-13Bquantization=2-bit post-training QuIP2024.07 | 4.51 | 0.4785 | 0.1444 | |
| LLaMA2-70Bquantization=2-bit post-training QuIP2024.07 | 6.39 | 0.5826 | 0.2504 | |
| Random Guess2024.07 | 14.97 | 0.9999 | 0.9968 |