Language Modeling on PG19 32K
7.96PerplexitySHAREDLLM
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SHAREDLLMBase Model=LLaMA-2, Supervised fine-tuning=true2026.03 | 7.96 | — | — | — | |
| Activation BeaconBase Model=LLaMA-2, Supervised fine-tuning=true2026.03 | 8.27 | — | — | — | |
| SHAREDLLMBase Model=Mistral-7B, Supervised fine-tuning=true2026.03 | 8.98 | — | — | — | |
| StreamingLLMBase Model=LLaMA-2, Supervised fine-tuning=true2026.03 | 9.24 | — | — | — | |
| Activation BeaconBase Model=Mistral-7B, Supervised fine-tuning=true2026.03 | 9.39 | — | — | — | |
| StreamingLLMBase Model=Mistral-7B, Supervised fine-tuning=true2026.03 | 9.52 | — | — | — | |
| DenseContext Length=32K, Number of chunks=202026.05 | 9.955 | — | — | — | |
| CertifiedContext Length=32K, Number of chunks=202026.05 | 9.956 | 0.001 | 0.002 | 1.0001 |