Language Modeling on ProofPile (4K)
3.26PerplexityLongAlpaca-16K
Evaluation results
| Method | Links | |
|---|---|---|
| LongAlpaca-16KBase Model=Mistral-7B, Supervised fine-tuning=true2026.03 | 3.26 | |
| SHAREDLLMBase Model=LLaMA-2, Supervised fine-tuning=true2026.03 | 3.36 | |
| StreamingLLMBase Model=LLaMA-2, Supervised fine-tuning=true2026.03 | 3.47 | |
| Activation BeaconBase Model=LLaMA-2, Supervised fine-tuning=true2026.03 | 3.47 | |
| SHAREDLLMBase Model=Mistral-7B, Supervised fine-tuning=true2026.03 | 3.58 | |
| LongAlpaca-16KBase Model=LLaMA-2, Supervised fine-tuning=true2026.03 | 3.82 | |
| Activation BeaconBase Model=Mistral-7B, Supervised fine-tuning=true2026.03 | 3.82 | |
| StreamingLLMBase Model=Mistral-7B, Supervised fine-tuning=true2026.03 | 4.08 |