Language Modeling on Proofpile (test)
3.68Performance (8K Context)YaRN
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| YaRNBase Model=LLaMA3-8B-Instruct, Context Window=8K, Scale (S)=162026.01 | 3.68 | 3.08 | 2.75 | 2.49 | 2.38 | |
| NTKBase Model=LLaMA3-8B-Instruct, Context Window=8K, Scale (S)=162026.01 | 3.67 | 3.72 | — | — | — | |
| MrRoPE-UniBase Model=LLaMA3-8B-Instruct, Context Window=8K, Scale (S)=162026.01 | 3.66 | 3.06 | 2.74 | 2.47 | 2.41 | |
| MrRoPE-ProBase Model=LLaMA3-8B-Instruct, Context Window=8K, Scale (S)=162026.01 | 3.63 | 3.03 | 2.71 | 2.45 | 2.34 | |
| NTKBase Model=Qwen2.5-3B-Instruct, Context Window=32K, Scale (S)=42026.01 | 3.548 | 3.778 | 7.782 | — | — | |
| NTKBase Model=Qwen2.5-3B-Instruct, Context Window=32K, Scale (S)=42026.01 | 3.548 | 3.778 | 7.782 | — | — | |
| YaRNBase Model=Qwen2.5-3B-Instruct, Context Window=32K, Scale (S)=42026.01 | 3.449 | 2.894 | 2.589 | 2.333 | 2.222 | |
| YaRNBase Model=Qwen2.5-3B-Instruct, Context Window=32K, Scale (S)=42026.01 | 3.449 | 2.894 | 2.589 | 2.333 | 2.222 | |
| MrRoPE-UniBase Model=Qwen2.5-3B-Instruct, Context Window=32K, Scale (S)=42026.01 | 3.44 | 2.885 | 2.581 | 2.329 | 2.224 | |
| MrRoPE-UniBase Model=Qwen2.5-3B-Instruct, Context Window=32K, Scale (S)=42026.01 | 3.44 | 2.885 | 2.581 | 2.329 | 2.224 | |
| MrRoPE-ProBase Model=Qwen2.5-3B-Instruct, Context Window=32K, Scale (S)=42026.01 | 3.433 | 2.881 | 2.579 | 2.328 | 2.221 | |
| MrRoPE-ProBase Model=Qwen2.5-3B-Instruct, Context Window=32K, Scale (S)=42026.01 | 3.433 | 2.881 | 2.579 | 2.328 | 2.221 |