Language Modeling on Wikipedia 20k sentences
9.06Perplexity (Wikipedia 20k)Original
Evaluation Results
| Method | Links | |
|---|---|---|
| OriginalModel=Llama3-8B, Sequential=false2025.10 | 9.06 | |
| Linear-AcTModel=Llama3-8B, Sequential=false2025.10 | 9.35 | |
| Mean-AcTModel=Llama3-8B, Sequential=false2025.10 | 9.36 | |
| CAAModel=Llama3-8B, Sequential=false2025.10 | 9.4 | |
| ITI-CModel=Llama3-8B, Sequential=false2025.10 | 9.48 | |
| Linear-AcTModel=Llama3-8B, Sequential=true2025.10 | 9.48 | |
| AURAModel=Llama3-8B, Sequential=false2025.10 | 9.52 | |
| PID-AcTModel=Llama3-8B, Sequential=true2025.10 | 9.56 | |
| ActADDModel=Llama3-8B, Sequential=false2025.10 | 9.71 | |
| Mean-AcTModel=Llama3-8B, Sequential=true2025.10 | 9.83 | |
| OriginalModel=Gemma2-2B, Sequential=false2025.10 | 13.98 | |
| AURAModel=Gemma2-2B, Sequential=false2025.10 | 14.18 | |
| Mean-AcTModel=Gemma2-2B, Sequential=false2025.10 | 14.53 | |
| CAAModel=Gemma2-2B, Sequential=false2025.10 | 14.6 | |
| ActADDModel=Gemma2-2B, Sequential=false2025.10 | 14.69 | |
| Linear-AcTModel=Gemma2-2B, Sequential=false2025.10 | 14.75 | |
| ITI-CModel=Gemma2-2B, Sequential=false2025.10 | 14.9 | |
| Mean-AcTModel=Gemma2-2B, Sequential=true2025.10 | 14.92 | |
| Linear-AcTModel=Gemma2-2B, Sequential=true2025.10 | 14.98 | |
| PID-AcTModel=Gemma2-2B, Sequential=true2025.10 | 15.22 |