Language Modeling on WikiText-103 and Dolma (held-out evaluation sets)
4.91NLLDiHAL, geometry-selected
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DiHAL, geometry-selectedModel=Llama-3.1-8B, Layer=32026.05 | 4.91 | 135.64 | 0.73 | |
| Validation-loss oracleModel=Qwen3-8B, Layer=12026.05 | 4.94 | 139.77 | 0.54 | |
| DiHAL, geometry-selectedModel=Qwen3-8B, Layer=22026.05 | 4.97 | 144.03 | 0.53 | |
| Validation-loss oracleModel=Llama-3.1-8B, Layer=12026.05 | 5.11 | 165.67 | 0.62 | |
| Worst layerModel=Llama-3.1-8B, Layer=312026.05 | 5.17 | 175.91 | 1.32 | |
| CoDARModel=Qwen3-8B (setting), Layer=N/A2026.05 | 5.18 | 177.87 | — | |
| Worst layerModel=Qwen3-8B, Layer=352026.05 | 5.23 | 186.79 | 1.46 |