Closed-ended Task Evaluation on L-Eval closed-ended tasks
41.86Coursera ScoreResonance YaRN
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Resonance YaRNBase Model=LLaMA2-Chat 13B, Scaling Factor (s)=4, Fine-Tuning=FT@4K, 400 epochs, Context Length=16K2024.02 | 41.86 | 35 | 42.57 | 65.8 | 5.56 | 48.44 | 39.87 | |
| YaRNBase Model=LLaMA2-Chat 13B, Scaling Factor (s)=4, Fine-Tuning=FT@4K, 400 epochs, Context Length=16K2024.02 | 41.72 | 34 | 41.09 | 66.91 | 2.22 | 48.44 | 39.06 | |
| NTK-AwareBase Model=LLaMA2-Chat 13B, Scaling Factor (s)=4, Fine-Tuning=no FT, Context Length=16K2024.02 | 40.26 | 21 | 38.12 | 65.43 | 1.11 | 46.88 | 35.47 | |
| Resonance YaRNBase Model=LLaMA2-Chat 13B, Scaling Factor (s)=4, Fine-Tuning=FT@16K, 100 epochs, Context Length=16K2024.02 | 38.66 | 39 | 43.56 | 65.06 | 1.11 | 62.5 | 41.65 | |
| YaRNBase Model=LLaMA2-Chat 13B, Scaling Factor (s)=4, Fine-Tuning=FT@16K, 100 epochs, Context Length=16K2024.02 | 38.08 | 39 | 43.07 | 65.43 | 0 | 63.28 | 41.48 | |
| NTK-AwareBase Model=LLaMA2-Chat 7B, Scaling Factor (s)=8, Fine-Tuning=no FT, Context Length=32K2024.02 | 36.77 | 3 | 26.73 | 34.2 | 1.11 | 50.78 | 25.43 | |
| Resonance YaRNBase Model=LLaMA2-Chat 7B, Scaling Factor (s)=8, Fine-Tuning=FT@32K, 50 epochs, Context Length=32K2024.02 | 36.48 | 22 | 34.16 | 55.76 | 0 | 57.03 | 34.24 | |
| Resonance YaRNBase Model=LLaMA2-Chat 7B, Scaling Factor (s)=8, Fine-Tuning=FT@4K, 400 epochs, Context Length=32K2024.02 | 36.34 | 27 | 40.59 | 56.51 | 3.33 | 61.72 | 37.58 | |
| YaRNBase Model=LLaMA2-Chat 7B, Scaling Factor (s)=8, Fine-Tuning=FT@32K, 50 epochs, Context Length=32K2024.02 | 36.05 | 19 | 33.17 | 50.56 | 4.44 | 56.25 | 33.24 | |
| YaRNBase Model=LLaMA2-Chat 7B, Scaling Factor (s)=8, Fine-Tuning=FT@4K, 400 epochs, Context Length=32K2024.02 | 35.03 | 24 | 37.62 | 57.62 | 4.44 | 60.94 | 36.61 | |
| Dynamic NTK-AwareBase Model=LLaMA2-Chat 7B, Fine-Tuning=no FT, Context Length=32K2024.02 | 31.98 | 32 | 34.65 | 59.11 | 1.11 | 36.72 | 32.59 | |
| Dynamic NTK-AwareBase Model=LLaMA2-Chat 13B, Fine-Tuning=no FT, Context Length=16K2024.02 | 29.22 | 39 | 40.59 | 63.94 | 1.11 | 39.84 | 35.62 |