Long-context language understanding on L-Eval (test)

58.28CourseraNTK-32k

Evaluation results

Method
NTK-32k2025.02	58.28	83	63.86	81.04	59.38	1.11	57.78
ChunkLlama2025.02	56.24	54	63.86	83.27	70.31	5.56	55.54
YaRN-16k2025.02	55.96	75	63.37	79.93	62.5	5.56	57.83
SelfExtend-16k2025.02	55.38	63	62.87	82.16	64.06	5.56	55.51
YaRN-32k2025.02	55.23	76	62.38	79.18	67.19	7.78	57.96
GALI2025.02	54.65	59.09	61.88	83.33	65.63	6.67	55.42
ChunkLlama-32k2025.02	54.36	55	60.89	81.78	64.06	5.56	53.61
Dyn-NTK-32k2025.02	54.36	55	61.88	82.16	64.06	6.67	54.02
GALI2025.02	54.17	59.09	62.38	82.68	66.41	7.78	55.29
SelfExtend-32k2025.02	53.92	77	63.37	79.93	65.63	3.33	57.2
ChunkLlama-16k2025.02	53.34	54	60.89	81.78	61.72	5.56	53.53
Original2025.02	53.05	58	61.88	82.16	60.16	4.44	53.31
NTK-16k2025.02	52.03	77	65.35	81.04	42.97	0	56.58
Dyn-NTK-16k2025.02	52.03	55	61.88	82.16	52.34	2.22	52.89
Greedy Attention Logit Interpolation2025.02	36.49	18	42.08	57.62	42.97	7.78	34.15
SelfExtend-16k2025.02	35.76	25	41.09	55.39	57.81	1.11	36.02
GALI2025.02	35.32	29	39.11	54.65	51.43	4.44	35.66
SelfExtend-16k2025.02	32.99	29	40.59	57.62	57.81	2.22	36.71
NTK-16k2025.02	32.71	19	33.16	52.78	64.84	0	33.75
ChunkLlama2025.02	32.12	31	35.14	57.62	61.72	2.22	36.64
Original2025.02	29.8	29	37.62	58.36	60.16	1.11	36.01
Original2025.02	29.21	19	37.62	51.67	60.15	1.11	33.12
ChunkLlama-16k2025.02	28.92	31	43.07	58.36	60.94	2.22	37.42
Dyn-NTK2025.02	26.89	16	33.66	60.97	41.41	0	29.82
YaRN-16k2025.02	15.41	13	33.17	54.65	54.69	1.11	28.67
Dyn-NTK-16k2025.02	13.95	13	30.69	52.27	57.02	1.11	28.01