General language understanding and reasoning on TRACE benchmark leaderboard · SOTA2 Research

General language understanding and reasoning on TRACE

59C-STANCE AccuracyMagMax

Evaluation Results

Method	Links
MagMax2025.02		59	45	0.37	0.32	43	42	45	0.4	0.42
Task-Arith.2025.02		51	2	0.39	0.56	29	40	52	0.38	0.38
Avg. (Offline)2025.02		50	21	0.38	0.59	35	38	49	0.37	0.41
Task-Arith.2025.02		50	47	0.38	0.6	32	42	52	0.4	0.45
Replay2025.02		49	21	0.38	0.5	47	35	41	0.39	0.4
Replay2025.02		49	13	0.37	0.57	21	38	56	0.37	0.38
Task-Arith.2025.02		49	1	0.39	0.59	32	40	53	0.39	0.39
Task-Arith.2025.02		49	28	0.39	0.57	40	44	48	0.38	0.43
MagMax2025.02		49	44	0.25	0.26	37	31	48	0.39	0.37
Avg. (Offline)2025.02		48	2	0.37	0.58	29	34	51	0.37	0.36
Replay2025.02		47	13	0.38	0.55	18	40	56	0.37	0.38
MagMax2025.02		46	41	0.39	0.26	35	43	43	0.4	0.39
Weight Space Consolidation2025.02		46	47	0.37	0.55	46	36	53	0.4	0.45
SLERP2025.02		45	42	0.26	0.53	33	29	48	0.43	0.4
SLERP2025.02		45	18	0.37	0.58	39	37	53	0.38	0.41
Avg. (Online)2025.02		45	50	0.41	0.55	35	34	52	0.39	0.44
Avg. (Online)2025.02		45	49	0.35	0.52	16	35	47	0.36	0.39
Avg. (Offline)2025.02		45	3	0.38	0.58	33	40	48	0.37	0.38
Avg. (Online)2025.02		44	46	0.33	0.52	39	31	37	0.37	0.4
Avg. (Offline)2025.02		44	56	0.38	0.59	36	38	53	0.36	0.45
Weight Space Consolidation2025.02		44	57	0.35	0.52	47	38	55	0.39	0.46
Replay2025.02		43	1	0.35	0.5	29	32	38	0.37	0.33
Fine-tune2025.02		42	25	0.33	0.49	21	28	51	0.37	0.35
SLERP2025.02		42	45	0.31	0.54	36	29	42	0.4	0.39
SLERP2025.02		42	56	0.38	0.57	30	34	54	0.38	0.43
Avg. (Online)2025.02		42	41	0.35	0.55	43	28	43	0.44	0.41
Weight Space Consolidation2025.02		41	48	0.38	0.55	45	41	30	0.39	0.42
Weight Space Consolidation2025.02		39	13	0.36	0.51	32	40	56	0.39	0.38
MagMax2025.02		32	54	0.23	0.25	32	29	51	0.39	0.36