Long document summarization on ScriptBase (test) benchmark leaderboard · SOTA2 Research

Long document summarization on ScriptBase (test)

50.2ROUGE 1CachED

Evaluation Results

Method	Links
CachED2025.01		50.2	14.9	20.4	64.6	—	—
SELECT & SUMM2024.04		49.98	12.11	47.95	57.46	57.64	57.29
CachED2025.01		48.9	14.4	19.8	64.1	—	—
SELECT & SUMM2024.04		48.29	11.4	46.62	57.31	57.39	57.2
Two-Stage Heuristic2024.04		46.89	11.11	44.65	56.34	56.39	56.3
Full Text2024.04		46.2	10.58	45.35	57.05	57.1	57.01
Full Text2024.04		46.15	10.62	44.46	56.04	56.32	55.77
SFT2025.01		45.5	10.5	19.4	60.8	—	—
SLED2025.01		45.2	11.9	17.8	58.3	—	—
Unlimiformer2025.01		44.8	12.3	18.3	58.7	—	—
SFT2025.01		42.9	8.9	17.8	60.1	—	—
Random Selection2024.04		41.28	8.99	39.81	54.04	54.13	53.96
SFT2025.01		39.2	8.2	17.2	57.8	—	—
CachED2025.01		36.7	7.7	17.6	56.9	—	—
SFT2025.01		33.4	4.5	14.5	55.9	—	—
Unlimiformer2024.04		31.12	4.15	30.25	46.08	41.74	51.55
FLAN-UL22024.04		25.86	5.42	24.71	50.71	51.4	48.79
SUMMN2024.04		22.65	3.01	22.42	39.68	40.87	38.6
GPT-3.5 Turbo2024.04		22.53	3.69	20.49	48.01	49.71	46.49
Lead-10242024.04		17.69	2.1	16.78	48.1	49.43	46.86
Vicuna-13b-1.52024.04		16.26	3.67	15.39	50.8	53.22	48.6
CreativeSumm2024.04		14.92	1.46	13.73	42.58	42.98	42.38
FLAN-T5-XXL2024.04		10.82	1.13	10.54	39.48	46.22	35.81
Lead-5122024.04		10.3	1.22	9.73	46.68	49.89	43.88