Opinion Summarization Evaluation on SUMMEVAL-OP Round-I benchmark leaderboard · SOTA2 Research

Opinion Summarization Evaluation on SUMMEVAL-OP Round-I

0.63Relevance (RE) Spearman RhoOP-MISTRAL

Evaluation Results

Method	Links
OP-MISTRAL2024.02		0.63	—	—	—	—	0.55	0.59	0.45	0.57	0.46	0.8	0.67	0.6	0.49
OP-I-MISTRAL2024.02		0.59	0.54	0.45	0.58	0.47	0.47	0.63	0.51	0.82	0.7	0.73	0.61	0.71	0.58
OP-I-GPT-3.52024.02		0.56	—	—	0.51	0.69	0.71	0.59	0.8	0.68	0.73	0.61	0.74	0.61	0.71
G-EVAL 3.52024.02		0.49	0.45	0.36	0.28	0.52	0.68	0.56	0.7	0.58	0.79	0.67	0.73	0.61	0.75
G-EVAL-MISTRAL2024.02		0.38	0.5	0.43	0.54	0.45	0.32	0.58	0.47	0.54	0.44	0.61	0.49	0.55	0.46
SUMMAC2024.02		0.3	0.23	0.2	0.18	0.14	0.25	0.25	0.21	0.24	0.19	0.25	0.2	0.26	0.21
UNIEVAL2024.02		0.25	0.59	0.52	0.42	0.33	0.17	0.14	—	—	—	—	—	—	—
ROUGE-22024.02		-0.17	-0.23	-0.18	-0.14	-0.1	-0.12	-0.21	-0.16	-0.26	-0.19	-0.24	-0.18	-0.14	-0.09
BERTSCORE2024.02		-0.29	-0.32	-0.27	-0.28	-0.22	-0.22	-0.34	-0.26	-0.51	-0.43	-0.41	-0.33	-0.37	-0.28
BARTSCORE2024.02		-0.29	-0.19	-0.15	-0.19	-0.14	-0.22	-0.33	-0.25	-0.45	-0.35	-0.37	-0.28	-0.36	-0.27
ROUGE-12024.02		-0.31	-0.36	-0.28	-0.3	-0.24	-0.23	-0.35	-0.26	-0.44	-0.32	-0.38	-0.29	-0.3	-0.23
ROUGE-L2024.02		-0.34	-0.39	-0.32	-0.3	-0.23	-0.25	-0.4	-0.3	-0.51	-0.37	-0.45	-0.33	-0.38	-0.27