AMR Similarity Consistency on BAMBOO (test) benchmark leaderboard · SOTA2 Research

AMR Similarity Consistency on BAMBOO (test)

66.94Main - STS-BWWLK_theta

Evaluation Results

Method	Links
WWLK_theta2021.08		66.94	67.64	37.91	64.34	65.49	39.23	60.11	62.29	35.15	55.03	75.06	29.64	54.9	50.26
SEMBLEU k=12021.08		66.03	62.88	39.72	61.76	62.1	38.17	61.83	58.83	37.1	1.99	1.47	1.4	41.11	5.78
WLK2021.08		64.86	61.52	37.35	62.69	62.55	36.49	59.41	56.6	33.71	45.89	64.7	19.47	50.44	44.35
rematch2024.04		64.72	66.54	34.88	63.49	62.55	35.82	59.75	61.54	32.7	42.38	67.28	15.37	50.59	—
WLK2024.04		63.68	62.32	35.18	61.31	63.03	35.65	57.9	56.6	31.43	44.72	66.39	17.46	49.64	—
WWLK2021.08		63.15	65.58	37.55	59.78	65.53	35.81	59.4	59.98	32.86	13.98	42.79	7.16	45.3	28.83
SEMBLEU k=22021.08		60.62	59.86	36.88	57.68	59.64	36.24	57.34	56.18	33.26	44.54	67.54	16.6	48.87	42.13
S2MATCH2021.08		58.82	60.42	42.55	58.08	62.25	40.6	56.7	57.92	41.22	48.79	71.41	27.83	52.22	49.07
SMATCH2021.08		58.45	59.72	41.25	57.98	61.81	39.66	56.14	57.39	39.58	48.05	70.53	24.75	51.28	47.5
sembleu2024.04		57.02	58.76	31.95	54.73	59.92	31.92	53.42	54.66	27.95	45.69	66.74	21.36	47.01	—
SEMBLEU k=32021.08		56.49	57.76	32.47	54.84	57.7	33.25	52.82	53.47	28.44	49.06	69.49	24.27	47.5	42.82
S2MATCH default2021.08		56.38	58.15	42.16	55.65	60.04	40.41	56.05	57.17	40.92	46.51	70.9	26.58	50.91	47.8
SEMA2021.08		55.9	53.32	33.43	55.51	56.16	32.33	50.16	48.87	29.11	49.73	68.18	22.79	46.29	41.85
s2match2024.04		55.87	57.38	41.92	55.41	59.46	40.78	54.86	56.12	40.59	48.23	73.89	26.19	50.89	—
SEMBLEU k=42021.08		53.19	56.69	29.61	52.28	56.12	30.11	49.31	52.11	25.56	49.75	69.58	29.44	46.15	41.75
WSMATCH2021.08		53.06	59.24	38.64	53.39	61.17	37.49	51.41	57.56	37.85	42.47	66.79	22.68	48.48	44.58
smatch2024.04		53.01	57.65	40.96	53.02	59.74	40.08	51.76	55.42	39.6	54.03	75.2	24.78	50.44	—