Complex Named Entity Recognition on MultiCoNER (test)

46.59Score (Bn)ACLM

Evaluation results

Method
ACLM2023.06	46.59	64.5	52.14	67.65	64.02	59.09	67.03	57.82	53.25	50.6	58.27
ACLM2023.06	43.79	62.32	50.32	63.94	62.05	56.82	64.41	55.09	51.83	48.44	55.9
Gold-Only2023.06	43.28	64.14	50.11	66.18	63.17	57.31	65.75	56.94	51.17	49.77	57.78
ACLM2023.06	42.61	59.75	48.52	61.49	59.05	53.46	61.59	53.34	49.96	44.72	53.45
ACLM (ours)2023.06	42.42	60.14	48.2	63.8	58.33	55.55	61.22	54.31	48.23	45.19	53.74
ACLM random2023.06	41	58.87	46.27	61.19	57.29	53.61	59.52	52.77	45.01	43.6	51.91
ACLM2023.06	41	58.74	46.22	59.13	56.93	51.22	60.3	50.26	49.32	40.93	51.4
Gold-Only2023.06	40.94	61.1	48.2	61.67	59.84	54.56	62.36	53.33	48.77	45.82	53.66
ACLM (ours)2023.06	40.26	58.31	41.48	59.35	55.69	51.56	56.31	49.4	43.57	41.23	49.72
LWTR2023.06	39.9	61.67	45.28	63.13	60.21	53.43	63.37	54.07	48.38	45.36	53.48
Gold-Only2023.06	39.84	58.67	46.34	59.65	58.5	50.7	60.79	51.66	47.12	40.98	51.42
LwTR2023.06	39.65	59.1	43.9	61.28	57.29	51.37	59.25	52.04	44.33	43.71	51.19
LWTR2023.06	38.47	55.65	43.44	54.71	53.95	44.78	56.5	46.93	45.41	31.56	47.14
ACLM (ours)2023.06	38.39	54.99	40.55	53.36	49.57	44.32	53.19	43.97	39.71	39.31	45.74
LWTR2023.06	38.37	59.09	43.8	59.37	57.76	50.38	60.42	51	46.53	42.87	50.96
ACLM random2023.06	38.24	57.92	39.33	57.14	53.24	49.81	55.06	48.27	42.22	40.55	48.18
ACLM only entity2023.06	38.13	55.46	41.84	60.05	56.99	53.32	58.22	50.17	45.11	39.62	49.89
ACLM (ours)2023.06	36.64	—	45.22	54.51	—	—	—	—	—	855	36.23
LWTR2023.06	35.93	51.78	38.87	52.73	51.59	42.55	54.49	43.99	41.23	35.19	44.83
ACLM (ours)2023.06	35.7	—	44.59	56.74	—	—	—	—	—	894	36.49
ACLM only entity2023.06	35.69	57.55	35.82	56.15	53.64	50.2	53.07	46.4	41.58	38.65	46.87
Gold-Only2023.06	35.66	56.21	42.16	55.71	54.98	45.14	57.48	46.13	44.4	30.72	46.86
ACLM (ours)2023.06	35.59	—	44.36	54.04	—	—	—	—	—	1,627	37.57
LwTR2023.06	35.42	56.97	37.83	55.91	54.74	49.36	56.1	46.82	39	38.55	47.07
ACLM random2023.06	35.26	52.69	39.83	51.14	48.7	42.19	48.71	39.68	37.26	34.22	42.96
Gold-only2023.06	34.6	55.51	38.66	55.95	51.52	48.57	50.97	45.14	38.83	38.84	45.86
LwTR2023.06	34.6	—	43.14	51.61	—	—	—	—	—	1,140	35.19
ACLM random2023.06	34.14	—	43.96	53.37	—	—	—	—	—	725	34.68
LwTR2023.06	33.74	—	43.32	53.32	—	—	—	—	—	738	34.44
MulDA2023.06	33.73	56.35	40.71	56.9	55.35	48.42	58.39	49.25	42.06	40.19	48.14
Gold-only2023.06	33.27	—	43.44	54.61	—	—	—	—	—	534	34.17
LwTR2023.06	32.36	—	40.07	48.95	—	—	—	—	—	604	31.85
ACLM random2023.06	32.16	—	41.72	52.27	—	—	—	—	—	1,363	34.95
DAGA2023.06	32.09	50.24	35.02	51.45	49.47	42.41	51.88	41.56	33.18	39.51	42.68
MulDA2023.06	31.45	48.89	36.76	48.41	48.3	39.78	51.09	42.01	35.98	31.65	41.43
MulDA2023.06	30.67	51.79	35.79	51.87	50.92	43.08	53.95	44.61	38.86	36.72	43.83
MELM2023.06	30.27	61.55	42.61	61.05	61.87	55.71	63.17	53	48.48	44.71	52.24
Gold-only2023.06	30.2	57.22	39.55	60.18	55.86	53.39	60.91	49.93	43.67	43.05	44.4
ACLM only entity2023.06	29.77	—	37.38	41.1	—	—	—	—	—	649	28.69
ACLM only entity2023.06	29.37	—	35.48	49.1	—	—	—	—	—	799	30.48
MulDA2023.06	29.25	46.87	34.52	45.92	45.55	33.91	48.21	38.65	35.56	27.33	38.58
Gold-only2023.06	27.44	—	36.9	48.7	—	—	—	—	—	376	29.2
ACLM random2023.06	27.19	—	36.52	47.73	—	—	—	—	—	712	29.64
MELM2023.06	27.17	—	34.97	44.31	—	—	—	—	—	731	28.44
DAGA2023.06	26.22	—	33.12	42.13	—	—	—	—	—	515	26.65
MELM2023.06	26.17	58.67	41.88	53.05	57.26	51.97	61.49	43.73	40.22	40.12	47.66
Gold-only2023.06	25.64	—	35.93	50.13	—	—	—	—	—	723	29.73
MELM2023.06	25.64	—	35.23	46.5	—	—	—	—	—	822	28.9
ACLM only entity2023.06	25.58	50.06	37.78	50.95	48.21	43.39	48.46	34.87	34.92	28.2	40.24
LwTR2023.06	24.59	—	32.36	46.05	—	—	—	—	—	211	26.28
MELM2023.06	24.27	52.53	40.1	49.69	52.42	43.56	47.28	44.35	40.62	34.28	47.45
ACLM (ours)2023.06	23.91	—	32.52	46.48	—	—	—	—	—	358	26.62
LwTR2023.06	23.85	52.88	34.27	50.31	47.01	42.77	52.01	40.18	35.92	30.57	40.98
MELM2023.06	23.43	53.27	41.55	48.17	51.28	39.23	51.37	45.73	41.97	30.67	42.67
ACLM (ours)2023.06	23.09	48.76	33.53	48.8	44.14	38.35	46.22	39.48	37.2	35.12	39.47
ACLM only entity2023.06	22.53	—	30.76	44.17	—	—	—	—	—	650	25.99
DAGA2023.06	22.36	44.62	24.3	43.02	42.77	36.23	47.11	30.94	30.84	33.79	35.6
ACLM random2023.06	21.64	—	29.68	45.27	—	—	—	—	—	305	24.91
DAGA2023.06	21.52	—	26.5	37.89	—	—	—	—	—	482	22.68
MELM2023.06	20.33	—	26.37	34.32	—	—	—	—	—	271	20.93
LwTR2023.06	20.25	48.6	29.95	48.38	44.08	35.09	43	39.22	30.58	27.7	36.68
ACLM random2023.06	20.13	43.59	28.04	45.83	42.27	33.64	41.82	38.2	36.79	25.99	35.63
Gold-only2023.06	19.31	51.83	33.68	49.62	45.16	42.51	47.83	31.55	26.76	32.34	38.06
MELM2023.06	18.8	—	27.55	41.1	—	—	—	—	—	621	23.41
ACLM only entity2023.06	17.55	14.06	19.6	29.72	38.1	31.57	38.47	27.4	35.62	26.34	27.84
DAGA2023.06	17.12	33.3	19.58	35.1	33.56	26.5	38.04	29.83	23.35	25.66	28.2
ACLM only entity2023.06	16.55	—	21.72	30.93	—	—	—	—	—	158	17.69
Gold-only2023.06	14.49	29.36	18.8	37.04	36.3	12.76	38.78	23.89	24.13	14.18	24.97
DAGA2023.06	14.37	—	18.92	29.32	—	—	—	—	—	179	16.1
Gold-only2023.06	12.15	—	16.36	29.71	—	—	—	—	—	31	14.63
MELM2023.06	9.46	52.57	31.57	53.57	46.4	45.01	51.9	46.73	38.26	39.64	41.51
MELM2023.06	6.88	53.48	37.02	58.69	52.43	50.5	56.25	48.99	36.83	38.88	44
MELM2023.06	6.22	40.12	27.84	43.94	37.45	34.1	37.82	32.38	20.13	25.11	30.51
DAGA2023.06	5.87	16.24	10.4	32.44	27.78	19.28	15.44	11.14	16.17	10.33	16.51
MELM2023.06	5.47	47.83	29.67	45.85	42.08	36.62	49.47	41.84	31.25	32.27	36.24
DAGA2023.06	3.28	—	4.54	14.21	—	—	—	—	—	13	5.54