Knowledge-Intensive Language Tasks on KILT (test) benchmark leaderboard · SOTA2 Research

Knowledge-Intensive Language Tasks on KILT (test)

21.6WoW F1 ScoreATLAS

Evaluation Results

Method	Links
ATLAS2022.08		21.6	93.5	85.1	80.8	61.3	50.6	84	90.6	—	—	—	—	—	—
FID with RS2022.08		20.6	92.2	85.2	83.7	61.2	39.1	84.6	—	—	—	—	—	—	—
Re2G2022.08		18.9	89.6	87.7	—	51.7	—	76.3	—	—	—	—	—	—	—
UpperBound2023.05		18.39	86.42	52.5	32.6	45.03	24.54	—	—	52.92	62.5	69.82	34.66	12.33	41.79
SEAL2022.08		18.3	89.5	83.6	74.6	53.7	40.5	70.9	—	—	—	—	—	—	—
UpperBound2023.05		17.92	88.2	49.96	33.65	42.6	23.84	—	—	50.86	61.77	69.14	33.71	11.8	41.22
PlugD2023.05		17.61	86.34	52.22	30.99	42.53	23.46	—	—	50.42	59.46	67.07	33.07	12.3	40.61
PlugD (w/o PT)2023.05		17.4	85.97	52.4	29.94	42.25	23.05	—	—	49.8	58.88	66.6	32.2	12.16	40.26
EmbRecy2023.05		17.3	84.34	50.12	30.1	42.71	23.57	—	—	50.55	59.31	66.67	33.46	12.01	39.93
PlugD2023.05		17.15	86.56	48.38	32.12	41.54	23.04	—	—	49.76	57.29	65.43	32.51	11.37	39.68
PlugD (w/o PT)2023.05		17.07	86.33	48.26	30.64	40.24	22.04	—	—	47.72	57.67	64.91	31.44	11.67	39.24
EmbRecy2023.05		16.91	84.59	44.16	27.2	37.42	18.98	—	—	45.43	53.02	61.05	27.7	11.57	36.73
EmbRecy2023.05		16.69	85.04	50.28	27.82	39.89	21.59	—	—	47.58	57.91	65.37	30.92	11.92	38.89
EmbRecy2023.05		16.65	84.59	41.22	20.76	32.13	18.18	—	—	40.17	47.59	55.37	26.79	11.92	34.13
ED2LM2023.05		16.1	80.59	50.84	31.77	42.07	22.8	—	—	49.79	58.94	66.68	32.32	11.66	39.35
ED2LM2023.05		15.83	84.17	48.04	24.19	40.84	21.61	—	—	48.57	57.05	64.92	30.7	11.94	37.96
Sphere2022.08		15.5	89	81.7	74.2	51.6	38.3	72.7	—	—	—	—	—	—	—
ATLAS2022.08		15.5	87.1	58.9	74.9	43.6	34.7	76.4	66.5	—	—	—	—	—	—
ED2LM2023.05		15.24	83.13	46.34	31.15	38.34	19.84	—	—	46.04	53.84	62.05	28.63	11.24	37.39
ED2LM2023.05		15.2	81.81	44.78	27.09	35.62	19.07	—	—	44.18	52.01	59.82	27.81	11.01	35.82
Hindsight2022.04		0.192	—	—	—	—	—	—	—	—	—	—	—	—	—
KGI2022.04		0.186	0.856	0.844	0.726	0.452	—	0.61	—	—	—	—	—	—	—
SEAL+FiD2022.04		0.183	0.895	0.836	0.747	0.537	40.5	0.709	—	—	—	—	—	—	—
SEAL+FiD2022.04		0.175	0.879	0.837	0.742	0.473	37.6	0.658	—	—	—	—	—	—	—
MT-DPR+FiD2022.04		0.157	0.89	0.825	0.717	0.499	36.9	0.71	—	—	—	—	—	—	—
MT-DPR-WEB+FiD2022.04		0.155	0.89	0.817	0.742	0.516	38.3	0.727	—	—	—	—	—	—	—
MT-DPR+BART2022.04		0.153	0.863	—	0.58	0.398	31.8	0.596	—	—	—	—	—	—	—
DPR+BART2022.04		0.152	0.867	0.592	0.304	0.413	25.2	0.586	—	—	—	—	—	—	—
RAG2022.04		0.131	0.863	0.592	0.447	0.444	27	0.713	—	—	—	—	—	—	—
GENRE2022.08		—	—	—	—	—	—	—	89.9	—	—	—	—	—	—