Knowledge Graph Prediction on JerichoWorld 1.0 (val test) benchmark leaderboard · SOTA2 Research

Knowledge Graph Prediction on JerichoWorld 1.0 (val test)

48.31Zork1 ScoreQ*BERT (Question Answering)

Evaluation Results

Method	Links
Q*BERT (Question Answering)2021.06		48.31	—	—	52.76	63.21	86.18	69.54	49.82	60.95	9.84	49.17	55.74
Worldformer2021.06		48.12	—	—	41.98	59.13	62.22	49	52.24	51.29	50.04	54.96	52.45
Q*BERT (Question Answering)2021.06		43.93	—	—	49.78	60.28	85.81	65.02	49.44	57.58	9.31	48.98	53.58
Worldformer2021.06		42.88	—	—	41.98	54.39	62.22	49	50.8	51.29	50.04	53.81	51.32
GATA-W2021.06		33.93	—	—	34.86	25.8	38.68	39.59	38.88	37.16	32.48	43.97	37.1
GATA-W2021.06		33.09	—	—	33.88	25.64	34.64	37.71	35.81	35.94	32.48	40.89	35.31
GATA-W2021.06		25.34	—	—	26.47	22.14	26.54	27.63	27	24.55	23.76	24.92	25.19
Q*BERT (Question Answering)2021.06		24.88	—	—	31.46	36.23	41.85	30.12	26.26	46.74	4.66	39.86	35.48
Q*BERT (Question Answering)2021.06		24.56	—	—	29.14	34.45	41.22	28.96	22.17	41.44	4.42	36.84	32.79
Worldformer2021.06		24.44	—	—	34.39	44.53	52.43	34.3	42.2	41.65	42.74	45.17	41.06
GATA-W2021.06		22.3	—	—	24.72	21.72	23.68	22.81	27	24.55	23.76	24.52	24.06
Worldformer2021.06		21.62	—	—	34.39	41.05	50.41	30	41.56	40.1	41.87	42.43	39.15
Seq2Seq2021.06		21.12	—	—	20.84	35.86	18.86	27.72	17.87	15.42	13.25	24.34	19.96
Seq2Seq2021.06		18.01	—	—	20.26	35.86	17.6	25.48	17.19	14.8	13.25	22.48	18.8
Seq2Seq2021.06		12.96	—	—	18.89	29.48	9.04	23.54	16.89	14.18	10.47	18.52	15.54
Seq2Seq2021.06		12.44	—	—	18.42	26.86	8.19	22.18	16.89	12.94	8.38	16.48	14.29
Rules2021.06		8.42	—	—	26.74	10.23	36.09	23.36	21.74	18.48	3.86	9.44	17.5
Rules2021.06		6.08	—	—	10.33	7.51	32.53	16.48	14.4	14.47	3.34	7.42	13.08
Rules2021.06		4.46	—	—	12.87	4.55	11.9	10.22	10.06	8.37	0.64	3.36	7.25
Rules2021.06		3.72	—	—	7.61	1.39	9.17	6.44	4.94	5.1	0.49	2.48	4.7