NLU and Reasoning on BIG-bench (unseen) benchmark leaderboard · SOTA2 Research

NLU and Reasoning on BIG-bench (unseen)

86.96Known Unknowns ScoreFLIPPED

Evaluation Results

Method	Links
FLIPPED2022.10		86.96	42.5	53.23	52.57	48.49	60	64.93	26.87	65.57	50.39	54.34	64.56	55.08	55.17	46.88
FLIPPED2022.10		71.74	41.7	53.19	47.43	47.93	45	61.64	19.01	57.07	50.56	46.58	59.82	53.33	48.57	25
PALM2022.10		67.39	42.2	69	94.86	57.2	61.67	80.02	37.3	70.4	52.2	77.63	34.42	47.5	60.8	59.38
T02023.02		65.22	33.67	54.67	42.86	57.33	51.67	71.72	18.33	57.33	48.33	52.97	54.67	55	51.06	—
DIRECT2022.10		63.04	35.9	53.28	50.29	47.15	33.33	58.14	22.01	63.83	49.85	50.23	38.06	44.33	43.75	3.13
GPT-32023.02		60.87	31.2	52.3	22.57	51.4	31.67	26.78	15.9	12.3	50.5	47.95	23.42	51.5	37.57	—
GPT-32022.10		60.87	31.2	52.3	32.57	51.4	31.67	26.78	15.9	12.3	50.5	47.95	23.42	51.5	38.23	46.88
Cos PE2023.02		58.7	30.7	42.36	51.43	46.72	66.67	72.92	25.95	46.55	50	47.03	42.4	44.33	48.13	—
T02022.10		58.7	38.3	52.75	29.71	53.69	43.33	69.29	20.2	64.73	51.81	50	54.86	52.11	46.79	15.63
PaLM2023.02		56.52	32.1	64	56	49.1	25	59.26	20.1	14.1	49.9	47.47	24.22	45.3	41.77	—
PALM2022.10		56.52	32.1	64	56	49.1	25	59.26	20.1	14.1	49.9	47.49	24.22	45.3	42.14	46.88
CHANNEL2022.10		52.17	30.9	53.01	16.57	51.06	71.67	35.67	11.55	15.73	50.43	47.79	25.73	55.36	38.07	15.63
GPT-32022.10		50	31.1	57.1	58.29	49.4	31.67	35.75	10.9	52.7	52.8	60.27	33.93	56.5	45.48	56.25
T02023.02		47.83	32.1	53.23	34.86	53.22	53.33	67.25	14.94	58.18	52.27	52.05	45.33	44.29	46.84	—
T02022.10		47.83	41.1	52.79	25.71	52.85	46.67	45.52	14.84	58.89	52.94	50.23	46.64	44.29	42.56	15.63