General Language Understanding on P3 v1 (unseen) benchmark leaderboard · SOTA2 Research

General Language Understanding on P3 v1 (unseen)

80.83RTE AccuracyT0-11B

Evaluation Results

Method	Links
T0-11B2023.05		80.83	70.12	43.56	38.68	41.26	90.02	33.58	92.4	59.94	61.45	56.58	60.76
T0-3B + COT FT2023.05		80.79	65	39.49	35.13	38.58	88.27	41.04	92.13	56.4	65.96	53.6	59.67
T5-3B + COT FT2023.05		80.61	69.17	40.24	36.67	40.13	90.1	41.08	93	56.47	55.1	56.73	59.94
T0-3B + COT FT2023.05		80.25	72.62	41.71	37.22	41.89	90.88	39.5	94.47	57.47	50.58	54.27	60.08
KiC-770M2023.05		74	67.9	36.3	35	37.6	85.3	29.6	94.4	55.3	65.4	52.4	57.56
FLIPPED-3B2023.05		71.05	57.74	39.99	37.05	37.73	89.88	41.64	95.88	58.56	58.37	50.42	58.03
T5-3B + COT FT2023.05		69.96	58.69	37.58	36	37.44	84.59	40.92	90.47	55.4	64.33	51.53	56.99
RoE-3B2023.05		64.01	43.57	35.49	34.64	31.22	79.25	34.6	86.33	61.6	62.21	52.97	53.48
GPT-3 (175B)2023.05		63.5	46.4	34.6	35.4	34.5	91	78.9	83.2	70.2	65.4	45.92	59
T0-3B2023.05		60.61	48.81	35.1	33.27	33.52	75.13	27.18	84.91	50.91	65	51.27	51.43
T5-3B2023.05		53.03	34.34	32.89	33.76	33.82	54.88	27	48.16	50.64	54.09	50.3	42.99