Three-way probability ranking on COMMON2SENSE paired benchmark leaderboard · SOTA2 Research

Three-way probability ranking on COMMON2SENSE paired

76.2F1 (C1)IDEA

Evaluation Results

Method	Links
IDEA2026.04		76.2	74.5	57.2	69.3
IDEA w/o MC2026.04		73.2	59	53	61.7
IDEA2026.04		72.8	71.1	65.2	69.7
IDEA w/o Inter2026.04		71.5	69.8	51.8	64.4
IDEA w/o EM2026.04		70.6	68.9	50.1	63.2
IDEA w/o MC2026.04		68.3	67.1	48.8	61.4
IDEA w/o Inter2026.04		67.5	66	50.1	61.2
IDEA w/o EM2026.04		65.8	64.2	47.2	59.1
PWC2026.04		61.5	52.8	41.5	51.9
BIRD2026.04		59	60.2	37.1	52.1
IDEA2026.04		57.5	55.3	38.5	50.4
BIRD2026.04		55.2	56.7	34.6	48.8
PWC2026.04		54.8	56.4	33	48.1
PWC2026.04		54.2	61	41.5	52.2
IDEA w/o MC2026.04		53.6	51	33.2	45.9
IDEA w/o Inter2026.04		52.2	49.4	35.8	45.8
IDEA w/o EM2026.04		50.5	47.6	16.8	38.3
CoT2026.04		45.5	43.8	29.5	40.2
Vanilla2026.04		43.8	41.5	30.8	38.7
Vanilla2026.04		41.8	39.2	30.1	37
BIRD2026.04		39.8	41.5	19	33.4
Vanilla2026.04		39.2	36.1	28.4	34.6
CoT2026.04		38.2	36.5	27	33.9
CoT2026.04		36.5	35.2	24.8	32.2
CoT2026.04		35.2	32.5	26.1	31.3
Logit2026.04		34.5	33.3	24.9	30.9
Random2026.04		33.3	33.3	33.3	33.3
Logit2026.04		32.2	31.4	23.6	29.1
CoT2026.04		31.8	33.5	20.5	28.6
Logit2026.04		28.1	29.2	23.8	27