Linguistic Minimal Pair Evaluation on BLiMP (test) benchmark leaderboard · SOTA2 Research

Linguistic Minimal Pair Evaluation on BLiMP (test)

100NPI lic. (2)Sonnet

Evaluation Results

Method	Links
Sonnet2025.06		100	96	98	98.4	62	85.6	81.3	100	90.2
Sonnet2025.06		99.3	88.7	91	93.8	98	97.6	71.6	98	92.2
Sonnet2025.06		99.3	96.7	99	95.8	98	97.8	74.9	100	95.2
gpt-4o2025.06		98.7	84.7	97.7	97.3	59.3	84.7	80.5	100	87.9
Llama (GPb+CT)2025.06		98.3	76.4	90.3	91.3	87.5	97.6	76.5	91.3	88.7
gpt-4o2025.06		97	88.7	92.3	94.7	92	95.6	81.6	100	92.7
Haiku2025.06		96.7	86	70.7	90.4	87.3	94.7	74.3	91.7	86.5
Sonnet2025.06		96.7	96	98	93.8	77	90.4	72.8	97.7	90.3
Haiku2025.06		96	82.7	74.3	94.2	73.7	73.1	73.3	91.3	82.3
gpt-4o2025.06		93.7	86.7	77.7	94.9	88.7	94.4	82.1	99.3	89.7
gpt-3.52025.06		88.7	74.7	68.7	94.4	72	59.6	74.4	91	77.9
Llama (GPb)2025.06		87.7	70.7	85.3	87.6	69.3	94.4	71.9	81	81
gpt-4o2025.06		87.7	90.7	80.3	92.4	69.7	81.3	79.1	100	85.1
gpt-3.52025.06		87	66	62	82.9	79.7	86	70.9	92.7	78.4
gpt-4o2025.06		81.7	88	90	89.8	85.7	80.7	77.1	86.3	84.9
Sonnet2025.06		79	95.3	91	88.2	88	71.3	65.9	63.7	80.3
Haiku2025.06		77	86.7	64.7	75.1	59.3	62.4	60.5	79	70.6
Sonnet2025.06		75.7	95.3	90.7	84.7	81	80.2	68.8	62.3	79.8
Haiku2025.06		73.3	85.3	63.7	83.1	75	62.9	64.7	68.3	72
gpt-4o2025.06		73.3	88	72.7	86	70.3	75.3	77.2	81	78
gpt-3.52025.06		69.7	79.3	67.7	92.9	72.7	58.9	65.1	83	73.6
gpt-3.52025.06		67.3	66	57.3	79.1	69.7	48.4	61.6	52	62.7
gpt-3.52025.06		67.3	46.7	57.3	87.8	74	88.9	63.1	95	72.5
Llama (CT)2025.06		66.7	82.6	81	88	70.3	61.3	70.8	57.1	72.2
Haiku2025.06		65.3	74	50.7	70.7	50	87.6	59.1	81.3	67.3
Llama (Base)2025.06		62.3	80.7	87.7	85.3	54	55.7	65.1	60.3	68.9
gpt-3.52025.06		60	77.3	65.3	83.1	76	52.7	68.9	59.7	67.9
Haiku2025.06		57.3	74.7	56.7	69.3	60.3	55.6	60.9	54.7	61.2