Linguistic Acceptability on URBLIMP benchmark leaderboard · SOTA2 Research

Linguistic Acceptability on URBLIMP

100Aspect AccuracyDunbaaBERT32k

Evaluation Results

Method	Links
DunbaaBERT32k2026.05		100	95.6	99.2	84.5	99.6	88.5	95.8	85.6	99	95	98.5	98	93.2	98	95.7	97.8	98.5	91.1	93.1	—	95.1
DunbaaBERT52k2026.05		100	92.5	98.5	95.6	99.8	90	98.3	90.8	99.5	94	99	98	96.1	98	97.3	99.3	98.5	99	98	—	97
Urdu-RoBERTasmall2026.05		99.8	86.9	90.9	96	99.6	86.5	95.8	71.2	92.5	89	99	84.3	84.5	99	89.7	81.6	89.8	95.1	88.1	—	90.5
XLM-Rbase2026.05		99.5	93.1	94.7	96.9	99.3	72.5	92.8	61.4	88.5	89	99	91.2	89.3	94.1	79.4	84.6	88.3	90.2	98	—	89.6
mmBERTsmall2026.05		99.3	88.1	96.2	93.3	98.4	82	73.8	65.4	87	91	93.3	95.1	84.5	96.1	89.7	85.4	91.3	92.1	99	—	89.5
XLM-Rlarge2026.05		99.3	95	97.7	90	99.8	92	95.3	79.7	93.5	92	98.4	92.2	88.3	97.1	92	93.6	96.1	98.7	98	—	94.1
mBERT2026.05		98.8	65	53	85.1	95.7	63	18.5	58.2	86.5	77	93.7	74.5	78.6	95.1	82.4	73.8	88.3	55.4	91.1	—	75.5
DunbaaBERT96k2026.05		98.5	95.6	95.5	94.5	98.7	91	90.6	92.8	97	88	97.2	95.1	94.2	96.1	97	93.3	97.6	98	87.1	87.1	94.6
HPLT-BERTur2026.05		98.3	98.8	99.2	98.4	99.9	97	96.5	94.8	97.5	95	99.2	95.1	95.1	96.1	98	98.1	97.1	97	98	—	97.3
mmBERTbase2026.05		98	90	97.7	94.5	99.6	89	89.4	71.2	94	92	97.6	95.1	85.4	94.1	92	91.4	91.7	95.7	97	—	92.4