Relevance Classification on Six entity-pair tasks T1-T6 (test) benchmark leaderboard · SOTA2 Research

Relevance Classification on Six entity-pair tasks T1-T6 (test)

89.96Overall ScoreMHP Ensemble

Evaluation Results

Method	Links
MHP Ensemble2026.06		89.96	86.19	84.64	94.38	64.21	89.07	87.61	97.26
ModernBERT2026.06		89.26	85.55	82.58	93.9	62.3	88.56	88.13	97.81
ELECTRA2026.06		88.38	84.73	82.14	93.28	63.58	87.36	85.8	96.57
ModernBERT2026.06		88.31	83.95	81.5	93	60.2	87.86	84.2	96.94
Llama-3-8b2026.06		88.27	84.25	82.69	92.73	61.74	87.38	85.04	95.93
ModernBERT2026.06		88.24	83.88	81.43	92.98	60.14	87.76	84.11	96.88
Gemma-2-2b2026.06		88.04	83.76	82.49	92.33	61.18	86.99	83.59	95.99
SH Ensemble2026.06		88.04	84.46	82.46	92.9	63.41	86.54	85.46	96.02
Llama-3-8b2026.06		87.88	84.63	83.02	93.25	64.67	86.08	85.74	95.02
Gemma-2-2b2026.06		87.14	83.56	81.96	92.45	62.91	85.83	83.89	94.3
ModernBERT2026.06		85.73	82.2	84.21	91.45	57.38	86.12	78.23	95.82
Llama-3-8b2026.06		84.89	82.4	87.45	86.12	74.83	85.91	71.47	88.63
Gemma-2-2b2026.06		83.42	81.04	86.71	85.23	72.15	84.38	69.82	87.94
Llama-3-8b2026.06		80.96	79.97	77.9	92.66	62.22	72.27	84.5	90.28
GPT-4 (few-shot)2026.06		67.2	66.22	72.1	69.4	58.3	65.8	61.2	70.5
Llama-3-8b (zero)2026.06		61.3	60.18	65.4	63.5	52.8	59.7	55.4	64.3
Gemma-2-2b†2026.06		55.59	46.43	47.36	33.16	33.14	44.81	33.41	86.71