Judgment Bias Evaluation on JudgeBiasBench (test) benchmark leaderboard · SOTA2 Research

Judgment Bias Evaluation on JudgeBiasBench (test)

73.8Length Bias ScoreAuto-J-13B

Evaluation Results

Method	Links
Auto-J-13B2026.03		73.8	42.4	61.9	45.5	50.3	32.7	75.1	18.1	23.8	31.6	19.2	15.5	38.5
GPT-3.5-Turbo2026.03		66	28.2	41.1	35.3	39.7	22.3	45.5	16.3	21.7	43.6	13.5	52.8	35.2
JudgeLM-7B2026.03		59.1	28.9	35.9	26.3	38.8	23.9	30.7	27.3	40.4	40.8	9.7	31.5	31.6
Prometheus-7B-V2.02026.03		51.2	62.5	53.2	68.4	45.4	51.4	68.6	27.8	30.1	25.2	15.2	30.8	42.4
Qwen3 8B2026.03		41.2	12.9	22.4	17.7	26.9	10.6	12.5	21.7	29.5	21.7	7.1	45.8	22.2
Llama-3.1-8B-Instruct-RM-RB22026.03		38.2	6.3	35.2	20.7	19.8	8.6	18.9	32.5	39.3	—	—	—	23.3
Qwen2.5-7B-Instruct2026.03		36.2	20.9	42.2	23.3	27.6	12.3	15.9	22	28.8	31	33.7	30	26.9
Llama-3.1-8B-Base-RM-RB22026.03		31.8	11.7	38.4	20.6	18.8	9.3	38.2	35.2	43.1	—	—	—	26.6
Llama-3.1-Tulu-3-8B-RL-RM-RB22026.03		30.8	9.8	40.4	18.6	20.5	12.4	14.8	33.5	55	—	—	—	25
o1-mini2026.03		30.5	5.6	13.7	10.3	14.8	8.3	5.6	18.9	25.5	8.8	8.5	9.1	12.3
Selene-1-Mini-Llama-3.1-8B2026.03		23.9	21.6	47.9	43.8	30	20.6	25.9	17.4	19.5	18.6	15.6	15.1	23.9
DeepSeek-R12026.03		22.5	10.6	17.6	11.6	9.2	6.7	10	8.9	13.7	15.1	11.4	7.9	11.8
Skywork-Reward-Llama-3.1-8B-v0.22026.03		22.2	6.6	46.8	21.6	13.6	14.9	18.3	23.2	24	—	—	—	20.6
GRM-Llama3-8B-rewardmodel-ft2026.03		21.4	12.4	50.2	28.1	16.6	12.4	26.8	22.4	20.3	—	—	—	23
Skywork-Reward-V2-Llama-3.1-8B2026.03		15.5	3.6	20.4	6.3	13.7	4.7	7.9	15.2	21.1	—	—	—	11.4
Kimi-K2-Instruct2026.03		13	5.8	12.2	9	11.9	3	1.4	13.4	16.9	22.6	8.7	25.4	12.4
Skywork-Reward-V2-Qwen3-8B2026.03		12.5	3.5	22.2	7	10.5	6.5	7.8	19	18	—	—	—	11.4
Claude-3.7-Sonnet2026.03		11.2	3.1	22.1	5.9	8.1	5.5	1.7	10.3	12.7	19.5	11.9	9.6	10.2