Evaluator Reliability Assessment on GlobalHealthAtlas 100 human-annotated scores 1.0 (test) benchmark leaderboard · SOTA2 Research

Evaluator Reliability Assessment on GlobalHealthAtlas 100 human-annotated scores 1.0 (test)

1.4216MAEqwen-plus

Evaluation Results

Method	Links
qwen-plus2026.01		1.4216	4.255	2.0001	0.9586	34.5	0.3825	4.17
Public-Evaluator2026.01		1.4259	4.1296	1.9558	0.9735	55.33	0.2772	3.67
gpt-5-mini2026.01		1.4383	4.1617	1.9431	0.9733	30.67	0.359	2
qwen3-32b2026.01		1.4811	4.519	2.0768	0.9519	14.56	0.6827	15.44
qwen3-max2026.01		1.4833	4.2767	2.0104	0.9593	35.88	0.3182	1.53
claude-3-5-sonnet-202410222026.01		1.5233	4.65	2.1234	0.9186	37.67	0.318	2
glm-4.72026.01		1.575	4.805	2.1488	0.9512	46.33	0.3226	3.5
deepseek-v3.22026.01		1.5883	4.8283	2.1459	0.9515	12.5	0.7674	20.5
qwen3-8b2026.01		1.734	5.9124	2.3722	0.9133	32.64	0.4224	4.51
Clinical-Judge2026.01		2.0122	7.1094	2.6377	0.3591	48.67	0.481	10.67
gemini-3-flash-preview2026.01		2.0375	7.2687	2.6232	0.897	35.33	0.5837	17.5
Expert-Guided Evaluator2026.01		2.1633	9.7833	3.084	0.8541	42.33	0.4677	7.5
qwen-flash2026.01		2.4033	10.48	3.1473	0.8744	36.67	0.7461	27.44