LLM-as-a-Judge on PandaLM Human Annotations (test)
0.7683AgreementFairJudge-8B
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| FairJudge-8BCapacity=8B, Backbone=Qwen3-VL2026.02 | 0.7683 | 0.7187 | 0.7254 | 0.7218 | |
| Qwen2.5-72BCapacity=72B2026.02 | 0.728 | 0.6625 | 0.636 | 0.6458 | |
| FairJudge-2BCapacity=2B, Backbone=Qwen3-VL2026.02 | 0.7153 | 0.6775 | 0.5999 | 0.6168 | |
| FlexVLCapacity=7B, Specialized Judge=true2026.02 | 0.7134 | 0.643 | 0.6218 | 0.63 | |
| Qwen3-VL-8BCapacity=8B2026.02 | 0.7005 | 0.6369 | 0.648 | 0.6379 | |
| GLM-4-9BCapacity=9B2026.02 | 0.7002 | 0.6986 | 0.5769 | 0.5955 | |
| FairJudge-4BCapacity=4B, Backbone=Qwen3-VL2026.02 | 0.6931 | 0.6465 | 0.6102 | 0.6218 | |
| JudgeLMCapacity=7B, Specialized Judge=true2026.02 | 0.6677 | 0.6383 | 0.7195 | 0.6392 | |
| InternVL3-14BCapacity=14B2026.02 | 0.6667 | 0.6507 | 0.6867 | 0.6227 | |
| PandaLMCapacity=7B, Specialized Judge=true2026.02 | 0.6582 | 0.4456 | 0.4893 | 0.4624 | |
| DeepSeek-V3-671BCapacity=671B2026.02 | 0.6476 | 0.5848 | 0.6113 | 0.5883 | |
| LLaVA-1.5-7BCapacity=7B2026.02 | 0.3713 | 0.4011 | 0.3924 | 0.2898 | |
| LLaVA-1.5-13BCapacity=13B2026.02 | 0.2989 | 0.4316 | 0.4167 | 0.2828 |