LLM-as-a-Judge on FairJudge Benchmark 1K (test)
71.5AgreementFairJudge-8B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| FairJudge-8BCapacity=8B, Backbone=Qwen3-VL2026.02 | 71.5 | 71.15 | 66.92 | 67.63 | |
| JudgeLMCapacity=7B, Specialized Judge=true2026.02 | 69.56 | 65.27 | 67.41 | 66.11 | |
| FairJudge-4BCapacity=4B, Backbone=Qwen3-VL2026.02 | 67.74 | 72.19 | 59.97 | 62.57 | |
| Qwen2.5-72BCapacity=72B2026.02 | 67.45 | 69.23 | 57.39 | 59.72 | |
| GLM-4-9BCapacity=9B2026.02 | 64.4 | 60.79 | 56.29 | 57.75 | |
| Qwen3-VL-8BCapacity=8B2026.02 | 63.93 | 62.85 | 56.45 | 58.35 | |
| FairJudge-2BCapacity=2B, Backbone=Qwen3-VL2026.02 | 62.17 | 58.89 | 58.15 | 56.37 | |
| FlexVLCapacity=7B, Specialized Judge=true2026.02 | 59.58 | 49.56 | 46.01 | 44.72 | |
| InternVL3-14BCapacity=14B2026.02 | 58.31 | 57.76 | 62.3 | 55.11 | |
| DeepSeek-V3-671BCapacity=671B2026.02 | 55.97 | 49.97 | 51.66 | 50.26 | |
| PandaLMCapacity=7B, Specialized Judge=true2026.02 | 52.46 | 35 | 39.23 | 36.99 | |
| LLaVA-1.5-13BCapacity=13B2026.02 | 45.86 | 38.93 | 41.95 | 39.03 | |
| LLaVA-1.5-7BCapacity=7B2026.02 | 40.34 | 37.42 | 35.49 | 32.57 |