List your company
ResearchFormerly WizWand

Pairwise evaluation on BIGGEN

78.33Human AgreementHuman-crafted (existing) Rubrics
63.728467.519271.3175.1008May 28, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Human-crafted (existing) RubricsJudge Model=Claude Sonnet 4, Rubric Granularity=Instance-specific2026.0578.33
Fine-tuned Rubric GeneratorJudge=Claude Sonnet 4, Iteration=22026.0576.96
Fine-tuned Rubric GeneratorJudge=Claude Sonnet 4, Iteration=22026.0576.96
Human-crafted (existing) RubricsJudge Model=Qwen3 14B, Rubric Granularity=Instance-specific2026.0576.68
Fine-tuned Rubric GeneratorJudge=Claude Sonnet 4, Iteration=12026.0576.28
Fine-tuned Rubric GeneratorJudge=Claude Sonnet 4, Iteration=12026.0576.28
Human-crafted (existing) RubricsJudge Model=Llama 3.1 70B, Rubric Granularity=Instance-specific2026.0576.15
Training-free Rubric GenerationJudge Model=Claude Sonnet 4, Rubric Granularity=Instance-specific2026.0574.89
Highest Training-free baselineJudge=Claude Sonnet 42026.0574.89
Highest Training-freeJudge=Claude Sonnet 42026.0574.89
DnA-EvalJudge Model=Claude Sonnet 4, Approach Type=Training-free Approach2026.0574.82
Training-free Rubric GenerationJudge Model=Claude Sonnet 4, Rubric Granularity=Dataset-specific2026.0574.75
Fine-tuned Rubric GeneratorJudge=Qwen3 14B, Iteration=12026.0574.67
Fine-tuned Rubric GeneratorJudge=Qwen3 14B, Iteration=12026.0574.67
Fine-tuned Rubric GeneratorJudge=Qwen3 14B, Iteration=22026.0574.46
Fine-tuned Rubric GeneratorJudge=Qwen3 14B, Iteration=22026.0574.46
Human-crafted (existing) RubricsJudge Model=Claude Sonnet 4, Rubric Granularity=Dataset-specific2026.0574.36
Training-free Rubric GenerationJudge Model=Qwen3 14B, Rubric Granularity=Instance-specific2026.0572.96
Highest Training-free baselineJudge=Qwen3 14B2026.0572.96
Highest Training-freeJudge=Qwen3 14B2026.0572.96
Human-crafted (existing) RubricsJudge Model=Qwen3 14B, Rubric Granularity=Dataset-specific2026.0572.53
Training-free Rubric GenerationJudge Model=Qwen3 14B, Rubric Granularity=Dataset-specific2026.0572.42
RubricHubJudge Model=Claude Sonnet 4, Approach Type=Training-free Approach2026.0572.22
Fine-tuned Rubric GeneratorJudge=Llama 3.1 70B, Iteration=22026.0571.92
Fine-tuned Rubric GeneratorJudge=Llama 3.1 70B, Iteration=22026.0571.92
Training-free Rubric GenerationJudge Model=Llama 3.1 70B, Rubric Granularity=Instance-specific2026.0571.74
Highest Training-free baselineJudge=Llama 3.1 70B2026.0571.74
Highest Training-freeJudge=Llama 3.1 70B2026.0571.74
Fine-tuned Rubric GeneratorJudge=Llama 3.1 70B, Iteration=12026.0571.1
Fine-tuned Rubric GeneratorJudge=Llama 3.1 70B, Iteration=12026.0571.1
DnA-EvalJudge Model=Qwen3 14B, Approach Type=Training-free Approach2026.0571.01
DnA-EvalJudge Model=Llama 3.1 70B, Approach Type=Training-free Approach2026.0570.34
Human-crafted (existing) RubricsJudge Model=Llama 3.1 70B, Rubric Granularity=Dataset-specific2026.0570.2
Prometheus 8x7BApproach Type=Fine-tuned Model, Rubric Granularity=Instance-specific2026.0569.73
Training-free Rubric GenerationJudge Model=Llama 3.1 70B, Rubric Granularity=Dataset-specific2026.0569.73
RubricHubJudge Model=Llama 3.1 70B, Approach Type=Training-free Approach2026.0569.68
CheckEvalJudge Model=Claude Sonnet 4, Approach Type=Training-free Approach2026.0568.82
Prometheus 8x7BApproach Type=Fine-tuned Model, Rubric Granularity=Dataset-specific2026.0568.3
RubricHubJudge Model=Qwen3 14B, Approach Type=Training-free Approach2026.0568.28
CheckEvalJudge Model=Qwen3 14B, Approach Type=Training-free Approach2026.0565.01
CheckEvalJudge Model=Llama 3.1 70B, Approach Type=Training-free Approach2026.0564.29