Judge Alignment on WMT ZH-EN
62.4Pairwise AccuracyDistribution-Calibrated Aggregation
Evaluation results
| Method | Links | |
|---|---|---|
| Distribution-Calibrated AggregationRater ID=R8, Number of samples (n)=122025.12 | 62.4 | |
| Distribution-Calibrated AggregationRater ID=R8, Number of samples (n)=82025.12 | 62.1 | |
| Distribution-Calibrated AggregationRater ID=R3, Number of samples (n)=122025.12 | 60.9 | |
| Distribution-Calibrated AggregationRater ID=R3, Number of samples (n)=82025.12 | 60.8 | |
| Human RaterRater ID=R32025.12 | 60.6 | |
| Distribution-Calibrated AggregationRater ID=R3, Number of samples (n)=42025.12 | 59.8 | |
| Distribution-Calibrated AggregationRater ID=R8, Number of samples (n)=42025.12 | 58.9 | |
| Distribution-Calibrated AggregationRater ID=R7, Number of samples (n)=122025.12 | 58.9 | |
| Distribution-Calibrated AggregationRater ID=R3, Number of samples (n)=22025.12 | 58.5 | |
| Distribution-Calibrated AggregationRater ID=R7, Number of samples (n)=82025.12 | 58.2 | |
| Distribution-Calibrated AggregationRater ID=R7, Number of samples (n)=42025.12 | 57.9 | |
| Distribution-Calibrated AggregationRater ID=R6, Number of samples (n)=82025.12 | 57.4 | |
| Distribution-Calibrated AggregationRater ID=R2, Number of samples (n)=122025.12 | 57.3 | |
| Distribution-Calibrated AggregationRater ID=R6, Number of samples (n)=122025.12 | 57 | |
| Human RaterRater ID=R22025.12 | 56.7 | |
| Distribution-Calibrated AggregationRater ID=R7, Number of samples (n)=22025.12 | 56.3 | |
| Distribution-Calibrated AggregationRater ID=R8, Number of samples (n)=22025.12 | 56.2 | |
| Distribution-Calibrated AggregationRater ID=R5, Number of samples (n)=82025.12 | 55.4 | |
| Distribution-Calibrated AggregationRater ID=R5, Number of samples (n)=122025.12 | 55.4 | |
| Distribution-Calibrated AggregationRater ID=R6, Number of samples (n)=42025.12 | 55.3 | |
| Distribution-Calibrated AggregationRater ID=R2, Number of samples (n)=42025.12 | 54.9 | |
| Distribution-Calibrated AggregationRater ID=R4, Number of samples (n)=122025.12 | 54.9 | |
| Distribution-Calibrated AggregationRater ID=R5, Number of samples (n)=42025.12 | 54.8 | |
| Distribution-Calibrated AggregationRater ID=R2, Number of samples (n)=82025.12 | 54.7 | |
| Human RaterRater ID=R12025.12 | 54.6 | |
| Distribution-Calibrated AggregationRater ID=R4, Number of samples (n)=82025.12 | 54.6 | |
| Distribution-Calibrated AggregationRater ID=R2, Number of samples (n)=22025.12 | 53.6 | |
| Distribution-Calibrated AggregationRater ID=R4, Number of samples (n)=42025.12 | 53.6 | |
| Human RaterRater ID=R42025.12 | 53 | |
| Distribution-Calibrated AggregationRater ID=R6, Number of samples (n)=22025.12 | 51.8 | |
| Distribution-Calibrated AggregationRater ID=R5, Number of samples (n)=22025.12 | 51.6 | |
| Human RaterRater ID=R72025.12 | 51.1 | |
| Distribution-Calibrated AggregationRater ID=R1, Number of samples (n)=122025.12 | 51.1 | |
| Human RaterRater ID=R52025.12 | 50.4 | |
| Human RaterRater ID=R82025.12 | 50.3 | |
| Distribution-Calibrated AggregationRater ID=R1, Number of samples (n)=82025.12 | 50.1 | |
| Distribution-Calibrated AggregationRater ID=R4, Number of samples (n)=22025.12 | 49.9 | |
| Human RaterRater ID=R62025.12 | 49.7 | |
| Distribution-Calibrated AggregationRater ID=R1, Number of samples (n)=42025.12 | 48.9 | |
| Distribution-Calibrated AggregationRater ID=R1, Number of samples (n)=22025.12 | 45.7 |