Judge Agreement on MT-bench Second Turn
95AgreementG4-Pair
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| G4-PairSetup=S2 (R=50%), Compared against=G4-Single2023.06 | 95 | 727 | |
| G4-PSetup=S2 (R=50%), Comparison Judge=G4-S2023.06 | 95 | 727 | |
| HumanSetup=S2 (R=50%), Comparison Judge=Human-M2023.06 | 91 | 629 | |
| G4-SSetup=S2 (R=50%), Comparison Judge=Author2023.06 | 89 | 193 | |
| G4-PSetup=S2 (R=50%), Comparison Judge=Author2023.06 | 88 | 205 | |
| AuthorSetup=S2 (R=50%), Comparison Judge=Author2023.06 | 87 | 31 | |
| AuthorSetup=S2 (R=50%), Comparison Judge=Human2023.06 | 86 | 273 | |
| G4-PairSetup=S2 (R=50%), Compared against=Human2023.06 | 85 | 864 | |
| G4-PSetup=S2 (R=50%), Comparison Judge=Human2023.06 | 85 | 864 | |
| G4-PSetup=S2 (R=50%), Comparison Judge=Human-M2023.06 | 85 | 557 | |
| G4-SSetup=S2 (R=50%), Comparison Judge=Human-M2023.06 | 85 | 506 | |
| G4-SingleSetup=S2 (R=50%), Compared against=Human2023.06 | 84 | 776 | |
| G4-SSetup=S2 (R=50%), Comparison Judge=Human2023.06 | 84 | 776 | |
| AuthorSetup=S2 (R=50%), Comparison Judge=Human-M2023.06 | 84 | 54 | |
| HumanSetup=S1 (R=33%), Comparison Judge=Human-M2023.06 | 83 | 877 | |
| HumanSetup=S2 (R=50%), Compared against=G4-Single2023.06 | 82 | 474 | |
| HumanSetup=S2 (R=50%), Comparison Judge=Human2023.06 | 82 | 474 | |
| G4-PairSetup=S1 (R=33%), Compared against=G4-Single2023.06 | 70 | 1,161 | |
| G4-PSetup=S1 (R=33%), Comparison Judge=G4-S2023.06 | 70 | 1,161 | |
| G4-PSetup=S1 (R=33%), Comparison Judge=Human-M2023.06 | 68 | 812 | |
| AuthorSetup=S1 (R=33%), Comparison Judge=Human2023.06 | 68 | 413 | |
| HumanSetup=S1 (R=33%), Compared against=G4-Single2023.06 | 67 | 707 | |
| AuthorSetup=S1 (R=33%), Comparison Judge=Author2023.06 | 67 | 49 | |
| HumanSetup=S1 (R=33%), Comparison Judge=Human2023.06 | 67 | 707 | |
| G4-PairSetup=S1 (R=33%), Compared against=Human2023.06 | 66 | 1,325 | |
| G4-PSetup=S1 (R=33%), Comparison Judge=Author2023.06 | 66 | 341 | |
| G4-PSetup=S1 (R=33%), Comparison Judge=Human2023.06 | 66 | 1,325 | |
| G4-SSetup=S1 (R=33%), Comparison Judge=Author2023.06 | 65 | 331 | |
| AuthorSetup=S1 (R=33%), Comparison Judge=Human-M2023.06 | 63 | 87 | |
| G4-SSetup=S1 (R=33%), Comparison Judge=Human-M2023.06 | 61 | 783 | |
| G4-SingleSetup=S1 (R=33%), Compared against=Human2023.06 | 59 | 1,285 | |
| G4-SSetup=S1 (R=33%), Comparison Judge=Human2023.06 | 59 | 1,285 |