LLM Evaluation Agreement on MT-bench First Turn
97AgreementG4-Pair
Evaluation results
| Method | Links | ||
|---|---|---|---|
| G4-PairSetup=S2 (R = 50%), Reference Judge=G4-Single2023.06 | 97 | 662 | |
| G4-PairSetup=S2 (R = 50%), Reference Judge=Human2023.06 | 85 | 859 | |
| G4-SingleSetup=S2 (R = 50%), Reference Judge=Human2023.06 | 85 | 739 | |
| HumanSetup=S2 (R = 50%), Reference Judge=Human2023.06 | 81 | 479 | |
| G4-PairSetup=S1 (R = 33%), Reference Judge=G4-Single2023.06 | 70 | 1,138 | |
| G4-PairSetup=S1 (R = 33%), Reference Judge=Human2023.06 | 66 | 1,343 | |
| HumanSetup=S1 (R = 33%), Reference Judge=Human2023.06 | 63 | 521 | |
| G4-SingleSetup=S1 (R = 33%), Reference Judge=Human2023.06 | 60 | 1,280 |