Judge Agreement on Chatbot Arena Random = 33% (S1)
72Agreement RateGPT-4 (Pairwise)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GPT-4 (Pairwise)Target Judge=GPT-4 (Single-answer)2023.06 | 72 | 2,968 | |
| GPT-3.5 (Pairwise)Target Judge=Claude (Pairwise)2023.06 | 68 | 3,057 | |
| GPT-4 (Pairwise)Target Judge=GPT-3.5 (Pairwise)2023.06 | 66 | 3,061 | |
| GPT-4 (Pairwise)Target Judge=Claude (Pairwise)2023.06 | 66 | 3,062 | |
| GPT-4 (Pairwise)Target Judge=Human2023.06 | 64 | 3,066 | |
| GPT-4 (Single-answer)Target Judge=Claude (Pairwise)2023.06 | 62 | 2,964 | |
| GPT-4 (Single-answer)Target Judge=GPT-3.5 (Pairwise)2023.06 | 60 | 2,964 | |
| GPT-4 (Single-answer)Target Judge=Human2023.06 | 60 | 2,968 | |
| GPT-3.5 (Pairwise)Target Judge=Human2023.06 | 54 | 3,061 | |
| Claude (Pairwise)Target Judge=Human2023.06 | 53 | 3,062 |