Judge Agreement on Chatbot Arena Random = 50% (S2)
96AgreementGPT-3.5 (Pairwise)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GPT-3.5 (Pairwise)Target Judge=Claude (Pairwise)2023.06 | 96 | 1,497 | |
| GPT-4 (Pairwise)Target Judge=GPT-4 (Single-answer)2023.06 | 95 | 1,967 | |
| GPT-4 (Pairwise)Target Judge=Claude (Pairwise)2023.06 | 95 | 1,712 | |
| GPT-4 (Pairwise)Target Judge=GPT-3.5 (Pairwise)2023.06 | 94 | 1,788 | |
| GPT-4 (Single-answer)Target Judge=Claude (Pairwise)2023.06 | 91 | 1,538 | |
| GPT-4 (Single-answer)Target Judge=GPT-3.5 (Pairwise)2023.06 | 89 | 1,593 | |
| GPT-4 (Pairwise)Target Judge=Human2023.06 | 87 | 1,944 | |
| GPT-4 (Single-answer)Target Judge=Human2023.06 | 85 | 1,761 | |
| Claude (Pairwise)Target Judge=Human2023.06 | 84 | 1,475 | |
| GPT-3.5 (Pairwise)Target Judge=Human2023.06 | 83 | 1,567 |