List your company
ResearchFormerly WizWand

Judge Agreement on MT-Bench Second Turn 1.0

95Agreement RateGPT-4 (Pairwise)
-3.186422.304347.79573.2857Jun 9, 2023
Updated 8mo ago

Evaluation results

MethodLinks
GPT-4 (Pairwise)Setup=S2 (R=50%), Reference Judge=GPT-4 (Single-answer)2023.0695727
Expert LabelerSetup=S2 (R=50%), Reference Judge=Human Majority Vote2023.0691629
GPT-4 (Single-answer)Setup=S2 (R=50%), Reference Judge=Paper Author2023.0689193
GPT-4 (Pairwise)Setup=S2 (R=50%), Reference Judge=Paper Author2023.0688205
Paper AuthorSetup=S2 (R=50%), Reference Judge=Paper Author2023.068731
Paper AuthorSetup=S2 (R=50%), Reference Judge=Expert Labeler2023.0686273
GPT-4 (Pairwise)Setup=S2 (R=50%), Reference Judge=Expert Labeler2023.0685864
GPT-4 (Pairwise)Setup=S2 (R=50%), Reference Judge=Human Majority Vote2023.0685557
GPT-4 (Single-answer)Setup=S2 (R=50%), Reference Judge=Human Majority Vote2023.0685506
GPT-4 (Single-answer)Setup=S2 (R=50%), Reference Judge=Expert Labeler2023.0684776
Paper AuthorSetup=S2 (R=50%), Reference Judge=Human Majority Vote2023.068454
Expert LabelerSetup=S1 (R=33%), Reference Judge=Human Majority Vote2023.0683877
Expert LabelerSetup=S2 (R=50%), Reference Judge=Expert Labeler2023.0682474
GPT-4 (Pairwise)Setup=S1 (R=33%), Reference Judge=GPT-4 (Single-answer)2023.06701,161
GPT-4 (Pairwise)Setup=S1 (R=33%), Reference Judge=Human Majority Vote2023.0668812
Paper AuthorSetup=S1 (R=33%), Reference Judge=Expert Labeler2023.0668413
Paper AuthorSetup=S1 (R=33%), Reference Judge=Paper Author2023.066749
Expert LabelerSetup=S1 (R=33%), Reference Judge=Expert Labeler2023.0667707
GPT-4 (Pairwise)Setup=S1 (R=33%), Reference Judge=Paper Author2023.0666341
GPT-4 (Pairwise)Setup=S1 (R=33%), Reference Judge=Expert Labeler2023.06661,325
GPT-4 (Single-answer)Setup=S1 (R=33%), Reference Judge=Paper Author2023.0665331
Paper AuthorSetup=S1 (R=33%), Reference Judge=Human Majority Vote2023.066387
GPT-4 (Single-answer)Setup=S1 (R=33%), Reference Judge=Human Majority Vote2023.0661783
GPT-4 (Single-answer)Setup=S1 (R=33%), Reference Judge=Expert Labeler2023.06591,285
G4-PSetup=S2 (R = 50%), Reference Judge=G4-S2023.060.95727
HumanSetup=S2 (R = 50%), Reference Judge=Human-M2023.060.91629
G4-SSetup=S2 (R = 50%), Reference Judge=Author2023.060.89193
G4-PSetup=S2 (R = 50%), Reference Judge=Author2023.060.88205
AuthorSetup=S2 (R = 50%), Reference Judge=Human2023.060.87273
AuthorSetup=S2 (R = 50%), Reference Judge=Human-M2023.060.8654
G4-PSetup=S2 (R = 50%), Reference Judge=Human2023.060.85864
G4-PSetup=S2 (R = 50%), Reference Judge=Human-M2023.060.85557
G4-SSetup=S2 (R = 50%), Reference Judge=Human-M2023.060.85506
G4-SSetup=S2 (R = 50%), Reference Judge=Human2023.060.84776
HumanSetup=S1 (R = 33%), Reference Judge=Human-M2023.060.83877
HumanSetup=S2 (R = 50%), Reference Judge=Human2023.060.82474
G4-PSetup=S1 (R = 33%), Reference Judge=G4-S2023.060.71,161
G4-PSetup=S1 (R = 33%), Reference Judge=Human-M2023.060.68812
AuthorSetup=S1 (R = 33%), Reference Judge=Human2023.060.68413
HumanSetup=S1 (R = 33%), Reference Judge=Human2023.060.67707
G4-PSetup=S1 (R = 33%), Reference Judge=Author2023.060.66341
G4-PSetup=S1 (R = 33%), Reference Judge=Human2023.060.661,325
G4-SSetup=S1 (R = 33%), Reference Judge=Author2023.060.65331
AuthorSetup=S1 (R = 33%), Reference Judge=Human-M2023.060.6387
G4-SSetup=S1 (R = 33%), Reference Judge=Human-M2023.060.61783
G4-SSetup=S1 (R = 33%), Reference Judge=Human2023.060.591,285