LLM Judge Agreement on MT-bench First Turn
0.97Agreement RateG4-P
Evaluation results
| Method | Links | ||
|---|---|---|---|
| G4-PSetup=S2 (R=50%), Comparison Judge=G4-S2023.06 | 0.97 | 662 | |
| G4-PSetup=S2 (R=50%), Comparison Judge=Claude2023.06 | 0.94 | 582 | |
| G4-SSetup=S2 (R=50%), Comparison Judge=Author2023.06 | 0.94 | 175 | |
| G4-PSetup=S2 (R=50%), Comparison Judge=Author2023.06 | 0.92 | 201 | |
| G4-SSetup=S2 (R=50%), Comparison Judge=Claude2023.06 | 0.9 | 563 | |
| HumanSetup=S2 (R=50%), Comparison Judge=Human-M2023.06 | 0.9 | 631 | |
| ClaudeSetup=S2 (R=50%), Comparison Judge=Author2023.06 | 0.89 | 141 | |
| AuthorSetup=S2 (R=50%), Comparison Judge=Author2023.06 | 0.87 | 31 | |
| ClaudeSetup=S2 (R=50%), Comparison Judge=Human-M2023.06 | 0.86 | 414 | |
| G4-PSetup=S2 (R=50%), Comparison Judge=Human2023.06 | 0.85 | 859 | |
| G4-PSetup=S2 (R=50%), Comparison Judge=Human-M2023.06 | 0.85 | 546 | |
| G4-SSetup=S2 (R=50%), Comparison Judge=Human2023.06 | 0.85 | 739 | |
| G4-SSetup=S2 (R=50%), Comparison Judge=Human-M2023.06 | 0.85 | 473 | |
| ClaudeSetup=S2 (R=50%), Comparison Judge=Human2023.06 | 0.85 | 648 | |
| AuthorSetup=S2 (R=50%), Comparison Judge=Human2023.06 | 0.83 | 262 | |
| HumanSetup=S1 (R=33%), Comparison Judge=Human-M2023.06 | 0.81 | 892 | |
| HumanSetup=S2 (R=50%), Comparison Judge=Human2023.06 | 0.81 | 479 | |
| AuthorSetup=S2 (R=50%), Comparison Judge=Human-M2023.06 | 0.76 | 46 | |
| G4-PSetup=S1 (R=33%), Comparison Judge=G4-S2023.06 | 0.7 | 1,138 | |
| G4-PSetup=S1 (R=33%), Comparison Judge=Author2023.06 | 0.69 | 345 | |
| AuthorSetup=S1 (R=33%), Comparison Judge=Author2023.06 | 0.69 | 49 | |
| G4-PSetup=S1 (R=33%), Comparison Judge=Human-M2023.06 | 0.67 | 821 | |
| G4-SSetup=S1 (R=33%), Comparison Judge=Author2023.06 | 0.67 | 324 | |
| G4-PSetup=S1 (R=33%), Comparison Judge=Human2023.06 | 0.66 | 1,343 | |
| G4-SSetup=S1 (R=33%), Comparison Judge=Claude2023.06 | 0.66 | 1,136 | |
| AuthorSetup=S1 (R=33%), Comparison Judge=Human2023.06 | 0.65 | 428 | |
| G4-PSetup=S1 (R=33%), Comparison Judge=Claude2023.06 | 0.63 | 1,198 | |
| HumanSetup=S1 (R=33%), Comparison Judge=Human2023.06 | 0.63 | 721 | |
| G4-SSetup=S1 (R=33%), Comparison Judge=Human2023.06 | 0.6 | 1,280 | |
| G4-SSetup=S1 (R=33%), Comparison Judge=Human-M2023.06 | 0.6 | 781 | |
| ClaudeSetup=S1 (R=33%), Comparison Judge=Author2023.06 | 0.58 | 343 | |
| ClaudeSetup=S1 (R=33%), Comparison Judge=Human-M2023.06 | 0.55 | 820 | |
| AuthorSetup=S1 (R=33%), Comparison Judge=Human-M2023.06 | 0.55 | 93 | |
| ClaudeSetup=S1 (R=33%), Comparison Judge=Human2023.06 | 0.54 | 1,341 |