Chatbot Arena
Benchmarks
TaskDatasetSOTA resultTrendResults
Chatbot Arena latest (test)
72.18Accuracy
51
Chatbot Arena Personalized
75.92Accuracy
42
Chatbot Arena
1Rank
40
Chatbot Arena (test)
0.342RK
16
Chatbot Arena
100Coverage
16
Chatbot Arena (test)
68.13Accuracy
14
Chatbot Arena
94.3Coverage
12
Chatbot Arena
0.352Rank Correlation (RK)
11
Chatbot Arena Random = 50% (S2)
96Agreement
10
Chatbot Arena Random = 33% (S1)
72Agreement Rate
10
Chatbot-Arena
58Accuracy
9
Chatbot Arena English-only
91.67Spearman Correlation
9
Chatbot Arena 15 LLMs after extension
0.921Spearman Correlation
7
Chatbot Arena
60.2Accuracy
5
Chatbot Arena inspired qualitative prompts (val)
1,150.78ELO
4