LMSYS-Chat
Benchmarks
TaskDatasetSOTA resultTrendResults
LMSYS-Chat-1M
0.1FPR on LMSYS
50
LMSYS-Chat-1M
0.47Average PTLA (s/token)
28
LMSYS-Chat-1M
0.143Average Per-token Latency (s/token)
16
LMSYS-Chat (Cluster 9: Casual Q&A)
0.1Compression Ratio
6
LMSYS-Chat Cluster 8: Translation Language
0.08Compression Ratio
6
LMSYS-Chat Cluster 7: Science/Math
0.39Compression Ratio
6
LMSYS-Chat Cluster 6: Philosophy/Ethics
0.09Compression Ratio
6
LMSYS-Chat (Cluster 5: Business/Professional)
0.09Compression Ratio
6
LMSYS-Chat Cluster 4: Roleplay Fiction
0.4Compression Ratio
6
LMSYS-Chat Cluster 3: Academic/Education
0.09Compression Ratio
6
LMSYS-Chat Cluster 2: Code/Technical
0.1Compression Ratio
6
LMSYS-Chat Cluster 1: Creative Writing
0.11Compression Ratio
6
LMSYS-Chat Cluster 0: General Chat
0.11Compression Ratio
6
LMSYS-Chat Overall
0.09Compression Ratio
6
LMSYS-Chat-1M Llama
0.65Kendall's Tau
4
LMSYS-Chat-1M
0.967Accuracy
4
LMSYS-Chat-1M Non-stationary rate-shift workload midpoint shift (λ = 0.4 to 0.6) Vidur Simulation Framework
1.47Average EEL
3
LMSYS-CHAT-1M
90.1ROC-AUC
3
LMSYS-Chat-1M 144 trajectories with more than 50 turns
3.67Overall MAL
2