ShareGPT
Benchmarks
TaskDatasetSOTA resultTrendResults
ShareGPT
0.36Average PTLA (s/token)
36
ShareGPT
27P50 Latency
32
ShareGPT
94.11Success Rate (SR)
24
ShareGPT v3
8.85Throughput (req/s)
24
ShareGPT
2.01Speedup vs AR
19
ShareGPT (test)
52.66LLM-Judge
17
ShareGPT
96.4Response Similarity
12
ShareGPT Llama-3.1-8B 1.0 (test)
3.212MT-Bench Score
10
ShareGPT
9,800FSR
10
ShareGPT-LF Llama Series Set (cross-domain (legal and financial))
90.07Success Rate (SR)
9
ShareGPT-LF Qwen Series Set cross-domain (legal and financial)
91.46Success Rate (SR)
9
ShareGPT-4o-Image SD3-Medium
35.185CLIP Score
7
ShareGPT
58.33Throughput (RPS)
6
ShareGPT Mixed candidate set (Qwen and Llama)
94.99SR
6
ShareGPT 3 Turn 6491 tokens
2.79PPL
6
ShareGPT 2 Turn, 3006 tokens
2.91PPL
6
ShareGPT 1 Turn, 765 tokens
4.01Perplexity
6
ShareGPT Vicuna unfiltered
117.5Throughput (tok/s)
5
ShareGPT N=94 conversations
78.3Fact Recovery Score
5
ShareGPT (trace replay)
2.33OOM Rate
5
ShareGPT
2.4Latency (ms/token)
5
ShareGPT
3.99MT-Bench Score
5
ShareGPT
30.5Match Rate
4
ShareGPT
2.01Throughput
2
ShareGPT
1.5TPOT Speedup vs DeepGEMM
2