Alpaca
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
Alpaca (test)
3.15SR Score
21
Alpaca harmful subset (test)
26.6Harmful Score
21
Alpaca
90Helpfulness Score
20
Alpaca GPT4
75.43Reasoning
20
Alpaca
0.158P90 Per-token Latency (s/token)
20
Alpaca (test)
71.87Alpaca LC Win Rate
20
Alpaca instruction-tuning 52k
116Pairwise Winning Score
19
Alpaca Cleaned (lower-level)
0.835Evaluation Loss
18
Alpaca
68.7Accuracy
18
Alpaca GPT4 (test)
100Accuracy
17
Alpaca GPT4
5.33MT-Bench Score
16
Alpaca
1.79Success Rate (SR)
16
Alpaca
5.68Speedup Ratio
15
Alpaca
-1.577Avg LogProb per Answer
14
Alpaca
43.24BLEU-1
14
Alpaca
3.58Accepted Length (τ)
12
Alpaca
111.46Average Latency (ms/token)
12
Alpaca
3.59Speedup
12
Alpaca (test)
7.38Test Perplexity
12
Alpaca (train)
0.562Loss
12
Alpaca instruction-following (test)
3.85PPL
12
Alpaca
0.601BLEU
12
Alpaca
0.243LDS Correlation
10
Alpaca (test)
7.7BE Score
10
Alpaca (test)
1,117Avg Length (Original)
10