Alpaca
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
Alpaca Dataset
100FSR
10
Alpaca
29.04Accuracy (Alpaca)
9
Alpaca GPT4 (held-out)
0.898Loss
8
Alpaca-Eval
72.59LC Win Rate
8
Alpaca
5.633Useful Score
8
Alpaca
0ASR (Alpaca)
8
Alpaca
1.35Evaluation Loss
7
ALPACA 1000 simulated patient rollouts
3.38Cumulative Reward
7
Alpaca In-Domain
0.72AUROC
7
Alpaca Mix (Unsafe)
93Refusal Rate
6
Alpaca Mix Safe
0Refusal Rate
6
Alpaca Audio
64.24GPT Score
6
Alpaca Qwen3 32B
38.43Memory (GB/GPU)
6
Alpaca
3.59Acceptance Length
6
Alpaca Vicuna-7B (test)
1,874Average Length
6
Alpaca Samantha-7B (test)
1,944Average Length
6
Alpaca Llama2-7B (test)
191Average Length
6
Alpaca-57k (OOD)
41.4Delta ASR
6
Alpaca 57k (Seen)
96.7Delta ASR
6
Alpaca 5% retention
157.162SUM
6
Alpaca
56Accuracy
6
Alpaca-P CoSER evaluation framework (test 20 randomly sampled instances)
4.38Anthropomorphism
5
Alpaca
3,348.18Prefill Throughput (tokens/s)
5
Alpaca-GPT4 Style
85.3Win Rate
5
Alpaca-GPT4 (Expertise)
76.97Win Rate
5