Llama2
Benchmarks
TaskDatasetSOTA resultTrendResults
Llama2 7B
1.4Iteration Time (s)
4
LLaMA2-7B
24Averaged Quantization Time (s)
4
Llama2-7b (q=32, k=32) (1k)
80.03Throughput (TFLOPS)
3
Llama2-7b (q=32, k=32) (1k)
134.56Throughput (TFLOPS)
3
Llama2-7b (q=32, k=32) (1k)
150.88TFLOPS
3
Llama2 7B
7.626PPL (Trial 1)
3
Llama2-7B
91.8F1 Score
3
LLaMA2 70B
1,450Latency (ms)
3
LLaMA2-13B Sequential batch-editing setup
84.7S Score
3
Llama2-7B tpu-v5p-512
2.5Iteration Time (s)
3
LLaMA2-7B
6.74Parameters (B)
2
Llama2 70B (tpu-v5p-1024)
11.6Iteration Time (s)
2
Llama2 70B 64 x Trainium2-16
11.2Iteration Time (s)
1
Llama2 7B 64 x Trainium2-16
1.2Iteration Time (s)
1
Llama2 7B
2.05Peak Performance
1