Llama2
Benchmarks
TaskDatasetSOTA resultTrendResults
Llama2 7B (32 Q-heads/32 KV-heads/128 Head-dimension)
207.3Attention TFLOPS
42
LLaMA2 7B
11.09TTFT (ms)
33
LLaMA2 13B
0Number of Recomputed Layers
21
Llama2-7b five finetuned variants
0Average ASR
16
LLaMA2-7B zero-shot
67.18Zero-Shot Accuracy
16
Llama2-7B Generation Evaluation Set
93.14Completion Accuracy (CA)
15
Llama2-7B Generation Evaluation Set
90.15Accuracy (CA)
15
LLaMA2-7B 300 tokens (test)
7.049Perplexity
14
LLaMA2-7B 12/128 tokens
1.889Latency
13
llama2-7b v1 (pretrained)
0ASR
13
Llama2-7B Copy-paste attack
97.8F1 Score
11
Llama2 7B
100AUC
10
Llama2-7B Paragraphing
91.6F1 Score
8
Llama2-7B Synonymous substitution
98.5F1 Score
8
Llama2-7B Clean
100F1 Score
8
LLaMA2-7B
1.34Memory Usage (GB)
7
LLaMA2-7B Adaptive AutoDAN-T attack
17ASR
6
LLaMA2-7B Adaptive PAIR attack
0Attack Success Rate (ASR)
6
Llama2-DA
57SRF
4
Llama2-7b (q=32, k=32) (1k)
114.85TFLOPS (Relative Pos. Attention)
4
Llama2-7b (q=32, k=32) (1k)
39.81TFLOPS (Share QK Mask Attention, 1k)
4
Llama2-7b q=32, k=32 (1k)
67.36TFLOPS
4
Llama2-7b (q=32, k=32) (8k)
163.7TFLOPS (PrefixLM Attention)
4
LLaMA2-7B 1,024 tokens
20Latency (ms)
4
Llama2-70B (64 x H100-8)
7.8Iteration Time (s)
4