Mistral
Benchmarks
TaskDatasetSOTA resultTrendResults
Mistral 7B
100ASR
45
Mistral-7B-Instruct (test)
1.37Perplexity (PPL)
34
Mistral-7B
5.45Perplexity (Mistral-7B)
24
Mistral-7B
40NR
20
Mistral
78.95Recall@k
15
Mistral 7B
100Attack Success Rate (ASR)
13
Mistral-7B
1AUC
10
Mistral
0.985BERT Score
9
Mistral v0.3
0.983Entropy (bit/token)
9
Mistral-7B
1AUC
7
Mistral
98.1Detection Rate
7
Mistral-7B
0Unsafe Rate (%)
7
Mistral-RB
58SRF
6
Mistral-7B-Instruct
4GCG Attack Count
6
Mistral
100Origin Detectability Score
5
Mistral CB
72Success Rate First (SRF)
4
Mistral (unseen)
68.32Object Count
4
Mistral-Sm-24B
70.6Peak VRAM (RTX)
4
Mistral-7B-Instruct
5.76PLS
3
Mistral-7B
0.336Average Attack Similarity
3
Mistral-7B
56.7Attack Success Rate (ASR)
3
Mistral-7B 8-bit quantized
0Unsafe Rate
3
Mistral
0.001ΔPPL
3
Mistral workload 7B
99.3Pairwise Accuracy
2
Mistral family
0.99TPR
1