Bio-threat knowledge evaluation on WMDP Bio
72.43Accuracy (All)Llama 3.1 8B Instruct
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Llama 3.1 8B InstructParameters=8B, Instruct=true2025.08 | 72.43 | 89.14 | 64.63 | 46.28 | |
| Tulu-3 8BParameters=8B2025.08 | 69.6 | 87.16 | 61.41 | 49.81 | |
| OLMO-2 7BParameters=7B, Instruct=false2025.08 | 67.32 | 86.67 | 58.29 | 42.94 | |
| OLMO-2 7B InstructParameters=7B, Instruct=true2025.08 | 66.38 | 87.65 | 56.45 | 45.35 | |
| Gemma-3 4B InstructParameters=4B, Instruct=true2025.08 | 63.86 | 85.68 | 53.69 | 38.1 | |
| Gemma-3 4BParameters=4B, Instruct=false2025.08 | 61.04 | 79.51 | 52.42 | 36.15 | |
| Our Unfiltered Model (Baseline)status=baseline2025.08 | 52.79 | 73.33 | 43.2 | 36.34 |