Multiple-Choice Question Answering on PubHealthBench
99.5Blood Safety & STIs/HIV AccuracyLlama-3.3-70B
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Llama-3.3-70Bmodel used to generate benchmark=true, number of parameters=70B2026.07 | 99.5 | 99.7 | 100 | 99.4 | 100 | 99.8 | 99.2 | 100 | 99.2 | 99.7 | 99.5 | |
| MedGemma-27Bnumber of parameters=27B2026.07 | 99.5 | 99.5 | 100 | 98.8 | 99.2 | 99.6 | 99.6 | 98.3 | 99 | 99.3 | 99.2 | |
| Gemma-3-27Bnumber of parameters=27B2026.07 | 99.2 | 99.2 | 100 | 98.8 | 99.2 | 99.6 | 99.2 | 98.7 | 99 | 98.8 | 98.9 | |
| Command-R-32Bnumber of parameters=32B2026.07 | 98.9 | 98.8 | 99.7 | 97.9 | 99.2 | 99.2 | 98.3 | 97.4 | 98.6 | 99.1 | 98.7 | |
| Gemma-3-12Bnumber of parameters=12B2026.07 | 98.9 | 99.4 | 99.7 | 98.5 | 98.4 | 99.3 | 99.2 | 99.6 | 98.5 | 99.2 | 98.9 | |
| Phi-4-14Bnumber of parameters=14B2026.07 | 98.6 | 99.2 | 100 | 98.5 | 98.6 | 99.2 | 97.9 | 99.1 | 99.1 | 99 | 98.9 | |
| Gemma-2-27Bnumber of parameters=27B2026.07 | 98.4 | 98.8 | 99.7 | 98.8 | 98.6 | 99 | 97.5 | 97.4 | 98.4 | 98.7 | 98.5 | |
| Llama-3.1-8Bnumber of parameters=8B2026.07 | 97.6 | 98 | 99.7 | 97.9 | 97.8 | 98.9 | 97.9 | 97.9 | 97.6 | 98.2 | 97.9 | |
| Gemma-3-4Bnumber of parameters=4B2026.07 | 95.7 | 97 | 98 | 96.2 | 95.3 | 96.5 | 96.2 | 94.9 | 95.8 | 95.6 | 95.6 | |
| Phi-4-4Bnumber of parameters=4B2026.07 | 95.4 | 96.9 | 98.7 | 95.9 | 97 | 98 | 95.8 | 97.9 | 97.5 | 97.3 | 97.1 | |
| Gemma-3-1Bnumber of parameters=1B2026.07 | 65.1 | 69.2 | 64.7 | 64.9 | 61.9 | 67.9 | 73.2 | 63.4 | 65.8 | 65.6 | 66.4 |