Question Answering on PubMedQA (Accuracy and ECE)
82.2AccuracyGemini 3 Pro
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Gemini 3 ProTemperature=default2026.04 | 82.2 | — | |
| Gemini 3 FlashTemperature=default2026.04 | 80.8 | — | |
| HPS-CPTBackbone=Qwen3-14B-Base, CPT Strategy=High-PPL Sampling2025.12 | 78.6 | — | |
| DOS-CPTBackbone=Qwen3-14B-Base, CPT Strategy=Distance-to-Optimum Selection2025.12 | 77.4 | — | |
| MedGemma 1Model Scale=27B, Temperature=0.02026.04 | 77.2 | — | |
| RS-CPTBackbone=Qwen3-14B-Base, CPT Strategy=Random Data Sampling2025.12 | 76.7 | — | |
| Qwen3-14B-BaseBackbone=Qwen3-14B-Base, CPT Strategy=None2025.12 | 76.6 | — | |
| LPS-CPTBackbone=Qwen3-14B-Base, CPT Strategy=Low-PPL Sampling2025.12 | 76.4 | — | |
| Qwen3 VLModel Scale=4B, Temperature=default2026.04 | 74.8 | — | |
| MedGemma 1Model Scale=4B, Temperature=0.02026.04 | 73.4 | — | |
| MedGemma 1.5Model Scale=4B, Temperature=0.02026.04 | 67.6 | — | |
| UAT-LITESeed=Single2026.02 | 64 | 0.103 | |
| BaselineSeed=Single2026.02 | 62 | 0.114 | |
| Temperature ScalingSeed=Single2026.02 | 62 | 0.128 | |
| Deep EnsembleEnsemble Size=5, Seed=Single2026.02 | 52 | 0.051 |