Question Answering on WMDP Biology
64.5Default ScoreBase model
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Base modelRefinement=Original, Attack forget-set source=WMDP-Cyber2025.01 | 64.5 | — | — | — | — | |
| Base modelRefinement=Original, Attack forget-set source=WMDP-Biology2025.01 | 64.5 | — | — | — | — | |
| DPO+MSERefinement=w/ RNA, Attack forget-set source=WMDP-Cyber2025.01 | 33.3 | — | 49.6 | 49.4 | 38 | |
| DPO+MSERefinement=w/ RNA, Attack forget-set source=WMDP-Biology2025.01 | 33.3 | 33.2 | 51.7 | 49.6 | 52.3 | |
| SimNPO+MSERefinement=w/ RNA, Attack forget-set source=WMDP-Cyber2025.01 | 31.5 | — | 60.6 | 37.9 | 60.6 | |
| SimNPO+MSERefinement=w/ RNA, Attack forget-set source=WMDP-Biology2025.01 | 31.5 | 32.6 | 60.5 | 37.5 | 57 | |
| Adaptive RMURefinement=w/ RNA, Attack forget-set source=WMDP-Cyber2025.01 | 31.3 | — | 55.9 | 34.2 | 57.3 | |
| RSVRefinement=w/ RNA, Attack forget-set source=WMDP-Cyber2025.01 | 31.3 | — | 48.5 | 40.1 | 36.9 | |
| Adaptive RMURefinement=w/ RNA, Attack forget-set source=WMDP-Biology2025.01 | 31.3 | 30.2 | 58.8 | 34.6 | 56.1 | |
| RSVRefinement=w/ RNA, Attack forget-set source=WMDP-Biology2025.01 | 31.3 | 30.1 | 59.8 | 37.6 | 58.1 | |
| NPO+MSERefinement=w/ RNA, Attack forget-set source=WMDP-Cyber2025.01 | 30.6 | — | 61 | 34.1 | 44.9 | |
| NPO+MSERefinement=w/ RNA, Attack forget-set source=WMDP-Biology2025.01 | 30.6 | 29.8 | 59.7 | 32.4 | 55.8 | |
| RMURefinement=w/ RNA, Attack forget-set source=WMDP-Cyber2025.01 | 30.5 | — | 58.1 | 30.7 | 50 | |
| RMURefinement=w/ RNA, Attack forget-set source=WMDP-Biology2025.01 | 30.5 | 29.1 | 59.4 | 42.8 | 56.8 | |
| NPO+KLRefinement=w/ RNA, Attack forget-set source=WMDP-Cyber2025.01 | 29.6 | — | 62.7 | 36.1 | 58.5 | |
| SimNPO+KLRefinement=w/ RNA, Attack forget-set source=WMDP-Cyber2025.01 | 29.6 | — | 63.1 | 33.7 | 60 | |
| NPO+KLRefinement=w/ RNA, Attack forget-set source=WMDP-Biology2025.01 | 29.6 | 29.1 | 61.7 | 49.7 | 56.3 | |
| SimNPO+KLRefinement=w/ RNA, Attack forget-set source=WMDP-Biology2025.01 | 29.6 | 30.4 | 62.5 | 39.2 | 57.3 | |
| Adaptive RMURefinement=Original, Attack forget-set source=WMDP-Cyber2025.01 | 29.4 | — | 61.4 | 41.3 | 53.1 | |
| DPO+KLRefinement=w/ RNA, Attack forget-set source=WMDP-Cyber2025.01 | 29.4 | — | 59.4 | 43.7 | 45.6 | |
| Adaptive RMURefinement=Original, Attack forget-set source=WMDP-Biology2025.01 | 29.4 | 27.9 | 64.8 | 43.9 | 57 | |
| DPO+KLRefinement=w/ RNA, Attack forget-set source=WMDP-Biology2025.01 | 29.4 | 28.4 | 59.6 | 36.1 | 55 | |
| DPO+KLRefinement=Original, Attack forget-set source=WMDP-Cyber2025.01 | 29.1 | — | 60.5 | 34.2 | 53.7 | |
| DPO+KLRefinement=Original, Attack forget-set source=WMDP-Biology2025.01 | 29.1 | 27.9 | 59.7 | 43.4 | 53.9 | |
| NPO+KLRefinement=Original, Attack forget-set source=WMDP-Cyber2025.01 | 28.8 | — | 55.5 | 34.5 | 49.8 | |
| NPO+KLRefinement=Original, Attack forget-set source=WMDP-Biology2025.01 | 28.8 | 27.7 | 62.4 | 32.2 | 56.4 | |
| RMURefinement=Original, Attack forget-set source=WMDP-Cyber2025.01 | 28 | — | 61.2 | 27.4 | 53.9 | |
| DPO+MSERefinement=Original, Attack forget-set source=WMDP-Cyber2025.01 | 28 | — | 46.4 | 32.5 | 44.9 | |
| RMURefinement=Original, Attack forget-set source=WMDP-Biology2025.01 | 28 | 27.7 | 64 | 30.9 | 57.7 | |
| DPO+MSERefinement=Original, Attack forget-set source=WMDP-Biology2025.01 | 28 | 27.3 | 55.1 | 33.5 | 55.1 | |
| RSVRefinement=Original, Attack forget-set source=WMDP-Cyber2025.01 | 27.8 | — | 50.4 | 32.6 | 37.2 | |
| RSVRefinement=Original, Attack forget-set source=WMDP-Biology2025.01 | 27.8 | 27.2 | 61.9 | 37 | 57.8 | |
| SimNPO+MSERefinement=Original, Attack forget-set source=WMDP-Cyber2025.01 | 27.6 | — | 61.7 | 27.2 | 54.2 | |
| SimNPO+MSERefinement=Original, Attack forget-set source=WMDP-Biology2025.01 | 27.6 | 27.2 | 61.2 | 32.3 | 55.1 | |
| NPO+MSERefinement=Original, Attack forget-set source=WMDP-Cyber2025.01 | 27 | — | 60.5 | 31 | 52.7 | |
| NPO+MSERefinement=Original, Attack forget-set source=WMDP-Biology2025.01 | 27 | 28.3 | 61 | 52.9 | 55.9 | |
| SimNPO+KLRefinement=Original, Attack forget-set source=WMDP-Cyber2025.01 | 26.5 | — | 61.6 | 32.6 | 45.3 | |
| SimNPO+KLRefinement=Original, Attack forget-set source=WMDP-Biology2025.01 | 26.5 | 27 | 62.2 | 29.7 | 55.7 |