# Knowledge Retention on WMDP bio (retain) Source: [Knowledge Retention on WMDP bio (retain)]() Collection index: [llms.txt]() Data updated: 2026-05-15T09:59:37.313Z Evaluation of the model's ability to retain knowledge, tested specifically on the WMDP bio (retain) dataset, measured using the Rt metric. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Knowledge Retention | | Dataset (including any stated split) | WMDP bio (retain) | | Metric | Rt | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Rt](): Higher is better; unit: PERCENT. ## Recorded results | Method | Rt | Result date | Report paper | | --- | --- | --- | --- | | Zero-shot | 80.3 | Not recorded | 6a06a8c9dbc836b87d82a365 | | Zero-shot | 76.3 | Not recorded | 6a06a8c9dbc836b87d82a365 | | MANSU | 67.1 | Not recorded | 6a06a8c9dbc836b87d82a365 | | LUNAR | 65.5 | Not recorded | 6a06a8c9dbc836b87d82a365 | | LUNAR | 61.9 | Not recorded | 6a06a8c9dbc836b87d82a365 | | Surgical GA | 56 | Not recorded | 6a06a8c9dbc836b87d82a365 | | MANSU | 52.3 | Not recorded | 6a06a8c9dbc836b87d82a365 | | NPO | 50.3 | Not recorded | 6a06a8c9dbc836b87d82a365 | | Surgical GA | 30.3 | Not recorded | 6a06a8c9dbc836b87d82a365 | | NPO | 30.3 | Not recorded | 6a06a8c9dbc836b87d82a365 | | GU+SimNPO | 27.7 | Not recorded | 6a06a8c9dbc836b87d82a365 | | Global GA | 26 | Not recorded | 6a06a8c9dbc836b87d82a365 | | SimNPO | 25.7 | Not recorded | 6a06a8c9dbc836b87d82a365 | | GU+SimNPO | 24.7 | Not recorded | 6a06a8c9dbc836b87d82a365 | | Global GA | 24.7 | Not recorded | 6a06a8c9dbc836b87d82a365 | | SimNPO | 21 | Not recorded | 6a06a8c9dbc836b87d82a365 | ### Zero-shot Rt: 80.3. Result ID: 6a06ee7247f34ce924ef1c27. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen-3-8B"}. This record does not assert independent reproduction. ### Zero-shot Rt: 76.3. Result ID: 6a06ee7247f34ce924ef1c1f. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### MANSU Rt: 67.1. Result ID: 6a06ee7247f34ce924ef1c2e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-3-8B"}. This record does not assert independent reproduction. ### LUNAR Rt: 65.5. Result ID: 6a06ee7247f34ce924ef1c2d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-3-8B"}. This record does not assert independent reproduction. ### LUNAR Rt: 61.9. Result ID: 6a06ee7247f34ce924ef1c25. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### Surgical GA Rt: 56. Result ID: 6a06ee7247f34ce924ef1c21. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### MANSU Rt: 52.3. Result ID: 6a06ee7247f34ce924ef1c26. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### NPO Rt: 50.3. Result ID: 6a06ee7247f34ce924ef1c22. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### Surgical GA Rt: 30.3. Result ID: 6a06ee7247f34ce924ef1c29. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen-3-8B"}. This record does not assert independent reproduction. ### NPO Rt: 30.3. Result ID: 6a06ee7247f34ce924ef1c2a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-3-8B"}. This record does not assert independent reproduction. ### GU+SimNPO Rt: 27.7. Result ID: 6a06ee7247f34ce924ef1c2c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-3-8B"}. This record does not assert independent reproduction. ### Global GA Rt: 26. Result ID: 6a06ee7247f34ce924ef1c20. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### SimNPO Rt: 25.7. Result ID: 6a06ee7247f34ce924ef1c2b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-3-8B"}. This record does not assert independent reproduction. ### GU+SimNPO Rt: 24.7. Result ID: 6a06ee7247f34ce924ef1c24. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### Global GA Rt: 24.7. Result ID: 6a06ee7247f34ce924ef1c28. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen-3-8B"}. This record does not assert independent reproduction. ### SimNPO Rt: 21. Result ID: 6a06ee7247f34ce924ef1c23. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.