# Knowledge Retention on MUSE Retain set (Dr)

Source: [Knowledge Retention on MUSE Retain set (Dr)](<https://www.sota2.com/research/sota/knowledge-retention-on-muse-retain-set-dr>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:47:55.374Z

Evaluation of model performance on the MUSE Retain set (Dr), measuring Knowledge Retention using the KnowMem metric.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Knowledge Retention |
| Dataset (including any stated split) | MUSE Retain set (Dr) |
| Metric | KnowMem |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 9 of 9 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [KnowMem](<https://www.sota2.com/research/sota/knowledge-retention-on-muse-retain-set-dr.md?metric=KnowMem>): Higher is better; unit: PERCENT.

## Recorded results

| Method | KnowMem | Result date | Report paper |
| --- | --- | --- | --- |
| GT | 56 | Not recorded | 697c3b76a8b01e76cc8da7b2 |
| Full | 55.5 | Not recorded | 697c3b76a8b01e76cc8da7b2 |
| NPO+ | 52.5 | Not recorded | 697c3b76a8b01e76cc8da7b2 |
| NPO | 52.1 | Not recorded | 697c3b76a8b01e76cc8da7b2 |
| PerTA-fisher | 46.4 | Not recorded | 697c3b76a8b01e76cc8da7b2 |
| PerTA-grad | 41.6 | Not recorded | 697c3b76a8b01e76cc8da7b2 |
| GD | 25.4 | Not recorded | 697c3b76a8b01e76cc8da7b2 |
| TV | 2.3 | Not recorded | 697c3b76a8b01e76cc8da7b2 |
| GA | 0.8 | Not recorded | 697c3b76a8b01e76cc8da7b2 |

### GT

KnowMem: 56. Result ID: 6998f6125698665ec45b1bee.

[Reporting paper](<https://www.sota2.com/research/paper/697c3b76a8b01e76cc8da7b2.md>)

Recorded evaluation configuration: {"Backbone Model":"Llama-3.2 1B Instruct","Unlearning Ratio":"10%"}. This record does not assert independent reproduction.

### Full

KnowMem: 55.5. Result ID: 6998f6125698665ec45b1bed.

[Reporting paper](<https://www.sota2.com/research/paper/697c3b76a8b01e76cc8da7b2.md>)

Recorded evaluation configuration: {"Backbone Model":"Llama-3.2 1B Instruct","Unlearning Ratio":"10%"}. This record does not assert independent reproduction.

### NPO+

KnowMem: 52.5. Result ID: 6998f6125698665ec45b1bf2.

[Reporting paper](<https://www.sota2.com/research/paper/697c3b76a8b01e76cc8da7b2.md>)

Recorded evaluation configuration: {"Method Category":"Training-based","Backbone Model":"Llama-3.2 1B Instruct","Unlearning Ratio":"10%"}. This record does not assert independent reproduction.

### NPO

KnowMem: 52.1. Result ID: 6998f6125698665ec45b1bf1.

[Reporting paper](<https://www.sota2.com/research/paper/697c3b76a8b01e76cc8da7b2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa753a1e209131cea66964.md>)

Recorded evaluation configuration: {"Method Category":"Training-based","Backbone Model":"Llama-3.2 1B Instruct","Unlearning Ratio":"10%"}. This record does not assert independent reproduction.

### PerTA-fisher

KnowMem: 46.4. Result ID: 6998f6125698665ec45b1bf5.

[Reporting paper](<https://www.sota2.com/research/paper/697c3b76a8b01e76cc8da7b2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/697c3b76a8b01e76cc8da7b2.md>)

Recorded evaluation configuration: {"Method Category":"Task Arithmetic-based","Backbone Model":"Llama-3.2 1B Instruct","Unlearning Ratio":"10%"}. This record does not assert independent reproduction.

### PerTA-grad

KnowMem: 41.6. Result ID: 6998f6125698665ec45b1bf4.

[Reporting paper](<https://www.sota2.com/research/paper/697c3b76a8b01e76cc8da7b2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/697c3b76a8b01e76cc8da7b2.md>)

Recorded evaluation configuration: {"Method Category":"Task Arithmetic-based","Backbone Model":"Llama-3.2 1B Instruct","Unlearning Ratio":"10%"}. This record does not assert independent reproduction.

### GD

KnowMem: 25.4. Result ID: 6998f6125698665ec45b1bf0.

[Reporting paper](<https://www.sota2.com/research/paper/697c3b76a8b01e76cc8da7b2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6d3a1e209131cea38943.md>)

Recorded evaluation configuration: {"Method Category":"Training-based","Backbone Model":"Llama-3.2 1B Instruct","Unlearning Ratio":"10%"}. This record does not assert independent reproduction.

### TV

KnowMem: 2.3. Result ID: 6998f6125698665ec45b1bf3.

[Reporting paper](<https://www.sota2.com/research/paper/697c3b76a8b01e76cc8da7b2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6ef01e209131cea43012.md>)

Recorded evaluation configuration: {"Method Category":"Task Arithmetic-based","Backbone Model":"Llama-3.2 1B Instruct","Unlearning Ratio":"10%"}. This record does not assert independent reproduction.

### GA

KnowMem: 0.8. Result ID: 6998f6125698665ec45b1bef.

[Reporting paper](<https://www.sota2.com/research/paper/697c3b76a8b01e76cc8da7b2.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72381e209131cea550a5.md>)

Recorded evaluation configuration: {"Method Category":"Training-based","Backbone Model":"Llama-3.2 1B Instruct","Unlearning Ratio":"10%"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.