# Model Editing on CounterFact (Efficacy, Paraphrase, Specificity, Avg.)

Source: [Model Editing on CounterFact (Efficacy, Paraphrase, Specificity, Avg.)](<https://www.sota2.com/research/sota/model-editing-on-counterfact-efficacy-paraphrase-specificity-avg>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-10T12:15:30.309Z

Evaluation of model editing performance on the CounterFact dataset, focusing on Efficacy, Paraphrase, Specificity, and Average scores.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Model Editing |
| Dataset (including any stated split) | CounterFact |
| Metric | Efficacy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 10 of 10 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Efficacy](<https://www.sota2.com/research/sota/model-editing-on-counterfact-efficacy-paraphrase-specificity-avg.md?metric=Efficacy>): Higher is better; unit: PERCENT.

- [Paraphrase](<https://www.sota2.com/research/sota/model-editing-on-counterfact-efficacy-paraphrase-specificity-avg.md?metric=Paraphrase>): Higher is better; unit: PERCENT.

- [Specificity](<https://www.sota2.com/research/sota/model-editing-on-counterfact-efficacy-paraphrase-specificity-avg.md?metric=Specificity>): Higher is better; unit: PERCENT.

- [Average Score](<https://www.sota2.com/research/sota/model-editing-on-counterfact-efficacy-paraphrase-specificity-avg.md?metric=Average%20Score>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Efficacy | Result date | Report paper |
| --- | --- | --- | --- |
| FT | 98.1 | Not recorded | 68fa7c061e209131cea8bf14 |
| FT | 97.9 | Not recorded | 68fa7c061e209131cea8bf14 |
| ACE | 91.2 | Not recorded | 68fa7c061e209131cea8bf14 |
| ACE | 89.7 | Not recorded | 68fa7c061e209131cea8bf14 |
| PMET | 74.6 | Not recorded | 68fa7c061e209131cea8bf14 |
| PMET | 70.7 | Not recorded | 68fa7c061e209131cea8bf14 |
| MEMIT | 57 | Not recorded | 68fa7c061e209131cea8bf14 |
| ROME | 54.1 | Not recorded | 68fa7c061e209131cea8bf14 |
| MEMIT | 50.3 | Not recorded | 68fa7c061e209131cea8bf14 |
| ROME | 45.2 | Not recorded | 68fa7c061e209131cea8bf14 |

### FT

Efficacy: 98.1. Result ID: 69b00b516dbb7075b959c9bd.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c061e209131cea8bf14.md>)

Recorded evaluation configuration: {"Base Model":"GPT-J"}. This record does not assert independent reproduction.

### FT

Efficacy: 97.9. Result ID: 69b00b516dbb7075b959c9be.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c061e209131cea8bf14.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-8B"}. This record does not assert independent reproduction.

### ACE

Efficacy: 91.2. Result ID: 69b00b516dbb7075b959c9c6.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c061e209131cea8bf14.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c061e209131cea8bf14.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-8B"}. This record does not assert independent reproduction.

### ACE

Efficacy: 89.7. Result ID: 69b00b516dbb7075b959c9c5.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c061e209131cea8bf14.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c061e209131cea8bf14.md>)

Recorded evaluation configuration: {"Base Model":"GPT-J"}. This record does not assert independent reproduction.

### PMET

Efficacy: 74.6. Result ID: 69b00b516dbb7075b959c9c3.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c061e209131cea8bf14.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72711e209131cea5656d.md>)

Recorded evaluation configuration: {"Base Model":"GPT-J"}. This record does not assert independent reproduction.

### PMET

Efficacy: 70.7. Result ID: 69b00b516dbb7075b959c9c4.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c061e209131cea8bf14.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72711e209131cea5656d.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-8B"}. This record does not assert independent reproduction.

### MEMIT

Efficacy: 57. Result ID: 69b00b516dbb7075b959c9c1.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c061e209131cea8bf14.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa703d1e209131cea49dc7.md>)

Recorded evaluation configuration: {"Base Model":"GPT-J"}. This record does not assert independent reproduction.

### ROME

Efficacy: 54.1. Result ID: 69b00b516dbb7075b959c9bf.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c061e209131cea8bf14.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e591e209131cea3f3c9.md>)

[Implementation](<https://rome.baulab.info/>)

Recorded evaluation configuration: {"Base Model":"GPT-J"}. This record does not assert independent reproduction.

### MEMIT

Efficacy: 50.3. Result ID: 69b00b516dbb7075b959c9c2.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c061e209131cea8bf14.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa703d1e209131cea49dc7.md>)

Recorded evaluation configuration: {"Base Model":"Qwen3-8B"}. This record does not assert independent reproduction.

### ROME

Efficacy: 45.2. Result ID: 69b00b516dbb7075b959c9c0.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c061e209131cea8bf14.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e591e209131cea3f3c9.md>)

[Implementation](<https://rome.baulab.info/>)

Recorded evaluation configuration: {"Base Model":"Qwen3-8B"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.