# Prompt recovery attack on CounterFact Source: [Prompt recovery attack on CounterFact]() Collection index: [llms.txt]() Data updated: 2026-02-26T06:07:07.780Z Evaluation of model performance on the Prompt Recovery Attack task using the CounterFact dataset, measured by Top-1, Top-5, Top-20 accuracy, and Similarity. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Prompt recovery attack | | Dataset (including any stated split) | CounterFact | | Metric | Top-1 Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 12 of 12 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Top-1 Accuracy](): Higher is better; unit: PERCENT. - [Top-5 Accuracy](): Higher is better; unit: PERCENT. - [Top-20 Accuracy](): Higher is better; unit: PERCENT. - [Similarity](): Higher is better; unit: PERCENT. ## Recorded results | Method | Top-1 Accuracy | Result date | Report paper | | --- | --- | --- | --- | | Prompt Recovery Attack | 60 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Prompt Recovery Attack | 60 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | KSTER | 57 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | KSTER | 53 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | KSTER | 51 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | KSTER | 51 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | KSTER | 50 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | KSTER | 49 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | KSTER | 47 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | KSTER | 42 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | KSTER | 35 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Prompt Recovery Attack | 20 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | ### Prompt Recovery Attack Top-1 Accuracy: 60. Result ID: 6996db375698665ec427a982. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-J","Editing Algorithm":"ROME"}. This record does not assert independent reproduction. ### Prompt Recovery Attack Top-1 Accuracy: 60. Result ID: 6996db375698665ec427a983. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-Instruct","Editing Algorithm":"ROME"}. This record does not assert independent reproduction. ### KSTER Top-1 Accuracy: 57. Result ID: 6996db375698665ec427a970. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-J","N":"10","Target Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### KSTER Top-1 Accuracy: 53. Result ID: 6996db375698665ec427a973. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-Instruct","N":"10","Target Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### KSTER Top-1 Accuracy: 51. Result ID: 6996db375698665ec427a971. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-J","N":"50","Target Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### KSTER Top-1 Accuracy: 51. Result ID: 6996db375698665ec427a975. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-Instruct","N":"100","Target Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### KSTER Top-1 Accuracy: 50. Result ID: 6996db375698665ec427a972. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-J","N":"100","Target Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### KSTER Top-1 Accuracy: 49. Result ID: 6996db375698665ec427a974. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-Instruct","N":"50","Target Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### KSTER Top-1 Accuracy: 47. Result ID: 6996db375698665ec427a978. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-2.5-Instruct","N":"100","Target Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### KSTER Top-1 Accuracy: 42. Result ID: 6996db375698665ec427a977. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-2.5-Instruct","N":"50","Target Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### KSTER Top-1 Accuracy: 35. Result ID: 6996db375698665ec427a976. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-2.5-Instruct","N":"10","Target Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### Prompt Recovery Attack Top-1 Accuracy: 20. Result ID: 6996db375698665ec427a984. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-2.5-Instruct","Editing Algorithm":"ROME"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.