# Subject Inference Attack on CounterFact batch-edit tasks Source: [Subject Inference Attack on CounterFact batch-edit tasks]() Collection index: [llms.txt]() Data updated: 2026-02-26T06:01:14.590Z Evaluation of a Subject Inference Attack methodology on the CounterFact dataset, specifically focusing on batch-edit scenarios, measured by recall. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Subject inference attack | | Dataset (including any stated split) | CounterFact batch-edit tasks | | Metric | Recall | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 36 of 36 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Recall](): Higher is better; unit: PERCENT. ## Recorded results | Method | Recall | Result date | Report paper | | --- | --- | --- | --- | | Ours (White-box) | 100 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Ours (White-box) | 100 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Ours (White-box) | 100 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Ours (White-box) | 100 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Ours (White-box) | 99 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Ours (White-box) | 99 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Ours (White-box) | 99 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Ours (White-box) | 99 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Ours (White-box) | 98 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Ours (White-box) | 98 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Gray-box | 96 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Gray-box | 96 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Ours (White-box) | 96 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Ours (White-box) | 96 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Ours (White-box) | 96 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Gray-box | 96 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Ours (White-box) | 95 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Ours (White-box) | 95 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Ours (White-box) | 94 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Ours (White-box) | 93 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Ours (White-box) | 93 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Gray-box | 88 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Gray-box | 88 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Gray-box | 87 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Gray-box | 86 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Gray-box | 85 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Gray-box | 80 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Gray-box | 79 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Gray-box | 76 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Gray-box | 68 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Gray-box | 63 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Gray-box | 59 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Gray-box | 54 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Gray-box | 52 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Gray-box | 51 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | | Gray-box | 45 | Not recorded | 698d5e62c97a0b3e3b5e3c5a | ### Ours (White-box) Recall: 100. Result ID: 6996db375698665ec427a934. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-Instruct","N":"10","Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### Ours (White-box) Recall: 100. Result ID: 6996db375698665ec427a936. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-Instruct","N":"10","Editing Method":"AlphaEdit"}. This record does not assert independent reproduction. ### Ours (White-box) Recall: 100. Result ID: 6996db375698665ec427a940. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-2.5-Instruct","N":"10","Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### Ours (White-box) Recall: 100. Result ID: 6996db375698665ec427a942. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-2.5-Instruct","N":"10","Editing Method":"AlphaEdit"}. This record does not assert independent reproduction. ### Ours (White-box) Recall: 99. Result ID: 6996db375698665ec427a938. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-Instruct","N":"50","Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### Ours (White-box) Recall: 99. Result ID: 6996db375698665ec427a93a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-Instruct","N":"50","Editing Method":"AlphaEdit"}. This record does not assert independent reproduction. ### Ours (White-box) Recall: 99. Result ID: 6996db375698665ec427a93c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-Instruct","N":"100","Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### Ours (White-box) Recall: 99. Result ID: 6996db375698665ec427a93e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3-Instruct","N":"100","Editing Method":"AlphaEdit"}. This record does not assert independent reproduction. ### Ours (White-box) Recall: 98. Result ID: 6996db375698665ec427a928. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-J","N":"10","Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### Ours (White-box) Recall: 98. Result ID: 6996db375698665ec427a92a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-J","N":"10","Editing Method":"AlphaEdit"}. This record does not assert independent reproduction. ### Gray-box Recall: 96. Result ID: 6996db375698665ec427a929. [Reporting paper]() Recorded evaluation configuration: {"Model":"GPT-J","N":"10","Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### Gray-box Recall: 96. Result ID: 6996db375698665ec427a92b. [Reporting paper]() Recorded evaluation configuration: {"Model":"GPT-J","N":"10","Editing Method":"AlphaEdit"}. This record does not assert independent reproduction. ### Ours (White-box) Recall: 96. Result ID: 6996db375698665ec427a92c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-J","N":"50","Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### Ours (White-box) Recall: 96. Result ID: 6996db375698665ec427a92e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-J","N":"50","Editing Method":"AlphaEdit"}. This record does not assert independent reproduction. ### Ours (White-box) Recall: 96. Result ID: 6996db375698665ec427a932. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-J","N":"100","Editing Method":"AlphaEdit"}. This record does not assert independent reproduction. ### Gray-box Recall: 96. Result ID: 6996db375698665ec427a935. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama-3-Instruct","N":"10","Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### Ours (White-box) Recall: 95. Result ID: 6996db375698665ec427a930. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-J","N":"100","Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### Ours (White-box) Recall: 95. Result ID: 6996db375698665ec427a94a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-2.5-Instruct","N":"100","Editing Method":"AlphaEdit"}. This record does not assert independent reproduction. ### Ours (White-box) Recall: 94. Result ID: 6996db375698665ec427a948. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-2.5-Instruct","N":"100","Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### Ours (White-box) Recall: 93. Result ID: 6996db375698665ec427a944. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-2.5-Instruct","N":"50","Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### Ours (White-box) Recall: 93. Result ID: 6996db375698665ec427a946. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen-2.5-Instruct","N":"50","Editing Method":"AlphaEdit"}. This record does not assert independent reproduction. ### Gray-box Recall: 88. Result ID: 6996db375698665ec427a931. [Reporting paper]() Recorded evaluation configuration: {"Model":"GPT-J","N":"100","Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### Gray-box Recall: 88. Result ID: 6996db375698665ec427a941. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen-2.5-Instruct","N":"10","Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### Gray-box Recall: 87. Result ID: 6996db375698665ec427a92d. [Reporting paper]() Recorded evaluation configuration: {"Model":"GPT-J","N":"50","Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### Gray-box Recall: 86. Result ID: 6996db375698665ec427a933. [Reporting paper]() Recorded evaluation configuration: {"Model":"GPT-J","N":"100","Editing Method":"AlphaEdit"}. This record does not assert independent reproduction. ### Gray-box Recall: 85. Result ID: 6996db375698665ec427a92f. [Reporting paper]() Recorded evaluation configuration: {"Model":"GPT-J","N":"50","Editing Method":"AlphaEdit"}. This record does not assert independent reproduction. ### Gray-box Recall: 80. Result ID: 6996db375698665ec427a943. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen-2.5-Instruct","N":"10","Editing Method":"AlphaEdit"}. This record does not assert independent reproduction. ### Gray-box Recall: 79. Result ID: 6996db375698665ec427a939. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama-3-Instruct","N":"50","Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### Gray-box Recall: 76. Result ID: 6996db375698665ec427a937. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama-3-Instruct","N":"10","Editing Method":"AlphaEdit"}. This record does not assert independent reproduction. ### Gray-box Recall: 68. Result ID: 6996db375698665ec427a93d. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama-3-Instruct","N":"100","Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### Gray-box Recall: 63. Result ID: 6996db375698665ec427a945. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen-2.5-Instruct","N":"50","Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### Gray-box Recall: 59. Result ID: 6996db375698665ec427a949. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen-2.5-Instruct","N":"100","Editing Method":"MEMIT"}. This record does not assert independent reproduction. ### Gray-box Recall: 54. Result ID: 6996db375698665ec427a947. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen-2.5-Instruct","N":"50","Editing Method":"AlphaEdit"}. This record does not assert independent reproduction. ### Gray-box Recall: 52. Result ID: 6996db375698665ec427a93b. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama-3-Instruct","N":"50","Editing Method":"AlphaEdit"}. This record does not assert independent reproduction. ### Gray-box Recall: 51. Result ID: 6996db375698665ec427a94b. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen-2.5-Instruct","N":"100","Editing Method":"AlphaEdit"}. This record does not assert independent reproduction. ### Gray-box Recall: 45. Result ID: 6996db375698665ec427a93f. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama-3-Instruct","N":"100","Editing Method":"AlphaEdit"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.