# Membership Inference Attack on PubMed Pythia Source: [Membership Inference Attack on PubMed Pythia]() Collection index: [llms.txt]() Data updated: 2026-02-26T06:00:24.052Z Evaluation of Membership Inference Attack performance on the PubMed Pythia dataset, measured using metrics such as ROC AUC. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Membership Inference Attack | | Dataset (including any stated split) | PubMed Pythia | | Metric | ROC AUC | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 36 of 36 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [ROC AUC](): Higher is better; unit: PERCENT. ## Recorded results | Method | ROC AUC | Result date | Report paper | | --- | --- | --- | --- | | AttenMIA (Perturbed) | 97 | Not recorded | 697846c1a2b28e761d87c68e | | AttenMIA (Perturbed) | 96 | Not recorded | 697846c1a2b28e761d87c68e | | AttenMIA (Perturbed) | 95 | Not recorded | 697846c1a2b28e761d87c68e | | AttenMIA (Transitional) | 85 | Not recorded | 697846c1a2b28e761d87c68e | | AttenMIA (Transitional) | 85 | Not recorded | 697846c1a2b28e761d87c68e | | AttenMIA (Transitional) | 84 | Not recorded | 697846c1a2b28e761d87c68e | | zlib attack | 72 | Not recorded | 697846c1a2b28e761d87c68e | | zlib attack | 71 | Not recorded | 697846c1a2b28e761d87c68e | | PPL attack | 70 | Not recorded | 697846c1a2b28e761d87c68e | | zlib attack | 70 | Not recorded | 697846c1a2b28e761d87c68e | | PPL attack | 69 | Not recorded | 697846c1a2b28e761d87c68e | | neighborhood attack | 69 | Not recorded | 697846c1a2b28e761d87c68e | | neighborhood attack | 69 | Not recorded | 697846c1a2b28e761d87c68e | | PPL attack | 68 | Not recorded | 697846c1a2b28e761d87c68e | | neighborhood attack | 68 | Not recorded | 697846c1a2b28e761d87c68e | | PETAL | 66 | Not recorded | 697846c1a2b28e761d87c68e | | reference attack | 65 | Not recorded | 697846c1a2b28e761d87c68e | | reference attack | 65 | Not recorded | 697846c1a2b28e761d87c68e | | PETAL | 65 | Not recorded | 697846c1a2b28e761d87c68e | | PETAL | 65 | Not recorded | 697846c1a2b28e761d87c68e | | reference attack | 64 | Not recorded | 697846c1a2b28e761d87c68e | | RS attack | 56 | Not recorded | 697846c1a2b28e761d87c68e | | BT attack | 56 | Not recorded | 697846c1a2b28e761d87c68e | | MIN-K% PROB | 55 | Not recorded | 697846c1a2b28e761d87c68e | | WS attack | 54 | Not recorded | 697846c1a2b28e761d87c68e | | WS attack | 54 | Not recorded | 697846c1a2b28e761d87c68e | | RS attack | 54 | Not recorded | 697846c1a2b28e761d87c68e | | RS attack | 54 | Not recorded | 697846c1a2b28e761d87c68e | | MIN-K% PROB | 53 | Not recorded | 697846c1a2b28e761d87c68e | | MIN-K% PROB | 53 | Not recorded | 697846c1a2b28e761d87c68e | | BT attack | 53 | Not recorded | 697846c1a2b28e761d87c68e | | WS attack | 52 | Not recorded | 697846c1a2b28e761d87c68e | | BT attack | 52 | Not recorded | 697846c1a2b28e761d87c68e | | RECALL | 52 | Not recorded | 697846c1a2b28e761d87c68e | | RECALL | 51 | Not recorded | 697846c1a2b28e761d87c68e | | RECALL | 50 | Not recorded | 697846c1a2b28e761d87c68e | ### AttenMIA (Perturbed) ROC AUC: 97. Result ID: 699918bb5698665ec47f1625. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"6.9b"}. This record does not assert independent reproduction. ### AttenMIA (Perturbed) ROC AUC: 96. Result ID: 699918bb5698665ec47f1624. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"2.8b"}. This record does not assert independent reproduction. ### AttenMIA (Perturbed) ROC AUC: 95. Result ID: 699918bb5698665ec47f1623. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"1.4b"}. This record does not assert independent reproduction. ### AttenMIA (Transitional) ROC AUC: 85. Result ID: 699918bb5698665ec47f1620. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"1.4b"}. This record does not assert independent reproduction. ### AttenMIA (Transitional) ROC AUC: 85. Result ID: 699918bb5698665ec47f1621. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"2.8b"}. This record does not assert independent reproduction. ### AttenMIA (Transitional) ROC AUC: 84. Result ID: 699918bb5698665ec47f1622. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"6.9b"}. This record does not assert independent reproduction. ### zlib attack ROC AUC: 72. Result ID: 699918bb5698665ec47f160a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"6.9b"}. This record does not assert independent reproduction. ### zlib attack ROC AUC: 71. Result ID: 699918bb5698665ec47f1608. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"1.4b"}. This record does not assert independent reproduction. ### PPL attack ROC AUC: 70. Result ID: 699918bb5698665ec47f1604. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"6.9b"}. This record does not assert independent reproduction. ### zlib attack ROC AUC: 70. Result ID: 699918bb5698665ec47f1609. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"2.8b"}. This record does not assert independent reproduction. ### PPL attack ROC AUC: 69. Result ID: 699918bb5698665ec47f1602. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"1.4b"}. This record does not assert independent reproduction. ### neighborhood attack ROC AUC: 69. Result ID: 699918bb5698665ec47f160b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model size":"1.4b"}. This record does not assert independent reproduction. ### neighborhood attack ROC AUC: 69. Result ID: 699918bb5698665ec47f160d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model size":"6.9b"}. This record does not assert independent reproduction. ### PPL attack ROC AUC: 68. Result ID: 699918bb5698665ec47f1603. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"2.8b"}. This record does not assert independent reproduction. ### neighborhood attack ROC AUC: 68. Result ID: 699918bb5698665ec47f160c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model size":"2.8b"}. This record does not assert independent reproduction. ### PETAL ROC AUC: 66. Result ID: 699918bb5698665ec47f161c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"6.9b"}. This record does not assert independent reproduction. ### reference attack ROC AUC: 65. Result ID: 699918bb5698665ec47f1605. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"1.4b"}. This record does not assert independent reproduction. ### reference attack ROC AUC: 65. Result ID: 699918bb5698665ec47f1607. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"6.9b"}. This record does not assert independent reproduction. ### PETAL ROC AUC: 65. Result ID: 699918bb5698665ec47f161a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"1.4b"}. This record does not assert independent reproduction. ### PETAL ROC AUC: 65. Result ID: 699918bb5698665ec47f161b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"2.8b"}. This record does not assert independent reproduction. ### reference attack ROC AUC: 64. Result ID: 699918bb5698665ec47f1606. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"2.8b"}. This record does not assert independent reproduction. ### RS attack ROC AUC: 56. Result ID: 699918bb5698665ec47f1616. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"6.9b"}. This record does not assert independent reproduction. ### BT attack ROC AUC: 56. Result ID: 699918bb5698665ec47f1619. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"6.9b"}. This record does not assert independent reproduction. ### MIN-K% PROB ROC AUC: 55. Result ID: 699918bb5698665ec47f1610. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"6.9b"}. This record does not assert independent reproduction. ### WS attack ROC AUC: 54. Result ID: 699918bb5698665ec47f1612. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"2.8b"}. This record does not assert independent reproduction. ### WS attack ROC AUC: 54. Result ID: 699918bb5698665ec47f1613. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"6.9b"}. This record does not assert independent reproduction. ### RS attack ROC AUC: 54. Result ID: 699918bb5698665ec47f1614. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"1.4b"}. This record does not assert independent reproduction. ### RS attack ROC AUC: 54. Result ID: 699918bb5698665ec47f1615. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"2.8b"}. This record does not assert independent reproduction. ### MIN-K% PROB ROC AUC: 53. Result ID: 699918bb5698665ec47f160e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"1.4b"}. This record does not assert independent reproduction. ### MIN-K% PROB ROC AUC: 53. Result ID: 699918bb5698665ec47f160f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"2.8b"}. This record does not assert independent reproduction. ### BT attack ROC AUC: 53. Result ID: 699918bb5698665ec47f1618. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"2.8b"}. This record does not assert independent reproduction. ### WS attack ROC AUC: 52. Result ID: 699918bb5698665ec47f1611. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"1.4b"}. This record does not assert independent reproduction. ### BT attack ROC AUC: 52. Result ID: 699918bb5698665ec47f1617. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"1.4b"}. This record does not assert independent reproduction. ### RECALL ROC AUC: 52. Result ID: 699918bb5698665ec47f161d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"1.4b"}. This record does not assert independent reproduction. ### RECALL ROC AUC: 51. Result ID: 699918bb5698665ec47f161f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"6.9b"}. This record does not assert independent reproduction. ### RECALL ROC AUC: 50. Result ID: 699918bb5698665ec47f161e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model size":"2.8b"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.