Membership Inference Attack on DOLMa corpus
54Coverage (Cov.)N-GRAM COVERAGE ATTACK
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| N-GRAM COVERAGE ATTACKModel=OLMo-1B2025.08 | 54 | 54 | — | — | — | — | — | — | — | — | |
| N-GRAM COVERAGE ATTACKModel=OLMo-7B2025.08 | 54 | 54 | — | — | — | — | — | — | — | — | |
| N-GRAM COVERAGE ATTACKModel=OLMo-7B-SFT2025.08 | 52 | 52 | — | — | — | — | — | — | — | — | |
| N-GRAM COVERAGE ATTACKModel=OLMo-7B-Instruct2025.08 | 52 | 52 | — | — | — | — | — | — | — | — | |
| DE-COPModel=OLMo-1B2025.08 | — | — | — | — | 0.49 | — | — | — | — | — | |
| DE-COPModel=OLMo-7B2025.08 | — | — | — | — | 0.5 | — | — | — | — | — | |
| DE-COPModel=OLMo-7B-SFT2025.08 | — | — | — | — | 0.5 | — | — | — | — | — | |
| DE-COPModel=OLMo-7B-Instruct2025.08 | — | — | — | — | 0.5 | — | — | — | — | — | |
| LCScModel=OLMo-1B2025.08 | — | — | 0.51 | — | — | — | — | — | — | — | |
| LCScModel=OLMo-7B2025.08 | — | — | 0.54 | — | — | — | — | — | — | — | |
| LCScModel=OLMo-7B-SFT2025.08 | — | — | 0.53 | — | — | — | — | — | — | — | |
| LCScModel=OLMo-7B-Instruct2025.08 | — | — | 0.52 | — | — | — | — | — | — | — | |
| LCSwModel=OLMo-1B2025.08 | — | — | — | 0.5 | — | — | — | — | — | — | |
| LCSwModel=OLMo-7B2025.08 | — | — | — | 0.51 | — | — | — | — | — | — | |
| LCSwModel=OLMo-7B-SFT2025.08 | — | — | — | 0.51 | — | — | — | — | — | — | |
| LCSwModel=OLMo-7B-Instruct2025.08 | — | — | — | 0.51 | — | — | — | — | — | — | |
| LossModel=OLMo-1B2025.08 | — | — | — | — | — | 0.47 | — | — | — | — | |
| LossModel=OLMo-7B2025.08 | — | — | — | — | — | 0.47 | — | — | — | — | |
| LossModel=OLMo-7B-SFT2025.08 | — | — | — | — | — | 0.47 | — | — | — | — | |
| LossModel=OLMo-7B-Instruct2025.08 | — | — | — | — | — | 0.47 | — | — | — | — | |
| Min-K%Model=OLMo-1B2025.08 | — | — | — | — | — | — | — | — | 0.45 | — | |
| Min-K%Model=OLMo-7B2025.08 | — | — | — | — | — | — | — | — | 0.46 | — | |
| Min-K%Model=OLMo-7B-SFT2025.08 | — | — | — | — | — | — | — | — | 0.46 | — | |
| Min-K%Model=OLMo-7B-Instruct2025.08 | — | — | — | — | — | — | — | — | 0.46 | — | |
| RandomModel=OLMo-7B2025.08 | — | — | — | — | — | — | — | — | — | 0.49 | |
| Reference LossModel=OLMo-7B2025.08 | — | — | — | — | — | — | 0.53 | — | — | — | |
| Reference LossModel=OLMo-7B-SFT2025.08 | — | — | — | — | — | — | 0.53 | — | — | — | |
| Reference LossModel=OLMo-7B-Instruct2025.08 | — | — | — | — | — | — | 0.52 | — | — | — | |
| zlibModel=OLMo-1B2025.08 | — | — | — | — | — | — | — | 0.51 | — | — | |
| zlibModel=OLMo-7B2025.08 | — | — | — | — | — | — | — | 0.51 | — | — | |
| zlibModel=OLMo-7B-SFT2025.08 | — | — | — | — | — | — | — | 0.51 | — | — | |
| zlibModel=OLMo-7B-Instruct2025.08 | — | — | — | — | — | — | — | 0.51 | — | — |