Membership Auditing on Math Reasoning Tasks
83.5AUCDIBA
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| DIBATarget Algorithm=DAPO, Model Scale=3B2025.11 | 83.5 | 15.4 | |
| DIBATarget Algorithm=DAPO, Model Scale=7B2025.11 | 81.9 | 10.7 | |
| DIBATarget Algorithm=GRPO, Model Scale=7B2025.11 | 74.9 | 8.4 | |
| DIBATarget Algorithm=GRPO, Model Scale=3B2025.11 | 71.3 | 7.3 | |
| EntropyTarget Algorithm=DAPO, Model Scale=7B, Evaluation Setting=Self-generated, Rollouts (N)=82025.11 | 64.5 | 0.8 | |
| EntropyTarget Algorithm=DAPO, Model Scale=3B, Evaluation Setting=Self-generated, Rollouts (N)=82025.11 | 60.6 | 1.1 | |
| EntropyTarget Algorithm=GRPO, Model Scale=7B, Evaluation Setting=Calibrated2025.11 | 59.8 | 0.8 | |
| ZlibTarget Algorithm=GRPO, Model Scale=3B, Evaluation Setting=Self-generated, Rollouts (N)=82025.11 | 58.8 | 0.2 | |
| ZlibTarget Algorithm=DAPO, Model Scale=3B, Evaluation Setting=Self-generated, Rollouts (N)=82025.11 | 58.8 | 0.3 | |
| EntropyTarget Algorithm=GRPO, Model Scale=7B, Evaluation Setting=Static-target2025.11 | 58.8 | 0.5 | |
| EntropyTarget Algorithm=GRPO, Model Scale=3B, Evaluation Setting=Static-target2025.11 | 58.6 | 0.5 | |
| EntropyTarget Algorithm=DAPO, Model Scale=3B, Evaluation Setting=Static-target2025.11 | 58.6 | 0.5 | |
| EntropyTarget Algorithm=DAPO, Model Scale=7B, Evaluation Setting=Static-target2025.11 | 58.4 | 0.2 | |
| ZlibTarget Algorithm=DAPO, Model Scale=7B, Evaluation Setting=Self-generated, Rollouts (N)=82025.11 | 58.3 | 1.6 | |
| LOSSTarget Algorithm=DAPO, Model Scale=7B, Evaluation Setting=Static-target2025.11 | 58.2 | 0 | |
| EntropyTarget Algorithm=DAPO, Model Scale=7B, Evaluation Setting=Calibrated2025.11 | 57.9 | 0.6 | |
| LOSSTarget Algorithm=GRPO, Model Scale=7B, Evaluation Setting=Static-target2025.11 | 57.5 | 0.2 | |
| LOSSTarget Algorithm=DAPO, Model Scale=7B, Evaluation Setting=Calibrated2025.11 | 57.1 | 0.5 | |
| LOSSTarget Algorithm=GRPO, Model Scale=3B, Evaluation Setting=Self-generated, Rollouts (N)=82025.11 | 57.1 | 0.2 | |
| ZlibTarget Algorithm=GRPO, Model Scale=7B, Evaluation Setting=Self-generated, Rollouts (N)=82025.11 | 57.1 | 0.3 | |
| LOSSTarget Algorithm=GRPO, Model Scale=3B, Evaluation Setting=Static-target2025.11 | 56.8 | 0.6 | |
| Min-K%Target Algorithm=DAPO, Model Scale=3B, Evaluation Setting=Calibrated2025.11 | 56.7 | 0.6 | |
| LOSSTarget Algorithm=DAPO, Model Scale=3B, Evaluation Setting=Static-target2025.11 | 56.2 | 0.8 | |
| LOSSTarget Algorithm=GRPO, Model Scale=7B, Evaluation Setting=Self-generated, Rollouts (N)=82025.11 | 55.4 | 0.3 | |
| LOSSTarget Algorithm=GRPO, Model Scale=7B, Evaluation Setting=Calibrated2025.11 | 55.3 | 0.4 | |
| EntropyTarget Algorithm=GRPO, Model Scale=3B, Evaluation Setting=Self-generated, Rollouts (N)=82025.11 | 55.3 | 0.9 | |
| Min-K%Target Algorithm=DAPO, Model Scale=3B, Evaluation Setting=Static-target2025.11 | 54.8 | 0.3 | |
| Min-K%Target Algorithm=GRPO, Model Scale=3B, Evaluation Setting=Calibrated2025.11 | 54.7 | 1.6 | |
| LOSSTarget Algorithm=DAPO, Model Scale=7B, Evaluation Setting=Self-generated, Rollouts (N)=82025.11 | 54.6 | 1.1 | |
| ZlibTarget Algorithm=DAPO, Model Scale=7B, Evaluation Setting=Static-target2025.11 | 54.3 | 0.1 | |
| ZlibTarget Algorithm=GRPO, Model Scale=7B, Evaluation Setting=Static-target2025.11 | 54.1 | 0 | |
| ZlibTarget Algorithm=DAPO, Model Scale=7B, Evaluation Setting=Calibrated2025.11 | 54.1 | 0.3 | |
| ZlibTarget Algorithm=GRPO, Model Scale=3B, Evaluation Setting=Static-target2025.11 | 53.6 | 0.2 | |
| EntropyTarget Algorithm=GRPO, Model Scale=3B, Evaluation Setting=Calibrated2025.11 | 53.3 | 0.6 | |
| EntropyTarget Algorithm=GRPO, Model Scale=7B, Evaluation Setting=Self-generated, Rollouts (N)=82025.11 | 53.3 | 0.7 | |
| ZlibTarget Algorithm=DAPO, Model Scale=3B, Evaluation Setting=Static-target2025.11 | 53 | 0.3 | |
| LOSSTarget Algorithm=DAPO, Model Scale=3B, Evaluation Setting=Calibrated2025.11 | 52.7 | 0.7 | |
| ZlibTarget Algorithm=GRPO, Model Scale=7B, Evaluation Setting=Calibrated2025.11 | 52.7 | 0.1 | |
| Min-K%Target Algorithm=DAPO, Model Scale=7B, Evaluation Setting=Self-generated, Rollouts (N)=82025.11 | 52.6 | 0.4 | |
| ZlibTarget Algorithm=DAPO, Model Scale=3B, Evaluation Setting=Calibrated2025.11 | 52.5 | 1 | |
| Min-K%Target Algorithm=GRPO, Model Scale=3B, Evaluation Setting=Self-generated, Rollouts (N)=82025.11 | 52.5 | 0 | |
| Min-K%Target Algorithm=GRPO, Model Scale=7B, Evaluation Setting=Self-generated, Rollouts (N)=82025.11 | 52.5 | 0.2 | |
| Min-K%Target Algorithm=GRPO, Model Scale=3B, Evaluation Setting=Static-target2025.11 | 52.4 | 0 | |
| Min-K%Target Algorithm=GRPO, Model Scale=7B, Evaluation Setting=Calibrated2025.11 | 52.4 | 0.8 | |
| NeighborTarget Algorithm=GRPO, Model Scale=3B, Evaluation Setting=Static-target2025.11 | 52.3 | 0.5 | |
| Min-K%Target Algorithm=DAPO, Model Scale=3B, Evaluation Setting=Self-generated, Rollouts (N)=82025.11 | 52.2 | 0.5 | |
| Min-K%Target Algorithm=DAPO, Model Scale=7B, Evaluation Setting=Static-target2025.11 | 52.1 | 0.2 | |
| ZlibTarget Algorithm=GRPO, Model Scale=3B, Evaluation Setting=Calibrated2025.11 | 51.9 | 0.4 | |
| Min-K%Target Algorithm=GRPO, Model Scale=7B, Evaluation Setting=Static-target2025.11 | 51.9 | 0.2 | |
| NeighborTarget Algorithm=DAPO, Model Scale=3B, Evaluation Setting=Static-target2025.11 | 51.8 | 0.4 | |
| NeighborTarget Algorithm=DAPO, Model Scale=7B, Evaluation Setting=Static-target2025.11 | 51.8 | 0.6 | |
| Min-K%Target Algorithm=DAPO, Model Scale=7B, Evaluation Setting=Calibrated2025.11 | 51.7 | 0.3 | |
| NeighborTarget Algorithm=GRPO, Model Scale=7B, Evaluation Setting=Static-target2025.11 | 51.4 | 0.2 | |
| LOSSTarget Algorithm=GRPO, Model Scale=3B, Evaluation Setting=Calibrated2025.11 | 51.3 | 0.9 | |
| EntropyTarget Algorithm=DAPO, Model Scale=3B, Evaluation Setting=Calibrated2025.11 | 51.2 | 0.4 | |
| LOSSTarget Algorithm=DAPO, Model Scale=3B, Evaluation Setting=Self-generated, Rollouts (N)=82025.11 | 50.4 | 0.7 |