Membership Inference Attack on arXivReasoning Sequence-level
68ACCBlackSpectrum
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BlackSpectrumTarget Model=QwQ-32B, # Q=12026.01 | 68 | 0.736 | 27.8 | |
| BlackSpectrumTarget Model=Claude-sonnet-4, # Q=12026.01 | 67.1 | 0.733 | 18.9 | |
| BlackSpectrumTarget Model=Gemini-2.5-flash, # Q=12026.01 | 66.4 | 0.721 | 18.8 | |
| BlackSpectrumTarget Model=Avg Performance, # Q=12026.01 | 64.5 | 0.694 | 18.3 | |
| DE-COPTarget Model=QwQ-32B, # Q=242026.01 | 62 | 0.65 | 8 | |
| LLM-based judgementTarget Model=QwQ-32B, # Q=22026.01 | 60 | 0.584 | 13.9 | |
| LLM-based judgementTarget Model=Gemini-2.5-flash, # Q=22026.01 | 59.6 | 0.537 | 11.4 | |
| Tr-Consistency (Token)Target Model=Gemini-2.5-flash, # Q=32026.01 | 57.3 | 0.582 | 6.6 | |
| Compression RateTarget Model=Gemini-2.5-flash, # Q=12026.01 | 56.8 | 0.583 | 9.3 | |
| DE-COPTarget Model=Avg Performance, # Q=242026.01 | 56.5 | 0.577 | 6.5 | |
| BlackSpectrumTarget Model=GPT-5-mini, # Q=12026.01 | 56.5 | 0.585 | 7.6 | |
| Tr-Consistency (Char)Target Model=Gemini-2.5-flash, # Q=32026.01 | 56.1 | 0.571 | 8.8 | |
| LLM-based judgementTarget Model=Avg Performance, # Q=22026.01 | 55.8 | 0.499 | 9.4 | |
| CDDTarget Model=Claude-sonnet-4, # Q=52026.01 | 55.3 | 0.555 | 11.4 | |
| Tr-Consistency (Token)Target Model=GPT-5-mini, # Q=32026.01 | 55.3 | 0.556 | 3.1 | |
| SaMIATarget Model=QwQ-32B, # Q=52026.01 | 54.9 | 0.551 | 5.6 | |
| Thinking TokenTarget Model=Gemini-2.5-flash, # Q=12026.01 | 54.9 | 0.564 | 6 | |
| Tr-Consistency (Token)Target Model=Claude-sonnet-4, # Q=32026.01 | 54.5 | 0.544 | 7.3 | |
| Tr-Consistency (Token)Target Model=Avg Performance, # Q=32026.01 | 54.5 | 0.537 | 4.7 | |
| Tr-Consistency (Char)Target Model=GPT-5-mini, # Q=32026.01 | 54.1 | 0.541 | 4.2 | |
| Compression RateTarget Model=Claude-sonnet-4, # Q=12026.01 | 53.4 | 0.54 | 8.7 | |
| Tr-Consistency (Char)Target Model=Avg Performance, # Q=32026.01 | 53.2 | 0.511 | 4.9 | |
| Compression RateTarget Model=Avg Performance, # Q=12026.01 | 52.8 | 0.48 | 6 | |
| Tr-Consistency (Char)Target Model=Claude-sonnet-4, # Q=32026.01 | 52.5 | 0.522 | 5.1 | |
| SaMIATarget Model=Avg Performance, # Q=52026.01 | 52.4 | 0.497 | 3.9 | |
| SaMIATarget Model=Claude-sonnet-4, # Q=52026.01 | 52.3 | 0.521 | 6 | |
| SaMIATarget Model=GPT-5-mini, # Q=52026.01 | 52.3 | 0.511 | 3.3 | |
| LLM-based judgementTarget Model=Claude-sonnet-4, # Q=22026.01 | 51.8 | 0.379 | 7.4 | |
| LLM-based judgementTarget Model=GPT-5-mini, # Q=22026.01 | 51.7 | 0.494 | 4.7 | |
| CDDTarget Model=Avg Performance, # Q=52026.01 | 51.6 | 0.471 | 5.9 | |
| Thinking TokenTarget Model=Avg Performance, # Q=12026.01 | 51.6 | 0.481 | 5 | |
| CDDTarget Model=GPT-5-mini, # Q=52026.01 | 51.3 | 0.503 | 5.4 | |
| Tr-Consistency (Token)Target Model=QwQ-32B, # Q=32026.01 | 51 | 0.465 | 1.8 | |
| DE-COPTarget Model=Gemini-2.5-flash, # Q=242026.01 | 50.9 | 0.503 | 5 | |
| Thinking TokenTarget Model=Claude-sonnet-4, # Q=12026.01 | 50.6 | 0.492 | 4.1 | |
| Thinking TokenTarget Model=QwQ-32B, # Q=12026.01 | 50.5 | 0.42 | 5.8 | |
| Compression RateTarget Model=GPT-5-mini, # Q=12026.01 | 50.5 | 0.399 | 2.2 | |
| Thinking TokenTarget Model=GPT-5-mini, # Q=12026.01 | 50.3 | 0.448 | 4 | |
| Compression RateTarget Model=QwQ-32B, # Q=12026.01 | 50.3 | 0.397 | 3.7 | |
| Tr-Consistency (Char)Target Model=QwQ-32B, # Q=32026.01 | 50.1 | 0.409 | 1.4 | |
| SaMIATarget Model=Gemini-2.5-flash, # Q=52026.01 | 50 | 0.405 | 0.5 | |
| CDDTarget Model=QwQ-32B, # Q=52026.01 | 50 | 0.451 | 4 | |
| CDDTarget Model=Gemini-2.5-flash, # Q=52026.01 | 50 | 0.376 | 2.6 |