Memorization Detection on AIME Math500 Translated distribution
84.4Contaminated ScorePerplexity
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| PerplexityModel=Qwen-3-8B2025.09 | 84.4 | 64.8 | |
| LogitTraceModel=Qwen-3-8B2025.09 | 78.1 | 10.1 | |
| LogitTraceModel=Qwen-2.5-Math-7B2025.09 | 64.2 | 9.4 | |
| LogitTraceModel=Llama-3.1-8B2025.09 | 46.4 | 4.5 | |
| Output DistributionModel=Llama-3.1-8B2025.09 | 39.3 | 20.2 | |
| Output DistributionModel=Qwen-3-8B2025.09 | 33.1 | 19.1 | |
| PerplexityModel=Llama-3.1-8B2025.09 | 31.5 | 8.7 | |
| PerplexityModel=Qwen-2.5-Math-7B2025.09 | 30.9 | 3 | |
| Output DistributionModel=Qwen-2.5-7B2025.09 | 26.4 | 16.5 | |
| LogitTraceModel=Qwen-2.5-7B2025.09 | 20 | 1.5 | |
| Output DistributionModel=Qwen-2.5-Math-7B2025.09 | 17.8 | 11.2 | |
| Completion-basedModel=Qwen-3-8B2025.09 | 13.8 | 2.6 | |
| Completion-basedModel=Qwen-2.5-Math-7B2025.09 | 13.6 | 2.6 | |
| Completion-basedModel=Qwen-2.5-7B2025.09 | 12.2 | 2.6 | |
| PerplexityModel=Qwen-2.5-7B2025.09 | 10.8 | 0.7 | |
| TS-GuessingModel=Qwen-3-8B2025.09 | 3.8 | 1.9 | |
| TS-GuessingModel=Qwen-2.5-7B2025.09 | 3.6 | 0 | |
| Completion-basedModel=Llama-3.1-8B2025.09 | 3.2 | 3.4 | |
| TS-GuessingModel=Qwen-2.5-Math-7B2025.09 | 1.5 | 0 | |
| TS-GuessingModel=Llama-3.1-8B2025.09 | 0.3 | 0.2 |