Memorization Detection on AIME Math500 Original distribution
100Contaminated ScoreCompletion-based
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Completion-basedModel=Qwen-2.5-7B2025.09 | 100 | 11.6 | |
| Completion-basedModel=Qwen-2.5-Math-7B2025.09 | 100 | 8.7 | |
| Completion-basedModel=Llama-3.1-8B2025.09 | 100 | 14.4 | |
| Completion-basedModel=Qwen-3-8B2025.09 | 100 | 14.2 | |
| TS-GuessingModel=Llama-3.1-8B2025.09 | 98.2 | 17.3 | |
| TS-GuessingModel=Qwen-2.5-Math-7B2025.09 | 96.4 | 11.4 | |
| PerplexityModel=Qwen-2.5-Math-7B2025.09 | 96.1 | 3.8 | |
| LogitTraceModel=Llama-3.1-8B2025.09 | 95.7 | 12.8 | |
| LogitTraceModel=Qwen-2.5-7B2025.09 | 95.6 | 5.6 | |
| PerplexityModel=Llama-3.1-8B2025.09 | 93.7 | 11.2 | |
| TS-GuessingModel=Qwen-2.5-7B2025.09 | 92.7 | 14.5 | |
| TS-GuessingModel=Qwen-3-8B2025.09 | 91.3 | 21.7 | |
| LogitTraceModel=Qwen-2.5-Math-7B2025.09 | 89.4 | 5.6 | |
| PerplexityModel=Qwen-2.5-7B2025.09 | 87.6 | 1.1 | |
| LogitTraceModel=Qwen-3-8B2025.09 | 87.5 | 14.8 | |
| Output DistributionModel=Qwen-3-8B2025.09 | 78 | 33 | |
| Output DistributionModel=Qwen-2.5-Math-7B2025.09 | 71.3 | 19.3 | |
| PerplexityModel=Qwen-3-8B2025.09 | 70 | 39.7 | |
| Output DistributionModel=Qwen-2.5-7B2025.09 | 64.7 | 28 | |
| Output DistributionModel=Llama-3.1-8B2025.09 | 60 | 18 |