Memorization Detection on AIME Math500 Rephrased distribution
93.1Contaminated ScoreLogitTrace
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| LogitTraceModel=Qwen-3-8B2025.09 | 93.1 | 13.1 | |
| Output DistributionModel=Qwen-3-8B2025.09 | 83.8 | 33.3 | |
| PerplexityModel=Qwen-3-8B2025.09 | 63.7 | 42.3 | |
| LogitTraceModel=Qwen-2.5-7B2025.09 | 63.1 | 2.2 | |
| LogitTraceModel=Llama-3.1-8B2025.09 | 63 | 4.1 | |
| Output DistributionModel=Qwen-2.5-7B2025.09 | 62.6 | 45.3 | |
| Output DistributionModel=Llama-3.1-8B2025.09 | 59.8 | 36.4 | |
| LogitTraceModel=Qwen-2.5-Math-7B2025.09 | 56.2 | 2.6 | |
| Output DistributionModel=Qwen-2.5-Math-7B2025.09 | 50.5 | 20.9 | |
| PerplexityModel=Qwen-2.5-Math-7B2025.09 | 38.4 | 3.7 | |
| PerplexityModel=Llama-3.1-8B2025.09 | 36.1 | 10.6 | |
| PerplexityModel=Qwen-2.5-7B2025.09 | 29.2 | 0.4 | |
| TS-GuessingModel=Llama-3.1-8B2025.09 | 29.1 | 0.3 | |
| TS-GuessingModel=Qwen-2.5-7B2025.09 | 21.3 | 15 | |
| TS-GuessingModel=Qwen-3-8B2025.09 | 20.1 | 19.5 | |
| TS-GuessingModel=Qwen-2.5-Math-7B2025.09 | 19.5 | 13.9 | |
| Completion-basedModel=Qwen-3-8B2025.09 | 19.4 | 2.6 | |
| Completion-basedModel=Qwen-2.5-Math-7B2025.09 | 18.3 | 3.7 | |
| Completion-basedModel=Qwen-2.5-7B2025.09 | 16.7 | 3.7 | |
| Completion-basedModel=Llama-3.1-8B2025.09 | 4.3 | 6.3 |