Memorization Detection on AIME Math500 Perturbed distribution
64.4Contaminated ScoreOutput Distribution
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Output DistributionModel=Qwen-3-8B2025.09 | 64.4 | 32.9 | |
| LogitTraceModel=Qwen-3-8B2025.09 | 64.4 | 13.9 | |
| Output DistributionModel=Llama-3.1-8B2025.09 | 62.8 | 34.8 | |
| LogitTraceModel=Llama-3.1-8B2025.09 | 61 | 4.5 | |
| LogitTraceModel=Qwen-2.5-7B2025.09 | 60.1 | 1.5 | |
| Output DistributionModel=Qwen-2.5-7B2025.09 | 58.9 | 39.3 | |
| PerplexityModel=Qwen-3-8B2025.09 | 57.5 | 36.7 | |
| LogitTraceModel=Qwen-2.5-Math-7B2025.09 | 51.7 | 1.9 | |
| Output DistributionModel=Qwen-2.5-Math-7B2025.09 | 50.5 | 26.6 | |
| PerplexityModel=Qwen-2.5-Math-7B2025.09 | 26.7 | 3 | |
| PerplexityModel=Llama-3.1-8B2025.09 | 24.9 | 8.1 | |
| PerplexityModel=Qwen-2.5-7B2025.09 | 19 | 0 | |
| TS-GuessingModel=Qwen-3-8B2025.09 | 19 | 17.6 | |
| Completion-basedModel=Qwen-3-8B2025.09 | 18.8 | 1.9 | |
| TS-GuessingModel=Qwen-2.5-7B2025.09 | 17.7 | 14.6 | |
| TS-GuessingModel=Qwen-2.5-Math-7B2025.09 | 17.1 | 12.4 | |
| Completion-basedModel=Qwen-2.5-7B2025.09 | 15.8 | 3 | |
| Completion-basedModel=Qwen-2.5-Math-7B2025.09 | 15.4 | 1.9 | |
| Completion-basedModel=Llama-3.1-8B2025.09 | 2.9 | 4.8 | |
| TS-GuessingModel=Llama-3.1-8B2025.09 | 0.3 | 0.2 |