List your company
ResearchFormerly WizWand

Memorization Detection on AIME Math500 Rephrased distribution

93.1Contaminated ScoreLogitTrace
0.74824.72448.772.676Sep 25, 2025
Updated 5mo ago

Evaluation results

MethodLinks
LogitTraceModel=Qwen-3-8B2025.0993.113.1
Output DistributionModel=Qwen-3-8B2025.0983.833.3
PerplexityModel=Qwen-3-8B2025.0963.742.3
LogitTraceModel=Qwen-2.5-7B2025.0963.12.2
LogitTraceModel=Llama-3.1-8B2025.09634.1
Output DistributionModel=Qwen-2.5-7B2025.0962.645.3
Output DistributionModel=Llama-3.1-8B2025.0959.836.4
LogitTraceModel=Qwen-2.5-Math-7B2025.0956.22.6
Output DistributionModel=Qwen-2.5-Math-7B2025.0950.520.9
PerplexityModel=Qwen-2.5-Math-7B2025.0938.43.7
PerplexityModel=Llama-3.1-8B2025.0936.110.6
PerplexityModel=Qwen-2.5-7B2025.0929.20.4
TS-GuessingModel=Llama-3.1-8B2025.0929.10.3
TS-GuessingModel=Qwen-2.5-7B2025.0921.315
TS-GuessingModel=Qwen-3-8B2025.0920.119.5
TS-GuessingModel=Qwen-2.5-Math-7B2025.0919.513.9
Completion-basedModel=Qwen-3-8B2025.0919.42.6
Completion-basedModel=Qwen-2.5-Math-7B2025.0918.33.7
Completion-basedModel=Qwen-2.5-7B2025.0916.73.7
Completion-basedModel=Llama-3.1-8B2025.094.36.3