List your company
ResearchFormerly WizWand

Memorization Detection on AIME Math500 Translated distribution

84.4Contaminated ScorePerplexity
-3.06419.64342.3565.057Sep 25, 2025
Updated 5mo ago

Evaluation results

MethodLinks
PerplexityModel=Qwen-3-8B2025.0984.464.8
LogitTraceModel=Qwen-3-8B2025.0978.110.1
LogitTraceModel=Qwen-2.5-Math-7B2025.0964.29.4
LogitTraceModel=Llama-3.1-8B2025.0946.44.5
Output DistributionModel=Llama-3.1-8B2025.0939.320.2
Output DistributionModel=Qwen-3-8B2025.0933.119.1
PerplexityModel=Llama-3.1-8B2025.0931.58.7
PerplexityModel=Qwen-2.5-Math-7B2025.0930.93
Output DistributionModel=Qwen-2.5-7B2025.0926.416.5
LogitTraceModel=Qwen-2.5-7B2025.09201.5
Output DistributionModel=Qwen-2.5-Math-7B2025.0917.811.2
Completion-basedModel=Qwen-3-8B2025.0913.82.6
Completion-basedModel=Qwen-2.5-Math-7B2025.0913.62.6
Completion-basedModel=Qwen-2.5-7B2025.0912.22.6
PerplexityModel=Qwen-2.5-7B2025.0910.80.7
TS-GuessingModel=Qwen-3-8B2025.093.81.9
TS-GuessingModel=Qwen-2.5-7B2025.093.60
Completion-basedModel=Llama-3.1-8B2025.093.23.4
TS-GuessingModel=Qwen-2.5-Math-7B2025.091.50
TS-GuessingModel=Llama-3.1-8B2025.090.30.2