List your company
ResearchFormerly WizWand

Memorization Detection on AIME Math500 Original distribution

100Contaminated ScoreCompletion-based
58.469.28090.8Sep 25, 2025
Updated 5mo ago

Evaluation results

MethodLinks
Completion-basedModel=Qwen-2.5-7B2025.0910011.6
Completion-basedModel=Qwen-2.5-Math-7B2025.091008.7
Completion-basedModel=Llama-3.1-8B2025.0910014.4
Completion-basedModel=Qwen-3-8B2025.0910014.2
TS-GuessingModel=Llama-3.1-8B2025.0998.217.3
TS-GuessingModel=Qwen-2.5-Math-7B2025.0996.411.4
PerplexityModel=Qwen-2.5-Math-7B2025.0996.13.8
LogitTraceModel=Llama-3.1-8B2025.0995.712.8
LogitTraceModel=Qwen-2.5-7B2025.0995.65.6
PerplexityModel=Llama-3.1-8B2025.0993.711.2
TS-GuessingModel=Qwen-2.5-7B2025.0992.714.5
TS-GuessingModel=Qwen-3-8B2025.0991.321.7
LogitTraceModel=Qwen-2.5-Math-7B2025.0989.45.6
PerplexityModel=Qwen-2.5-7B2025.0987.61.1
LogitTraceModel=Qwen-3-8B2025.0987.514.8
Output DistributionModel=Qwen-3-8B2025.097833
Output DistributionModel=Qwen-2.5-Math-7B2025.0971.319.3
PerplexityModel=Qwen-3-8B2025.097039.7
Output DistributionModel=Qwen-2.5-7B2025.0964.728
Output DistributionModel=Llama-3.1-8B2025.096018