List your company
ResearchFormerly WizWand

Memorization Detection on AIME Math500 Perturbed distribution

64.4Contaminated ScoreOutput Distribution
-2.26415.04332.3549.657Sep 25, 2025
Updated 5mo ago

Evaluation results

MethodLinks
Output DistributionModel=Qwen-3-8B2025.0964.432.9
LogitTraceModel=Qwen-3-8B2025.0964.413.9
Output DistributionModel=Llama-3.1-8B2025.0962.834.8
LogitTraceModel=Llama-3.1-8B2025.09614.5
LogitTraceModel=Qwen-2.5-7B2025.0960.11.5
Output DistributionModel=Qwen-2.5-7B2025.0958.939.3
PerplexityModel=Qwen-3-8B2025.0957.536.7
LogitTraceModel=Qwen-2.5-Math-7B2025.0951.71.9
Output DistributionModel=Qwen-2.5-Math-7B2025.0950.526.6
PerplexityModel=Qwen-2.5-Math-7B2025.0926.73
PerplexityModel=Llama-3.1-8B2025.0924.98.1
PerplexityModel=Qwen-2.5-7B2025.09190
TS-GuessingModel=Qwen-3-8B2025.091917.6
Completion-basedModel=Qwen-3-8B2025.0918.81.9
TS-GuessingModel=Qwen-2.5-7B2025.0917.714.6
TS-GuessingModel=Qwen-2.5-Math-7B2025.0917.112.4
Completion-basedModel=Qwen-2.5-7B2025.0915.83
Completion-basedModel=Qwen-2.5-Math-7B2025.0915.41.9
Completion-basedModel=Llama-3.1-8B2025.092.94.8
TS-GuessingModel=Llama-3.1-8B2025.090.30.2