Leakage Detection on APPS
90.53AccuracySrDetection
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SrDetectionEpochs=5, Target Model=Qwen2.5-7B, Evaluation Setting=Gray-box2026.06 | 90.53 | 91.56 | 90.49 | 90.47 | |
| SrDetectionEpochs=3, Target Model=Qwen2.5-7B, Evaluation Setting=Gray-box2026.06 | 88.59 | 89.7 | 88.55 | 88.5 | |
| SrDetectionEpochs=5, Target Model=Llama3.1-8B, Evaluation Setting=Gray-box2026.06 | 84.47 | 87.96 | 84.41 | 84.09 | |
| SrDetectionEpochs=3, Target Model=Llama3.1-8B, Evaluation Setting=Gray-box2026.06 | 82.43 | 86.42 | 82.36 | 81.92 | |
| Min-K%++ ProbEpochs=5, Target Model=Qwen2.5-7B, Evaluation Setting=Gray-box2026.06 | 78.55 | 84.36 | 78.45 | 77.56 | |
| SrDetectionEpochs=1, Target Model=Qwen2.5-7B, Evaluation Setting=Gray-box2026.06 | 74.95 | 75 | 75.81 | 74.76 | |
| Min-K%++ ProbEpochs=3, Target Model=Qwen2.5-7B, Evaluation Setting=Gray-box2026.06 | 70.83 | 80.62 | 70.7 | 68.22 | |
| SrDetectionEpochs=1, Target Model=Llama3.1-8B, Evaluation Setting=Gray-box2026.06 | 70.01 | 70.1 | 70.03 | 69.99 | |
| LowercaseEpochs=1, Target Model=Llama3.1-8B, Evaluation Setting=Gray-box2026.06 | 66.95 | 69.16 | 66.88 | 65.91 | |
| Min-K%++ ProbEpochs=5, Target Model=Llama3.1-8B, Evaluation Setting=Gray-box2026.06 | 60.29 | 77.49 | 60.11 | 52.73 | |
| LowercaseEpochs=5, Target Model=Qwen2.5-7B, Evaluation Setting=Gray-box2026.06 | 59.8 | 77.06 | 59.61 | 51.97 | |
| LowercaseEpochs=1, Target Model=Qwen2.5-7B, Evaluation Setting=Gray-box2026.06 | 59.09 | 69.93 | 58.92 | 52.44 | |
| LowercaseEpochs=5, Target Model=Llama3.1-8B, Evaluation Setting=Gray-box2026.06 | 57.91 | 77.06 | 57.72 | 48.63 | |
| Min-K%++ ProbEpochs=1, Target Model=Llama3.1-8B, Evaluation Setting=Gray-box2026.06 | 55.94 | 69.67 | 55.75 | 46.34 | |
| Min-K%++ ProbEpochs=3, Target Model=Llama3.1-8B, Evaluation Setting=Gray-box2026.06 | 55.21 | 76.21 | 55.01 | 43.72 | |
| DC-PDDEpochs=3, Target Model=Qwen2.5-7B, Evaluation Setting=Gray-box2026.06 | 54.88 | 75.84 | 54.66 | 43.1 | |
| DC-PDDEpochs=5, Target Model=Qwen2.5-7B, Evaluation Setting=Gray-box2026.06 | 54.69 | 75.27 | 54.47 | 42.79 | |
| DC-PDDEpochs=5, Target Model=Llama3.1-8B, Evaluation Setting=Gray-box2026.06 | 54.56 | 75.49 | 54.35 | 42.52 | |
| Min-K%++ ProbEpochs=1, Target Model=Qwen2.5-7B, Evaluation Setting=Gray-box2026.06 | 54.31 | 64.03 | 54.11 | 44.36 | |
| LowercaseEpochs=3, Target Model=Llama3.1-8B, Evaluation Setting=Gray-box2026.06 | 53.94 | 75.78 | 53.73 | 41.26 | |
| DC-PDDEpochs=1, Target Model=Llama3.1-8B, Evaluation Setting=Gray-box2026.06 | 53.54 | 69.8 | 53.33 | 41.18 | |
| DC-PDDEpochs=3, Target Model=Llama3.1-8B, Evaluation Setting=Gray-box2026.06 | 52.4 | 74.67 | 52.19 | 38.18 | |
| LowercaseEpochs=3, Target Model=Qwen2.5-7B, Evaluation Setting=Gray-box2026.06 | 52.27 | 73.51 | 52.05 | 37.98 | |
| Min-K% ProbEpochs=5, Target Model=Llama3.1-8B, Evaluation Setting=Gray-box2026.06 | 52.2 | 75.06 | 51.98 | 37.72 | |
| DC-PDDEpochs=1, Target Model=Qwen2.5-7B, Evaluation Setting=Gray-box2026.06 | 51.47 | 70.75 | 51.24 | 36.36 | |
| Min-K% ProbEpochs=5, Target Model=Qwen2.5-7B, Evaluation Setting=Gray-box2026.06 | 51.37 | 75.41 | 51.14 | 35.93 | |
| Min-K% ProbEpochs=3, Target Model=Llama3.1-8B, Evaluation Setting=Gray-box2026.06 | 50.93 | 75.29 | 50.71 | 34.98 | |
| Min-K% ProbEpochs=3, Target Model=Qwen2.5-7B, Evaluation Setting=Gray-box2026.06 | 50.9 | 75.29 | 50.67 | 34.9 | |
| PPLEpochs=1, Target Model=Qwen2.5-7B, Evaluation Setting=Gray-box2026.06 | 50.85 | 71.94 | 50.62 | 34.88 | |
| PPLEpochs=5, Target Model=Llama3.1-8B, Evaluation Setting=Gray-box2026.06 | 50.66 | 75.22 | 50.43 | 34.39 | |
| Min-K% ProbEpochs=1, Target Model=Llama3.1-8B, Evaluation Setting=Gray-box2026.06 | 50.57 | 68.06 | 50.34 | 34.31 | |
| Min-K% ProbEpochs=1, Target Model=Qwen2.5-7B, Evaluation Setting=Gray-box2026.06 | 50.43 | 66.83 | 50.19 | 33.94 | |
| PPLEpochs=5, Target Model=Qwen2.5-7B, Evaluation Setting=Gray-box2026.06 | 50.43 | 75.17 | 50.19 | 33.86 | |
| PPLEpochs=3, Target Model=Llama3.1-8B, Evaluation Setting=Gray-box2026.06 | 50.39 | 75.15 | 50.16 | 33.79 | |
| PPLEpochs=3, Target Model=Qwen2.5-7B, Evaluation Setting=Gray-box2026.06 | 50.38 | 75.15 | 50.14 | 33.75 | |
| PPLEpochs=1, Target Model=Llama3.1-8B, Evaluation Setting=Gray-box2026.06 | 50.34 | 64.03 | 50.11 | 33.77 | |
| ZlibEpochs=3, Target Model=Llama3.1-8B, Evaluation Setting=Gray-box2026.06 | 50.29 | 75.13 | 50.07 | 33.59 | |
| ZlibEpochs=3, Target Model=Qwen2.5-7B, Evaluation Setting=Gray-box2026.06 | 50.28 | 75.13 | 50.05 | 33.54 | |
| ZlibEpochs=5, Target Model=Qwen2.5-7B, Evaluation Setting=Gray-box2026.06 | 50.28 | 75.13 | 50.05 | 33.54 | |
| ZlibEpochs=5, Target Model=Llama3.1-8B, Evaluation Setting=Gray-box2026.06 | 50.27 | 75.12 | 50.05 | 33.54 | |
| ZlibEpochs=1, Target Model=Llama3.1-8B, Evaluation Setting=Gray-box2026.06 | 50.25 | 58.45 | 50.02 | 33.52 | |
| ZlibEpochs=1, Target Model=Qwen2.5-7B, Evaluation Setting=Gray-box2026.06 | 50.24 | 25.12 | 50 | 33.44 |