Drift Detection on Controlled (post-cutoff)
0.95AUROCOur probe
Evaluation Results
| Method | Links | |
|---|---|---|
| Our probeModel=Llama-22026.05 | 0.95 | |
| Our probeModel=Llama-3.12026.05 | 0.93 | |
| MLP probe (drift)Model=Mistral2026.05 | 0.92 | |
| MLP probe (drift)Model=Llama-3.12026.05 | 0.92 | |
| Our probeModel=Mistral2026.05 | 0.92 | |
| MLP probe (drift)Model=Llama-22026.05 | 0.89 | |
| Our probeModel=Mean2026.05 | 0.89 | |
| Our probeModel=Qwen-2.52026.05 | 0.85 | |
| Our probeModel=Gemma-2B2026.05 | 0.84 | |
| Our probeModel=Gemma-9B2026.05 | 0.83 | |
| MLP probe (drift)Model=Mean2026.05 | 0.82 | |
| MLP probe (drift)Model=Gemma-2B2026.05 | 0.78 | |
| SEPModel=Gemma-2B2026.05 | 0.75 | |
| MLP probe (drift)Model=Gemma-9B2026.05 | 0.75 | |
| Generation lengthModel=Llama-3.12026.05 | 0.63 | |
| MLP probe (drift)Model=Qwen-2.52026.05 | 0.63 | |
| Scalar ensembleModel=Llama-3.12026.05 | 0.62 | |
| Generation lengthModel=Gemma-9B2026.05 | 0.6 | |
| SAPLMAModel=Llama-3.12026.05 | 0.6 | |
| SAPLMAModel=Qwen-2.52026.05 | 0.58 | |
| Token entropyModel=Llama-3.12026.05 | 0.57 | |
| Token entropyModel=Gemma-2B2026.05 | 0.57 | |
| SEPModel=Mean2026.05 | 0.57 | |
| Sequence entropyModel=Gemma-2B2026.05 | 0.56 | |
| CCSModel=Gemma-9B2026.05 | 0.56 | |
| Token entropyModel=Gemma-9B2026.05 | 0.55 | |
| Token probabilityModel=Llama-3.12026.05 | 0.55 | |
| Sequence log-probModel=Mistral2026.05 | 0.55 | |
| Sequence log-probModel=Gemma-2B2026.05 | 0.55 | |
| SEModel=Gemma-9B2026.05 | 0.55 | |
| SEModel=Gemma-2B2026.05 | 0.55 | |
| SEPModel=Llama-3.12026.05 | 0.55 | |
| SEPModel=Gemma-9B2026.05 | 0.55 | |
| CCSModel=Llama-22026.05 | 0.55 | |
| Token probabilityModel=Gemma-9B2026.05 | 0.54 | |
| Token probabilityModel=Gemma-2B2026.05 | 0.54 | |
| Sequence entropyModel=Mistral2026.05 | 0.54 | |
| Generation lengthModel=Mean2026.05 | 0.54 | |
| SEModel=Llama-3.12026.05 | 0.54 | |
| SEModel=Mean2026.05 | 0.54 | |
| Scalar ensembleModel=Gemma-9B2026.05 | 0.54 | |
| CCSModel=Qwen-2.52026.05 | 0.54 | |
| CCSModel=Gemma-2B2026.05 | 0.54 | |
| CCSModel=Mean2026.05 | 0.54 | |
| Token entropyModel=Mean2026.05 | 0.53 | |
| SEModel=Mistral2026.05 | 0.53 | |
| SEModel=Qwen-2.52026.05 | 0.53 | |
| Scalar ensembleModel=Llama-22026.05 | 0.53 | |
| Scalar ensembleModel=Mean2026.05 | 0.53 | |
| SEPModel=Llama-22026.05 | 0.53 | |
| SEPModel=Mistral2026.05 | 0.53 | |
| SEPModel=Qwen-2.52026.05 | 0.53 | |
| SAPLMAModel=Llama-22026.05 | 0.53 | |
| SAPLMAModel=Mean2026.05 | 0.53 | |
| CCSModel=Llama-3.12026.05 | 0.53 | |
| Token entropyModel=Llama-22026.05 | 0.52 | |
| Token probabilityModel=Mean2026.05 | 0.52 | |
| Top-k marginModel=Gemma-9B2026.05 | 0.52 | |
| Generation lengthModel=Mistral2026.05 | 0.52 | |
| SEModel=Llama-22026.05 | 0.52 | |
| Token probabilityModel=Llama-22026.05 | 0.51 | |
| Sequence log-probModel=Gemma-9B2026.05 | 0.51 | |
| Sequence log-probModel=Mean2026.05 | 0.51 | |
| Sequence entropyModel=Llama-22026.05 | 0.51 | |
| Sequence entropyModel=Mean2026.05 | 0.51 | |
| Top-k marginModel=Llama-22026.05 | 0.51 | |
| Top-k marginModel=Llama-3.12026.05 | 0.51 | |
| SAPLMAModel=Gemma-2B2026.05 | 0.51 | |
| Token entropyModel=Mistral2026.05 | 0.5 | |
| Sequence log-probModel=Llama-22026.05 | 0.5 | |
| Sequence entropyModel=Gemma-9B2026.05 | 0.5 | |
| Generation lengthModel=Llama-22026.05 | 0.5 | |
| Generation lengthModel=Gemma-2B2026.05 | 0.5 | |
| Scalar ensembleModel=Gemma-2B2026.05 | 0.5 | |
| CCSModel=Mistral2026.05 | 0.5 | |
| Token probabilityModel=Mistral2026.05 | 0.49 | |
| Top-k marginModel=Mistral2026.05 | 0.49 | |
| Top-k marginModel=Mean2026.05 | 0.49 | |
| Scalar ensembleModel=Mistral2026.05 | 0.49 | |
| SAPLMAModel=Gemma-9B2026.05 | 0.49 | |
| Token entropyModel=Qwen-2.52026.05 | 0.47 | |
| Token probabilityModel=Qwen-2.52026.05 | 0.47 | |
| Sequence log-probModel=Llama-3.12026.05 | 0.47 | |
| Sequence log-probModel=Qwen-2.52026.05 | 0.47 | |
| Sequence entropyModel=Llama-3.12026.05 | 0.47 | |
| Sequence entropyModel=Qwen-2.52026.05 | 0.47 | |
| Generation lengthModel=Qwen-2.52026.05 | 0.47 | |
| Scalar ensembleModel=Qwen-2.52026.05 | 0.47 | |
| SAPLMAModel=Mistral2026.05 | 0.47 | |
| Top-k marginModel=Qwen-2.52026.05 | 0.46 | |
| Top-k marginModel=Gemma-2B2026.05 | 0.44 |