Offline OOD Detection on Mathematical Reasoning Far-shift OOD
96.54AUROCTV Score w/ DiSmo
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| TV Score w/ DiSmoModel=GPT2-XL2024.05 | 96.54 | 9.89 | |
| TV ScoreModel=GPT2-XL2024.05 | 93.47 | 24.1 | |
| TV ScoreModel=Llama2-7B2024.05 | 92.64 | 28.39 | |
| Input EmbeddingModel=GPT2-XL2024.05 | 86.26 | 49.33 | |
| PerplexityModel=GPT2-XL2024.05 | 80.82 | 64.53 | |
| Output EmbeddingModel=GPT2-XL2024.05 | 77.95 | 65.64 | |
| Max Softmax Prob.Model=GPT2-XL2024.05 | 70.54 | 78.29 | |
| Monte-Carlo DropoutModel=GPT2-XL2024.05 | 66.18 | 84.69 | |
| Input EmbeddingModel=Llama2-7B2024.05 | 60.33 | 84.65 | |
| Max Softmax Prob.Model=Llama2-7B2024.05 | 60.14 | 88.91 | |
| PerplexityModel=Llama2-7B2024.05 | 59.35 | 86.09 | |
| TV Score w/ DiSmoModel=Llama2-7B2024.05 | 56.99 | 88.01 | |
| Monte-Carlo DropoutModel=Llama2-7B2024.05 | 52.33 | 91.92 | |
| Output EmbeddingModel=Llama2-7B2024.05 | 44.5 | 86.46 |