List your company
ResearchFormerly WizWand

OOD Quality Estimation on Mathematical Reasoning OOD (Near-shift)

0.159Kendall TauTV Score
0.030040.063520.0970.13048May 22, 2024
Updated 8mo ago

Evaluation results

MethodLinks
TV ScoreModel=Llama2-7B2024.050.1590.158
TV ScoreModel=GPT2-XL2024.050.1310.146
TV Score w/ DiSmoModel=GPT2-XL2024.050.1230.154
TV Score w/ DiSmoModel=Llama2-7B2024.050.1130.134
PerplexityModel=Llama2-7B2024.050.0740.05
Input EmbeddingModel=GPT2-XL2024.050.0590.098
Max Softmax Prob.Model=GPT2-XL2024.050.0570.057
Max Softmax Prob.Model=Llama2-7B2024.050.0380.026
Output EmbeddingModel=Llama2-7B2024.050.0380.012
Input EmbeddingModel=Llama2-7B2024.050.0360.115
Output EmbeddingModel=GPT2-XL2024.050.0360.029
PerplexityModel=GPT2-XL2024.050.0350.058