Natural Language Understanding on AdvGLUE
81.96RTE AccuracyTruthV(argmax)
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| TruthV(argmax)Backbone Model=Qwen3-4B2025.09 | 81.96 | 65.85 | 77.54 | 78.24 | 79.17 | 72.59 | 52.8 | |
| TruthV(argmin)Backbone Model=Qwen3-4B2025.09 | 81.76 | 65.08 | 77.29 | 78.24 | 77.71 | 72.15 | 52.8 | |
| TruthV(argmin)Backbone Model=Gemma-2-2B-it2025.09 | 81.57 | 71.27 | 70.08 | 74.84 | 82.71 | 72.26 | 53.11 | |
| TruthV(argmax)Backbone Model=Gemma-2-2B-it2025.09 | 80.59 | 71.1 | 69.92 | 74.62 | 82.5 | 71.9 | 52.65 | |
| TruthV(argmin)Backbone Model=Llama-3.2-3B-Instruct2025.09 | 78.82 | 64.92 | 62.88 | 50.11 | 82.29 | 65.14 | 51.82 | |
| TruthV(argmax)Backbone Model=Llama-2-7B-Chat2025.09 | 74.9 | 71.61 | 64.75 | 52.42 | 79.38 | 64.02 | 41.06 | |
| TruthV(argmax)Backbone Model=Llama-3.2-3B-Instruct2025.09 | 74.9 | 65.34 | 62.88 | 50.22 | 84.58 | 64.97 | 51.89 | |
| TruthV(argmin)Backbone Model=Llama-2-7B-Chat2025.09 | 73.33 | 70.93 | 65.25 | 52.86 | 79.58 | 63.86 | 41.21 | |
| NoVoBackbone Model=Qwen3-4B2025.09 | 69.02 | 61.53 | 65.08 | 52.31 | 58.54 | 57.97 | 41.36 | |
| SEBackbone=RoBERTa-large2023.05 | 67.9 | 48.6 | 58.1 | 34.6 | 55.1 | 52.9 | — | |
| BaselineBackbone=RoBERTa-large2023.05 | 64.1 | 43.9 | 61.5 | 33.9 | 44.9 | 49.7 | — | |
| NoVoBackbone Model=Gemma-2-2B-it2025.09 | 61.18 | 65.25 | 53.05 | 54.07 | 59.38 | 55.21 | 38.33 | |
| NoVoBackbone Model=Llama-2-7B-Chat2025.09 | 57.06 | 61.95 | 52.54 | 39.78 | 56.67 | 50.16 | 32.95 | |
| NoVoBackbone Model=Llama-3.2-3B-Instruct2025.09 | 55.88 | 63.47 | 50 | 40.33 | 58.75 | 50.47 | 34.39 | |
| SEBackbone=BERT-large2023.05 | 53.1 | 35.1 | 45.3 | 24.7 | 50 | 41.6 | — | |
| BaselineBackbone=RoBERTa-base2023.05 | 47.1 | 41.9 | 36.2 | 21 | 30.9 | 35.4 | — | |
| SEBackbone=RoBERTa-base2023.05 | 45.7 | 37.8 | 32.4 | 23.5 | 38.5 | 35.6 | — | |
| BaselineBackbone=BERT-large2023.05 | 45.6 | 35.8 | 41.4 | 25.3 | 45.4 | 38.7 | — | |
| Log-LikelihoodBackbone Model=Llama-2-7B-Chat2025.09 | 43.21 | 42.57 | 50 | 42.98 | 41.03 | 42.7 | 36.42 | |
| Log-LikelihoodBackbone Model=Gemma-2-2B-it2025.09 | 43.21 | 52.7 | 50 | 61.16 | 41.03 | 48.65 | 43.83 | |
| Log-LikelihoodBackbone Model=Llama-3.2-3B-Instruct2025.09 | 43.21 | 63.51 | 50 | 32.23 | 41.03 | 42.96 | 27.78 | |
| Log-LikelihoodBackbone Model=Qwen3-4B2025.09 | 43.21 | 53.38 | 50 | 45.45 | 41.03 | 45.02 | 37.04 | |
| SEBackbone=BERT-base2023.05 | 33.3 | 28.4 | 42.6 | 23.5 | 42.3 | 34 | — | |
| BaselineBackbone=BERT-base2023.05 | 32.8 | 29.7 | 40.5 | 22.8 | 38.5 | 32.9 | — |