Natural Language Inference on ANLI (Accuracy and Calibration)
74.02AccuracyZero-shot-EI
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Zero-shot-EIModel=Gemini-1.5-Pro, Model Role=Teacher Model2026.02 | 74.02 | — | — | — | — | — | |
| Zero-shotModel=Gemini-1.5-Pro, Model Role=Teacher Model2026.02 | 71 | — | — | — | — | — | |
| RevThinkModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 65.1 | — | — | — | — | — | |
| Answer AugModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 64.83 | — | — | — | — | — | |
| Question AugModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 64.45 | — | — | — | — | — | |
| ExGRPOModel=Qwen2.5-7B-Instruct, Model Role=Student Model, Adapter=On-Policy Adapt.2026.02 | 64.42 | — | — | — | — | — | |
| On-Policy DistillationModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 64.18 | — | — | — | — | — | |
| ExGRPOModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 64 | — | — | — | — | — | |
| Rephrase QuestionModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 63.95 | — | — | — | — | — | |
| Divide-or-ConquerModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 63.89 | — | — | — | — | — | |
| Distill Step-by-StepModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 63.58 | — | — | — | — | — | |
| SKDModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 62.73 | — | — | — | — | — | |
| Zero-shotModel=Qwen2.5-7B-Instruct, Model Role=Student Model2026.02 | 61.67 | — | — | — | — | — | |
| ExGRPOModel=Gemma-7B-it, Model Role=Student Model, Adapter=On-Policy Adapt.2026.02 | 56.34 | — | — | — | — | — | |
| ExGRPOModel=Gemma-7B-it, Model Role=Student Model2026.02 | 55.75 | — | — | — | — | — | |
| MUPPET2022.12 | 52.59 | — | — | — | — | — | |
| On-Policy DistillationModel=Gemma-7B-it, Model Role=Student Model2026.02 | 52.18 | — | — | — | — | — | |
| ColD-Fusion2022.12 | 52 | — | — | — | — | — | |
| Finetune2022.12 | 51.51 | — | — | — | — | — | |
| Multitask2022.12 | 51.48 | — | — | — | — | — | |
| MI-based distillationBackbone=T5-base2024.03 | 51.2 | — | — | — | — | — | |
| DSSBackbone=T5-base2024.03 | 49.58 | — | — | — | — | — | |
| ZipCalModel=gemma-2-9b-it, Calibration Category=Mean2026.03 | 47.58 | — | — | — | — | -4.32 | |
| Divide-or-ConquerModel=Gemma-7B-it, Model Role=Student Model2026.02 | 47.45 | — | — | — | — | — | |
| RevThinkModel=Gemma-7B-it, Model Role=Student Model2026.02 | 47.36 | — | — | — | — | — | |
| SITE (M = 50)# Trainable Parameters=1.02K, Backbone=Llama-3.1-8B, M=502025.07 | 47.31 | — | — | — | — | — | |
| (IA)3# Trainable Parameters=524.29K, Backbone=Llama-3.1-8B2025.07 | 47.03 | — | — | — | — | — | |
| SITE (M = 1)# Trainable Parameters=1.02K, Backbone=Llama-3.1-8B, M=12025.07 | 46.35 | — | — | — | — | — | |
| LoRA# Trainable Parameters=3407.87K, Backbone=Llama-3.1-8B2025.07 | 45.82 | — | — | — | — | — | |
| SKDModel=Gemma-7B-it, Model Role=Student Model2026.02 | 45.42 | — | — | — | — | — | |
| Distill Step-by-StepModel=Gemma-7B-it, Model Role=Student Model2026.02 | 44.23 | — | — | — | — | — | |
| 10-shot ICLBackbone=Llama-3.1-8B2025.07 | 43.96 | — | — | — | — | — | |
| FinetuningBackbone=T5-base2024.03 | 43.58 | — | — | — | — | — | |
| Single-taskBackbone=T5-base2024.03 | 43.5 | — | — | — | — | — | |
| Rephrase QuestionModel=Gemma-7B-it, Model Role=Student Model2026.02 | 43.07 | — | — | — | — | — | |
| Answer AugModel=Gemma-7B-it, Model Role=Student Model2026.02 | 42.72 | — | — | — | — | — | |
| Question AugModel=Gemma-7B-it, Model Role=Student Model2026.02 | 41.22 | — | — | — | — | — | |
| QLoRAModel=Llama-2-7B, Average Bit Budget=3-bit, Zero-shot=true2025.09 | 39.06 | — | — | — | — | — | |
| CoA-LoRAModel=Qwen2.5-3B, Average Bit Budget=3-bit, Zero-shot=true2025.09 | 38.28 | — | — | — | — | — | |
| Shared-LoRAModel=Llama-2-7B, Average Bit Budget=3-bit, Zero-shot=true2025.09 | 38.28 | — | — | — | — | — | |
| CoA-LoRAModel=Llama-2-7B, Average Bit Budget=3-bit, Zero-shot=true2025.09 | 38.28 | — | — | — | — | — | |
| DOPA*Backbone=Qwen3-8B2026.04 | 38.1 | — | — | — | — | — | |
| Zero-shotModel=Gemma-7B-it, Model Role=Student Model2026.02 | 37.92 | — | — | — | — | — | |
| ZipCalModel=Llama-3.1-8B-Instruct, Calibration Category=Mean2026.03 | 37.61 | — | — | — | — | -6.89 | |
| DOPA*Backbone=LLaMA3.1-8B2026.04 | 37.53 | — | — | — | — | — | |
| KNNBackbone=LLaMA3.1-8B2026.04 | 37.4 | — | — | — | — | — | |
| InfICLBackbone=LLaMA3.1-8B2026.04 | 37.23 | — | — | — | — | — | |
| RandomBackbone=LLaMA3.1-8B2026.04 | 36.67 | — | — | — | — | — | |
| RewriteBackbone=Qwen3-1.7B2026.04 | 36.63 | — | — | — | — | — | |
| DOPA*Backbone=Qwen3-1.7B2026.04 | 36.47 | — | — | — | — | — | |
| DICLBackbone=LLaMA3.1-8B2026.04 | 36.37 | — | — | — | — | — | |
| DrICLBackbone=Qwen3-8B2026.04 | 36.1 | — | — | — | — | — | |
| InfICLBackbone=LLaMA3.2-3B2026.04 | 36 | — | — | — | — | — | |
| RewriteBackbone=LLaMA3.1-8B2026.04 | 36 | — | — | — | — | — | |
| DrICLBackbone=LLaMA3.1-8B2026.04 | 35.9 | — | — | — | — | — | |
| DOPA*Backbone=LLaMA3.2-3B2026.04 | 35.87 | — | — | — | — | — | |
| DICLBackbone=Qwen3-1.7B2026.04 | 35.83 | — | — | — | — | — | |
| RandomBackbone=Qwen3-8B2026.04 | 35.77 | — | — | — | — | — | |
| RewriteBackbone=LLaMA3.2-3B2026.04 | 35.6 | — | — | — | — | — | |
| InfICLBackbone=Qwen3-8B2026.04 | 35.6 | — | — | — | — | — | |
| InfICLBackbone=Qwen3-1.7B2026.04 | 35.57 | — | — | — | — | — | |
| RandomBackbone=LLaMA3.2-3B2026.04 | 35.5 | — | — | — | — | — | |
| DrICLBackbone=Qwen3-1.7B2026.04 | 35.33 | — | — | — | — | — | |
| DICLBackbone=Qwen3-8B2026.04 | 35.33 | — | — | — | — | — | |
| KNNBackbone=Qwen3-1.7B2026.04 | 35.2 | — | — | — | — | — | |
| RandomBackbone=Qwen3-1.7B2026.04 | 35.17 | — | — | — | — | — | |
| KNNBackbone=Qwen3-8B2026.04 | 34.93 | — | — | — | — | — | |
| MTV# Trainable Parameters=1.02K, Backbone=Llama-3.1-8B2025.07 | 34.7 | — | — | — | — | — | |
| KNNBackbone=LLaMA3.2-3B2026.04 | 34.67 | — | — | — | — | — | |
| RewriteBackbone=Qwen3-8B2026.04 | 34.63 | — | — | — | — | — | |
| DICLBackbone=LLaMA3.2-3B2026.04 | 34.6 | — | — | — | — | — | |
| LQ-LoRAModel=Qwen2.5-3B, Average Bit Budget=3-bit, Zero-shot=true2025.09 | 34.38 | — | — | — | — | — | |
| DrICLBackbone=LLaMA3.2-3B2026.04 | 33.87 | — | — | — | — | — | |
| DrICLBackbone=Gemma2-2B2026.04 | 33.57 | — | — | — | — | — | |
| KNNBackbone=GPT2-xl2026.04 | 33.5 | — | — | — | — | — | |
| DICLBackbone=GPT2-xl2026.04 | 33.47 | — | — | — | — | — | |
| DOPA*Backbone=GPT2-xl2026.04 | 33.43 | — | — | — | — | — | |
| DrICLBackbone=GPT2-xl2026.04 | 33.33 | — | — | — | — | — | |
| RewriteBackbone=Gemma2-2B2026.04 | 33.33 | — | — | — | — | — | |
| DICLBackbone=Gemma2-2B2026.04 | 33.33 | — | — | — | — | — | |
| Prompt Tuning# Trainable Parameters=81.92K, Backbone=Llama-3.1-8B2025.07 | 33.22 | — | — | — | — | — | |
| TVBackbone=Llama-3.1-8B2025.07 | 33.17 | — | — | — | — | — | |
| DOPA*Backbone=Gemma2-2B2026.04 | 33.07 | — | — | — | — | — | |
| RandomBackbone=Gemma2-2B2026.04 | 33 | — | — | — | — | — | |
| KNNBackbone=Gemma2-2B2026.04 | 32.93 | — | — | — | — | — | |
| InfICLBackbone=Gemma2-2B2026.04 | 32.93 | — | — | — | — | — | |
| RewriteBackbone=GPT2-xl2026.04 | 32.77 | — | — | — | — | — | |
| InfICLBackbone=GPT2-xl2026.04 | 32.4 | — | — | — | — | — | |
| RandomBackbone=GPT2-xl2026.04 | 32.23 | — | — | — | — | — | |
| IVBackbone=Llama-3.1-8B2025.07 | 31.52 | — | — | — | — | — | |
| EnsembleCalibration category=Unlearnable, Backbone=T52022.10 | 31.41 | 85.49 | 54.09 | 16.49 | 86.4 | — | |
| I-VanillaCalibration category=Learnable, Backbone=T52022.10 | 31.41 | 40.92 | 38.3 | 65.43 | 43.82 | — | |
| VanillaCalibration category=Unlearnable, Backbone=T52022.10 | 31.31 | 75.48 | 44.17 | 26.87 | 90.3 | — | |
| TSCalibration category=Unlearnable, Backbone=T52022.10 | 31.31 | 85.58 | 54.27 | 16.22 | 86.41 | — | |
| E-MLPCalibration category=Learnable, Backbone=T52022.10 | 31.31 | 81.65 | 50.74 | 18.39 | 81.66 | — | |
| E-T5Calibration category=Learnable, Backbone=T52022.10 | 31.31 | 41.67 | 38.68 | 65.84 | 45.11 | — | |
| Shared-LoRAModel=Qwen2.5-3B, Average Bit Budget=3-bit, Zero-shot=true2025.09 | 31.25 | — | — | — | — | — | |
| LQ-LoRAModel=Llama-2-7B, Average Bit Budget=3-bit, Zero-shot=true2025.09 | 31.25 | — | — | — | — | — | |
| I-IterCalibration category=Learnable, Backbone=T52022.10 | 30.69 | 37.02 | 28.37 | 68.84 | 39.62 | — | |
| I-SimulCalibration category=Learnable, Backbone=T52022.10 | 30.66 | 38.65 | 46.06 | 68.4 | 41.76 | — |