Natural Language Inference on RTE (Delta Accuracy and Calibration Change)
-0.03Delta AccuracyAutoPrompt
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| AutoPromptTarget Model=Qwen2-1.5B, Suffix length (K)=42025.12 | -0.03 | 0.25 | — | — | |
| Universal TriggerTarget Model=Qwen2-1.5B, Suffix length (K)=42025.12 | -0.15 | 0.2 | — | — | |
| Proposed methodTarget Model=Qwen2-1.5B, Suffix length (K)=42025.12 | -0.22 | 0.26 | — | — | |
| ZipCalModel=Llama-3.1-8B-Instruct, Calibration Category=Mean2026.03 | — | — | 65.87 | -1.17 | |
| ZipCalModel=gemma-2-9b-it, Calibration Category=Mean2026.03 | — | — | 66.3 | 0.3 |