Boolean Question Answering on BoolQ (Delta Acc, Delta CalLogP)
-0.01Delta Accuracysoft prompt
Evaluation results
| Method | Links | ||
|---|---|---|---|
| soft promptTarget Model=Qwen2-1.5B, Shot Setting=0-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.01 | -0.54 | |
| UATTarget Model=Qwen2-1.5B, Shot Setting=4-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.01 | 0.22 | |
| soft promptTarget Model=Phi-1.5, Shot Setting=0-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.01 | 0.17 | |
| Calibrated Soft Suffix LearningTarget Model=Phi-1.5, Shot Setting=0-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.01 | -0.02 | |
| UATTarget Model=Qwen2-1.5B, Shot Setting=0-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.02 | -0.19 | |
| AutoPromptTarget Model=Qwen2-1.5B, Shot Setting=0-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.02 | -0.63 | |
| AutoPromptTarget Model=Qwen2-1.5B, Shot Setting=4-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.02 | 0.52 | |
| soft promptTarget Model=Qwen2-1.5B, Shot Setting=4-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.02 | -0.14 | |
| UATTarget Model=Phi-1.5, Shot Setting=0-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.02 | 0.21 | |
| UATTarget Model=Phi-1.5, Shot Setting=4-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.02 | 1.06 | |
| soft promptTarget Model=Phi-1.5, Shot Setting=4-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.05 | 0.5 | |
| Calibrated Soft Suffix LearningTarget Model=TinyLlama, Shot Setting=4-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.07 | -1.46 | |
| AutoPromptTarget Model=Phi-1.5, Shot Setting=0-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.08 | 0.19 | |
| Calibrated Soft Suffix LearningTarget Model=Phi-1.5, Shot Setting=4-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.08 | -0.43 | |
| UATTarget Model=TinyLlama, Shot Setting=0-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.12 | -0.05 | |
| UATTarget Model=TinyLlama, Shot Setting=4-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.12 | 0.57 | |
| AutoPromptTarget Model=Phi-1.5, Shot Setting=4-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.14 | 0.97 | |
| Calibrated Soft Suffix LearningTarget Model=TinyLlama, Shot Setting=0-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.14 | -0.16 | |
| Calibrated Soft Suffix LearningTarget Model=Qwen2-1.5B, Shot Setting=0-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.16 | 0.62 | |
| soft promptTarget Model=TinyLlama, Shot Setting=0-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.18 | 0.7 | |
| AutoPromptTarget Model=TinyLlama, Shot Setting=0-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.2 | -1.7 | |
| soft promptTarget Model=TinyLlama, Shot Setting=4-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.23 | 1.48 | |
| AutoPromptTarget Model=TinyLlama, Shot Setting=4-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.27 | -1.9 | |
| Calibrated Soft Suffix LearningTarget Model=Qwen2-1.5B, Shot Setting=4-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.3 | 0.08 |