Delta Evaluation on PIQA (Physical Commonsense Reasoning)
0Delta AccuracyAutoPrompt
Evaluation results
| Method | Links | ||
|---|---|---|---|
| AutoPromptTarget Model=Phi-1.5, Shot Setting=0-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | 0 | 0.26 | |
| soft promptTarget Model=Phi-1.5, Shot Setting=0-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | 0 | 0.15 | |
| UATTarget Model=TinyLlama, Shot Setting=0-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | 0 | -1.24 | |
| soft promptTarget Model=TinyLlama, Shot Setting=4-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | 0 | 0.11 | |
| soft promptTarget Model=Qwen2-1.5B, Shot Setting=0-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.01 | 1.75 | |
| UATTarget Model=Phi-1.5, Shot Setting=0-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.01 | 0.31 | |
| Calibrated Soft Suffix LearningTarget Model=Phi-1.5, Shot Setting=0-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.01 | -1.23 | |
| UATTarget Model=Phi-1.5, Shot Setting=4-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.01 | 2.01 | |
| AutoPromptTarget Model=Phi-1.5, Shot Setting=4-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.01 | 0.48 | |
| Calibrated Soft Suffix LearningTarget Model=Phi-1.5, Shot Setting=4-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.01 | -1.22 | |
| UATTarget Model=TinyLlama, Shot Setting=4-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.01 | 0.17 | |
| AutoPromptTarget Model=TinyLlama, Shot Setting=4-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.01 | 0.72 | |
| UATTarget Model=Qwen2-1.5B, Shot Setting=0-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.02 | 1.17 | |
| AutoPromptTarget Model=Qwen2-1.5B, Shot Setting=0-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.02 | 0.62 | |
| AutoPromptTarget Model=Qwen2-1.5B, Shot Setting=4-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.02 | -0.35 | |
| AutoPromptTarget Model=TinyLlama, Shot Setting=0-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.02 | -0.62 | |
| soft promptTarget Model=TinyLlama, Shot Setting=0-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.02 | 1.39 | |
| Calibrated Soft Suffix LearningTarget Model=TinyLlama, Shot Setting=0-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.03 | 1.34 | |
| Calibrated Soft Suffix LearningTarget Model=TinyLlama, Shot Setting=4-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.03 | -0.07 | |
| UATTarget Model=Qwen2-1.5B, Shot Setting=4-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.04 | 1.24 | |
| soft promptTarget Model=Phi-1.5, Shot Setting=4-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.04 | 1.78 | |
| Calibrated Soft Suffix LearningTarget Model=Qwen2-1.5B, Shot Setting=0-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.06 | 0.28 | |
| soft promptTarget Model=Qwen2-1.5B, Shot Setting=4-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.07 | -2.53 | |
| Calibrated Soft Suffix LearningTarget Model=Qwen2-1.5B, Shot Setting=4-shot, Token Length (K)=4, Seen Model=Qwen2-1.5B-Instruct2025.12 | -0.08 | -0.23 |