Question Answering on PIQA (B-1, R-L)
0.5Accuracy DeltaGAIN
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| GAINModel=Mistral-7B, Params=131K2026.04 | 0.5 | — | — | — | |
| LoRAModel=Mistral-7B, Params=262K, Learning Rate=10−32026.04 | 0.3 | — | — | — | |
| LoRAModel=Mistral-7B, Params=262K, Learning Rate=3×10−42026.04 | 0.3 | — | — | — | |
| LoRAModel=Llama-2-13B, Params=410K, Learning Rate=10−32026.04 | 0.2 | — | — | — | |
| GAINModel=Llama-2-70B, Params=655K2026.04 | 0.2 | — | — | — | |
| GAINModel=Llama-2-13B, Params=205K2026.04 | 0 | — | — | — | |
| LoRAModel=Llama-2-13B, Params=410K, Learning Rate=3×10−42026.04 | 0 | — | — | — | |
| LoRAModel=Llama-2-70B, Params=1.3M, Learning Rate=10−32026.04 | 0 | — | — | — | |
| LoRAModel=Llama-2-70B, Params=1.3M, Learning Rate=3×10−42026.04 | 0 | — | — | — | |
| AutoPromptTarget Model=Qwen2-1.5B, Suffix length (K)=42025.12 | -0.04 | — | — | 0.1 | |
| Proposed methodTarget Model=Qwen2-1.5B, Suffix length (K)=42025.12 | -0.04 | — | — | 0.45 | |
| Universal TriggerTarget Model=Qwen2-1.5B, Suffix length (K)=42025.12 | -0.16 | — | — | 0.3 | |
| Diverse DecodingTraining Strategy=Inference-time optimization (LM FT / -)2022.04 | — | 11.3 | 15.5 | — | |
| FiD-T5Model Scale=base, Backbone=T5, Training Strategy=LM FT2022.04 | — | 12.9 | 11.9 | — | |
| FiD-T5Model Scale=large, Backbone=T5, Training Strategy=LM FT2022.04 | — | 14.8 | 17.4 | — | |
| KIDGPT2Backbone=GPT2, Implementation=Knowledge trie, Training Strategy=Inference-time optimization2022.04 | — | 17.7 | 18.4 | — | |
| Little KIDImplementation=naive (plain list), Training Strategy=Inference-time optimization2022.04 | — | 12.2 | 13.5 | — | |
| PPLMTraining Strategy=Inference-time optimization (LM FT / -)2022.04 | — | 10.3 | 11.9 | — |