Instruction Following on SuperNI Hold-In v1.0 (test)
62.47ROUGE-L ScorePACIT
Evaluation results
| Method | Links | |
|---|---|---|
| PACITBackbone=LLAMA2-7B, Inference Setting=Few-Shot2023.10 | 62.47 | |
| PACITBackbone=T5-3B, Inference Setting=Few-Shot2023.10 | 55.21 | |
| PACITBackbone=LLAMA2-7B, Inference Setting=Zero-Shot2023.10 | 54.05 | |
| PACITBackbone=T5-770M, Inference Setting=Few-Shot2023.10 | 53.31 | |
| SuperNI (Few-Shot)Backbone=T5-770M, Inference Setting=Few-Shot2023.10 | 52.96 | |
| SuperNI (Few-Shot)Backbone=T5-3B, Inference Setting=Few-Shot2023.10 | 52.96 | |
| SuperNI (Few-Shot)Backbone=LLAMA2-7B, Inference Setting=Few-Shot2023.10 | 52.68 | |
| SuperNI (Zero-Shot)Backbone=T5-3B, Inference Setting=Zero-Shot2023.10 | 49.95 | |
| SuperNI (Zero-Shot)Backbone=LLAMA2-7B, Inference Setting=Zero-Shot2023.10 | 49.36 | |
| SuperNI (Zero-Shot)Backbone=LLAMA2-7B, Inference Setting=Few-Shot2023.10 | 48.85 | |
| SuperNI (Zero-Shot)Backbone=T5-3B, Inference Setting=Few-Shot2023.10 | 47.59 | |
| PACITBackbone=T5-3B, Inference Setting=Zero-Shot2023.10 | 47.29 | |
| SuperNI (Zero-Shot)Backbone=T5-770M, Inference Setting=Zero-Shot2023.10 | 46.22 | |
| SuperNI (Few-Shot)Backbone=LLAMA2-7B, Inference Setting=Zero-Shot2023.10 | 45.53 | |
| PACITBackbone=T5-770M, Inference Setting=Zero-Shot2023.10 | 44.67 | |
| SuperNI (Few-Shot)Backbone=T5-770M, Inference Setting=Zero-Shot2023.10 | 43.59 | |
| SuperNI (Zero-Shot)Backbone=T5-770M, Inference Setting=Few-Shot2023.10 | 42.59 | |
| SuperNI (Few-Shot)Backbone=T5-3B, Inference Setting=Zero-Shot2023.10 | 41.49 |