List your company
ResearchFormerly WizWand

Instruction Following on SuperNI Hold-Out v1.0 (test)

53.53ROUGE-LPACIT
32.490837.952943.41548.8771Oct 2, 2023
Updated 8mo ago

Evaluation results

MethodLinks
PACITBackbone=LLAMA2-7B, Inference Setting=Few-Shot2023.1053.53
PACITBackbone=T5-3B, Inference Setting=Few-Shot2023.1052.11
SuperNI (Few-Shot)Backbone=T5-3B, Inference Setting=Few-Shot2023.1051.08
SuperNI (Few-Shot)Backbone=LLAMA2-7B, Inference Setting=Few-Shot2023.1050.71
SuperNI (Zero-Shot)Backbone=LLAMA2-7B, Inference Setting=Few-Shot2023.1049.35
PACITBackbone=T5-770M, Inference Setting=Few-Shot2023.1046.66
SuperNI (Zero-Shot)Backbone=T5-3B, Inference Setting=Few-Shot2023.1045.73
PACITBackbone=LLAMA2-7B, Inference Setting=Zero-Shot2023.1045.62
SuperNI (Few-Shot)Backbone=T5-770M, Inference Setting=Few-Shot2023.1045.08
SuperNI (Zero-Shot)Backbone=LLAMA2-7B, Inference Setting=Zero-Shot2023.1044.81
PACITBackbone=T5-3B, Inference Setting=Zero-Shot2023.1043.09
SuperNI (Zero-Shot)Backbone=T5-3B, Inference Setting=Zero-Shot2023.1042.89
SuperNI (Few-Shot)Backbone=LLAMA2-7B, Inference Setting=Zero-Shot2023.1042.14
SuperNI (Zero-Shot)Backbone=T5-770M, Inference Setting=Few-Shot2023.1040.59
SuperNI (Few-Shot)Backbone=T5-3B, Inference Setting=Zero-Shot2023.1038.54
SuperNI (Zero-Shot)Backbone=T5-770M, Inference Setting=Zero-Shot2023.1038.02
PACITBackbone=T5-770M, Inference Setting=Zero-Shot2023.1033.59
SuperNI (Few-Shot)Backbone=T5-770M, Inference Setting=Zero-Shot2023.1033.3