Instruction Awareness on InstructSTSB (test)
22.07Spearman CorrelationINBEDDER
Evaluation results
| Method | Links | |
|---|---|---|
| INBEDDERBackbone=llama-2-7b, Encoding Method=1st-gen2024.02 | 22.07 | |
| llama-2-7bProcessing=without, Encoding Method=1st-gen2024.02 | 21.1 | |
| llama-2-13b-chatInference Protocol=re-enc2024.02 | 19.76 | |
| llama-2-7b-chatInference Protocol=re-enc2024.02 | 16.56 | |
| INBEDDERBackbone=roberta-large, Encoding Method=avg-gen2024.02 | 14.81 | |
| INBEDDERBackbone=opt-2.7b, Encoding Method=1st-gen2024.02 | 10.45 | |
| AlpacaBackbone=roberta-large, Encoding Method=avg-gen2024.02 | 8.43 | |
| INBEDDERBackbone=opt-1.3b, Encoding Method=1st-gen2024.02 | 7.47 | |
| AlpacaBackbone=opt-2.7b, Encoding Method=avg-gen2024.02 | 3.95 | |
| e5-large-v2instruction=without2024.02 | 0 | |
| AlpacaBackbone=opt-1.3b, Encoding Method=avg-gen2024.02 | -1.81 | |
| instructor-large2024.02 | -15.02 |