Instruction Awareness on NYT (test)
64.65Spearman CorrelationINBEDDER
Evaluation Results
| Method | Links | |
|---|---|---|
| INBEDDERBackbone=llama-2-7b, Encoding Method=1st-gen2024.02 | 64.65 | |
| INBEDDERBackbone=opt-2.7b, Encoding Method=1st-gen2024.02 | 59.43 | |
| INBEDDERBackbone=opt-1.3b, Encoding Method=1st-gen2024.02 | 53.13 | |
| llama-2-7bProcessing=without, Encoding Method=1st-gen2024.02 | 52.72 | |
| INBEDDERBackbone=roberta-large, Encoding Method=avg-gen2024.02 | 51.18 | |
| e5-large-v2instruction=without2024.02 | 50.07 | |
| instructor-large2024.02 | 49.96 | |
| llama-2-13b-chatInference Protocol=re-enc2024.02 | 32.74 | |
| llama-2-7b-chatInference Protocol=re-enc2024.02 | 29.41 | |
| AlpacaBackbone=roberta-large, Encoding Method=avg-gen2024.02 | 21.6 | |
| AlpacaBackbone=opt-2.7b, Encoding Method=avg-gen2024.02 | 13.32 | |
| AlpacaBackbone=opt-1.3b, Encoding Method=avg-gen2024.02 | 12.88 |