Question Answering on BBQ v1.0 (test)
93.1BBQ SES ScoreCRISPR
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| CRISPRBackbone=Flan-T5-xl, Parameter Count=3B, Zero-shot instruction-following=true2023.11 | 93.1 | 88.54 | 87.85 | |
| CRISPRBackbone=Flan-T5-large, Parameter Count=780M, Zero-shot instruction-following=true2023.11 | 85.11 | 73.6 | 76.13 | |
| OriginalBackbone=Flan-T5-xl, Parameter Count=3B, Zero-shot instruction-following=true2023.11 | 82.92 | 77.03 | 67.54 | |
| CRISPRBackbone=Flan-T5-base, Parameter Count=250M, Zero-shot instruction-following=true2023.11 | 71.68 | 60.32 | 62.88 | |
| CRISPRBackbone=T-Zero, Parameter Count=3B, Zero-shot instruction-following=true2023.11 | 67.03 | 55.88 | 54.04 | |
| OriginalBackbone=Flan-T5-large, Parameter Count=780M, Zero-shot instruction-following=true2023.11 | 66.67 | 53.62 | 53.26 | |
| OriginalBackbone=Flan-T5-base, Parameter Count=250M, Zero-shot instruction-following=true2023.11 | 65.63 | 43.6 | 43.44 | |
| CCBackbone=Flan-T5-xl, Parameter Count=3B, Zero-shot instruction-following=true2023.11 | 59.65 | 67.7 | 51.97 | |
| DCBackbone=Flan-T5-xl, Parameter Count=3B, Zero-shot instruction-following=true2023.11 | 56.15 | 71.04 | 51.94 | |
| CCBackbone=Flan-T5-large, Parameter Count=780M, Zero-shot instruction-following=true2023.11 | 48.95 | 49.01 | 48.22 | |
| DCBackbone=Flan-T5-base, Parameter Count=250M, Zero-shot instruction-following=true2023.11 | 47.78 | 40.01 | 40.46 | |
| DCBackbone=Flan-T5-large, Parameter Count=780M, Zero-shot instruction-following=true2023.11 | 47.56 | 50.33 | 46.47 | |
| DCBackbone=T-Zero, Parameter Count=3B, Zero-shot instruction-following=true2023.11 | 46.82 | 45.01 | 42.74 | |
| CCBackbone=T-Zero, Parameter Count=3B, Zero-shot instruction-following=true2023.11 | 46.18 | 44.38 | 41.34 | |
| OriginalBackbone=T-Zero, Parameter Count=3B, Zero-shot instruction-following=true2023.11 | 45.01 | 42.98 | 40.13 | |
| CCBackbone=Flan-T5-base, Parameter Count=250M, Zero-shot instruction-following=true2023.11 | 43.95 | 39.13 | 39.65 |