Commonsense knowledge probing on ViComTe (test)
49.6Color Spearman CorrelationGPT-J-6B
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| GPT-J-6Bmethod_variant=Z-LaVI, zero-shot=true2022.10 | 49.6 | 50.9 | 47.1 | 70 | 37.9 | 46.5 | |
| OPT-30Bmethod_variant=Z-LaVI, zero-shot=true2022.10 | 48.1 | 52.1 | 48.4 | 72.1 | 38.1 | 47.5 | |
| GPT-Neo-1.3Bmethod_variant=Z-LaVI, zero-shot=true2022.10 | 47.4 | 48.4 | 46.1 | 64.3 | 36.4 | 41.2 | |
| GPT-Neo-2.7Bmethod_variant=Z-LaVI, zero-shot=true2022.10 | 47 | 43.6 | 45.2 | 62.1 | 36.1 | 37.7 | |
| GPT-J-6Bmethod_variant=Original, zero-shot=true2022.10 | 44.3 | 38 | 46.9 | 65.7 | 38.5 | 42.3 | |
| OPT-30Bmethod_variant=Original, zero-shot=true2022.10 | 41.9 | 41.3 | 46.4 | 60 | 38.3 | 44.7 | |
| RoBERTa-L-mnlimethod_variant=Z-LaVI, zero-shot=true2022.10 | 41.4 | 38.2 | 43.5 | 67.9 | 31.6 | 37.7 | |
| GPT-Neo-2.7Bmethod_variant=Original, zero-shot=true2022.10 | 41.3 | 29.3 | 43.5 | 50 | 35.5 | 30.6 | |
| GPT-Neo-1.3Bmethod_variant=Original, zero-shot=true2022.10 | 40.1 | 31.7 | 44.2 | 52.9 | 35.1 | 34.5 | |
| BART-L-mnlimethod_variant=Z-LaVI, zero-shot=true2022.10 | 38.2 | 32.6 | 42 | 69.3 | 32.6 | 40.5 | |
| SBERTmethod_variant=Z-LaVI, zero-shot=true2022.10 | 38.2 | 40.6 | 33.6 | 35 | 30.4 | 34.9 | |
| BERT-Lmethod_variant=Original, zero-shot=true2022.10 | 37.6 | 30.3 | 42.7 | 28.4 | 36.6 | 35.7 | |
| Z-LaVI w/o LMmethod_variant=Vision-only, zero-shot=true2022.10 | 37.2 | 39.4 | 33.8 | 32.1 | 24.9 | 32.7 | |
| SimCSEmethod_variant=Z-LaVI, zero-shot=true2022.10 | 36.3 | 40.4 | 34.7 | 40.7 | 29.2 | 33.5 | |
| RoBERTa-L-mnlimethod_variant=Original, zero-shot=true2022.10 | 35.3 | 25.1 | 41.7 | 66.4 | 28.3 | 34.9 | |
| BART-L-mnlimethod_variant=Original, zero-shot=true2022.10 | 34.6 | 27.5 | 41.5 | 68.6 | 30.5 | 37 | |
| Oscar-Lmethod_variant=Original, zero-shot=true2022.10 | 31.8 | 17.1 | 40 | 38.1 | 39.2 | 40.5 | |
| SimCSEmethod_variant=Original, zero-shot=true2022.10 | 30.7 | 34.8 | 30.1 | 28.6 | 24.6 | 26.4 | |
| SBERTmethod_variant=Original, zero-shot=true2022.10 | 22.6 | 26.5 | 20.3 | 13.6 | 24.1 | 22.5 | |
| Randommethod_variant=Original, zero-shot=true2022.10 | -0.1 | 6.6 | 1.3 | 7.1 | -1.5 | 6 |