Cross-lingual Vision-Language Understanding and Retrieval on IGLUE 1.0 (test)
78.95XVNLI AccuracyCCLM_large
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| CCLM_largeEvaluation Protocol=Zero-Shot2022.06 | 78.95 | 56.25 | 74.83 | 83.78 | 83.46 | 43.74 | 44.88 | |
| CCLM base 3MEvaluation Protocol=Few-Shot, Fine-tuning Strategy=English fine-tuned on target languages2022.06 | 75.15 | 50.94 | 70.53 | 66.04 | 68.15 | — | — | |
| CCLM_baseEvaluation Protocol=Zero-Shot2022.06 | 74.78 | 48.12 | 68.49 | 76.94 | 76.22 | 33.9 | 35.26 | |
| CCLM_3M_baseEvaluation Protocol=Zero-Shot2022.06 | 74.64 | 42.36 | 65.91 | 67.35 | 65.37 | 27.46 | 28.66 | |
| VisualBERTEvaluation Protocol=Translate-Test2022.06 | 74.12 | 48.72 | 62.35 | 41.64 | 36.44 | 15.36 | 15.75 | |
| VL-BERTEvaluation Protocol=Translate-Test2022.06 | 73.86 | 49.78 | 64.16 | 38.18 | 31.84 | 15.11 | 16.09 | |
| UNITEREvaluation Protocol=Translate-Test2022.06 | 73.65 | 50.62 | 61.92 | 41.04 | 37.49 | 15.43 | 16.01 | |
| ViLBERTEvaluation Protocol=Translate-Test2022.06 | 73.45 | 50.33 | 62.39 | 36.97 | 33.21 | 15.4 | 16.93 | |
| UC2Evaluation Protocol=Few-Shot, Fine-tuning Strategy=English fine-tuned on target languages2022.06 | 63.68 | 42.95 | 58.32 | 19.79 | 17.59 | — | — | |
| UC2Evaluation Protocol=Zero-Shot2022.06 | 62.05 | 29.35 | 57.28 | 20.31 | 17.89 | 7.83 | 9.09 | |
| xUNITEREvaluation Protocol=Few-Shot, Fine-tuning Strategy=English fine-tuned on target languages2022.06 | 60.55 | 40.68 | 57.46 | 14.3 | 13.54 | — | — | |
| M3PEvaluation Protocol=Few-Shot, Fine-tuning Strategy=English fine-tuned on target languages2022.06 | 59.36 | 41.04 | 49.79 | 13.21 | 12.26 | — | — | |
| xUNITEREvaluation Protocol=Zero-Shot2022.06 | 58.48 | 21.72 | 54.59 | 14.04 | 13.51 | 8.72 | 9.81 | |
| M3PEvaluation Protocol=Zero-Shot2022.06 | 58.25 | 28.17 | 56 | 12.91 | 11.9 | 8.12 | 9.98 | |
| mUNITEREvaluation Protocol=Few-Shot, Fine-tuning Strategy=English fine-tuned on target languages2022.06 | 53.95 | 37.21 | 53.41 | 8.54 | 9.32 | — | — | |
| mUNITEREvaluation Protocol=Zero-Shot2022.06 | 53.69 | 9.97 | 53.72 | 8.06 | 8.86 | 9.16 | 10.48 |