Natural Language Understanding on GLUE standard (val test)
94.6SST-2 AccuracyImagine to Hear (ITH)
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Imagine to Hear (ITH)Backbone=RoBERTa-base2025.03 | 94.6 | 91.9 | 91.4 | 87.4 | 88 | 90.1 | 90.6 | |
| VAWIBackbone=BERT-base2025.03 | 92.9 | 89.1 | 89.7 | 83 | 85.8 | 87.2 | 87.8 | |
| Imagine to Hear (ITH)Backbone=BERT-base2025.03 | 92.7 | 90.4 | 90.9 | 83.7 | 85.7 | 87.5 | 88.5 | |
| VOKENBackbone=BERT-base2025.03 | 92.2 | 88.6 | 88.6 | 82.6 | 83.5 | 86 | 86.83 | |
| iACEBackbone=BERT-base2025.03 | 91.7 | 88.6 | 89.1 | 82.8 | 85.8 | 86.6 | 87.43 | |
| VAWIBackbone=RoBERTa-base2025.03 | 91.7 | 90.6 | 87.9 | 82.6 | 88.5 | 88.3 | 88.21 | |
| iACEBackbone=RoBERTa-base2025.03 | 91.6 | 89.1 | 87.9 | 82.6 | 87.7 | 86.9 | 87.63 | |
| VOKENBackbone=RoBERTa-base2025.03 | 90.5 | 88.2 | 87.8 | 81 | 87 | 86.9 | 87.83 | |
| BERT (Baseline)Backbone=BERT-base2025.03 | 89.3 | 87.9 | 87.2 | 79.4 | 81.7 | 84.4 | 84.98 | |
| RoBERTa (Baseline)Backbone=RoBERTa-base2025.03 | 89.2 | 87.5 | 86.2 | 79 | 81.4 | 85.4 | 84.78 | |
| ALBEF14MArchitecture=Multimodal text encoder2025.03 | 78.9 | 78.2 | 79.4 | 73.4 | 76.5 | 77.5 | 77.31 | |
| BLIPArchitecture=Multimodal text encoder2025.03 | 76.3 | 77.4 | 78.8 | 72.5 | 77.8 | 76.4 | 76.53 | |
| CLIPArchitecture=Multimodal text encoder2025.03 | 73.3 | 74.5 | 72.8 | 68.4 | 74.3 | 73.8 | 72.85 |