Commonsense Plausibility Estimation (Attribute Ranking) on CoDa
62.87Spearman's ρEVA-CLIP
Evaluation Results
| Method | Links | |
|---|---|---|
| EVA-CLIPInference Parameters=695M, Model Category=VLM, Method Framework=ComPaSS2025.02 | 62.87 | |
| Qwen2_w/ CLInference Parameters=7B, Model Category=LM, Method Framework=ComPaSS, Contrastive Learning=Yes2025.02 | 59.16 | |
| VERA-T5Inference Parameters=5B, Model Category=CSM, Method Framework=Baseline2025.02 | 58.93 | |
| Mistral_w/ CLInference Parameters=7B, Model Category=LM, Method Framework=ComPaSS, Contrastive Learning=Yes2025.02 | 58.54 | |
| CLIPInference Parameters=124M, Model Category=VLM, Method Framework=ComPaSS2025.02 | 58.1 | |
| Qwen2+verbal.Inference Parameters=7B, Model Category=LM, Evaluation Method=Verbalization, Method Framework=Baseline2025.02 | 57.4 | |
| Mistral+likelihoodInference Parameters=7B, Model Category=LM, Evaluation Method=Likelihood, Method Framework=Baseline2025.02 | 51.3 | |
| Qwen2+likelihoodInference Parameters=7B, Model Category=LM, Evaluation Method=Likelihood, Method Framework=Baseline2025.02 | 50.25 | |
| Qwen2_w/ CL+likelihoodInference Parameters=7B, Model Category=LM, Evaluation Method=Likelihood, Method Framework=Baseline, Contrastive Learning=Yes2025.02 | 49.65 | |
| Mistral+verbal.Inference Parameters=7B, Model Category=LM, Evaluation Method=Verbalization, Method Framework=Baseline2025.02 | 46.64 | |
| RoBERTa_w/ CLInference Parameters=355M, Model Category=LM, Method Framework=ComPaSS, Contrastive Learning=Yes2025.02 | 44.59 | |
| RoBERTa+likelihoodInference Parameters=355M, Model Category=LM, Evaluation Method=Likelihood, Method Framework=Baseline2025.02 | 24.37 | |
| RoBERTa_w/ CL+likelihoodInference Parameters=355M, Model Category=LM, Evaluation Method=Likelihood, Method Framework=Baseline, Contrastive Learning=Yes2025.02 | 23.36 | |
| COMET-AtomicInference Parameters=440M, Model Category=CSM, Method Framework=Baseline2025.02 | 22.91 | |
| ACCENTInference Parameters=440M, Model Category=CSM, Method Framework=Baseline2025.02 | 10.07 |