Concept Attribution on CLEVR (test)
0.85F1 ScoreDirect Alignment
Evaluation results
| Method | Links | |
|---|---|---|
| Direct AlignmentConcept Type=Linsep, Model=Llama, Representation Type=Concept2025.12 | 0.85 | |
| LIMEConcept Type=Linsep, Model=Llama, Representation Type=SuperActivators2025.12 | 0.85 | |
| LIMEConcept Type=K-Linsep, Model=Llama, Representation Type=SuperActivators2025.12 | 0.83 | |
| LIMEConcept Type=Avg, Model=Llama, Representation Type=SuperActivators2025.12 | 0.81 | |
| LIMEConcept Type=K-Means, Model=Llama, Representation Type=SuperActivators2025.12 | 0.81 | |
| SHAPConcept Type=Avg, Model=Llama, Representation Type=SuperActivators2025.12 | 0.8 | |
| SHAPConcept Type=Linsep, Model=Llama, Representation Type=SuperActivators2025.12 | 0.8 | |
| SHAPConcept Type=K-Means, Model=Llama, Representation Type=SuperActivators2025.12 | 0.8 | |
| SHAPConcept Type=K-Linsep, Model=Llama, Representation Type=SuperActivators2025.12 | 0.8 | |
| IntGradConcept Type=Avg, Model=Llama, Representation Type=SuperActivators2025.12 | 0.8 | |
| Direct AlignmentConcept Type=Avg, Model=Llama, Representation Type=Concept2025.12 | 0.78 | |
| IntGradConcept Type=Linsep, Model=Llama, Representation Type=SuperActivators2025.12 | 0.78 | |
| LIMEConcept Type=K-Linsep, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.77 | |
| IntGradConcept Type=Avg, Model=Llama, Representation Type=Concept2025.12 | 0.77 | |
| LIMEConcept Type=Avg, Model=Llama, Representation Type=Concept2025.12 | 0.76 | |
| LIMEConcept Type=K-Means, Model=Llama, Representation Type=Concept2025.12 | 0.76 | |
| SHAPConcept Type=Avg, Model=Llama, Representation Type=Concept2025.12 | 0.75 | |
| SHAPConcept Type=Linsep, Model=Llama, Representation Type=Concept2025.12 | 0.75 | |
| SHAPConcept Type=K-Means, Model=Llama, Representation Type=Concept2025.12 | 0.75 | |
| SHAPConcept Type=K-Linsep, Model=Llama, Representation Type=Concept2025.12 | 0.75 | |
| IntGradConcept Type=Linsep, Model=Llama, Representation Type=Concept2025.12 | 0.72 | |
| LIMEConcept Type=Linsep, Model=Llama, Representation Type=Concept2025.12 | 0.7 | |
| LIMEConcept Type=Linsep, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.68 | |
| LIMEConcept Type=K-Linsep, Model=Llama, Representation Type=Concept2025.12 | 0.68 | |
| Direct AlignmentConcept Type=Linsep, Model=CLIP, Representation Type=Concept2025.12 | 0.65 | |
| Direct AlignmentConcept Type=K-Means, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.64 | |
| IntGradConcept Type=K-Linsep, Model=Llama, Representation Type=SuperActivators2025.12 | 0.64 | |
| Direct AlignmentConcept Type=K-Means, Model=CLIP, Representation Type=Concept2025.12 | 0.63 | |
| RISEConcept Type=Linsep, Model=Llama, Representation Type=SuperActivators2025.12 | 0.63 | |
| RISEConcept Type=K-Linsep, Model=Llama, Representation Type=SuperActivators2025.12 | 0.63 | |
| IntGradConcept Type=K-Linsep, Model=Llama, Representation Type=Concept2025.12 | 0.62 | |
| Direct AlignmentConcept Type=Linsep, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.61 | |
| LIMEConcept Type=K-Means, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.61 | |
| SHAP IQConcept Type=Linsep, Model=Llama, Representation Type=SuperActivators2025.12 | 0.61 | |
| SHAP IQConcept Type=K-Linsep, Model=Llama, Representation Type=SuperActivators2025.12 | 0.61 | |
| Direct AlignmentConcept Type=Avg, Model=CLIP, Representation Type=Concept2025.12 | 0.6 | |
| Direct AlignmentConcept Type=Avg, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.6 | |
| Direct AlignmentConcept Type=K-Linsep, Model=CLIP, Representation Type=Concept2025.12 | 0.6 | |
| RISEConcept Type=Linsep, Model=Llama, Representation Type=Concept2025.12 | 0.6 | |
| RISEConcept Type=K-Linsep, Model=Llama, Representation Type=Concept2025.12 | 0.6 | |
| SHAP IQConcept Type=Linsep, Model=Llama, Representation Type=Concept2025.12 | 0.6 | |
| SHAP IQConcept Type=K-Linsep, Model=Llama, Representation Type=Concept2025.12 | 0.6 | |
| Direct AlignmentConcept Type=K-Linsep, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.59 | |
| RISEConcept Type=Linsep, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.59 | |
| RISEConcept Type=K-Linsep, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.59 | |
| IntGradConcept Type=K-Linsep, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.59 | |
| SHAPConcept Type=Linsep, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.58 | |
| SHAPConcept Type=K-Linsep, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.58 | |
| RISEConcept Type=Linsep, Model=CLIP, Representation Type=Concept2025.12 | 0.58 | |
| RISEConcept Type=K-Linsep, Model=CLIP, Representation Type=Concept2025.12 | 0.58 | |
| SHAP IQConcept Type=Avg, Model=Llama, Representation Type=SuperActivators2025.12 | 0.58 | |
| SHAP IQConcept Type=Linsep, Model=CLIP, Representation Type=Concept2025.12 | 0.58 | |
| SHAP IQConcept Type=Linsep, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.58 | |
| SHAP IQConcept Type=K-Means, Model=Llama, Representation Type=SuperActivators2025.12 | 0.58 | |
| SHAP IQConcept Type=K-Linsep, Model=CLIP, Representation Type=Concept2025.12 | 0.58 | |
| SHAP IQConcept Type=K-Linsep, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.58 | |
| IntGradConcept Type=K-Means, Model=Llama, Representation Type=SuperActivators2025.12 | 0.58 | |
| IntGradConcept Type=K-Linsep, Model=CLIP, Representation Type=Concept2025.12 | 0.58 | |
| MFABAConcept Type=Linsep, Model=Llama, Representation Type=SuperActivators2025.12 | 0.58 | |
| CALMConcept Type=Linsep, Model=Llama, Representation Type=Concept2025.12 | 0.57 | |
| CALMConcept Type=Linsep, Model=Llama, Representation Type=SuperActivators2025.12 | 0.57 | |
| RISEConcept Type=Avg, Model=Llama, Representation Type=SuperActivators2025.12 | 0.56 | |
| RISEConcept Type=K-Means, Model=Llama, Representation Type=SuperActivators2025.12 | 0.56 | |
| IntGradConcept Type=K-Means, Model=Llama, Representation Type=Concept2025.12 | 0.56 | |
| CALMConcept Type=Linsep, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.56 | |
| MFABAConcept Type=Linsep, Model=Llama, Representation Type=Concept2025.12 | 0.56 | |
| Direct AlignmentConcept Type=Avg, Model=Llama, Representation Type=SuperActivators2025.12 | 0.55 | |
| LIMEConcept Type=Avg, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.55 | |
| RISEConcept Type=Avg, Model=Llama, Representation Type=Concept2025.12 | 0.55 | |
| RISEConcept Type=K-Means, Model=Llama, Representation Type=Concept2025.12 | 0.55 | |
| SHAP IQConcept Type=Avg, Model=Llama, Representation Type=Concept2025.12 | 0.55 | |
| SHAP IQConcept Type=K-Means, Model=Llama, Representation Type=Concept2025.12 | 0.55 | |
| IntGradConcept Type=Linsep, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.55 | |
| FullGradConcept Type=Linsep, Model=Llama, Representation Type=SuperActivators2025.12 | 0.55 | |
| CALMConcept Type=Linsep, Model=CLIP, Representation Type=Concept2025.12 | 0.55 | |
| MFABAConcept Type=Linsep, Model=CLIP, Representation Type=Concept2025.12 | 0.55 | |
| MFABAConcept Type=Linsep, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.55 | |
| MFABAConcept Type=K-Linsep, Model=Llama, Representation Type=SuperActivators2025.12 | 0.55 | |
| Direct AlignmentConcept Type=Linsep, Model=Llama, Representation Type=SuperActivators2025.12 | 0.54 | |
| CALMConcept Type=K-Linsep, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.54 | |
| CALMConcept Type=K-Linsep, Model=Llama, Representation Type=SuperActivators2025.12 | 0.54 | |
| MFABAConcept Type=K-Linsep, Model=Llama, Representation Type=Concept2025.12 | 0.54 | |
| SHAPConcept Type=Avg, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.53 | |
| SHAPConcept Type=K-Means, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.53 | |
| RISEConcept Type=Avg, Model=CLIP, Representation Type=Concept2025.12 | 0.53 | |
| RISEConcept Type=Avg, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.53 | |
| RISEConcept Type=K-Means, Model=CLIP, Representation Type=Concept2025.12 | 0.53 | |
| RISEConcept Type=K-Means, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.53 | |
| SHAP IQConcept Type=Avg, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.53 | |
| SHAP IQConcept Type=K-Means, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.53 | |
| IntGradConcept Type=Avg, Model=CLIP, Representation Type=SuperActivators2025.12 | 0.53 | |
| FullGradConcept Type=K-Linsep, Model=Llama, Representation Type=SuperActivators2025.12 | 0.53 | |
| CALMConcept Type=Avg, Model=Llama, Representation Type=SuperActivators2025.12 | 0.53 | |
| CALMConcept Type=K-Linsep, Model=Llama, Representation Type=Concept2025.12 | 0.53 | |
| MFABAConcept Type=Avg, Model=Llama, Representation Type=SuperActivators2025.12 | 0.53 | |
| LIMEConcept Type=K-Means, Model=CLIP, Representation Type=Concept2025.12 | 0.52 | |
| LIMEConcept Type=K-Linsep, Model=CLIP, Representation Type=Concept2025.12 | 0.52 | |
| SHAPConcept Type=Linsep, Model=CLIP, Representation Type=Concept2025.12 | 0.52 | |
| SHAPConcept Type=K-Linsep, Model=CLIP, Representation Type=Concept2025.12 | 0.52 | |
| SHAP IQConcept Type=Avg, Model=CLIP, Representation Type=Concept2025.12 | 0.52 |