Performance prediction on MNLI source domains (in-domain)
0.699ROC AUCCosine distance (fine-tuned)
Evaluation Results
| Method | Links | |
|---|---|---|
| Cosine distance (fine-tuned)Model dependency=model-dependent, Embedding status=fine-tuned2023.05 | 0.699 | |
| Vocabulary, structural, semantic driftModel dependency=model-agnostic2023.05 | 0.525 | |
| Combined prev. model-agnosticModel dependency=model-agnostic2023.05 | 0.52 | |
| Structural driftModel dependency=model-agnostic2023.05 | 0.516 | |
| Semantic driftModel dependency=model-agnostic2023.05 | 0.516 | |
| Baseline (no-performance drop)Model dependency=model-agnostic2023.05 | 0.5 | |
| Token frequency cross-entropyModel dependency=model-agnostic2023.05 | 0.5 | |
| Token frequency JS-divModel dependency=model-agnostic2023.05 | 0.496 | |
| Cosine distance (pre-trained)Model dependency=model-agnostic, Embedding status=pre-trained2023.05 | 0.484 | |
| Vocabulary driftModel dependency=model-agnostic2023.05 | 0.474 |