Performance prediction on MNLI source domains (out-of-domain)
0.683ROC AUCCosine distance (fine-tuned)
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Cosine distance (fine-tuned)Model dependency=model-dependent, Embedding status=fine-tuned2023.05 | 0.683 | 141.9 | |
| Structural driftModel dependency=model-agnostic2023.05 | 0.531 | 80.6 | |
| Vocabulary, structural, semantic driftModel dependency=model-agnostic2023.05 | 0.531 | 81 | |
| Semantic driftModel dependency=model-agnostic2023.05 | 0.521 | 79.1 | |
| Combined prev. model-agnosticModel dependency=model-agnostic2023.05 | 0.514 | 99.8 | |
| Token frequency cross-entropyModel dependency=model-agnostic2023.05 | 0.512 | 96.8 | |
| Cosine distance (pre-trained)Model dependency=model-agnostic, Embedding status=pre-trained2023.05 | 0.508 | 107.5 | |
| Token frequency JS-divModel dependency=model-agnostic2023.05 | 0.503 | 118.8 | |
| Baseline (no-performance drop)Model dependency=model-agnostic2023.05 | 0.5 | 100 | |
| Vocabulary driftModel dependency=model-agnostic2023.05 | 0.5 | 81.5 |