Ranking Correlation Analysis on LAM benchmark
0.879Kendall CorrelationCombined Embedding
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| Combined EmbeddingSubset Size (n)=2002026.04 | 0.879 | 0.826 | 49 | 350 | |
| Anchor PointsSubset Size (n)=2002026.04 | 0.861 | 0.816 | 55 | — | |
| Random SamplingSubset Size (n)=2002026.04 | 0.848 | 0.751 | 127 | 500 | |
| Combined EmbeddingSubset Size (n)=1002026.04 | 0.844 | 0.826 | 49 | 350 | |
| Difficulty-basedSubset Size (n)=2002026.04 | 0.841 | 0.766 | 93 | 500 | |
| Random-Sampling-LearnSubset Size (n)=2002026.04 | 0.829 | 0.72 | 158 | 900 | |
| IRT-basedSubset Size (n)=2002026.04 | 0.821 | 0.736 | 142 | 800 | |
| Random-Search-LearnSubset Size (n)=2002026.04 | 0.819 | 0.728 | 141 | — | |
| Anchor PointsSubset Size (n)=1002026.04 | 0.818 | 0.816 | 55 | — | |
| Semantic EmbeddingSubset Size (n)=2002026.04 | 0.802 | 0.723 | 99 | — | |
| Combined EmbeddingSubset Size (n)=502026.04 | 0.802 | 0.826 | 49 | 350 | |
| Semantic EmbeddingSubset Size (n)=1002026.04 | 0.8 | 0.723 | 99 | — | |
| Acoustic EmbeddingSubset Size (n)=2002026.04 | 0.797 | 0.709 | 166 | — | |
| Anchor PointsSubset Size (n)=502026.04 | 0.79 | 0.816 | 55 | — | |
| Difficulty-basedSubset Size (n)=1002026.04 | 0.775 | 0.766 | 93 | 500 | |
| Random SamplingSubset Size (n)=1002026.04 | 0.771 | 0.751 | 127 | 500 | |
| IRT-basedSubset Size (n)=1002026.04 | 0.768 | 0.736 | 142 | 800 | |
| Anchor PointsSubset Size (n)=302026.04 | 0.762 | 0.816 | 55 | — | |
| Acoustic EmbeddingSubset Size (n)=1002026.04 | 0.758 | 0.709 | 166 | — | |
| Random-Sampling-LearnSubset Size (n)=1002026.04 | 0.75 | 0.72 | 158 | 900 | |
| Random-Search-LearnSubset Size (n)=1002026.04 | 0.745 | 0.728 | 141 | — | |
| Anchor PointsSubset Size (n)=202026.04 | 0.731 | 0.816 | 55 | — | |
| Combined EmbeddingSubset Size (n)=302026.04 | 0.722 | 0.826 | 49 | 350 | |
| Semantic EmbeddingSubset Size (n)=502026.04 | 0.719 | 0.723 | 99 | — | |
| Acoustic EmbeddingSubset Size (n)=502026.04 | 0.719 | 0.709 | 166 | — | |
| Difficulty-basedSubset Size (n)=502026.04 | 0.704 | 0.766 | 93 | 500 | |
| IRT-basedSubset Size (n)=502026.04 | 0.696 | 0.736 | 142 | 800 | |
| Random SamplingSubset Size (n)=502026.04 | 0.692 | 0.751 | 127 | 500 | |
| Combined EmbeddingSubset Size (n)=202026.04 | 0.68 | 0.826 | 49 | 350 | |
| Anchor PointsSubset Size (n)=102026.04 | 0.659 | 0.816 | 55 | — | |
| Random-Search-LearnSubset Size (n)=502026.04 | 0.651 | 0.728 | 141 | — | |
| Variance-basedSubset Size (n)=2002026.04 | 0.648 | 0.591 | — | — | |
| Difficulty-basedSubset Size (n)=302026.04 | 0.643 | 0.766 | 93 | 500 | |
| Random-Sampling-LearnSubset Size (n)=502026.04 | 0.637 | 0.72 | 158 | 900 | |
| Random SamplingSubset Size (n)=302026.04 | 0.623 | 0.751 | 127 | 500 | |
| Semantic EmbeddingSubset Size (n)=302026.04 | 0.619 | 0.723 | 99 | — | |
| IRT-basedSubset Size (n)=302026.04 | 0.611 | 0.736 | 142 | 800 | |
| Variance-basedSubset Size (n)=1002026.04 | 0.601 | 0.591 | — | — | |
| Difficulty-basedSubset Size (n)=202026.04 | 0.595 | 0.766 | 93 | 500 | |
| Random-Search-LearnSubset Size (n)=302026.04 | 0.582 | 0.728 | 141 | — | |
| Random-Sampling-LearnSubset Size (n)=302026.04 | 0.576 | 0.72 | 158 | 900 | |
| Acoustic EmbeddingSubset Size (n)=102026.04 | 0.557 | 0.709 | 166 | — | |
| Variance-basedSubset Size (n)=502026.04 | 0.555 | 0.591 | — | — | |
| Random SamplingSubset Size (n)=202026.04 | 0.544 | 0.751 | 127 | 500 | |
| IRT-basedSubset Size (n)=202026.04 | 0.544 | 0.736 | 142 | 800 | |
| Random-Search-LearnSubset Size (n)=202026.04 | 0.54 | 0.728 | 141 | — | |
| Random-Sampling-LearnSubset Size (n)=202026.04 | 0.528 | 0.72 | 158 | 900 | |
| Acoustic EmbeddingSubset Size (n)=302026.04 | 0.528 | 0.709 | 166 | — | |
| Combined EmbeddingSubset Size (n)=102026.04 | 0.523 | 0.826 | 49 | 350 | |
| Variance-basedSubset Size (n)=302026.04 | 0.521 | 0.591 | — | — | |
| Variance-basedSubset Size (n)=202026.04 | 0.487 | 0.591 | — | — | |
| Random-Search-LearnSubset Size (n)=102026.04 | 0.48 | 0.728 | 141 | — | |
| Difficulty-basedSubset Size (n)=102026.04 | 0.465 | 0.766 | 93 | 500 | |
| Semantic EmbeddingSubset Size (n)=202026.04 | 0.46 | 0.723 | 99 | — | |
| Random-Sampling-LearnSubset Size (n)=102026.04 | 0.428 | 0.72 | 158 | 900 | |
| Random SamplingSubset Size (n)=102026.04 | 0.427 | 0.751 | 127 | 500 | |
| Variance-basedSubset Size (n)=102026.04 | 0.415 | 0.591 | — | — | |
| Semantic EmbeddingSubset Size (n)=102026.04 | 0.371 | 0.723 | 99 | — | |
| IRT-basedSubset Size (n)=102026.04 | 0.364 | 0.736 | 142 | 800 | |
| Acoustic EmbeddingSubset Size (n)=202026.04 | 0.33 | 0.709 | 166 | — |