Semantic Similarity on USEB (Universal Sentence Encoder Benchmark)
57.5AskU APSGPT-2.7B
Evaluation Results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SGPT-2.7BBatch Size=1024, BitFit=true, Pooling=weightedmean, Training=NLI2022.02 | 57.5 | 14 | 73.4 | 74.7 | 75.8 | 71 | 72.3 | 75.4 | 74.7 | 76.5 | 74.7 | |
| Curie SimilarityDate=Mar 2022, Training=OOD Unsupervised + OOD Supervised (NLI)2022.02 | 57.3 | 15.8 | 77.6 | 70.6 | 76.3 | 78.9 | 66.3 | 71.7 | 71.3 | 73 | 70.6 | |
| SGPT-5.8BBatch Size=1024, BitFit=true, Pooling=weightedmean, Training=NLI2022.02 | 57.1 | 16 | 76.3 | 77.2 | 76.5 | 76 | 75 | 78.2 | 77.1 | 78.3 | 77.2 | |
| SGPT-1.3BBatch Size=1024, BitFit=true, Pooling=weightedmean, Training=NLI2022.02 | 56 | 13.5 | 73.1 | 72.8 | 75.4 | 70.7 | 70.1 | 72.9 | 73.2 | 75 | 72.8 | |
| Ada SimilarityDate=Mar 2022, Training=OOD Unsupervised + OOD Supervised (NLI)2022.02 | 55.9 | 13.6 | 76.4 | 70.7 | 76.6 | 76.2 | 66.7 | 71 | 71.3 | 73.7 | 70.7 | |
| Davinci SimilarityDate=June 2022, Training=OOD Unsupervised + OOD Supervised (NLI)2022.02 | 55.9 | 16 | 76.7 | 70.6 | 76.3 | 77.1 | 66.3 | 71.3 | 71.3 | 73.3 | 70.6 | |
| SGPT-0.1BBatch Size=1024, BitFit=true, Pooling=weightedmean, Training=NLI2022.02 | 55.7 | 11.1 | 69.6 | 68 | 72.8 | 66.5 | 65.1 | 67.8 | 68.6 | 70.5 | 68 | |
| SGPT-0.1BBatch Size=64, Pooling=weightedmean, Training=NLI2022.02 | 54.9 | 11.2 | 69.3 | 68.9 | 72.7 | 66 | 66.2 | 68.9 | 68.9 | 71.7 | 68.9 | |
| SGPT-0.1BBatch Size=64, BitFit=true, Pooling=weightedmean, Training=NLI2022.02 | 54.9 | 10.8 | 68.8 | 67.8 | 72.3 | 65.3 | 64.7 | 67.4 | 68 | 70.8 | 67.8 | |
| SBERT-base-nli-v2Batch Size=64, BitFit=true, Training=OOD Unsupervised + OOD Supervised (NLI)2022.02 | 53.8 | 11.7 | 74.8 | 70.5 | 76.6 | 72.9 | 67.5 | 70.6 | 70.8 | 73 | 70.5 | |
| BM25Training=OOD Unsupervised2022.02 | 53.4 | 13.3 | 71.2 | 63.6 | 71.9 | 70.5 | 58.9 | 61.3 | 67.3 | 66.9 | 63.6 | |
| SBERT-base-nli-v2Batch Size=64, Training=OOD Unsupervised + OOD Supervised (NLI)2022.02 | 52.8 | 11.6 | 73.5 | 70.8 | 75.5 | 71.5 | 68 | 70.6 | 71.1 | 73.5 | 70.8 |