# Text Embedding on MTEB (8 Representative Tasks, test)

Source: [Text Embedding on MTEB (8 Representative Tasks, test)](<https://www.sota2.com/research/sota/text-embedding-on-mteb-8-representative-tasks-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-10T09:59:19.311Z

Evaluation of text embedding models using a specific subset of 8 representative tasks from the Massive Text Embedding Benchmark (MTEB) suite on the test split, covering Classification, Clustering, Retrieval, STS, and Pair Classification metrics.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Massive Text Embedding Evaluation |
| Dataset (including any stated split) | MTEB 8 representative tasks including Banking77, SprintDuplicateQuestions, TwitterSemEval, ArguAna (test) |
| Metric | Classification Acc |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 8 of 8 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Classification Acc](<https://www.sota2.com/research/sota/text-embedding-on-mteb-8-representative-tasks-test.md?metric=Classification%20Acc>): Higher is better; unit: PERCENT.

- [Clustering V-measure](<https://www.sota2.com/research/sota/text-embedding-on-mteb-8-representative-tasks-test.md?metric=Clustering%20V-measure>): Higher is better; unit: PERCENT.

- [Retrieval nDCG@10](<https://www.sota2.com/research/sota/text-embedding-on-mteb-8-representative-tasks-test.md?metric=Retrieval%20nDCG%4010>): Higher is better; unit: PERCENT.

- [STS Spearman Correlation](<https://www.sota2.com/research/sota/text-embedding-on-mteb-8-representative-tasks-test.md?metric=STS%20Spearman%20Correlation>): Higher is better; unit: PERCENT.

- [Pair Classification AP](<https://www.sota2.com/research/sota/text-embedding-on-mteb-8-representative-tasks-test.md?metric=Pair%20Classification%20AP>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Classification Acc | Result date | Report paper |
| --- | --- | --- | --- |
| Sentence-BERT | 0.752 | Not recorded | 6902f0f13ef7ccd74e5e4258 |
| Jina-v2-small | 0.738 | Not recorded | 6902f0f13ef7ccd74e5e4258 |
| GTE-tiny | 0.721 | Not recorded | 6902f0f13ef7ccd74e5e4258 |
| BGE-micro | 0.714 | Not recorded | 6902f0f13ef7ccd74e5e4258 |
| DistilBERT | 0.703 | Not recorded | 6902f0f13ef7ccd74e5e4258 |
| SwiftEmbed | 0.589 | Not recorded | 6902f0f13ef7ccd74e5e4258 |
| FastText | 0.582 | Not recorded | 6902f0f13ef7ccd74e5e4258 |
| GloVe-840B | 0.567 | Not recorded | 6902f0f13ef7ccd74e5e4258 |

### Sentence-BERT

Classification Acc: 0.752. Result ID: 69afeb6d6dbb7075b958cbef.

[Reporting paper](<https://www.sota2.com/research/paper/6902f0f13ef7ccd74e5e4258.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6ecf1e209131cea424f7.md>)

[Implementation](<https://github.com/embeddings-benchmark/mteb>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### Jina-v2-small

Classification Acc: 0.738. Result ID: 69afeb6d6dbb7075b958cbf2.

[Reporting paper](<https://www.sota2.com/research/paper/6902f0f13ef7ccd74e5e4258.md>)

Recorded evaluation configuration: {"Model size":"small","Version":"v2"}. This record does not assert independent reproduction.

### GTE-tiny

Classification Acc: 0.721. Result ID: 69afeb6d6dbb7075b958cbf0.

[Reporting paper](<https://www.sota2.com/research/paper/6902f0f13ef7ccd74e5e4258.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6ecf1e209131cea424f7.md>)

[Implementation](<https://github.com/embeddings-benchmark/mteb>)

Recorded evaluation configuration: {"Model size":"tiny"}. This record does not assert independent reproduction.

### BGE-micro

Classification Acc: 0.714. Result ID: 69afeb6d6dbb7075b958cbf1.

[Reporting paper](<https://www.sota2.com/research/paper/6902f0f13ef7ccd74e5e4258.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6ecf1e209131cea424f7.md>)

[Implementation](<https://github.com/embeddings-benchmark/mteb>)

Recorded evaluation configuration: {"Model size":"micro"}. This record does not assert independent reproduction.

### DistilBERT

Classification Acc: 0.703. Result ID: 69afeb6d6dbb7075b958cbf3.

[Reporting paper](<https://www.sota2.com/research/paper/6902f0f13ef7ccd74e5e4258.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6ecf1e209131cea424f7.md>)

[Implementation](<https://github.com/embeddings-benchmark/mteb>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### SwiftEmbed

Classification Acc: 0.589. Result ID: 69afeb6d6dbb7075b958cbf6.

[Reporting paper](<https://www.sota2.com/research/paper/6902f0f13ef7ccd74e5e4258.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6902f0f13ef7ccd74e5e4258.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### FastText

Classification Acc: 0.582. Result ID: 69afeb6d6dbb7075b958cbf4.

[Reporting paper](<https://www.sota2.com/research/paper/6902f0f13ef7ccd74e5e4258.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6ecf1e209131cea424f7.md>)

[Implementation](<https://github.com/embeddings-benchmark/mteb>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### GloVe-840B

Classification Acc: 0.567. Result ID: 69afeb6d6dbb7075b958cbf5.

[Reporting paper](<https://www.sota2.com/research/paper/6902f0f13ef7ccd74e5e4258.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6ecf1e209131cea424f7.md>)

[Implementation](<https://github.com/embeddings-benchmark/mteb>)

Recorded evaluation configuration: {"Vocab size":"840B"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
