Text Embedding on MTEB Multilingual V2 (test)
62.5Mean Score (TaskType)KaLM-Gemma3-12B
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| KaLM-Gemma3-12BParams=11.8B, Zero Shot=73%2026.04 | 62.5 | 83.8 | 77.9 | 55.8 | 5.5 | 33 | 84.7 | 67.3 | 75.7 | 79 | 72.3 | 1 | |
| GTE-Qwen2-7BParams=7.1B2026.04 | 55.9 | 73.9 | 61.5 | 52.8 | 4.9 | 25.5 | 85.1 | 65.5 | 60.1 | 74 | 62.5 | 22 | |
| BidirLM-Omni-2.5BParams=2.5B, Zero Shot=100%2026.04 | 55.1 | 72.2 | 65.5 | 51.6 | 0.9 | 26.6 | 80.7 | 63.4 | 59.4 | 75.7 | 63.1 | 17 | |
| BidirLM-1.7BParams=1.7B, Zero Shot=100%2026.04 | 54.8 | 72.2 | 65.7 | 51.5 | 0.5 | 26.6 | 80.2 | 62.1 | 59.9 | 74.2 | 62.9 | 18 | |
| BidirLM-1BParams=1.0B, Zero Shot=100%2026.04 | 53.6 | 71.8 | 65.9 | 50.3 | -0.4 | 25.1 | 79.8 | 58.6 | 56.5 | 74.6 | 62.1 | 24 | |
| GTE-Qwen2-1.5BParams=1.5B2026.04 | 52.7 | 62.5 | 58.3 | 52 | 0.7 | 24 | 81.6 | 62.6 | 60.8 | 71.6 | 59.5 | 40 | |
| BGE-M3Params=568M, Zero Shot=98%2026.04 | 52.2 | 79.1 | 60.4 | 40.9 | -3.1 | 20.1 | 80.8 | 62.8 | 54.6 | 74.1 | 59.6 | 38 | |
| BidirLM-0.6BParams=596M, Zero Shot=100%2026.04 | 51.9 | 66.7 | 62.2 | 49.6 | 0.8 | 23.8 | 78.4 | 58.1 | 56.5 | 70.8 | 59.6 | 36 | |
| Codestral Mamba2 7BFine-tuned=true, Architecture=Recurrent (embed-mamba2)2026.04 | 51.9 | — | — | — | — | — | — | — | — | — | 59.4 | — | |
| Mistral 7B v0.1Fine-tuned=true2026.04 | 50.5 | — | — | — | — | — | — | — | — | — | 58.6 | — | |
| KaLM-miniParams=494M, Zero Shot=92%2026.04 | 50 | 64.8 | 57.6 | 45.6 | -1.5 | 20.7 | 77.7 | 60.6 | 54.2 | 70.8 | 57 | 49 | |
| Stella-1.5BParams=1.5B, Zero Shot=90%2026.04 | 50 | 58.6 | 56.7 | 49.7 | 0.2 | 21.8 | 78.5 | 61.4 | 52.8 | 69.9 | 56.5 | 54 | |
| Qwen2 1.5BFine-tuned=true2026.04 | 48.7 | — | — | — | — | — | — | — | — | — | 56.2 | — | |
| BidirLM-270MParams=268M, Zero Shot=100%2026.04 | 48 | 59.5 | 58.8 | 45.9 | -2.1 | 19.7 | 76.7 | 56.8 | 48.7 | 67.7 | 55.5 | 58 | |
| Mamba2 1.3BFine-tuned=true, Architecture=Recurrent (embed-mamba2)2026.04 | 47.9 | — | — | — | — | — | — | — | — | — | 55.2 | — | |
| Nomic-v1Params=137M, Zero Shot=96%2026.04 | 39.1 | 28.4 | 49.4 | 40.8 | -3.2 | 17.8 | 71.7 | 46 | 37 | 64.2 | 45.7 | 87 |