Job and Skill Intelligence Tasks on WorkBench (test)
17.9Job2Skill MAPUWE
Evaluation Results
| Method | Links | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| UWEModel Category=Task-specific models, Param.=109M (×1), h=768, Latency (ms)=15.9 ± 0.6, FLOPS=3.0 ± 0.9B, Precision=bf16, Batch size=12025.11 | 17.9 | 37.8 | 37.3 | 49.4 | 39.3 | 60.8 | 88.9 | 95.9 | 10.4 | 33.8 | 52.3 | 73.5 | 42.3 | 64.2 | 40.2 | 57.7 | |
| ContextMatchModel Category=Task-specific models, Param.=109M, h=768, Latency (ms)=16.1 ± 0.7, FLOPS=1.6 ± 0.4B, Precision=bf16, Batch size=12025.11 | 14.6 | 34.5 | 32.2 | 43.3 | 36.8 | 55.3 | 86.5 | 94.3 | 7.7 | 24.6 | 53.6 | 73.9 | 42.2 | 65.8 | 37.6 | 53.7 | |
| Qwen3-8BModel Category=Gen. emb. models, Param.=8B (×73.4), h=4,096, Latency (ms)=60.2 ± 0.8, FLOPS=92.4 ± 19.0B, Precision=bf16, Batch size=12025.11 | 13.8 | 32.7 | 32.7 | 44.4 | 35.3 | 55.5 | 85.7 | 94.5 | 13 | 40.8 | 39.1 | 61 | 29.1 | 49 | 35.8 | 53.8 | |
| Qwen3-4BModel Category=Gen. emb. models, Param.=4B (×36.7), h=2,560, Latency (ms)=60.0 ± 0.1, FLOPS=48.4 ± 9.9B, Precision=bf16, Batch size=12025.11 | 13 | 31.6 | 31.9 | 43.8 | 33.6 | 53.7 | 85.1 | 94.1 | 12.3 | 37.9 | 37.6 | 58.4 | 28.9 | 46.4 | 34.8 | 52.3 | |
| Emb. GemmaModel Category=Gen. emb. models, Param.=308M (×2.8), h=768, Latency (ms)=58.9 ± 0.1, FLOPS=1.5 ± 0.3B, Precision=bf16, Batch size=12025.11 | 12.6 | 30.1 | 29.9 | 41.6 | 36.7 | 54.6 | 85.1 | 93.9 | 9.3 | 28.7 | 38.1 | 59.3 | 27.3 | 44.9 | 34.4 | 50.2 | |
| MPNetModel Category=Gen. emb. models, Param.=109M, h=768, Latency (ms)=13.2 ± 0.2, FLOPS=1.3 ± 0.2B, Precision=bf16, Batch size=12025.11 | 11.8 | 29 | 29 | 40.3 | 32.9 | 50.8 | 84.7 | 93.2 | 10.1 | 30.7 | 31.2 | 49.8 | 19.3 | 36.7 | 32.3 | 47.9 | |
| Qwen3-0.6BModel Category=Gen. emb. models, Param.=596M (×5.5), h=1,024, Latency (ms)=46.2 ± 0.4, FLOPS=5.9 ± 1.2B, Precision=bf16, Batch size=12025.11 | 11.7 | 29.1 | 28.3 | 39.7 | 30.1 | 49.5 | 81.7 | 92.2 | 10.4 | 32.7 | 35.1 | 54.6 | 25.7 | 41.8 | 32.1 | 48.6 | |
| E5-mistral-7BModel Category=Gen. emb. models, Param.=7B (×65.2), h=4,096, Latency (ms)=61.6 ± 8.6, FLOPS=112.0 ± 21.0B, Precision=bf16, Batch size=12025.11 | 11.1 | 28.7 | 29.2 | 40.4 | 35.3 | 54.4 | 83.4 | 92.9 | 12.2 | 37.7 | 32 | 45.8 | 19.6 | 33.7 | 32.8 | 49 | |
| JobBERT-v2Model Category=Task-specific models, Param.=109M, h=1,024, Latency (ms)=13.4 ± 0.2, FLOPS=1.3 ± 0.2B, Precision=bf16, Batch size=12025.11 | 0.4 | 0.3 | 0.5 | 0.4 | 39 | 58.5 | 83 | 91.8 | 9.7 | 30.4 | 35.1 | 48.5 | 20.8 | 38.1 | 26.8 | 37.4 |