Skill prediction on SkillsBench Real-task holdout n=65
74.2Set F1TF-IDF
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| TF-IDFRetrieval protocol=oracle-k2026.06 | 74.2 | 73.2 | 89.2 | 77.3 | 47.7 | |
| Qwen3-Emb.Retrieval protocol=oracle-k2026.06 | 73.4 | 72.7 | 90.9 | 76.9 | 47.7 | |
| SkillComposerType=Trained models2026.06 | 62.9 | 54.7 | 90.8 | 63.4 | 20 | |
| TF-IDFRetrieval protocol=best-k, k=22026.06 | 60.6 | 60.6 | 89.2 | 67.7 | 10.8 | |
| Gemini-2.5-flashType=LLM-judge2026.06 | 59.9 | 63.8 | 81.8 | 65.1 | 15.4 | |
| Qwen3-Emb.Retrieval protocol=best-k, k=32026.06 | 58.5 | 69.2 | 90.8 | 73.8 | 10.8 | |
| BM25Retrieval protocol=oracle-k2026.06 | 55.9 | 55.6 | 73.6 | 59.2 | 33.8 | |
| SkillComposerBaseType=Trained models2026.06 | 53.9 | 45 | 87.7 | 54.1 | 16.9 | |
| BM25Retrieval protocol=best-k, k=22026.06 | 47 | 48.7 | 72.3 | 54 | 7.7 | |
| Qwen3-0.6B-BaseType=Trained models2026.06 | 43.6 | 36.1 | 66.2 | 46 | 16.9 |