Skill prediction on SkillsBench Synthetic n=494 (test)
73.9Set F1SkillComposer
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| SkillComposerType=Trained models2026.06 | 73.9 | 72.4 | 86.5 | 75 | 41.3 | |
| Qwen3-0.6B-BaseType=Trained models2026.06 | 71.1 | 68.9 | 79.2 | 74.1 | 44.9 | |
| SkillComposerBaseType=Trained models2026.06 | 70.4 | 70.9 | 84.2 | 73.2 | 37.2 | |
| Gemini-2.5-flashType=LLM-judge2026.06 | 61 | 69 | 69.3 | 63.1 | 21.3 | |
| TF-IDFRetrieval protocol=oracle-k2026.06 | 58.4 | 58.4 | 81 | 63 | 28.7 | |
| TF-IDFRetrieval protocol=best-k, k=22026.06 | 52.5 | 53 | 82.2 | 58.9 | 4.3 | |
| Qwen3-Emb.Retrieval protocol=oracle-k2026.06 | 49 | 49 | 69.9 | 52 | 20.2 | |
| Qwen3-Emb.Retrieval protocol=best-k, k=32026.06 | 43.9 | 55 | 72.3 | 55.3 | 2.6 | |
| BM25Retrieval protocol=oracle-k2026.06 | 35.3 | 35.3 | 54.4 | 38.3 | 14.8 | |
| BM25Retrieval protocol=best-k, k=22026.06 | 33 | 33.7 | 53.9 | 37 | 2.2 |