Competency Tagging on UTC
57.1Micro-F1LBG (best K,τ)
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| LBG (best K,τ)Group=LLM-based2026.05 | 57.1 | 50.37 | 50.72 | 51.07 | 82.47 | |
| Few-shot LLMGroup=LLM-based2026.05 | 48.58 | 40.47 | 40.71 | 41.2 | 70.31 | |
| Zero-shot LLMGroup=LLM-based2026.05 | 44.82 | 36.89 | 36.98 | 38.77 | 60.2 | |
| BM25+Cross EncoderGroup=Retrieval & similarity-based2026.05 | 39.9 | 27.66 | 27.94 | — | 75.52 | |
| SBERT SimilarityGroup=Retrieval & similarity-based2026.05 | 14.59 | 14.34 | 7.45 | — | 73.57 | |
| LinearSVC + TF-IDFGroup=Supervised-based2026.05 | 12.18 | 12.01 | 12.11 | — | 34.62 | |
| LogReg + TF-IDFGroup=Supervised-based2026.05 | 11.89 | 11.74 | 11.87 | — | 39.17 | |
| Supervised TransformerGroup=Supervised-based2026.05 | 11.58 | 11.25 | 11.36 | — | 23.06 | |
| Dense (Bi-encoder)Group=Retrieval & similarity-based2026.05 | 8.06 | 7.99 | 8.04 | — | 73.57 | |
| Hybrid RRFGroup=Retrieval & similarity-based2026.05 | 7.73 | 7.67 | 7.7 | — | 56.41 | |
| BM25Group=Retrieval & similarity-based2026.05 | 7.7 | 7.58 | 7.62 | — | 34.07 |