Job-skill Extraction on KOMPETENCER
63.51PrecisionSRICL-Qwen2.5-14B (Our method)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SRICL-Qwen2.5-14B (Our method)Evaluation Protocol=RELAX / NER-STYLE, Model=Qwen2.5-14B2026.04 | 63.51 | 44.76 | 52.51 | |
| Qwen2.5 (5-shot + kNN + data-specific prompt)Evaluation Protocol=RELAX / NER-STYLE, Model=Qwen2.5, Setting=5-shot, Retrieval (kNN)=true, Prompting Strategy=data-specific prompt2026.04 | 45.12 | 35.24 | 39.57 | |
| Llama-3-8B-SRICL(B8-Our Method)Evaluation Protocol=STRICT / NER-STYLE, Model=Llama-3-8B, Method Variant=B82026.04 | 42.11 | 45.71 | 43.84 | |
| SRICL-Qwen2.5-14B (Our method)Evaluation Protocol=STRICT / NER-STYLE, Model=Qwen2.5-14B2026.04 | 39.19 | 27.62 | 32.4 | |
| +kNN+specificEvaluation Protocol=RELAX / NER-STYLE, Setting=5-shot, Retrieval (kNN)=true, Prompting Strategy=specific2026.04 | 30.15 | 59.41 | 40 | |
| 5-shotEvaluation Protocol=STRICT / NER-STYLE, Setting=5-shot2026.04 | 27.27 | 25 | 26.09 | |
| +kNNEvaluation Protocol=RELAX / NER-STYLE, Setting=5-shot, Retrieval (kNN)=true2026.04 | 23.53 | 22.22 | 22.86 | |
| 5-shotEvaluation Protocol=RELAX / NER-STYLE, Setting=5-shot2026.04 | 11.59 | 38.09 | 17.78 | |
| GPT-3.5 Version (+kNN+specific)Evaluation Protocol=STRICT / NER-STYLE, Retrieval (kNN)=true, Prompting Strategy=specific, Model=GPT-3.52026.04 | 11.2 | 29.2 | 16.1 | |
| +kNNEvaluation Protocol=STRICT / NER-STYLE, Setting=5-shot, Retrieval (kNN)=true2026.04 | 10.55 | 23.53 | 22.22 | |
| +kNN+specificEvaluation Protocol=STRICT / NER-STYLE, Setting=5-shot, Retrieval (kNN)=true, Prompting Strategy=specific2026.04 | 10.55 | 20.79 | 14 | |
| Qwen2.5 (5-shot + kNN + data-specific prompt)Evaluation Protocol=STRICT / NER-STYLE, Model=Qwen2.5, Setting=5-shot, Retrieval (kNN)=true, Prompting Strategy=data-specific prompt2026.04 | 6.1 | 4.76 | 5.35 | |
| Llama-3-8B (SRICL- Our Method)Evaluation Protocol=RELAX / NER-STYLE, Model=Llama-3-8B2026.04 | 6.1 | 4.76 | 5.35 | |
| zero-shot+specific (GPT-3.5)Evaluation Protocol=RELAX / NER-STYLE, Prompting Strategy=specific, Model=GPT-3.5, Setting=zero-shot2026.04 | 5.92 | 13.8 | 8.29 | |
| zero-shot+specific (GPT-3.5)Evaluation Protocol=STRICT / NER-STYLE, Prompting Strategy=specific, Model=GPT-3.5, Setting=zero-shot2026.04 | 1.54 | 4.76 | 2.33 |