Job-skill Extraction on SAYFULLINA
78.66PrecisionSRICL-Qwen2.5-14B (Our method)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SRICL-Qwen2.5-14B (Our method)Evaluation Protocol=RELAX / NER-STYLE, Model=Qwen2.5-14B2026.04 | 78.66 | 81.72 | 80.16 | |
| SRICL-Qwen2.5-14B (Our method)Evaluation Protocol=STRICT / NER-STYLE, Model=Qwen2.5-14B2026.04 | 61.3 | 61.07 | 61.18 | |
| 5-shotEvaluation Protocol=RELAX / NER-STYLE, Setting=5-shot2026.04 | 57.94 | 52.31 | 26.64 | |
| +kNN+specificEvaluation Protocol=RELAX / NER-STYLE, Setting=5-shot, Retrieval (kNN)=true, Prompting Strategy=specific2026.04 | 44.91 | 55.09 | 49.48 | |
| +kNNEvaluation Protocol=RELAX / NER-STYLE, Setting=5-shot, Retrieval (kNN)=true2026.04 | 42.1 | 50.71 | 46.01 | |
| Qwen2.5 (5-shot + kNN + data-specific prompt)Evaluation Protocol=RELAX / NER-STYLE, Model=Qwen2.5, Setting=5-shot, Retrieval (kNN)=true, Prompting Strategy=data-specific prompt2026.04 | 36.57 | 56.84 | 44.51 | |
| 5-shotEvaluation Protocol=STRICT / NER-STYLE, Setting=5-shot2026.04 | 30.35 | 36.88 | 33.2 | |
| GPT-3.5 Version (+kNN+specific)Evaluation Protocol=STRICT / NER-STYLE, Retrieval (kNN)=true, Prompting Strategy=specific, Model=GPT-3.52026.04 | 29.8 | 47.3 | 36.6 | |
| +kNN+specificEvaluation Protocol=STRICT / NER-STYLE, Setting=5-shot, Retrieval (kNN)=true, Prompting Strategy=specific2026.04 | 26.6 | 32.63 | 29.31 | |
| +kNNEvaluation Protocol=STRICT / NER-STYLE, Setting=5-shot, Retrieval (kNN)=true2026.04 | 24.97 | 30.08 | 27.29 | |
| Llama-3-8B-SRICL(B8-Our Method)Evaluation Protocol=STRICT / NER-STYLE, Model=Llama-3-8B, Method Variant=B82026.04 | 22.15 | 29.03 | 25.13 | |
| Qwen2.5 (5-shot + kNN + data-specific prompt)Evaluation Protocol=STRICT / NER-STYLE, Model=Qwen2.5, Setting=5-shot, Retrieval (kNN)=true, Prompting Strategy=data-specific prompt2026.04 | 18.44 | 28.54 | 22.41 | |
| Llama-3-8B (SRICL- Our Method)Evaluation Protocol=RELAX / NER-STYLE, Model=Llama-3-8B2026.04 | 18.44 | 28.54 | 22.41 | |
| zero-shot+specific (GPT-3.5)Evaluation Protocol=STRICT / NER-STYLE, Prompting Strategy=specific, Model=GPT-3.5, Setting=zero-shot2026.04 | 5.11 | 3.42 | 4.1 | |
| zero-shot+specific (GPT-3.5)Evaluation Protocol=RELAX / NER-STYLE, Prompting Strategy=specific, Model=GPT-3.5, Setting=zero-shot2026.04 | 3.08 | 11.94 | 4.9 |