Job-skill Extraction on FIJO
97.06Precision+kNN
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| +kNNEvaluation Protocol=RELAX / NER-STYLE, Setting=5-shot, Retrieval (kNN)=true2026.04 | 97.06 | 75 | 84.62 | |
| Qwen2.5 (5-shot + kNN + data-specific prompt)Evaluation Protocol=RELAX / NER-STYLE, Model=Qwen2.5, Setting=5-shot, Retrieval (kNN)=true, Prompting Strategy=data-specific prompt2026.04 | 88.07 | 98.38 | 93.01 | |
| SRICL-Qwen2.5-14B (Our method)Evaluation Protocol=RELAX / NER-STYLE, Model=Qwen2.5-14B2026.04 | 73.13 | 99.63 | 85.63 | |
| 5-shotEvaluation Protocol=RELAX / NER-STYLE, Setting=5-shot2026.04 | 67.74 | 69.24 | 47.68 | |
| +kNN+specificEvaluation Protocol=STRICT / NER-STYLE, Setting=5-shot, Retrieval (kNN)=true, Prompting Strategy=specific2026.04 | 55.31 | 48.48 | 51.67 | |
| +kNN+specificEvaluation Protocol=RELAX / NER-STYLE, Setting=5-shot, Retrieval (kNN)=true, Prompting Strategy=specific2026.04 | 55.31 | 48.48 | 51.67 | |
| GPT-3.5 Version (+kNN+specific)Evaluation Protocol=STRICT / NER-STYLE, Retrieval (kNN)=true, Prompting Strategy=specific, Model=GPT-3.52026.04 | 43.7 | 44.7 | 44.2 | |
| +kNNEvaluation Protocol=STRICT / NER-STYLE, Setting=5-shot, Retrieval (kNN)=true2026.04 | 41.18 | 31.82 | 35.9 | |
| SRICL-Qwen2.5-14B (Our method)Evaluation Protocol=STRICT / NER-STYLE, Model=Qwen2.5-14B2026.04 | 35.38 | 37.4 | 36.36 | |
| Llama-3-8B-SRICL(B8-Our Method)Evaluation Protocol=STRICT / NER-STYLE, Model=Llama-3-8B, Method Variant=B82026.04 | 23.94 | 27.64 | 25.66 | |
| zero-shot+specific (GPT-3.5)Evaluation Protocol=STRICT / NER-STYLE, Prompting Strategy=specific, Model=GPT-3.5, Setting=zero-shot2026.04 | 21.43 | 9.68 | 13.33 | |
| 5-shotEvaluation Protocol=STRICT / NER-STYLE, Setting=5-shot2026.04 | 20 | 9.09 | 12.5 | |
| Qwen2.5 (5-shot + kNN + data-specific prompt)Evaluation Protocol=STRICT / NER-STYLE, Model=Qwen2.5, Setting=5-shot, Retrieval (kNN)=true, Prompting Strategy=data-specific prompt2026.04 | 9.47 | 10.58 | 10 | |
| Llama-3-8B (SRICL- Our Method)Evaluation Protocol=RELAX / NER-STYLE, Model=Llama-3-8B2026.04 | 9.47 | 10.58 | 10 | |
| zero-shot+specific (GPT-3.5)Evaluation Protocol=RELAX / NER-STYLE, Prompting Strategy=specific, Model=GPT-3.5, Setting=zero-shot2026.04 | 7.89 | 2.1 | 3.315 |