Job-skill Extraction on GREEN
92.52Precision+kNN
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| +kNNEvaluation Protocol=RELAX / NER-STYLE, Setting=5-shot, Retrieval (kNN)=true2026.04 | 92.52 | 69.72 | 79.52 | |
| +kNN+specificEvaluation Protocol=RELAX / NER-STYLE, Setting=5-shot, Retrieval (kNN)=true, Prompting Strategy=specific2026.04 | 69.82 | 63.5 | 66.51 | |
| Qwen2.5 (5-shot + kNN + data-specific prompt)Evaluation Protocol=RELAX / NER-STYLE, Model=Qwen2.5, Setting=5-shot, Retrieval (kNN)=true, Prompting Strategy=data-specific prompt2026.04 | 67.15 | 43.48 | 52.78 | |
| SRICL-Qwen2.5-14B (Our method)Evaluation Protocol=RELAX / NER-STYLE, Model=Qwen2.5-14B2026.04 | 63.84 | 65.53 | 64.67 | |
| GPT-3.5 Version (+kNN+specific)Evaluation Protocol=STRICT / NER-STYLE, Retrieval (kNN)=true, Prompting Strategy=specific, Model=GPT-3.52026.04 | 30.8 | 32.8 | 31.8 | |
| +kNN+specificEvaluation Protocol=STRICT / NER-STYLE, Setting=5-shot, Retrieval (kNN)=true, Prompting Strategy=specific2026.04 | 30.02 | 27.3 | 28.59 | |
| SRICL-Qwen2.5-14B (Our method)Evaluation Protocol=STRICT / NER-STYLE, Model=Qwen2.5-14B2026.04 | 26.69 | 20.54 | 23.21 | |
| 5-shotEvaluation Protocol=RELAX / NER-STYLE, Setting=5-shot2026.04 | 25.9 | 60.14 | 70.82 | |
| 5-shotEvaluation Protocol=STRICT / NER-STYLE, Setting=5-shot2026.04 | 24.53 | 8.72 | 12.87 | |
| Llama-3-8B-SRICL(B8-Our Method)Evaluation Protocol=STRICT / NER-STYLE, Model=Llama-3-8B, Method Variant=B82026.04 | 23.98 | 20.98 | 22.38 | |
| +kNNEvaluation Protocol=STRICT / NER-STYLE, Setting=5-shot, Retrieval (kNN)=true2026.04 | 22.86 | 18.31 | 20.88 | |
| Qwen2.5 (5-shot + kNN + data-specific prompt)Evaluation Protocol=STRICT / NER-STYLE, Model=Qwen2.5, Setting=5-shot, Retrieval (kNN)=true, Prompting Strategy=data-specific prompt2026.04 | 18.94 | 11.76 | 14.51 | |
| Llama-3-8B (SRICL- Our Method)Evaluation Protocol=RELAX / NER-STYLE, Model=Llama-3-8B2026.04 | 18.94 | 11.76 | 14.51 | |
| zero-shot+specific (GPT-3.5)Evaluation Protocol=STRICT / NER-STYLE, Prompting Strategy=specific, Model=GPT-3.5, Setting=zero-shot2026.04 | 7.89 | 2.1 | 3.32 | |
| zero-shot+specific (GPT-3.5)Evaluation Protocol=RELAX / NER-STYLE, Prompting Strategy=specific, Model=GPT-3.5, Setting=zero-shot2026.04 | 1.54 | 4.76 | 2.326 |