3-category taxonomy classification (multiple responses allowed) on Mutation Dataset (Experiment C)
97.64Agreement Coefficient (AC)Deepseek-r1-7b
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Deepseek-r1-7bFew-shot setting=One-Shot2026.01 | 97.64 | 83.69 | 28.53 | 57.72 | |
| Deepseek-r1-7bFew-shot setting=Zero-Shot2026.01 | 94.68 | 83.26 | 30.39 | 57.84 | |
| Llama-8bFew-shot setting=Zero-Shot2026.01 | 84.93 | 96.82 | 27.19 | 56.03 | |
| Llama-70bFew-shot setting=Zero-Shot2026.01 | 79.76 | 40.04 | 31.72 | 46.33 | |
| Llama-8bFew-shot setting=One-Shot2026.01 | 77.55 | 94.7 | 20.51 | 50.02 | |
| Llama-70bFew-shot setting=One-Shot2026.01 | 75.63 | 23.73 | 23.4 | 37.64 | |
| Deepseek-r1-7bFew-shot setting=Two-Shot2026.01 | 66.47 | 67.16 | 68.57 | 67.74 | |
| Llama-70bFew-shot setting=Two-Shot2026.01 | 65.58 | 34.53 | 41.75 | 46.85 | |
| Gemini-2.5-ProFew-shot setting=Zero-Shot2026.01 | 59.53 | 94.28 | 99.55 | 87.7 | |
| Gemini-2.5-ProFew-shot setting=One-Shot2026.01 | 57.02 | 93.43 | 98.14 | 86.09 | |
| Llama-8bFew-shot setting=Two-Shot2026.01 | 55.39 | 70.97 | 48.51 | 54.63 | |
| Gemini-2.5-ProFew-shot setting=Two-Shot2026.01 | 49.93 | 93.86 | 98.22 | 84.29 |