IoT security static analysis vulnerability classification on Mutation Dataset Experiment D
63.81Accuracy (AC)Llama-70b
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Llama-70bShots=Zero-Shot2026.01 | 63.81 | 23.94 | 35.51 | 41 | |
| Llama-70bShots=Two-Shot2026.01 | 62.48 | 20.13 | 26.97 | 35.31 | |
| Llama-8bShots=Two-Shot2026.01 | 53.47 | 31.99 | 22.88 | 32.91 | |
| Llama-70bShots=One-Shot2026.01 | 53.32 | 23.94 | 33.58 | 37.11 | |
| Gemini-2.5-ProShots=Zero-Shot2026.01 | 46.53 | 92.37 | 73.11 | 69.54 | |
| Gemini-2.5-ProShots=One-Shot2026.01 | 45.49 | 91.95 | 75.63 | 70.54 | |
| Llama-8bShots=One-Shot2026.01 | 44.61 | 23.31 | 27.12 | 31.14 | |
| Gemini-2.5-ProShots=Two-Shot2026.01 | 44.46 | 79.24 | 80.24 | 70.34 | |
| Llama-8bShots=Zero-Shot2026.01 | 41.95 | 30.3 | 23.7 | 29.9 | |
| Deepseek-r1-7bShots=Zero-Shot2026.01 | 34.42 | 6.78 | 35.29 | 29.66 | |
| Deepseek-r1-7bShots=One-Shot2026.01 | 34.12 | 8.26 | 44.73 | 34.95 | |
| Deepseek-r1-7bShots=Two-Shot2026.01 | 26.14 | 9.96 | 47.18 | 34.43 |