Reasoning in IoT (RIT) Classification on Mutation Dataset (Experiment A)
82.06WACLlama-8b
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Llama-8bshot_count=Two-Shot2026.01 | 82.06 | 19.95 | 83.56 | 11.49 | 41.25 | 4.15 | 35.19 | |
| Deepseek-r1-7bshot_count=One-Shot2026.01 | 80.4 | 21.81 | 8.72 | 45.4 | 24.4 | 24.62 | 30.42 | |
| Deepseek-r1-7bshot_count=Zero-Shot2026.01 | 76.08 | 36.17 | 22.15 | 31.61 | 30.02 | 24.07 | 33.95 | |
| Deepseek-r1-7bshot_count=Two-Shot2026.01 | 76.08 | 13.83 | 11.74 | 35.06 | 19.1 | 25.73 | 27.33 | |
| Llama-8bshot_count=Zero-Shot2026.01 | 74.42 | 36.44 | 75.17 | 22.41 | 57.78 | 12.17 | 42.97 | |
| Llama-70bshot_count=Zero-Shot2026.01 | 72.43 | 9.84 | 35.91 | 13.79 | 53.13 | 2.07 | 29.34 | |
| Llama-8bshot_count=One-Shot2026.01 | 60.13 | 24.47 | 74.5 | 12.64 | 63.72 | 8.85 | 39.2 | |
| Gemini-1.5-Proshot_count=Zero-Shot2026.01 | 42.86 | 39.36 | 87.25 | 81.03 | 41.89 | 51.73 | 52.63 | |
| Llama-70bshot_count=Two-Shot2026.01 | 37.87 | 19.68 | 30.54 | 20.11 | 26.48 | 2.35 | 19.88 | |
| Gemini-1.5-Proshot_count=One-Shot2026.01 | 36.21 | 55.32 | 76.85 | 80.46 | 49.76 | 63.35 | 58.28 | |
| Gemini-1.5-Proshot_count=Two-Shot2026.01 | 35.88 | 60.11 | 19.13 | 89.08 | 48.31 | 57.95 | 50.74 | |
| Llama-70bshot_count=One-Shot2026.01 | 17.61 | 4.79 | 31.21 | 12.64 | 29.37 | 2.77 | 15.59 |