Binary classification on CTAF-KHAF Synthetic (test)
96.4Macro F1 ScoreQwen 2.5 7B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Qwen 2.5 7BPrompting Strategy=Direct prompting, Shot Setting=ZS2026.05 | 96.4 | 96.8 | 99.5 | |
| Qwen 2.5 7BPrompting Strategy=Chain-of-thought, Shot Setting=OS2026.05 | 96.4 | 96.8 | 97.9 | |
| Claude 3.5 SonnetPrompting Strategy=Chain-of-thought, Shot Setting=FS2026.05 | 95.2 | 95.7 | 99.4 | |
| Gemma 2 9BPrompting Strategy=Direct prompting, Shot Setting=OS2026.05 | 93.1 | 93.6 | 97.9 | |
| GPT-4oPrompting Strategy=Chain-of-thought, Shot Setting=ZS2026.05 | 93.1 | 93.6 | 97.7 | |
| GPT-4oPrompting Strategy=Chain-of-thought, Shot Setting=OS2026.05 | 93.1 | 93.6 | 98.8 | |
| Gemma 2 9BPrompting Strategy=Direct prompting, Shot Setting=FS2026.05 | 92 | 92.6 | 98.2 | |
| Gemma 2 9BPrompting Strategy=Chain-of-thought, Shot Setting=FS2026.05 | 92 | 92.6 | 96.8 | |
| Qwen 2.5 7BPrompting Strategy=Chain-of-thought, Shot Setting=FS2026.05 | 91.9 | 92.6 | 98.4 | |
| GPT-4oPrompting Strategy=Chain-of-thought, Shot Setting=FS2026.05 | 90.9 | 91.5 | 97.8 | |
| GPT-4oPrompting Strategy=Direct prompting, Shot Setting=FS2026.05 | 90.7 | 91.5 | 97.6 | |
| GPT-4oPrompting Strategy=Direct prompting, Shot Setting=OS2026.05 | 90 | 91.5 | 98.3 | |
| Mistral 7BPrompting Strategy=Chain-of-thought, Shot Setting=ZS2026.05 | 89.8 | 90.4 | 94.2 | |
| Gemma 2 9BPrompting Strategy=Direct prompting, Shot Setting=ZS2026.05 | 89.8 | 90.4 | 95.1 | |
| Claude 3.5 SonnetPrompting Strategy=Chain-of-thought, Shot Setting=OS2026.05 | 89.8 | 91.5 | 98.2 | |
| Claude 3.5 SonnetPrompting Strategy=Direct prompting, Shot Setting=FS2026.05 | 88.9 | 90.4 | 96.4 | |
| Mistral 7BPrompting Strategy=Direct prompting, Shot Setting=FS2026.05 | 87.7 | 88.3 | 98.4 | |
| Gemma 2 9BPrompting Strategy=Chain-of-thought, Shot Setting=ZS2026.05 | 87.7 | 88.3 | 97.8 | |
| Qwen 2.5 7BPrompting Strategy=Direct prompting, Shot Setting=OS2026.05 | 87.6 | 88.3 | 97.3 | |
| Gemma 2 9BPrompting Strategy=Chain-of-thought, Shot Setting=OS2026.05 | 87.6 | 88.3 | 96.9 | |
| GPT-4o miniPrompting Strategy=Chain-of-thought, Shot Setting=OS2026.05 | 86.7 | 89.4 | 95.2 | |
| Claude 3.5 SonnetPrompting Strategy=Chain-of-thought, Shot Setting=ZS2026.05 | 86.4 | 88.3 | 96.1 | |
| Mistral 7BPrompting Strategy=Direct prompting, Shot Setting=ZS2026.05 | 85.6 | 86.2 | 89.7 | |
| GPT-4o miniPrompting Strategy=Direct prompting, Shot Setting=OS2026.05 | 85.2 | 88.3 | 93.5 | |
| Claude 3.5 SonnetPrompting Strategy=Direct prompting, Shot Setting=OS2026.05 | 84.4 | 87.2 | 97.4 | |
| Qwen 2.5 7BPrompting Strategy=Direct prompting, Shot Setting=FS2026.05 | 82.5 | 83 | 98.3 | |
| Mistral 7BPrompting Strategy=Direct prompting, Shot Setting=OS2026.05 | 82.5 | 83 | 96.8 | |
| Claude 3.5 SonnetPrompting Strategy=Direct prompting, Shot Setting=ZS2026.05 | 80.4 | 83 | 93.7 | |
| GPT-4oPrompting Strategy=Direct prompting, Shot Setting=ZS2026.05 | 78.7 | 84 | 99.1 | |
| GPT-4o miniPrompting Strategy=Direct prompting, Shot Setting=FS2026.05 | 77.6 | 83 | 92.4 | |
| GPT-4o miniPrompting Strategy=Chain-of-thought, Shot Setting=FS2026.05 | 77.6 | 83 | 91.5 | |
| Qwen 2.5 7BPrompting Strategy=Chain-of-thought, Shot Setting=ZS2026.05 | 70 | 76.6 | 99.4 | |
| GPT-4o miniPrompting Strategy=Direct prompting, Shot Setting=ZS2026.05 | 67.2 | 77.7 | 90.2 | |
| GPT-4o miniPrompting Strategy=Chain-of-thought, Shot Setting=ZS2026.05 | 67.2 | 77.7 | 93.5 | |
| Mistral 7BPrompting Strategy=Chain-of-thought, Shot Setting=OS2026.05 | 33.2 | 38.3 | 54.8 | |
| Mistral 7BPrompting Strategy=Chain-of-thought, Shot Setting=FS2026.05 | 30 | 36.2 | 42.5 |