CTI Rule Generation on GTI (test)
89.5SAGenTI
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GenTIParams=7B2026.06 | 89.5 | 87.2 | 94.8 | 85.9 | 89.4 | |
| GPT-4oParams=Unknown2026.06 | 81.2 | 78.9 | 84.1 | 77.3 | 80.4 | |
| Claude-3.5-SonnetParams=175B2026.06 | 80.5 | 78.1 | 83.7 | 76.8 | 79.8 | |
| GPT-4-TurboParams=Unknown2026.06 | 79.9 | 77.3 | 82.9 | 75.9 | 79 | |
| Gemini-1.5-ProParams=Unknown2026.06 | 79.1 | 76.9 | 82.1 | 75.4 | 78.4 | |
| Qwen2.5-7BParams=7B2026.06 | 78.7 | 76.2 | 81.4 | 74.8 | 78 | |
| GPT-4Params=1.76T2026.06 | 78.3 | 75.8 | 80.9 | 74.4 | 77.3 | |
| Claude-3-OpusParams=175B2026.06 | 77.1 | 74.5 | 80.2 | 73.2 | 76.1 | |
| Gemini-1.0-ProParams=Unknown2026.06 | 75.9 | 73.4 | 79 | 72.1 | 74.9 | |
| Qwen2-7BParams=7B2026.06 | 75.4 | 72.6 | 78.5 | 71.3 | 74.4 | |
| DeepSeek-7BParams=7B2026.06 | 74.6 | 72.5 | 77.3 | 70.7 | 73.5 | |
| LLaMA-3.1-8BParams=8B2026.06 | 74.1 | 71.4 | 77.1 | 70.5 | 73 | |
| Mixtral-8x7BParams=45B2026.06 | 73.5 | 70.6 | 76.8 | 69.2 | 72.3 | |
| LLaMA-3-8BParams=8B2026.06 | 72.8 | 69.9 | 75.9 | 68.5 | 71.6 | |
| GPT-3.5 TurboParams=175B2026.06 | 72.1 | 68.6 | 75.4 | 70.3 | 71.5 | |
| Phi-3-MediumParams=14B2026.06 | 71.4 | 68.7 | 74.1 | 67.2 | 70.1 | |
| Mistral-7BParams=7B2026.06 | 70.5 | 67.9 | 73.3 | 66.3 | 69.5 |