Query Generation on SIEM executable queries AQL and YARA-L (test)
0.384BLEUSQM
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SQMArchitecture=Syntax-constrained meta-data driven query generation, Backbone=GPT-4o2026.04 | 0.384 | 0.731 | |
| Qwen3Model Type=Base Model2026.04 | 0.149 | 0.522 | |
| gpt-4oModel Type=Base Model2026.04 | 0.14 | 0.508 | |
| DeepSeek-V3Model Type=Base Model2026.04 | 0.137 | 0.492 | |
| gemmaModel Type=Base Model2026.04 | 0.121 | 0.495 | |
| gpt-4o-miniModel Type=Base Model2026.04 | 0.091 | 0.462 | |
| Llama-3.3Model Type=Base Model2026.04 | 0.081 | 0.429 |