Vulnerability Reasoning on Vulnerability Reasoning Basic Prompt (test)
76.11RAccDAGVUL
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| DAGVULBase Model=Qwen3-8B, Inference Strategy=DAG2026.02 | 76.11 | 91.67 | 57.89 | 70.97 | |
| BaselineBase Model=GPT-OSS-20B-High, Inference Strategy=DAG2026.02 | 75.71 | 80 | 55.17 | 65.31 | |
| BaselineBase Model=Qwen3-30B-Reasoning, Inference Strategy=DAG2026.02 | 75 | 94.44 | 54.84 | 69.39 | |
| RAGBase Model=Claude-Sonnet-4.5, Inference Strategy=CoT2026.02 | 74.56 | 78 | 68.42 | 72.9 | |
| BaselineBase Model=Claude-Sonnet-4.5, Inference Strategy=DAG2026.02 | 73.68 | 72.13 | 77.19 | 74.58 | |
| RAGBase Model=Qwen3-30B-Reasoning, Inference Strategy=CoT2026.02 | 72.73 | 87.88 | 55.77 | 68.24 | |
| RAGBase Model=GPT-OSS-20B-High, Inference Strategy=CoT2026.02 | 70.54 | 74.47 | 62.5 | 67.96 | |
| BaselineBase Model=Claude-Sonnet-4.5, Inference Strategy=CoT2026.02 | 70.18 | 69.49 | 71.93 | 70.69 | |
| BaselineBase Model=Claude-Sonnet-4.5, Inference Strategy=Simple prompting2026.02 | 69.3 | 68.97 | 70.18 | 69.57 | |
| SFTBase Model=Qwen3-8B, Inference Strategy=DAG2026.02 | 67.54 | 72.73 | 56.14 | 63.37 | |
| DAGVULBase Model=Llama3.1-Nemo-8B, Inference Strategy=DAG2026.02 | 67.54 | 76.32 | 50.88 | 61.05 | |
| BaselineBase Model=Qwen3-8B-Reasoning, Inference Strategy=DAG2026.02 | 66.07 | 67.27 | 64.91 | 66.07 | |
| BaselineBase Model=Qwen3-8B-Reasoning, Inference Strategy=CoT2026.02 | 65.79 | 64.52 | 70.18 | 67.23 | |
| BaselineBase Model=Qwen3-8B-Reasoning, Inference Strategy=Simple prompting2026.02 | 64.91 | 66.67 | 59.65 | 62.96 | |
| BaselineBase Model=GPT-OSS-20B-High, Inference Strategy=Simple prompting2026.02 | 63.16 | 62.86 | 59.46 | 61.11 | |
| BaselineBase Model=Qwen3-8B, Inference Strategy=DAG2026.02 | 62.83 | 65.31 | 56.14 | 60.38 | |
| SFTBase Model=Llama3.1-Nemo-8B, Inference Strategy=DAG2026.02 | 60.53 | 63.04 | 50.88 | 56.31 | |
| BaselineBase Model=Qwen3-8B, Inference Strategy=CoT2026.02 | 59.65 | 57.14 | 77.19 | 65.67 | |
| BaselineBase Model=GPT-OSS-20B-High, Inference Strategy=CoT2026.02 | 58.04 | 60.98 | 44.64 | 51.55 | |
| BaselineBase Model=Qwen3-30B-Reasoning, Inference Strategy=Simple prompting2026.02 | 57.94 | 57.78 | 50 | 53.61 | |
| RAGBase Model=Qwen3-8B-Reasoning, Inference Strategy=CoT2026.02 | 57.89 | 55.84 | 75.44 | 64.18 | |
| RAGBase Model=Qwen3-8B, Inference Strategy=CoT2026.02 | 57.02 | 55.71 | 68.42 | 61.42 | |
| RAGBase Model=Llama3.1-Nemo-8B, Inference Strategy=CoT2026.02 | 56.14 | 55.93 | 57.89 | 56.9 | |
| RAGBase Model=Llama3.1-Nemo-8B-Reasoning, Inference Strategy=CoT2026.02 | 56.14 | 55.74 | 59.65 | 57.63 | |
| SFTBase Model=Llama3.1-Nemo-8B, Inference Strategy=CoT2026.02 | 54.39 | 54.72 | 50.88 | 52.73 | |
| BaselineBase Model=Qwen3-30B-Reasoning, Inference Strategy=CoT2026.02 | 54.08 | 54.29 | 39.58 | 45.78 | |
| BaselineBase Model=Qwen3-8B, Inference Strategy=Simple prompting2026.02 | 53.51 | 53.45 | 54.39 | 53.91 | |
| BaselineBase Model=Llama3.1-Nemo-8B, Inference Strategy=DAG2026.02 | 53.27 | 53.85 | 26.92 | 35.9 | |
| SFTBase Model=Qwen3-8B, Inference Strategy=CoT2026.02 | 52.63 | 51.58 | 85.96 | 64.47 | |
| ORPOBase Model=Llama3.1-Nemo-8B, Inference Strategy=CoT2026.02 | 52.63 | 52.94 | 47.37 | 50 | |
| BaselineBase Model=Llama3.1-Nemo-8B-Reasoning, Inference Strategy=DAG2026.02 | 51.4 | 51.16 | 41.51 | 45.83 | |
| ORPOBase Model=Qwen3-8B, Inference Strategy=CoT2026.02 | 50 | 50 | 82.46 | 62.25 | |
| BaselineBase Model=Llama3.1-Nemo-8B, Inference Strategy=CoT2026.02 | 48.67 | 47.5 | 33.93 | 39.58 | |
| BaselineBase Model=Llama3.1-Nemo-8B-Reasoning, Inference Strategy=CoT2026.02 | 46.49 | 45.45 | 35.09 | 39.6 | |
| BaselineBase Model=Llama3.1-Nemo-8B-Reasoning, Inference Strategy=Simple prompting2026.02 | 34.21 | 37.14 | 45.61 | 40.94 | |
| BaselineBase Model=Llama3.1-Nemo-8B, Inference Strategy=Simple prompting2026.02 | 33.33 | 35.82 | 42.11 | 38.71 |