Code Analysis on PSSec (test)
93.4Is-secure AccuracyPSSec(Qwen3-1.7B-SFT)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| PSSec(Qwen3-1.7B-SFT)Backbone=Qwen3-1.7B, Training Strategy=SFT, Training Epochs=10, Sum. API Cost ($)=0.5, Sum. Time Cost (ms)=10111.62026.01 | 93.4 | 96.7 | 96.5 | 89.1 | 90.9 | |
| PSSec(Qwen3-8B-SFT)Backbone=Qwen3-8B, Training Strategy=SFT, Training Epochs=3, Sum. API Cost ($)=2.5, Sum. Time Cost (ms)=22922.22026.01 | 93.2 | 96.9 | 96.8 | 89.1 | 91.8 | |
| PSSec(Qwen3-1.7B-SFT_RL)Backbone=Qwen3-1.7B, Training Strategy=SFT+RL, Training Epochs=10 SFT + 5 RL, Sum. API Cost ($)=0.5, Sum. Time Cost (ms)=10111.62026.01 | 91.8 | 96.8 | 96.3 | 85.6 | 89.9 | |
| GPT-4o + External KnowledgeBackbone=GPT-4o, External Knowledge=true, Sum. API Cost ($)=115.0, Sum. Time Cost (ms)=31552.12026.01 | 67.8 | 67.8 | 64.8 | 38.5 | 47.9 | |
| GPT-4oBackbone=GPT-4o, Sum. API Cost ($)=50.8, Sum. Time Cost (ms)=29890.22026.01 | 62 | 35.7 | 34 | 15.1 | 23.8 |