Code Fix on PSSec (test)
0.935FSucRateGPT-4o + External Knowledge + PSScriptAnalyzer
Evaluation results
| Method | Links | |
|---|---|---|
| GPT-4o + External Knowledge + PSScriptAnalyzerBackbone=GPT-4o, External Knowledge=true, PSScriptAnalyzer=true, Sum. API Cost ($)=14.7, Sum. Time Cost (ms)=13355.72026.01 | 0.935 | |
| PSSec(Qwen3-8B-SFT)Backbone=Qwen3-8B, Training Strategy=SFT, Training Epochs=3, Sum. API Cost ($)=2.5, Sum. Time Cost (ms)=22922.22026.01 | 0.878 | |
| PSSec(Qwen3-1.7B-SFT_RL)Backbone=Qwen3-1.7B, Training Strategy=SFT+RL, Training Epochs=10 SFT + 5 RL, Sum. API Cost ($)=0.5, Sum. Time Cost (ms)=10111.62026.01 | 0.866 | |
| PSSec(Qwen3-1.7B-SFT)Backbone=Qwen3-1.7B, Training Strategy=SFT, Training Epochs=10, Sum. API Cost ($)=0.5, Sum. Time Cost (ms)=10111.62026.01 | 0.776 | |
| GPT-4o + External KnowledgeBackbone=GPT-4o, External Knowledge=true, Sum. API Cost ($)=115.0, Sum. Time Cost (ms)=31552.12026.01 | 0.659 | |
| GPT-4oBackbone=GPT-4o, Sum. API Cost ($)=50.8, Sum. Time Cost (ms)=29890.22026.01 | 0.548 |