Secret Detection on LLMSec-DB
82.7EMAGPT-5
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-5Open-source (O)=false, Cost per million input tokens (MT)=$1.25, Total cost estimate (ALL)=$8402025.10 | 82.7 | 91.7 | 50 | 3 | 0 | 42 | 16 | 2 | 38 | 9 | 4 | |
| Claude Sonnet 4Open-source (O)=false, Cost per million input tokens (MT)=$3.00, Total cost estimate (ALL)=$20162025.10 | 81 | 87.7 | 50 | 0 | 0 | 44 | 10 | 0 | 39 | 8 | 3 | |
| Claude 3.7 SonnetOpen-source (O)=false, Cost per million input tokens (MT)=$3.00, Total cost estimate (ALL)=$20162025.10 | 80.3 | 92 | 50 | 0 | 0 | 43 | 21 | 1 | 37 | 12 | 4 | |
| Qwen-2.5 72BOpen-source (O)=true, Cost per million input tokens (MT)=$0.07, Total cost estimate (ALL)=$472025.10 | 80 | 91 | 50 | 2 | 0 | 44 | 11 | 0 | 38 | 11 | 3 | |
| Llama-3.1 405BOpen-source (O)=true, Cost per million input tokens (MT)=$0.80, Total cost estimate (ALL)=$5382025.10 | 80 | 87.7 | 49 | 2 | 1 | 44 | 11 | 0 | 36 | 10 | 5 | |
| Qwen3 235B-A22bOpen-source (O)=true, Cost per million input tokens (MT)=$0.30, Total cost estimate (ALL)=$2022025.10 | 78.7 | 86 | 48 | 3 | 2 | 42 | 2 | 2 | 33 | 8 | 8 | |
| Gemini 2.5 Flash LiteOpen-source (O)=false, Cost per million input tokens (MT)=$0.10, Total cost estimate (ALL)=$672025.10 | 77 | 86 | 45 | 1 | 5 | 44 | 8 | 0 | 33 | 10 | 8 | |
| GPT-5 NanoOpen-source (O)=false, Cost per million input tokens (MT)=$0.05, Total cost estimate (ALL)=$342025.10 | 76.7 | 85 | 50 | 6 | 0 | 44 | 17 | 0 | 34 | 13 | 7 | |
| Llama-3.3 70BOpen-source (O)=true, Cost per million input tokens (MT)=$0.04, Total cost estimate (ALL)=$272025.10 | 76 | 86.3 | 49 | 5 | 1 | 44 | 7 | 0 | 35 | 12 | 6 | |
| Gemini 2.5 ProOpen-source (O)=false, Cost per million input tokens (MT)=$1.25, Total cost estimate (ALL)=$8402025.10 | 75 | 93.3 | 50 | 0 | 0 | 42 | 24 | 2 | 32 | 14 | 8 | |
| Deepseek R1Open-source (O)=true, Cost per million input tokens (MT)=$0.40, Total cost estimate (ALL)=$2692025.10 | 75 | 86.7 | 48 | 5 | 2 | 43 | 17 | 1 | 31 | 16 | 9 | |
| GPT-5-miniOpen-source (O)=false, Cost per million input tokens (MT)=$0.25, Total cost estimate (ALL)=$1682025.10 | 71.3 | 84.3 | 50 | 11 | 0 | 44 | 24 | 0 | 30 | 17 | 10 | |
| NVIDIA Nemotron 9BOpen-source (O)=true, Cost per million input tokens (MT)=$0.25, Total cost estimate (ALL)=$1682025.10 | 70.3 | 80.3 | 50 | 15 | 0 | 44 | 19 | 0 | 29 | 18 | 11 | |
| GPT-4o miniOpen-source (O)=false, Cost per million input tokens (MT)=$0.15, Total cost estimate (ALL)=$1012025.10 | 67.7 | 86.3 | 50 | 13 | 0 | 43 | 9 | 1 | 28 | 17 | 13 | |
| Grok4Open-source (O)=false, Cost per million input tokens (MT)=$3.00, Total cost estimate (ALL)=$20162025.10 | 67 | 81.7 | 49 | 0 | 1 | 42 | 20 | 2 | 27 | 20 | 14 | |
| Gemma 3 27BOpen-source (O)=true, Cost per million input tokens (MT)=$0.07, Total cost estimate (ALL)=$472025.10 | 65.7 | 82 | 50 | 18 | 0 | 44 | 25 | 0 | 27 | 19 | 15 | |
| Gemini Flash-1.5 8BOpen-source (O)=false, Cost per million input tokens (MT)=$0.30, Total cost estimate (ALL)=$2022025.10 | 65 | 74.3 | 48 | 10 | 2 | 43 | 5 | 1 | 25 | 20 | 17 | |
| Mixtral 8x22bOpen-source (O)=true, Cost per million input tokens (MT)=$0.90, Total cost estimate (ALL)=$6052025.10 | 63.3 | 77.3 | 43 | 3 | 7 | 44 | 25 | 0 | 22 | 24 | 20 | |
| Mistral 7BOpen-source (O)=true, Cost per million input tokens (MT)=$0.03, Total cost estimate (ALL)=$202025.10 | 59 | 76.7 | 46 | 11 | 4 | 40 | 13 | 4 | 17 | 29 | 25 | |
| Llama-4 ScoutOpen-source (O)=true, Cost per million input tokens (MT)=$0.08, Total cost estimate (ALL)=$542025.10 | 59 | 80 | 50 | 43 | 0 | 44 | 20 | 0 | 16 | 30 | 26 | |
| Claude-3.5 HaikuOpen-source (O)=false, Cost per million input tokens (MT)=$0.08, Total cost estimate (ALL)=$542025.10 | 56 | 79 | 50 | 12 | 0 | 44 | 42 | 0 | 18 | 28 | 24 | |
| Gemma-3 12BOpen-source (O)=true, Cost per million input tokens (MT)=$0.04, Total cost estimate (ALL)=$272025.10 | 55.7 | 80 | 50 | 27 | 0 | 44 | 24 | 0 | 19 | 27 | 23 | |
| Qwen2.5 7BOpen-source (O)=true, Cost per million input tokens (MT)=$0.04, Total cost estimate (ALL)=$272025.10 | 54.7 | 65.7 | 47 | 26 | 3 | 36 | 24 | 8 | 20 | 26 | 22 | |
| Mixtral 8x7bOpen-source (O)=true, Cost per million input tokens (MT)=$0.40, Total cost estimate (ALL)=$2692025.10 | 54 | 65.3 | 38 | 2 | 12 | 31 | 6 | 13 | 21 | 25 | 21 | |
| Ministral 3BOpen-source (O)=true, Cost per million input tokens (MT)=$0.04, Total cost estimate (ALL)=$272025.10 | 49.3 | 56.7 | 37 | 2 | 13 | 23 | 11 | 21 | 14 | 32 | 28 |