Scam Detection Benchmark Streaming Cross-App
98.4Hit Rate (HR)ORACLE
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| ORACLEBackbone=Qwen3-4B, Category=ORACLE2026.05 | 98.4 | 29.5 | 1.3 | 98.2 | |
| ORACLEBackbone=Qwen3-1.7B, Category=ORACLE2026.05 | 96 | 30.3 | 5.4 | 89.2 | |
| ORACLEBackbone=Qwen3-0.6B, Category=ORACLE2026.05 | 95.3 | 44.2 | 10.2 | 89.1 | |
| GPT-5.1Category=General LLMs2026.05 | 80.8 | 46.4 | 12.8 | 77.3 | |
| Claude-4.1-OpusCategory=General LLMs2026.05 | 79.6 | 47.8 | 11.5 | 74.5 | |
| Claude-4-SonnetCategory=General LLMs2026.05 | 78.8 | 48.9 | 12.2 | 73.1 | |
| Gemini-3-FlashCategory=General LLMs2026.05 | 78.3 | 50.2 | 13.6 | 68.2 | |
| Kimi-2.5Category=General LLMs2026.05 | 77.5 | 51 | 11.9 | 69.3 | |
| GLM-4.6Category=General LLMs2026.05 | 76.9 | 52.1 | 10.7 | 70.4 | |
| DeepSeek-V4Category=General LLMs2026.05 | 76.4 | 42.2 | 11.2 | 68.3 | |
| Grok-4Category=General LLMs2026.05 | 74.2 | 55.6 | 13.9 | 65.1 |