Risk Classification on JD.com Risk Investigation 1.0 (evaluation set)
95PrecisionLegacy Hybrid
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| Legacy HybridModel=Hybrid (rule engines, XGBoost, GNN), Inference Mode=Non-LLM production system2025.10 | 95 | 82 | 88 | 4 | |
| Sherlock JoyAI-10BModel=JoyAI-10B, Inference Mode=STRO-fine-tuned with domain KB and R&R module2025.10 | 89 | 84 | 86 | 10 | |
| Sherlock Qwen3-8BModel=Qwen3-8B, Inference Mode=STRO-fine-tuned with domain KB and R&R module2025.10 | 85 | 86 | 85 | 15 | |
| Vanilla JoyAI-750BModel=JoyAI-750B, Inference Mode=Zero-shot2025.10 | 55 | 71 | 62 | 58 | |
| Vanilla DeepSeek-R1Model=DeepSeek-R1, Inference Mode=Zero-shot2025.10 | 52 | 74 | 61 | 68 | |
| Vanilla Qwen3-8BModel=Qwen3-8B, Inference Mode=Zero-shot2025.10 | 45 | 68 | 54 | 83 | |
| Vanilla JoyAI-10BModel=JoyAI-10B, Inference Mode=Zero-shot2025.10 | 42 | 65 | 51 | 90 |